TheFu
February 2nd, 2018, 10:00 PM
TL;DR - fresh 16.04 upgrade from 14.04 on a x64 system with LVM and lots of storage. It is a storage and plex server and calibre and NFS and ... you get the idea. System reboots 12 seconds after the desktop is displayed. This is a physical host. I use lots and lots of virtual machines, but this is one of the few real physical installs that runs directly on hardware.
If I go in through rescue mode, manually bring up all the LVs, manually bring up networking, all seems fine. The system does not reboot.
No HW issues are showing up in log files. There weren't any HW issues before the 16.04 install.
My question is this. What could cause the system to shutdown cleanly - systemd seems to be calling the shutdown target automatically - without any user request?
More details than you want ...
Updated from 14.04 to 16.04 (istar is the hostname) this morning. Everything seemed to go fine.
Did the typical:
* make backup
* apt update
* apt dist-upgrade
* do-release-upgrade
* reboot
and walked away from istar for 10 minutes.
The machine turned off. Hum.... so I pressed the power, saw all the expected lights come on and walked away to use my desktop in a different room. About 10 min later, tried to ssh into istar. Nothing. ping istar. Nothing. Great, not!
Back to the console. Watch the boot this time. Login, open a terminal and bam, it starts shutting down. Huh? I didn't ask to reboot or shutdown. I just right-clicked and selected "terminal" from the openbox window-manager menu. Vaguely remember it has a 4.x kernel. The 14.04 kernel was 3.13.x
Tried to boot using the 3.13.x kernel. Same behavior. About 10-12 seconds after the desktop is displayed, the system starts rebooting.
I go into recovery mode, have to fsck the disks to get read-write access to storage, then drop into a maintenance/root shell and look around. syslog is showing a warning about "lvmetad" near the end, so I google that with 14.04-->16.04 upgrade ... someone suggests to turn that off in the /etc/lvm/lvm.conf file. Fine - do that. Reboot. 12 seconds after the desktop is shown, it reboots again. Time for more googling.
Some people say that after an upgrade to 16.04 the volume groups aren't being seen and they have a workaround by creating /etc/initramfs-tools/scripts/local-top/lvm with lvm vgchange -aay to force all LVs to be discovered at boot. Add that file, chmod +x it, update-initramfs ; update-grub; reboot ... 12 seconds after the desktop is shown, it reboots again.
I quickly save the dmesg output into a file before the reboot happens again, then go back into rescue mode, fsck, root shell, and view the dmesg stuff looking for hints. Nothing there seems odd.
If I go in via rescue mode, mount all the storage, manually bring up the networking, start autofs, then the system seems pretty stable. Obviously, some services (which are the entire point for this machine) aren't running, but it doesn't reboot. It is on the network. Storage is available.
This is a desktop system. It never sleeps. No hibernation. It doesn't have batteries to fail. I've never configured those things on it and certainly not within 30 seconds of boot. The CPU has never run hot. Monitoring shows it runs around 45 degC most of the time over the last 6 months.
My question is this. What could cause the system to shutdown cleanly - systemd seems to be calling the shutdown target automatically - without any user request?
I've googled, searched, and considered possibilities. No joy.
I can restore from prior backups, but I'm more likely to just do a fresh install with 16.04 and see what happens. Just wanted to see if someone here had some ideas for what it might be?
If I go in through rescue mode, manually bring up all the LVs, manually bring up networking, all seems fine. The system does not reboot.
No HW issues are showing up in log files. There weren't any HW issues before the 16.04 install.
My question is this. What could cause the system to shutdown cleanly - systemd seems to be calling the shutdown target automatically - without any user request?
More details than you want ...
Updated from 14.04 to 16.04 (istar is the hostname) this morning. Everything seemed to go fine.
Did the typical:
* make backup
* apt update
* apt dist-upgrade
* do-release-upgrade
* reboot
and walked away from istar for 10 minutes.
The machine turned off. Hum.... so I pressed the power, saw all the expected lights come on and walked away to use my desktop in a different room. About 10 min later, tried to ssh into istar. Nothing. ping istar. Nothing. Great, not!
Back to the console. Watch the boot this time. Login, open a terminal and bam, it starts shutting down. Huh? I didn't ask to reboot or shutdown. I just right-clicked and selected "terminal" from the openbox window-manager menu. Vaguely remember it has a 4.x kernel. The 14.04 kernel was 3.13.x
Tried to boot using the 3.13.x kernel. Same behavior. About 10-12 seconds after the desktop is displayed, the system starts rebooting.
I go into recovery mode, have to fsck the disks to get read-write access to storage, then drop into a maintenance/root shell and look around. syslog is showing a warning about "lvmetad" near the end, so I google that with 14.04-->16.04 upgrade ... someone suggests to turn that off in the /etc/lvm/lvm.conf file. Fine - do that. Reboot. 12 seconds after the desktop is shown, it reboots again. Time for more googling.
Some people say that after an upgrade to 16.04 the volume groups aren't being seen and they have a workaround by creating /etc/initramfs-tools/scripts/local-top/lvm with lvm vgchange -aay to force all LVs to be discovered at boot. Add that file, chmod +x it, update-initramfs ; update-grub; reboot ... 12 seconds after the desktop is shown, it reboots again.
I quickly save the dmesg output into a file before the reboot happens again, then go back into rescue mode, fsck, root shell, and view the dmesg stuff looking for hints. Nothing there seems odd.
If I go in via rescue mode, mount all the storage, manually bring up the networking, start autofs, then the system seems pretty stable. Obviously, some services (which are the entire point for this machine) aren't running, but it doesn't reboot. It is on the network. Storage is available.
This is a desktop system. It never sleeps. No hibernation. It doesn't have batteries to fail. I've never configured those things on it and certainly not within 30 seconds of boot. The CPU has never run hot. Monitoring shows it runs around 45 degC most of the time over the last 6 months.
My question is this. What could cause the system to shutdown cleanly - systemd seems to be calling the shutdown target automatically - without any user request?
I've googled, searched, and considered possibilities. No joy.
I can restore from prior backups, but I'm more likely to just do a fresh install with 16.04 and see what happens. Just wanted to see if someone here had some ideas for what it might be?