16.04: Mount existing boot partition from live USB to repair grub?

[Dáire Fagan]

I am using a custom install after following Paddy Laundau's ManualFullSystemEncryption (https://help.ubuntu.com/community/ManualFullSystemEncryption) guide (this is a lengthy article and just here for reference), it has been working fine, and it automatically detects specific updates related to specific parts of the system, and runs the refreshgrub script to make the required changes to grub so everything keeps working. Last week I ran a sudo apt update, which prompted me to run a sudo apt upgrade, and during this changes were made to grub, and I was prompted in what looked like a coloured terminal screen if I wanted to keep my current grub config or the maintainers version. I compared the files and they did not seem very different so I thought if there was any issue I could make the required changes, so I selected the maintainers version. There was some error during the update process, there was no notification of the refreshgrub script running - I should have manually ran it - and on reboot I see the following errors after selecting Ubuntu, same if I try to boot to recovery:

I have booted into a 16.04 live USB, and mounted the container containing boot and root. I open in a terminal the dir /media/ubuntu/root/usr/local/sbin and then input sudo ./refreshgrub but this outputs the error:

282757


cp: cannot stat '/boot/grub/x86_64-efi': No such file or directory
Failed to copy boot modules to EFI.

Is there a way I can mount boot somehow from the live USB so that the script will repair it for me? The script is normally called from within a working Ubuntu install, so I assume the boot partition would have to be mounted in such a way that the script sees it as such.

[Dáire Fagan]

Mod please delete this post.

[yancek]

The image you posted in your first post indicates that a specific UUID (listed in the image) does not exist. From the Ubuntu usb in a terminal, run the command below to get the actual UUIDs on the system to compare to the one in the error.


sudo blkid

You should certainly be able to create a mount point for boot. Is it on the / filesystem partition or do you have a separate boot partition? Just use sudo mkdir to create the mount point and then mount it. This is an EFI install, correct? You may need to chroot into the system from the usb.

[Dáire Fagan]

Hi yancek, thanks for the input.


The image you posted in your first post indicates that a specific UUID (listed in the image) does not exist. From the Ubuntu usb in a terminal, run the command below to get the actual UUIDs on the system to compare to the one in the error.


Unsure exactly what we need, I am going to risk providing too much info here, rather than too little. After just booting into the Live USB:


ubuntu@ubuntu:~$ sudo lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABELNAME FSTYPE SIZE MOUNTPOINT LABEL
loop0 squashfs 1.8G /rofs
loop1 squashfs 91M /snap/core/6350
loop2 squashfs 34.6M /snap/gtk-common-themes/818
loop3 squashfs 140.7M /snap/gnome-3-26-1604/74
loop4 squashfs 2.3M /snap/gnome-calculator/260
loop5 squashfs 13M /snap/gnome-characters/139
loop6 squashfs 14.5M /snap/gnome-logs/45
sda 232.9G
├─sda1 vfat 599.9M SYSTEM
├─sda2 16M
├─sda3 100G
├─sda4 ntfs 450M
├─sda5 crypto_LUKS 62.5G
└─sda6 crypto_LUKS 69.4G

ubuntu@ubuntu:~$ sudo blkid
/dev/sda1: LABEL="SYSTEM" UUID="E625-C979" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="8a07c467-0d92-43e2-b65c-0f380900c0e8"
/dev/sda4: UUID="549C56F89C56D458" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="947b53c9-2cae-4ab4-8597-eaa7d78d6cad"
/dev/sde1: LABEL="UBUNTU 18_0" UUID="6A33-63D0" TYPE="vfat" PARTUUID="002f0b65-01"
/dev/loop0: TYPE="squashfs"
/dev/loop1: TYPE="squashfs"
/dev/loop2: TYPE="squashfs"
/dev/loop3: TYPE="squashfs"
/dev/loop4: TYPE="squashfs"
/dev/loop5: TYPE="squashfs"
/dev/loop6: TYPE="squashfs"
/dev/loop7: TYPE="squashfs"
/dev/sda2: PARTLABEL="Microsoft reserved partition" PARTUUID="5e5c6ec1-0c93-4e2c-a55b-1820cadb6cdf"
/dev/sda3: PARTLABEL="Basic data partition" PARTUUID="90252cd2-5d58-418e-a368-2e29b02752e8"
/dev/sda5: UUID="fd9f3d69-ea4d-40e2-8f99-44cfe0a8307c" TYPE="crypto_LUKS" PARTLABEL="system" PARTUUID="f38563c6-55cf-4bf9-85cd-30b2d10bbbc7"
/dev/sda6: UUID="2fcffbba-fa50-47e3-9dcc-7bff7bf0623a" TYPE="crypto_LUKS" PARTLABEL="data" PARTUUID="b89dc015-0964-4cda-a048-9ac44ca7d5b4"

After mounting and decrypting with Nautilus sda5 (root and boot), and sda6 (home):


ubuntu@ubuntu:~$ sudo lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT,LABEL
NAME FSTYPE SIZE MOUNTPOINT LABEL
loop0 squashfs 1.8G /rofs
loop1 squashfs 91M /snap/core/6350
loop2 squashfs 34.6M /snap/gtk-common-themes/818
loop3 squashfs 140.7M /snap/gnome-3-26-1604/74
loop4 squashfs 2.3M /snap/gnome-calculator/260
loop5 squashfs 13M /snap/gnome-characters/139
loop6 squashfs 14.5M /snap/gnome-logs/45
loop7 squashfs 3.7M /snap/gnome-system-monitor/57
sda 232.9G
├─sda1 vfat 599.9M SYSTEM
├─sda2 16M
├─sda3 100G
├─sda4 ntfs 450M
├─sda5 crypto_LUKS 62.5G
│ └─luks-fd9f3d69-ea4d-40e2-8f99-44cfe0a8307c
│ LVM2_member 62.5G
│ ├─system-boot ext4 512M boot
│ ├─system-swap swap 32G
│ └─system-root ext4 30G root
└─sda6 crypto_LUKS 69.4G
└─luks-2fcffbba-fa50-47e3-9dcc-7bff7bf0623a
LVM2_member 69.4G
└─data-home ext4 69.4G home

ubuntu@ubuntu:~$ sudo blkid
/dev/sda1: LABEL="SYSTEM" UUID="E625-C979" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="8a07c467-0d92-43e2-b65c-0f380900c0e8"
/dev/sda4: UUID="549C56F89C56D458" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="947b53c9-2cae-4ab4-8597-eaa7d78d6cad"
/dev/sde1: LABEL="UBUNTU 18_0" UUID="6A33-63D0" TYPE="vfat" PARTUUID="002f0b65-01"
/dev/loop0: TYPE="squashfs"
/dev/loop1: TYPE="squashfs"
/dev/loop2: TYPE="squashfs"
/dev/loop3: TYPE="squashfs"
/dev/loop4: TYPE="squashfs"
/dev/loop5: TYPE="squashfs"
/dev/loop6: TYPE="squashfs"
/dev/loop7: TYPE="squashfs"
/dev/sda5: UUID="fd9f3d69-ea4d-40e2-8f99-44cfe0a8307c" TYPE="crypto_LUKS" PARTLABEL="system" PARTUUID="f38563c6-55cf-4bf9-85cd-30b2d10bbbc7"
/dev/sda6: UUID="2fcffbba-fa50-47e3-9dcc-7bff7bf0623a" TYPE="crypto_LUKS" PARTLABEL="data" PARTUUID="b89dc015-0964-4cda-a048-9ac44ca7d5b4"
/dev/sdc1: UUID="8861b3b9-b605-4677-8984-dcadb1c300c4" TYPE="crypto_LUKS" PARTUUID="e0ce77ae-95e1-d143-b7ee-98cf8ed57765"
/dev/sdd1: UUID="26a2e077-6806-4e42-96cd-924e75406a03" TYPE="crypto_LUKS" PARTUUID="7e4f4e89-9f4e-ea4d-8d96-05356fc3f958"
/dev/sda2: PARTLABEL="Microsoft reserved partition" PARTUUID="5e5c6ec1-0c93-4e2c-a55b-1820cadb6cdf"
/dev/sda3: PARTLABEL="Basic data partition" PARTUUID="90252cd2-5d58-418e-a368-2e29b02752e8"
/dev/sdb1: PARTLABEL="Basic data partition" PARTUUID="24de0cc7-75b0-471e-b8cf-052f4acae73d"
/dev/mapper/luks-fd9f3d69-ea4d-40e2-8f99-44cfe0a8307c: UUID="swt3sR-xEZd-szrb-Mxez-Kqjh-7Dxs-aJmhco" TYPE="LVM2_member"
/dev/mapper/system-boot: LABEL="boot" UUID="924a73b7-12e0-44d3-93fb-6d54c668dfc0" TYPE="ext4"
/dev/mapper/system-swap: UUID="bb3547e6-4dea-44c0-9408-1e04570ec070" TYPE="swap"
/dev/mapper/system-root: LABEL="root" UUID="d668eb62-0855-4e73-a703-cfdcb088b796" TYPE="ext4"
/dev/mapper/luks-2fcffbba-fa50-47e3-9dcc-7bff7bf0623a: UUID="NIDDRi-x2Vu-2zpj-3ezd-PtnP-0FT2-mLUjVB" TYPE="LVM2_member" /dev/mapper/data-home: LABEL="home" UUID="025be7eb-2017-478f-b4c7-5b35cdae71d5" TYPE="ext4"



You should certainly be able to create a mount point for boot. Is it on the / filesystem partition or do you have a separate boot partition? Just use sudo mkdir to create the mount point and then mount it. This is an EFI install, correct? You may need to chroot into the system from the usb.

Is sudo mkdir to create the mount point not the same as what Nautilus does? To be clear, before my original post, I decrypted and mounted with Nautilus by just selecting other places/locations from the left hand menu and double clicking encrypted sda5 and sda6 partitions. I had thought that perhaps boot needs to be mounted somewhere specific for the refreshgrub script to fix it, considering the error in my original post about something not being found?

[oldfred]

Nautilus uses UUID as default, or label if you have labeled partitions. I try to label all partitions, but only mount some with fstab.
If partition already mounted with fstab, you cannot mount it again.

You show you have a /boot partition and its UUID matches the error?
/dev/mapper/system-boot: LABEL="boot" UUID="924a73b7-12e0-44d3-93fb-6d54c668dfc0" TYPE="ext4"

I would reinstall grub, but make sure encrypted LVM is mounted and decrypted.

[Dáire Fagan]

Nautilus uses UUID as default, or label if you have labeled partitions. I try to label all partitions, but only mount some with fstab.
If partition already mounted with fstab, you cannot mount it again.

You show you have a /boot partition and its UUID matches the error?
/dev/mapper/system-boot: LABEL="boot" UUID="924a73b7-12e0-44d3-93fb-6d54c668dfc0" TYPE="ext4"

I would reinstall grub, but make sure encrypted LVM is mounted and decrypted.

Hi oldfred.

I mounted with Nautilus - from within the Ubuntu Live USB 'try without installing', because I cannot boot normally, even with recovery - not sure if you saw that in my original post?

That UUID is only visible after I unencrypted sda5 within Nautilus, so grub cannot see it...

Is it possible to reinstall grub on to a target system, booting from the Live USB? If so, please tell me how. Also, I am not sure a normal install of grub will fix this, as I believe the refreshgrub script is required to fix it - if I can just get back into the system, I can run the script from there, if not from the Live USB somehow?

[oldfred]

If you mount LVM and decrypt your install, then Boot-Repair should work, otherwise you have to chroot into your install. I think Boot-Repair in its advanced mode to totally reinstall grub, just walks you thru the minimum steps required to reinstall grub. Better to use Advanced mode than whatever it suggests as auto-fix.

May be best to see details, use ppa version with your live installer or any working install, not older Boot-Repair ISO:
Please copy & paste link to the Boot-info summary report ( do not post report), the auto fix sometimes can create more issues.
https://help.ubuntu.com/community/Boot-Repair

[Dáire Fagan]

I tried boot-repair several times, but it always ends with it telling me there was a problem during repair, and when I reboot I now just have a grub prompt, instead of a menu.

This is the report it output:

http://paste.ubuntu.com/p/93XQSjxyXG/

Any ideas?

[oldfred]

Are you using encrypted Windows? It shows a Veracrypt boot entry.

The NTFS Linux driver cannot open encrypted nor hibernated NTFS partitions, so grub will never be able to boot those.
Not sure if the chainload to veracrypt boot entry may work or not.

[Dáire Fagan]

Are you using encrypted Windows? It shows a Veracrypt boot entry.

The NTFS Linux driver cannot open encrypted nor hibernated NTFS partitions, so grub will never be able to boot those.
Not sure if the chainload to veracrypt boot entry may work or not.

Chain loading works, I have been using it the last few months, for reference: Using VeraCrypt with a UEFI dual boot setup (https://medium.com/@lankycyril/using-veracrypt-with-a-uefi-dual-boot-setup-27d1eacbf36b)

So after boot-repair did not resolve the issue, which was not surprising as it is a complex setup, I properly read the description of the fix-grub.sh (https://gist.github.com/lovromazgon/7d0a5b6ac8f7557059a8b97e8442720b) script, which is designed to resolve issues when a ManualFullSystemEncryption system fails to boot after upgrade or new installation, and I ran it while logged in from a Live USB. On rebooting, there was now a grub menu again, although with many entries:

282761

None of the Ubuntu entries worked, giving the same or similar errors as before. I contacted the developer of the fix-grub.sh script, and he noticed that I was missing a line from /boot/grub/grub.cfg, and so for my Ubuntu menuentry, I manually edited the file to now include above the line beginning 'set root=':

cryptomount -u fd9f3d69ea4d40e28f9944cfe0a8307c

The string in the command after -u is just the UUID of sda5, which in my case contains my system-boot, system-root, and system-swap partitions:

The author of fix-grub.sh also noticed my /etc/default/grub was missing the following line, so I added it to the end of that file:


GRUB_ENABLE_CRYPTODISK=y

The top of the file /boot/grub/grub.cfg warns it should not be edited directly, but I did not know how to edit the config from a Live USB (I am guessing this might be the kind of thing 'chrooting into a system' might allow), so I tried it. As expected, when in grub I pressed e with Ubuntu selected, I could see that the cryptomount -u line was not there, so very carefully I manually typed it in, and thankfully I was prompted for the passphrase and I was able to boot normally at last.

Once logged in I opened /usr/local/sbin/ in a terminal and input


sudo ./refreshgrub

This seemed to add the cryptomount -u line the correct way to all of the Ubuntu entries, recovery etc, in /etc/default/grub.

With everything in grub still working properly on reboot, all that remained was to remove the extra entries boot-repair had added, and I did this by manually removing them with grub-customizer, though I discovered the only way to make the changes apply after saving is to run refreshgrub again - not sudo update-grub.

This issue is now resolved, thank you to all who helped, and especially the author of the fix-grub.sh script.