Manual Full System Encryption has been updated and simplified

[Paddy Landau]

… install everything on the correct disk, my external USB Disk … I wasn't able to boot anymore.

I've tested this, and I admit defeat.

Here is what I've managed.

When I have a computer with something already installed on the hard drive

I install Ubuntu onto the USB. Thereafter, the only way that I can boot into the USB drive is to remove the hard drive (!), and even then it takes some fuss to get the USB to boot. After booting, it goes to the Ubuntu logo, and nothing more happens. However, if the hard drive is in the machine, it boots normally to the hard drive.

When I have a blank computer to start with

I install Ubuntu onto the USB. This works normally.
Then, I install something else onto the hard drive, ignoring the USB.
Thereafter, if the USB is in the machine, it installs normally onto the USB, and if the USB isn't in the machine, it installs normally onto the hard drive.

So, it seems that it works if you install onto the USB drive first, and then onto the hard drive.

You might have different results, of course, depending on hardware and the specifics of your installation.

Unfortunately, at this point, my lack of technical knowledge lets me down and I cannot take this any further. I don't understand why this happens, sorry.

[flyffies]

Thank you very much for testing this.

I, also admited defeat.
I installed ubuntu with the ubuntu Installer, enabling the encryption provided by the installer. That worked, so my data on my traveling disk is (somewhat) safe, should I loose it.


I noticed the installation also doesn't showup correctly on most boots. Though it creates an EFI entry for the bootloader and then I can select that one and it works.

Maybe that is missing in the full encryption setup?

Thank you again for investigating this

[Paddy Landau]

… it creates an EFI entry for the bootloader and then I can select that one and it works.

Specifically, which installation creates the new EFI entry — the original one on the disk or the one on your USB? I haven't been able to replicated it.

[flyffies]

The installation from the USB Drive.

I let the Ububtu installer install Ubuntu on my external drive. I disabled the internal SATA controller so only the install stick and the 2 Tb Drive would show up and let the installer just run. I mean by that, I didn't select "Something else" when asked where to install it.
I activated the check boxes for encryption and LVM.

Sometimes when I boot, the ubuntu boot option is not there, only the USB drive itself. When I boot it, it shows a message about creating the efi entry and reboots.
After that the ubuntu entry is back and I can boot without any problem. Tried it across 3 machines and it worked every time like that.

[Paddy Landau]

… I didn't select "Something else" when asked where to install it.

Ah, that's something else. That's the default installer, which works fine on a blank disk. It has a couple of significant disadvantages, namely that it deletes anything already on the disk (not a problem for you) and it doesn't encrypt /boot.

In other words, what you did was not the manual full system encryption discussed in this thread. So, it's useful to know if you are using a blank disk and don't care about encryption of /boot. I'm glad that you managed to sort it out, and I suspect that if you had left the SATA enabled, there is a chance that you would have had a problem.

[flyffies]

Yes, I had to have a working Ubuntu for traveling, so I settled with the default installer. Not happy with it, but the best I can currently have on my portable USB Drive, because the manual full encyption does loose the EFI entry for ubuntu, if the Drive is plugged into another pc or just another port (its very strange).

Thought the thing that the default installer does, where it creates the missing EFI entry could fix it for the manual method, but I am not experienced enough to figure that out...

I only disabled the internal SATA controller so my windows disk could not be accidentally be written to.

[Paddy Landau]

Thought the thing that the default installer does, where it creates the missing EFI entry could fix it for the manual method, but I am not experienced enough to figure that out...

You're probably right, but I too have no clue how to do this.

I only disabled the internal SATA controller so my windows disk could not be accidentally be written to.

That was a good idea, because the default Ubuntu installer might have wiped it. The default full-encryption option is rather poor, reminding me of the earlier Windows versions that also went ahead and wiped the entire disk without asking.

[3-david-o]

Hi Paddy,
I appreciate all the work you've put into getting this working, and making it easy to install. Unfortunately, things arent working out for me. I'm installing on a new machine, alongside Windows 10, with secure boot enabled. Everything goes smoothly until the "Check and finalise" part. When I reboot my machine, I get the blue MOK management screen. I follow the menus to enrol the key, which all goes smoothly. Then I reboot again. I do not get the EFI screen shown in your instructions, I just get the standard Grub menu. When I select Ubuntu, I do not get prompted for a passphrase, I get the error message:

No such device: ec61b9bf-6895-43fd-84fb-c018e117e17f.
No server is specified.
You need to load the kernel first

I have been through the Troubleshooting process - it didnt help. I have spent a long time trying to understand the problem. The UUID quoted is that of the boot partition inside the encrypted container. I've examined the grub.cfg file in detail. It seems to me that cryptomount is silently failing. I found that the UUID given to cryptomount doesnt have any hyphens in it. I tried adding them back in, and also using "cryptomount -a", but neither made any difference. I've tried using the grub command line to try out cryptomount manually - but it just does nothing: whatever parameters I pass to cryptomount or insmod I get a success exit status, so this gives me no clues about which commands will work successfully, and there's certainly no prompting for a passphrase.

I've gone back and done the whole installation process again, just to be sure I didnt make a mistake. Everything is exactly the same as before (though with a different UUID, obviously) (though I notice your instructions have changed a little - no more refreshGrub I see.)

I'm wondering what else I can do to troubleshoot this situation. Any ideas?

[Paddy Landau]

… When I reboot my machine, I get the blue MOK management screen.

Unfortunately, my lack of technical expertise gets in the way here. I didn't know what a MOK management screen was until I looked it up.

I wonder if this has anything to do with the secure-boot bug?

I found a troubleshooting guide. I have absolutely no idea whether or not it would work with you, and whether or not it would disable secure boot on your machine.

Is this a new machine? If so, it likely does adhere to standards, so that's probably not the problem.

I'm sorry that I can't be of more help; I really am clueless as to how the process works. I only documented what it does with extensive help from other people.

By the way, it still uses refreshgrub, but it now runs automatically behind the scenes.

Can I confirm that you are using Ubuntu 18.04 64-bit, rather than one of the derivatives?

[3-david-o]

Hi Paddy,
The MOK screen was behaving as expected (it only appears on the first boot after installing Ubuntu). Yes, its a new machine (Lenovo x380), and I'm installing Ubuntu 18.04.1.

I did some more exploring with grub and (to cut a long story short) discovered this bug, which clearly explains that "Ubuntu cannot boot from an encrypted volume with Secure Boot enabled" because the required grub modules (luks and cryptodisk) are not included in the distributed signed grub binary, and, for security reasons, grub will not load them subsequently (so the "insmod" commands fail - though they return a success exit status nevertheless).

So, I disabled UEFI Secure Boot, and found that I now get

Attempting to decrypt master key...
Enter passphrase for hd0,gpt5 (1ee8a2ca...):

All well and good. Except that after I enter the passphrase, nothing happens for 60 seconds. Then I get the Ubuntu startup screen, and then after another 65 seconds, I get the Busybox ... (initramfs) screen. "Exit" then tells me: "ALERT! /dev/mapper/system-root does not exist.", and I'm taken back to the initramfs prompt again. If I then do "ls /dev/mapper", I find that the only thing listed is "control". So, it seems that the boot process hasnt completed properly and found the (unlocked?) partitions with Ubuntu on.

I havent yet started searching for solutions to this, but I wondered if you had come across this yourself?