Page 4 of 14 FirstFirst ... 23456 ... LastLast
Results 31 to 40 of 139

Thread: Manual Full System Encryption has been updated and simplified

  1. #31
    Join Date
    May 2008
    Location
    United Kingdom
    Beans
    4,574
    Distro
    Lubuntu 18.04 Bionic Beaver

    Re: Manual Full System Encryption has been updated and simplified

    Quote Originally Posted by 3-david-o View Post
    I did some more exploring with grub and (to cut a long story short) discovered this bug
    Thank you for the information. I have upvoted the bug, and I've now included it in the documentation.

    It seems that Ubuntu has a little way to go still until the process works fully. It's a shame that it's not being taken more seriously.

    If you find a solution, please let us know. In the meantime, we rely on Canonical to take this up seriously.

  2. #32
    Join Date
    Sep 2018
    Beans
    6

    Re: Manual Full System Encryption has been updated and simplified

    Hi Paddy,
    I re-ran the encrypted installation process again, this time with Secure Boot turned off at the outset. At last I have a system that works successfully! But, I still find that there is a 60 second delay after I enter my passphrase before Ubuntu begins to boot. Is this something you've observed? Do you know if there's anything I can do about it?

    It's good that you've put a reference to the Secure Boot bug in your documentation. I think you should make it more explicit, though, to stop people falling into the same problem as me:
    Warning: You must disable Secure Boot before you carry out this installation procedure. If you install with Secure Boot enabled, your computer will not be able to boot (due to this bug). Even if you subsequently turn off secure boot, your computer will still not boot!

  3. #33
    Join Date
    May 2008
    Location
    United Kingdom
    Beans
    4,574
    Distro
    Lubuntu 18.04 Bionic Beaver

    Re: Manual Full System Encryption has been updated and simplified

    Quote Originally Posted by 3-david-o View Post
    I still find that there is a 60 second delay…
    As mentioned earlier, I suspect that what is happening is that some of the boot process is happening but being hidden behind the screen, because Grub and the kernel need to be loaded once the partition has been decrypted. I could be wrong, but I get this delay even when my computer isn't encrypted.

    Quote Originally Posted by 3-david-o View Post
    I think you should make it more explicit…
    Good idea. I've amended the documentation accordingly. Have you visited the bug to give your support? (Log in and select the green writing near the top right.)

  4. #34
    Join Date
    Sep 2018
    Beans
    6

    Re: Manual Full System Encryption has been updated and simplified

    Hi Paddy,
    A number of people have reported long delays in decrypting LUKS containers when they need to be decrypted by Grub. The cause of this is largely explained here and here. Essentially, cryptsetup "scrambles" your passphrase before it is stored (by hashing it many times, know as iterations). (This is to guard against brute-force attempts to guess the passphrase.) This scrambling must be repeated each time your passphrase is entered and used to unlock the encrypted device. By default, cryptsetup will spend 2 seconds scrambling your passphrase when it is first created. But when booting, in a grub environment, the scrambling can take 10-20 times longer, so it may take 20-40 seconds.

    On top of this, the encrypted installation uses two decryption keys - one for the passphrase the user enters, and one that's used by the machine in the second half of the boot process. When you enter a passphrase, it is checked against each key in turn. Guess what - the machine key is stored before the user's passphrase key, so time is wasted checking against that key, and so this doubles the delay.

    Fortunately, it is easy to change the ordering of the keys (so that the user's passphrase key is checked first, not second), and to change the amount of work the machine has to do to scramble the passphrase (specified using: cryptsetup --iter-time). The basic approach is described here, but it isnt explained very clearly. I can provide you with more detailed instructions of how I did this if you wish.

    On my machine (a Lenovo x380), I did some experiments, and found that the first boot phase (when grub is doing the work) takes 8 seconds, plus the time taken to scramble the passphrase. The second boot phase (when grub has passed control to Ubuntu) takes 2 seconds, plus the time taken to scramble the machine key. During the second phase, the scrambling is about 20 times faster than in the first phase. I have adjusted the way my keys are stored so that only a couple of seconds are added to each of the boot phases.

    There are potential security implications of reducing the amount of "scrambling" done to the passphrase, but so long as a sufficiently long/complex passphrase is used, there's no problem. This topic is well discussed here.

    I think that your encryptinstallation script could easily be improved so that the key for the user's passphrase is stored before the machine key - which would halve the delay. It would also be fairly easy to provide users with instructions (or, better still a utility script) that allows them to easily adjust the key "scrambling" time, in case they want to reduce the time taken during boot. I'm happy to help if I can.

  5. #35
    Join Date
    May 2008
    Location
    United Kingdom
    Beans
    4,574
    Distro
    Lubuntu 18.04 Bionic Beaver

    Re: Manual Full System Encryption has been updated and simplified

    @3-david-o Thank you for all of that information. When creating the LUKS partition, the user's key is used, while the system key is added afterwards. Are you saying that this order causes Grub to try the system key first? Or something a little different?

    If you can help me with an adjusted procedure, or tell me what commands you used to change the key order, I'll see if I can incorporate that into the encryption script. If it reduces the boot time, that will benefit everyone.

    Thanks again.

  6. #36
    Join Date
    Sep 2018
    Beans
    8

    Re: Manual Full System Encryption has been updated and simplified

    Hello, i'm not sure I understand the purpose of the reapplyGrubUpdates function, is that because boot is encrypted and we have to rely entirely on efi partition to boot while we usually rely mostly on a simple efi file that delegate the rest to content in /boot?

    If having /boot in a separate unencrypted1 partition (ideally on a removable storage), is this function still necessary?

    1) Encrypted, but data made clear for current runtime before calling grub.

    Your script calls wget to obtain other scripts from remote location, this is usually considered not secure, cannot the script be hosted on a
    git and have the user do one git clone instead of a wget that will call wget again later ? (Plus easier collaboration if someone can and want to share improvement made to your script).

    Script would then cp to /sbin instead of wget.

    Thank you for your contribution, I hope this will get popular enough to have something similar backed in officially with fancy passphrase files right into the installer's GUI.

  7. #37
    Join Date
    May 2008
    Location
    United Kingdom
    Beans
    4,574
    Distro
    Lubuntu 18.04 Bionic Beaver

    Re: Manual Full System Encryption has been updated and simplified

    Quote Originally Posted by archphoenix View Post
    … the purpose of the reapplyGrubUpdates function
    When any update changes the Grub or initramfs (e.g. a kernel update), Grub is incorrectly updated. This causes the next boot of the computer to fail. The script reapplyGrubUpdates refreshgrub detects this and then automatically runs, re-updating Grub and initramfs correctly.

    Quote Originally Posted by archphoenix View Post
    … is that because boot is encrypted…
    Sorry, I don't have the answer to that. I'm not technically competent enough to understand why

    Quote Originally Posted by archphoenix View Post
    If having /boot in a separate unencrypted1 partition (ideally on a removable storage), is this function still necessary?
    I haven't tested what will happen if you have an unencrypted /boot. However, if /boot is encrypted, you don't need it on removable storage.

    Quote Originally Posted by archphoenix View Post
    Your script calls wget to obtain other scripts from remote location, this is usually considered not secure, cannot the script be hosted on a git and have the user do one git clone instead of a wget that will call wget again later ?

    Ah, you're the first person to answer my question of where we could host the scripts! I asked elsewhere, had no answer, so decided on Dropbox.

    I have absolutely no idea how to use git. If you, or someone else, would guide my hand with this, I'd be happy to do so.

    Quote Originally Posted by archphoenix View Post
    … I hope this will get popular enough to have something similar backed in officially with fancy passphrase files right into the installer's GUI.
    If you haven't already done so, please upvote the four bugs that the instructions list: bug #1401532, bug #1514120, bug #1565950, and bug #1773457. Canonical won't bother unless it is persuaded of the need.
    Last edited by Paddy Landau; September 24th, 2018 at 02:49 PM. Reason: Replace reapplyGrubUpdates with refreshgrub

  8. #38
    Join Date
    Sep 2018
    Beans
    8

    Re: Manual Full System Encryption has been updated and simplified

    Hello,
    Ah, you're the first person to answer my question of where we could host the scripts! I asked elsewhere, had no answer, so decided on Dropbox.

    I have absolutely no idea how to use git. If you, or someone else, would guide my hand with this, I'd be happy to do so.
    I didn't notice you asked for this, sorry.

    If using fancy git hosting providers such as github, you have a full web GUI where you can add, remove or modify text files (script) without knowing how the git command works.
    In addition, such git host would allow you to review changes proposed by other before merging them or rejecting them and you will have a modification history allowing easy reverse of problematic changes.

    Since your project isn't closed source, most would probably tell you gitlab public git is the best place to host your work.
    As for my personal opinion, all major host works well, but you will have to dig into their documentation as they all have their own GUI take the one you feel the easiest to use for you as they're basically all free for open source projects. Most are really straightforward to use without reading the documentation as long as you interact only with the GUI.

    There's also bitbucket, sourceforge and many others I would not list.

    When any update changes the Grub or initramfs (e.g. a kernel update), Grub is incorrectly updated. This causes the next boot of the computer to fail. The script reapplyGrubUpdates detects this and then automatically runs, re-updating Grub and initramfs correctly.
    Does it happen when full disk encryption isn't used too ?

    I haven't tested what will happen if you have an unencrypted /boot. However, if /boot is encrypted, you don't need it on removable storage.
    That's for paranoid users, required when you hold certain type of data.

    If you haven't already done so, please upvote the four bugs that the instructions list: bug #1401532, bug #1514120, bug #1565950, and bug #1773457. Canonical won't bother unless it is persuaded of the need.
    Have marked i'm affected, unsure if it is what you are calling upvote.

  9. #39
    Join Date
    May 2008
    Location
    United Kingdom
    Beans
    4,574
    Distro
    Lubuntu 18.04 Bionic Beaver

    Re: Manual Full System Encryption has been updated and simplified

    Quote Originally Posted by archphoenix View Post
    If using fancy git hosting providers such as github
    Thanks. When I have some time, I'll look at both GIT and SourceForge.

    Quote Originally Posted by archphoenix View Post
    Does it happen when full disk encryption isn't used too ?
    refreshgrub is used with full-system encryption (the process that encrypts everything including /boot but not EFI, obviously, or any other OS on the disk). If you use the Ubuntu installer, it wipes the entire disk and uses its own method that does work, so refreshgrub isn't required. I think that that answers your question. Ask again if I misunderstood.

    Quote Originally Posted by archphoenix View Post
    That's for paranoid users, required when you hold certain type of data.
    Understood, although if /boot is encrypted along with the rest of the system and data, there's no need to hold it separately, because it's as safe as the data itself.

    Quote Originally Posted by archphoenix View Post
    Have marked i'm affected, unsure if it is what you are calling upvote.
    Thanks; your wording is correct and mine is incorrect.

  10. #40
    Join Date
    Sep 2018
    Beans
    6

    Re: Manual Full System Encryption has been updated and simplified

    Quote Originally Posted by Paddy Landau View Post
    ... When creating the LUKS partition, the user's key is used, while the system key is added afterwards. Are you saying that this order causes Grub to try the system key first? Or something a little different?
    Hi Paddy,
    I may have been wrong about the key order (it isnt immediately obvious which key is stored in which slot), and just jumping to conclusions.

    Here are the steps that I used to ensure the keys are in the optimum order, and also to adjust the number of iterations done on each key - which influences how quickly the device can be unlocked. The steps are fairly simple and quick, but you do have to enter your passphrase many times!

    # 1. specify your encrytped device (subtitute your device name)
    device=/dev/nvme0n1p5
    # 2. check what's in the encyption header - note the iterations for each key
    sudo cryptsetup luksDump $device
    # 3. create an extra passphrase as a back-stop. Enter your passphrase when prompted (3 times)
    sudo cryptsetup --key-slot 7 luksAddKey $device
    # 4. delete existing user key slot
    sudo cryptsetup luksKillSlot $device 0
    # 5. delete existing machine key slot
    sudo cryptsetup luksKillSlot $device 1
    # 6. Create a new key slot for your passphrase, with required iter-time:
    sudo cryptsetup --iter-time 500 --key-slot 0 luksAddKey $device
    # 7. Create a new key slot for the machine key, with required iter-time:
    sudo cryptsetup --iter-time 1000 --key-slot 1 luksAddKey $device /etc/crypt.system
    # 8. check what's in the encyption header - note the iterations for each key
    sudo cryptsetup luksDump $device
    # 9. Reboot, and check to see how fast the startup is.

    If you'd like to reduce the boot time, repeat steps 1, 4, 6 & 8, choosing a lower value for iter-time (the iter-time is the amount of time, in milliseconds, that the encryption process spends "hashing" (scrambling) your passphrase before it is stored. It would take the same amount of time to repeat this process each time you decrypt the device - but only if your processor can operate at the same speed during boot. That's not always the case - in my case it was 10 times slower.)
    You can also repeat steps 5 & 7, choosing a lower iter-time, to speed up the second part of the boot process, but this will make less of a difference.
    When you've finished, you can optionally delete the "back-stop" key in slot 7 (but there's no harm in leaving it).

    In my case, I found that an iter-time of 200 for my passphrase gave 185000 iterations, which took about 2 seconds during boot. For the machine key, I found that an iter-time of 1000 gave 900000 iterations, which took about 1 second during boot. The default iter-time is 2000. The relationship between iter-time and number of iterations depends on how fast your machine is. As I mentioned previously, there are potential security implications of reducing the number of iterations, but so long as a sufficiently long/complex passphrase is used, there's no problem. This topic is well discussed here.

    I hope that the above proves to be useful for anyone suffering agonisingly long boot times.

Page 4 of 14 FirstFirst ... 23456 ... LastLast

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •