Call for testing the current daily Groovy iso files

[guiverc]

I noted yesterday https://bugs.launchpad.net/ubuntu/+s...2/+bug/1892754
"Unable to boot in UEFI+secure boot mode"

so played with my sony crapbook and yep it has a secure option in settings. it fails too

sony vaio svp112a1cw (i5-9400u, 4gb, intel haswell-ULT)

I also had a recent Ubuntu desktop groovy ISO lying around, so booted it, it's impacted too, but it was too late for me to (want to) update my ubuntu daily ISO and thus record the failure on iso.qa.ubuntu.com, so I made mention of it on #lubuntu-devel

Turns out @sudodus had the same thought as no sooner had I jumped into bed, my phone beeped with an update to the mentioned bug report that Sudodus had written confirmation on the lp bug report of what I'd mentioned on IRC.

I suspect where bugs impact Ubuntu too, filing reports on iso.qa.ubuntu.com is a great way for them to get noticed, get more attention.

I'm not a developer so of course haven't see the "product" reports that are mailed out, but I'm aware that any bug reported on iso.qa.ubuntu.com seems to be known by developers on teams so I'd recommend everyone recording testing on iso.qa.ubuntu.com too, and testing bugs on other flavors (esp. main Ubuntu if impacted so Canonical people see the issues).

And thanks @sudodus. I'd looked for you on IRC (where I ~always am), plus a quick search here but failed to find this thread (I was tired), so I gave up & went to bed (do it tomorrow..).

I saw talk yesterday on #ubuntu-devel relating to server boot issues, missing.... so Canonical devs are aware of some issues (may not issue impacting us)

[guiverc]

Thanks likely to @sudodus & quite possibly the not-so-subtle push..

https://bugs.launchpad.net/ubuntu/+s...2/+bug/1892754 has been fixed, or at least fix committed.

For anyone interested in why, see #ubuntu-release (ie. groovy daily ISOs were pulling in a grub signed with old archive key; made invalid with recent boothole fixes...) Vorlon has fixed this.

In a day or so we should have bootable ISOs for thumb-drive media, with secure boot enabled devices

[zebra2]

I don't know exactly what the changes to the boot processes are but I am having no problem installing anything including Windows 10 with my bios set to legacy. One thing that has changed for me with the current 20.10 ISO is I am getting a "system may not boot, no EFI found" warning at the end of the installation. I ignore the warning and my system boots just fine. Many of the problems may be due to individual bios layout. If you are having a problem booting from USB, note that on some systems you must actually have a USB drive plugged into the USB port before your bios will recognize and allow a USB boot setting.

[sudodus]

@zebra2,

You are right, that there is no problem installing anything including Windows 10 with the bios set to legacy (alias BIOS mode alias CSM as opposite to UEFI mode). For example: I run my main computer in BIOS mode.

But I can see two problems:

- If Windows 10 was installed by the manufacturer of the computer, it is installed in UEFI mode, and users who want to keep Windows and install Ubuntu alongside it want to install in the same boot mode. It is not convenient to switch boot mode every time you want to switch between Ubuntu and Windows.

- Many people want the increased security offered by the secure boot option, and it is only available in UEFI mode.

[zebra2]

@sudodus,
I wouldn't want to argue any of your valid points however the OP is looking for "workarounds" . What I have is exactly that. For me it all comes down to what is primary. Am I a Linux user who only needs Windows for a "Utility" to accomplish a couple or annual chores. Or am I a Windows user that just wants to explore the usability of Linux.
In my case I am a full time user of Ubuntu on a few Ubuntu friendly Laptops that only needs Windows as a utility (like a garmin update). In addition, all of my Windows installs are registered with MS and have activation built into bios so Installing Windows 10 for me is not much more difficult than installing Ubuntu. Given my minor need for Windows in the first place and the fact that Windows 10 is (even without Secure Boot) more secure than in the past I don't need the aggravation of UEFI and Secure Boot. I don't want to rant on this but I think it is laughable that MS couldn't build a secure kernel so saddled the whole planet with this UEFI in hardware pain it the posterior. Windows 10 runs fine in NTFS, Ubuntu runs fine in EXT4, Grub runs fine in MBR, so with my limited need for Windows 10 why muddle around in that messy bog called UEFI. Thats my workaround and I'm sticking to it until my needs get larger than 2.2T.
You have my applause for your contribution to this really complex section of the forum. I suppose Canonical and Grub will develop a fix that works properly at some point.

This whole thing reminds me of a mechanic trying to fix a customers drive-ability problem with a new car when the problem is actually encoded into the car and everything the mechanic is doing is outside of the mfd specs (workarounds). Software solution for a hardware problem. Write all the software you want to, you still have crap hardware.

[sudodus]

[QUOTE=zebra2;13983576]@sudodus,
I wouldn't want to argue any of your valid points however the OP is looking for "workarounds" . What I have is exactly that./QUOTE]

I am the OP of this thread

And I think that your workaround to use legacy mode will be useful for many people

But I think we can also look for workarounds, or even better, solutions, that work also in UEFI mode with secure boot

[zebra2]

But I think we can also look for workarounds, or even better, solutions, that work also in UEFI mode with secure boot :-P

I know you are the OP but the OP can't by default be about what is know as MS Secure Boot (signed shim). Workaround means "not secure". I think you are right to put this in the Development section since crashing a Windows system can be a consequence of "looking", The new user shouldn't be led into an untenable position. For an experienced Ubuntu user, reinstall of Ubuntu and in most cases Windows is no big deal. For the "many" dual booting UEFI can be a *#@?* of a mess. The "Installation and Upgrades" section is getting to be about just that.
Anyway I think discourse on this whole matter is very much needed.
So thanks.

PS: O Yes, I forgot! If you are now in addition looking for a "solution", that would have to be a UEFI Secure Boot that doesn't have to be dependent on an MS signed shim. Right now Secure Boot isn't Ubuntu's monkey but it is becoming Ubuntu's circus.

[sudodus]

Thanks likely to @sudodus & quite possibly the not-so-subtle push..

Bug #1892754 has been fixed, or at least fix committed.

For anyone interested in why, see #ubuntu-release (ie. groovy daily ISOs were pulling in a grub signed with old archive key; made invalid with recent boothole fixes...) Vorlon has fixed this.

In a day or so we should have bootable ISOs for thumb-drive media, with secure boot enabled devices

Yes, Bug #1892754 has been fixed, USB drives with the live system cloned from the current daily Groovy iso files work also in UEFI mode with secure boot in most computers, for example in my Dell Precision M4800

But mkusb-dus can no longer create persistent live drives from these Groovy iso files, and I have not yet found how to fix it. This also means that the workaround to boot a Lenovo V130 with secure boot no longer works

In addition to cloning there is another method that also works in UEFI mode and with secure boot (for most computers),

mkusb-plug can make persistent live drives (as well as live-only drives with and without a data partition)

So far I have only tested Lubuntu Groovy, but this method is a semi-cloning process, so there is a good reason to think it will work with standard Ubuntu and all Ubuntu flavours.

[sudodus]

I can report progress: There are new compressed image files to be extracted and cloned to USB drives for UEFI and BIOS mode, that can boot also with secure boot in a computer updated to squash the boothole bug.

You find these image files at phillw.net/isos/linux-tools/uefi-n-bios, where you also find files with the corresponding md5sums.

These compressed image files are similar to the previous ones made recently, but the boot system is made by installing Ubuntu Focal from the current daily iso files (post 20.04.1 LTS) with up to date program packages, that include secure boot shims for the relevant components. This works also with a Lenovo V130, that refuses to boot USB boot drives cloned from current daily Groovy iso files in UEFI mode.

There is still a problem with usb-pack-efi and mkusb-dus to make persistent live drives that work with secure boot. It may take some time until I can fix it. Until then, secure boot must be turned off for them to run. The problem is that vital parts of the old tools have an old signature in the shims and lack the fix against the boothole bug.

Fortunately for users, who want a convenient tool with a graphical user interface to make persistent live drives, mkusb-plug is still working, where cloned drives are working [with secure boot], that is in most computers.

I think the proposed updates of the LTS release beyond the first point release are not very adventurous, and should not cause too many hiccups. Anyway, I am interested in your opinion about that.

---

- mkusb_grub-boot-template-for-uefi-n-bios_fat32_4GB_use-in-wins_2020-09-07.img.xz

This image is small, ~ 7.5 MB, and expands to slightly less than 4 GB in order to fit in a 4 GB USB pendrive or memory card.

When extracted, you mount the first partition, which is labeled 'ISODEVICE' and has the file system FAT32 (shown as 'vfat' by Ubuntu), and copy an Ubuntu family iso file [into this first partition 'ISODEVICE']. Rename the file to ubuntu.iso.

$ lsblk -fm /dev/sdc
NAME   FSTYPE LABEL     UUID                                 MOUNTPOINT                 SIZE OWNER GROUP MODE
sdc                                                                                    29,1G root  disk  brw-rw----
├─sdc1 vfat   ISODEVICE 0338-85F9                            /media/sudodus/ISODEVICE   3,4G root  disk  brw-rw----
├─sdc2                                                                                    1M root  disk  brw-rw----
├─sdc3 vfat   ESP       1841-531C                                                        64M root  disk  brw-rw----
└─sdc4 ext4   usb-boot  733906cb-a2c4-47f2-8fd0-a2a0bdef765a /media/sudodus/usb-boot    128M root  disk  brw-rw----

$ ls -lh /media/sududus/ISODEVICE
total 1,8G
-rw-r--r-- 1 sudodus sudodus 1,8G sep  7 21:51 ubuntu.iso

I have tested iso files of 20.04.1 LTS as well as current daily Groovy, and I can boot live systems in BIOS mode as well as in UEFI mode also with secure boot [in a computer updated to squash the boothole bug].

If you have a bigger drive, you can use gparted to create new partition with the label 'casper-rw' or 'writable' and the file system ext4 and get a persistent live drive. You can also create a data partition with the file system NTFS to store data and exchange data with current versions of WIndows 10, that can manage more than one partition in USB drives. See the attached screenshots. (Older versions of Windows can only manage partition #1, and this is preconfigured as 'ISODEVICE' with the file system FAT32.)

- dd_unb_ubuntu-20.04_15GB_2020-09-07_with-proposed.img.xz

This image contains a whole installed and fully updated & upgraded Ubuntu 20.04.1 LTS system. The compressed image is 1.6 MB (1.5 MiB), smaller than the iso file because several files are removed near the end of the installation, for example language files. The language selected is US English, but you can add another language, and you can install whatever drivers and program packages, when your system is running.

When extracted, this installed Ubuntu system expands to slightly less than 16 GB in order to o fit in a 16 GB USB pendrive or memory card. If you have a bigger drive, you can use gparted to grow the root partition to use the whole drive.

---

See also these links (which should soon be updated with the current compressed image files),

help.ubuntu.com/community/Installation/UEFI-and-BIOS

BIOS/UEFI Template Image for Booting ISO Files

[sudodus]

There is another step to create USB boot drives for secure boot with a fix for the boothole bug.

See this link: mkusb version 12.6.0 is uploaded to the unstable PPA

The 'usb-pack-efi' packages are upgraded with grub versions, that contain the crucial bugfix. Now a Lenovo V130, that has been unwilling to boot Groovy iso files can boot persistent live drives made by mkusb using 'usb-pack-efi'.