Development of the [Boot-Repair] and [Boot-Info] utilities

[YannBuntu]

hi all,

This is discussion about the dev of the tool, inform about new versions, call for testing, etc.
I especially appreciate the feedback from the forum 'helpers' , i.e. people like oldfred who daily help others with their boot issues, because those tools are mainly here to decrease their workload by providing a useful diagnostic (boot-info) and solving the frequent cases (boot-repair).

This is not a topic for user support. If you have an issue with the boot of your computer, please open a new thread in the Support section of the forums.

[YannBuntu]

To begin, here is a quick summary of the December updates:

ppa131: fixed https://bugs.launchpad.net/boot-repair/+bug/1905578 + added free spaces and gdisk sections to the Boot-Info report
ppa134: Boot-Repair will install grub-efi on an hidden ESP in one of the 2 following cases knowing that it prefers ESP located on the disc where the Linux we install grub-efi is:
1) if it detects a WindowsEFI, and there is no normal ESP on the same disc as the hidden ESP
2) if no Windows detected and there is no normal ESP on the same disc as the hidden ESP
ppa135: added an advice to remove the hidden flag in these cases
ppa138: removes the "Unknown MBRs" section from the boot-info report when grub-efi selected
and "StdErr Messages" et "Devices which don't seem to have a corresponding hard drive" , except if option --no-filter is enabled.
ppa139: improved gdisk filter + added mdadm & dmraid titles
ppa140: no more select 'win-legacy-basic-fix' if grub-efi
ppa141: select grub-efi-signed by defaut only when SecureBoot is enabled in the firmware.
ppa142: modifies GRUB_DISABLE_OS_PROBER=true to false by default , at the moment is included in the 'Display the boot menu' option, so enabled by default.
ppa143: solves empty pastebin https://paste2.org/ case
- extended the no more attempt to install grub-pc on GPT discs without bios-boot, blocks if main disc, silent for others (when install on several discs).
- remplace the purge grub*-common+shim-signed by the double command (grub-common* then grub2-common*) for compatibility with OS using grub-common:i386 and grub2-common:i386 packages

[MAFoElffen]

Welcome back.

I'm all-in to help...

[oldfred]

Also my welcome back.
I regularly update & noticed a lot of updates on the Boot-Repair packages.

Grub2 has now turned off os-prober for security reasons.
One suggest was to just turn it on, run it once to get other boot stanza & then turn it off again.
Not sure how newer users would add Windows to grub menu otherwise. I have preferred adding boot stanza to 40_custom, but that a bit more advanced.

Noticed next version of Ubuntu 22.04 will use grub 2.06, so we can expect more changes in 2022.

[MAFoElffen]

What do you needed tested and under what test conditions?

I have various hardware to test on. I can also replicate quite a myriad of test machines in virtual... I can recreate quite a few "problems' for test cases. I just need some parameter direction and boundaries of 'what' you want to test against...

x86, amd64, arm32, arm64, Intel, AMD, ARM, Mac, etc...

[YannBuntu]

@ MAFoElffen : thx for your help. In https://ubuntuforums.org/showthread....8#post14054758 you indicated that boot-repair fails with ZFS. Please could you describe what's wrong ?

ppa145: as discussed with oldfred, if Windows detected, Boot-Repair will not install grub in all MBRs (but only in the mbr of the selected Linux drive).

TODO: if several Linux available, B-R should try to select one which is not on same drive as Windows. (in order to avoid installing grub in the mbr of the Windows drive)

[YannBuntu]

hi,
I've been reported a regression ('syntax error near unexpected token' which removes chroot prefix from the commands) since ppa142, that I don't find the origin, so I'm going to revert back to ppa141 and redo all gradually.
ppa150: reverted major changes done since Monday (i.e. purge commands update, non install if no BIOSboot, non install in all MBRs if Windows).
Compared to 141, I left the resolution of empty pastebin and GRUB_DISABLE_OS_PROBER=false.

[MAFoElffen]

@ MAFoElffen : thx for your help. In https://ubuntuforums.org/showthread....8#post14054758 you indicated that boot-repair fails with ZFS. Please could you describe what's wrong ?

ppa145: as discussed with oldfred, if Windows detected, Boot-Repair will not install grub in all MBRs (but only in the mbr of the selected Linux drive).

TODO: if several Linux available, B-R should try to select one which is not on same drive as Windows. (in order to avoid installing grub in the mbr of the Windows drive)

Yes...

Right before that I had helped two Forum Users to mount and re-install Grub to their Ubuntu on ZFS systems. (There was One additional was a problem with a Kernel update, which affected ZFS related kernel modules.) Both had tried to use the Boot Repair ISO unsuccessfully on them. As soon as they could mount their systems, then they could reinstall Grub.

Honestly, I haven't seen/gone through any of your code of boot repair... So I do not even know if you do ZFS, or what and how you deal with things yet... But I have always had a lot of respect for you and what you have done.

My test environment for Ama-gi has 5 different systems installed on 5 Virtual disks, which one is booting on ZFS... Which I test on. Since it tests for multiple installed systems, (parsing if each is Linux > Debian Branch > Ubuntu, etc,) while parsing and analyzing, I build a data structure of what is found, and store certain things on each (including filesystem types and managers) then use that data structure to give the user a choice on what system they want/need to chroot into... I use the info I store in that data-structure to do the mount and chroot into that filesystem or file manager (mdadm, lvm, zfs). At least, that is how I dealt with it in that...

You mentioned MBR... Then CSM boot on GPT... which then has the 'bios_grub' flagged partition where that Grub boot microcode resides, that would be on CSM MBR disks before.. Then hybrid disks that have both bios_grub and EFI partitions, and both legacy and EFI versions of Grub installed to GPT... Just thinking out loud.

[oldfred]

I really wish MBR would go away.

The only place you have to have MBR, now is with Windows in BIOS mode. Which users should not be doing with new installs. Perhaps some old Windows 7 systems or updates to those would then still be BIOS/MBR. But those are now often 10 years old systems. A few downgraded Windows 9 to Windows 7, but those could have been UEFI.

Microsoft has required vendors to install in UEFI boot mode to gpt drives since 2012.
But Ubuntu lets uses use UEFI on MBR and that often prevents grub from dual booting if Windows is BIOS as drive can only have one boot flag, either Windows boot partition or ESP with UEFI. Grub may boot Windows for short time if fast start up is off.

I started conversion to gpt 10 years ago. XP on MBR and new install of Ubuntu using gpt.
When I knew I was eventually getting a new system with UEFI, I added both bios_grub & ESP to all new drives. But then you could only install one version of grub, grub-pc or grub-efi-amd64. I think sudodus had a work around where he installed one version then the other, but then eventually would get out of sync with updates.
Now I believe you can install both grub packages, but still only have one or the other installed.

[YannBuntu]

Happy new year !

ppa151: improved ZFS detection
ppa152: purge commands improvement, non install if no BIOSboot, non install in all MBRs if Windows

@MAFoElffen: please can you show the output of your script and the output of boot-info on your ZFS environment, so that we can compare ?

@oldfred: the Ubuntu installer (ubiquity) installs grub-pc in addition to grub-efi when signed, not sure why:

	case $grub_package in
	    *-signed)
		apt-install shim-signed || true
		apt-install grub-pc || true
		;;
	esac