[Call for testing] Script to populate grub.cfg with all EFI entries.
 

Hello,

The attached script writes a grub.cfg file including entries for all EFI applications (including EFI loaders) found on block devices attached to the system, mounted or not, including IDE and SATA disks, NVMe, USB connected drives, eMMC drives, whatever.

It could be useful for folks running multi-boot systems, including non Linux ones, for instance run after elilo.

I didn't test with eMMC drives because I don't have any and there is no virtual drives available for VirtualBox.

Please test and report your findings here.

The file is written in /tmp, not directly in /boot/grub, so there is no risk running the script, I think.

I also paste the code below.

Gave the script a try on my laptop. First here are all the *.efi files on my ESP

And this is the grub.cfg file that your script created

Not sure why the script doesn't create stanzas for all the *.efi files, although most of them are junk anyway. Was really surprised that it didn't create one for refind.

Thanks for testing.

I assume that this test: is too selective. To check, please provide the output of this command:

Here you go;

Thanks. So, this confirms my assumption.

Now, as the script's purpose is to build a boot menu, do you think that some of the missing .efi files should be included in the output file, and which ones?

Maybe /boot/efi/EFI/refind/refind_x64.efi? Others?

Well let see. For the EFI/Boot directory I would only add the bootx64.efi. This directory is suppose to be for a fallback boot option in the event that nothing else boots. In my opinion some distro miss-use it. Note that trueOS (BSD) added an efi file there.

Window - Only add entry for bootmgfw.efi.

Tools directory (EFI/tools) it would be nice to add all the tools, maybe is a sub menu if possible. That what is nice about refind, it adds smaller launch icons for tools in a separate line from the main OS launch applications.

For ubuntu, EFI/ubuntu, (its actually Mint) just add an entry for the grubx64.efi. The other file, fwupx64.efi is for launching a firmware update utility.

And finally rEFInd. I not sure that launching a boot manager from another boot manager make much sense, but yeah for the sake of completeness I think it should be added.

To check I installed refind in the VM and removed the restriction to "PE32+ executable (EFI application)" and ran the script then grub-install with that. I noted that the entries to files in
boot/efi/EFI/refind/drivers_x64/ do nothing. I assume that they are only used by refind itself internally (like the files in /usr/libexec, as an analogy).

/boot/efi/EFI/refind/tools_x64/gptsync_x64.efi works, but it's not our business an hybrid MBR anyway as this feature is provided "only for Apple Macs that dual-boot with Windows or some BIOS-mode OS."

So I am tempted to write stanzas in the file only if one of these conditions is met:

• The file is a "PE32+ executable" and also an "(EFI application)" according to the file utility.
• The file is named bootx64.efi

But, before I change this please generate grub.cfg with the "unrestricted" script below, copy or move it to /boot/grub and run grub-install.

It is possible than refind be still first started, in this case you can just refresh it and start grub.efi from it.

I would like to know which of the resulting entries are actually enabled from the boot menu after that.

Then, I am considering allowing the user to customize each stanza's label.

Hopefully I'll have some time today to test it. Regarding rEFInd, I forgot to add that you should ignore anything in the refind/driver directory. In fact I would suggest that you only add a stanza for refind/refind.efi

Here's the list of menu items that your script produced on my system.
Going from top to bottom, you can ignore the rEFInd driver directory as they don't boot nor are they meant to. The only image in the Microsoft directory that booted was bootmgfw.efi, which boots into Win 10. All three of the images in the Boot directory booted, Shell.efi to the UEFI Shell, bootx64.efi booted into a rEFInd menu and bootx64-trueos.efi to TrueOS (BSD). The boot images in the Salix, Slint and Slackware all boot into the appropriate OS. Every image in the EFI/Tool directory work, but I don't think the ipx images are configured correctly. The fwupx64.efi in the ubuntu directory doesn't boot. The grub64.efi boots into a grub menu for Mint. bootx64-trueos.efi boots into TrueOS BSD.

OK, now I have compared what says rEFInd (using default options) with what says this script.

Considering your last post I will select files whose name ends on .efi:

• that "file" characterize as both a PE32+ executable and an EFI application
• and/or that are found in EFI/BOOT which is somehow a fallback directory to store bootloaders if entries in the firmware's menu fail.

I wonder if it's useful to include files in /EFI/tools, as probably users who install them will start them from the rEFInd menu, as they are considered addons of rEFInd (though admittedly people already using rEFInd won't need this script anyway...), so that would uselessly clutter the list.

I have a (maybe last) question before beginning the real work: in your last post you didn't mention /boot/efi/EFI/Microsoft/Boot/bootx64.efi as bootig but wrote "The only image in the Microsoft directory that booted was bootmgfw.efi, which boots into Win 10" Isn't this other entry also booting?

Yes this is suppose to be a directory for a fallback bootable efi image, however it seems to me that the purpose of this directory is widely misunderstood. At first there were some efi firmware implementation that would only boot efi images from this directory. Now it seems like almost every distribution wants to throw a image into this directory and give it the generic bootx64.efi name.

Yes I think you should include them. Although it not prevalent, some systems (HP and Apple) come with this directory on the system's ESP. I personally would like to see UEFI tightening up the spec for what should go into the ESP and where it should go, with EFI/Tools becoming the standard directory where uefi tool images be stored. Right now tools such as memtest get install all over the place depending on which distro is installing them.

On my laptop, booting this images fails into a grub rescue prompt. Not sure how it got installed, but I do not think it was part of the Windows installation.

Thanks again Rich.

Which makes hard to identify the distribution is concern both for my script and for rEFInd. os-prober can sometimes do better, but not in all cases. We have to live with that, only thing we can do is include all these images, and allow the user to modify the stanza's label or title, I think.

OK I will include all of them. And maybe all /path/to/Shell*.efi although "file" just says it's a MS-DOS executable.

It will be included anyway. Missing a boot loader is worst than having a not working stanza in the boot menu, and I will provide a mean for the user to easily edit all stanzas (add, delete, modify) anyway.

Let me quote some lines of the EFI Specification, Version 2.7 Errata A :This leads to this question: should the script provide only stanzas for OS loaders or also for other applications like tools, shells, boot managers? Bearing in mind that in an ideal world all OS loaders at least should be accessible from the firmware's menu, but we are not in an ideal work (yet). Maybe include all we can and let the user remove what could be redundant is the way to go?

Update and plan
 

Update: I checked that an ESP on an NVMe devices be detected, installing the soon to be released openSUSE 15 in a dedicated NVMe virtual disk in EFI mode.

As an aside I tried this distribution because my friend Martin Brückner who maintains WackoWiki just wrote me that he likes it. Well, I am pleased with Plasma 5.12, not so with the installer that proposed to erase my other systems to make room for openSUSE until I give it the room it wanted on its own device ;)

Status report:

1. Write the EFI boot manager image. Done
2. Allow to write a stanza for the boot manager in the firmware's menu. Done, needs testing on bare metal.
3. Allow to write the EFI boot manager on an USB stick, as a "customized rescue boot manager". WIP.
4. Allow the user to edit the boot menu, after having checked the entries, so that the labels be more informative. The association between ESP/EFI image path and corresponding label would be stored in a separate file, read by the script when (re)building the EFI boot manager image. TBD
5. Check that the script be distribution agnostic, (portable, in other words) i.e. that it can be used at least on other Linux distributions, maybe on other at least partially POSIX compliant systems. TBD.

I will request more testing when step 3 and 4 will have been completed.

Status report
 

The new shell script, named EFI3M for "EFI multi-boot menu maker" allow the user to:

• Build and install the multi-boot menu on a mass storage device attached to the computer, and optionally in its firmware. The boot menu includes an entry for most .efi files found in ESP in all devices attached to the computer.
• Make a rescue USB stick providing the boot menu.
• Edit the boot menu: hide boot entries, edit their labels, change their display order.

It works at least on Slackware version 14.2 and Slint64-14.2.1.1.

On second thought (and though I tried hard to make it fully POSIX compliant) I won't try to make this script work on non Slackware-based distributions. Instead I plan to provide an hybrid ISO image, intended to be written on an USB stick, loading a an initramfs allowing to run EFI3M. This will allow to boot all OS installed on the computer on any kind of mass storage device, provided that an EFI boot loader be found for them on the computer. The initramfs won't even need to include GRUB, as BOOTx64.EFI (that weighs just 693K) would be already installed in an ESP of the USB stick, so only a few utilities allowing EFI3M to write and then customize grub.cfg would be necessary.

I will be grateful for comments about this plan but first for testing reports of EFI3M as it is, attached to this post and also stored in http://slint.fr/testing/

I will internationalize EFI3M but as English is not my native language, please do not hesitate to state any typo or bad wording or phrasing of the dialogs and propose enhancements to make the instructions easier to understand and follow.

I attach a few pics, just to give an idea.

PS Saturday 9 June 2018: I have removed the (buggy) file EFI3M.txt that was attached to this post.