GRUB is Consistently Inconsistent

stemwinder · 03-31-2019, 07:16 PM

This site has been very helpful over the last year answering questions for a newcomer without having to post. I have however had problems/questions regarding GRUB2 that I haven't been able to solve/answer.

1/ Over the last year I tried out many Linux distributions. I didn't delete any installations, I just kept adding partitions. I have 3 primary partitions and many logical partitions. All operating systems are installed in a single partition without a swap file. update-grub or grub-mkconfig ran correctly 99% of the time excluding 2 time frames in which updating GRUB did not properly probe for other operating systems about 30% of the time as below:

15% of the time the probe would stop at a random partition and exit normally without an error but without scanning all partitions. If I ran update-grub a second time it would run to the end. When the probe stopped short all the partitions listed in the GRUB menu would boot properly, the list was just not complete.

15% of the time I would get one or many "Failed to probe /dev/sda16 for filesystem type". If the probe generated even one "failed to probe" then the next run of update-grub would result in a "failure to probe" for every partition except the partition controlling GRUB, requiring a reboot. On reboot its again a 30% chance of an error.

These errors appeared and disappeared after replacing or upgrading installations other than the distro controlling GRUB, as I made many changes at once I was not able to single out a culprit. This behavior is for Debian based Distros, other distros had similar results but different error messages and the results were more erratic. At the present time update-grub runs without a hitch on all distros.

I'm less interested in a solution than curious as to what could cause this behavior and why GRUB gets it right on the second pass. Does GRUB realise it failed and tries different parameters.

2/ Some distros like Gecko probe very quickly like 3 times faster than some of the slowest. What accounts for this difference if its using the same version of GRUB.

3/ Some distros using the same GRUB version identify a disro as unknown distribution while others identify it properly. Similarly when I install 4 different flavors of Linux Mint 18.3 all show up on the menu with the same label. Where does this information come from and what changes can I make so the GRUB menu displays a unique label.

4/ I am trying to get GRUB to not display the recovery entries. I doubt I would ever use these entries, as I make very few changes to a distro I would just reinstall. I have googled this and searched on this site, the latest threads I found (2016) indicate that # GRUB_DISABLE_LINUX_RECOVERY="true" should work. The GRUB manual shows this option and it is a commented out entry in /etc/default/grub of most distros. I have tried GRUB_DISABLE_LINUX_RECOVERY="true" and GRUBB_DISABLE_RECOVERY="true" with and without quotes and it doesn't work. (I ran update-grub). Why doesn't this work, or am I misunderstanding and GRUB_DISABLE_RECOVERY="true" does something different. I have tried Grub Customizer and although it initially works, eventually it just gets fuddle duddled and I end up reinstalling.

Thanks

mrmazda · 04-01-2019, 02:52 AM

#3, part2: /etc/default/grub: GRUB_DISTRIBUTOR=

You might wish to exercise more control over the distro "controlling" Grub.

On my MBR systems, I have Grub on a primary partition (with its boot flag set) that never is mounted to /boot, so exclusively under my control, and operates from generic (Windows-compatible) MBR code. More than two of such systems here exceed /dev/sda40 in fdisk -l output. Average is probably in the neighborhood of sda30.

On my UEFI systems, I only have the ESP partition mounted when booted to the OS "controlling" Grub2, so the other distros can do whatever they want with bootloaders and os-probers without any impact on routine booting. For the "controlling" Grub2, I create my own /etc/grub2/custom.cfg from scratch, then rename /etc/grub.d/40_custom to 06_custom, which causes my own entries to head the distro-generated Grub's boot menu. My own custom.cfg relies on volume labels rather than UUIDs, and symlinks to the initrds and kernels named /boot/initrd and /boot/vmlinuz, created manually for those distros that don't make them automatically. Each stanza in custom.cfg for any given distro is comprised of exactly 5 lines, including the standalone "}". Current custom.cfg on the currently booted UEFI PC is 3497 bytes, compared to the 19948 bytes of the most recently generated grub.cfg. Here are two of the (nominally altered) stanzas contained in this custom.cfg (first listed is the default):

Code:

menuentry "memtest86 7.4 EFI" {
	search --no-floppy --label --set=root KINGSP01ESP
	chainloader /mt74x64.efi
}
menuentry "Tubuntu 18.04 defkernel" {
	search --no-floppy --set=root --hint-baremetal=ahci0,gpt10 --label kings-p12Ubionic
	linux	/boot/vmlinuz root=LABEL=kings-p12Ubionic noresume video=1400x900@60 3
	initrd	/boot/initrd
}

stemwinder · 04-01-2019, 12:59 PM

Quote:

Originally Posted by mrmazda

#3,

On my MBR systems, I have Grub on a primary partition (with its boot flag set) that never is mounted to /boot, so exclusively under my control, and operates from generic (Windows-compatible) MBR code. }[/code]

This is a sidebar to the topic at hand which I will reply to later. I am not familiar with a dedicated GRUB partition, but I will be. When I started with Linux I had a Windows 7 MBR and Debian in a primary partition. I booted the other Linux partitions from Debian's GRUB menu. Someone suggested this was not a good setup, so I switched to GRUB in the MBR. I know this is not the same setup you are describing but operationally what is the difference between a dedicated GRUB partition and the setup I started with. I'm singling this out because although not a solution it is certainly the best workaround to my previous (first) post "GRUB2 boots functioning Windows 7 hiding behind a corrupted grub splash screen".

Thanks

ondoho · 04-01-2019, 01:48 PM

Quote:

Originally Posted by stemwinder

I'm less interested in a solution than curious as to what could cause this behavior and why GRUB gets it right on the second pass. Does GRUB realise it failed and tries different parameters.

os-prober is a different package on most systems, so it's not really grub that's problematic.
os-prober does some weird magic finding out what's stored on unmounted partitions.
i had os-prober hang indefinitely when one of my partitions was unformatted (did not contain a filesystem) or wasn't a partition at all, just unused disk space.
so, do you have something like that, or something else that's really exotic?

maybe os-prober keeps some record of what it does, that could explain why things work slightly better next time around.

time to read the docs i guess.
try:
http://joeyh.name/code/os-prober/
or 'dpkg -L os-prober' then look at (inside) all files.

mrmazda · 04-01-2019, 02:20 PM

Quote:

Originally Posted by stemwinder

2/ Some distros like Gecko probe very quickly like 3 times faster than some of the slowest. What accounts for this difference if its using the same version of GRUB.

There may be a clue to this here.

mrmazda · 04-01-2019, 02:36 PM

Quote:

Originally Posted by stemwinder

what is the difference between a dedicated GRUB partition and the setup I started with.

How much control you relinquish to the bootloader configuration process of the OS "in control".

The dedicated master "boot" partition need not contain Grub. Any bootloader that can chainload or load kernels and initrds will do. On my PCs that have OS/2 included, IBM Boot Manager, born 3 decades ago (but in a much more recent version), is the master bootloader.

Without Grub on MBR, Windows updates, or inevitable Windows (re-)installation, won't usurp your boot configuration. This way the most damage Windows does is move the boot flag to its own boot partition, a two bit change in the master partition table. That is easily reverted from within Windows - no rescue boot is required to "fix" Grub on those infrequent occasions the flag gets moved.

colorpurple21859 · 04-01-2019, 05:23 PM

Some distros will modify the scripts in /etc/grub.d to achieve a desired result.

stemwinder · 04-01-2019, 11:45 PM

Quote:

Originally Posted by mrmazda

#3, part2: /etc/default/grub: GRUB_DISTRIBUTOR=

#3 Tried this unsuccessfully previously but I think I left out the quotes. Worked like a charm with quotes around the entry as in GRUB_DISTRIBUTOR="Linux Mint 18.3 Cinnamon".

Quote:

Originally Posted by mrmazda

You might wish to exercise more control over the distro "controlling" Grub.
On my MBR systems, I have Grub on a primary partition (with its boot flag set) that never is mounted to /boot, so exclusively under my control, and operates from generic (Windows-compatible) MBR code. More than two of such systems here exceed /dev/sda40 in fdisk -l output. Average is probably in the neighborhood of sda30.
On my UEFI systems, I only have the ESP partition mounted when booted to the OS "controlling"

As per advice previously gleaned from this forum I generally have Debian or Linux Mint controlling GRUB and they are in the 1st and 2nd primary partitions.

I was wondering if there was any kind of limit on the number of partitions, I don't think I went over 32 or so. The number of partitions didn't seem to be a problem just what was in the partitions.

At present I don't have any GPT setups, but plan to when I have eliminated all the glitches with my MBR setup. I will heed your UEFI advice at that time. I especially like the idea of using labels instead of UUIDs. I haven't made a lot of use of 40_custom except to setup FreeBSD.

Thanks

stemwinder · 04-02-2019, 12:15 AM

Quote:

Originally Posted by ondoho

os-prober is a different package on most systems, so it's not really grub that's problematic.
os-prober does some weird magic finding out what's stored on unmounted partitions.
i had os-prober hang indefinitely when one of my partitions was unformatted (did not contain a filesystem) or wasn't a partition at all, just unused disk space.
so, do you have something like that, or something else that's really exotic?

maybe os-prober keeps some record of what it does, that could explain why things work slightly better next time around.

time to read the docs i guess.
try:
http://joeyh.name/code/os-prober/
or 'dpkg -L os-prober' then look at (inside) all files.

That's quite interesting, because all of these pieces are referred to in the GRUB manual I assumed that any version of GRUB included all these files in one big ball of wax. I should have known better when the os-prober version is 1.70ubuntu3.3 for example. That clears a few things up, different results because they're different programs and files.

Thanks

stemwinder · 04-02-2019, 12:50 AM

Quote:

Originally Posted by colorpurple21859

Some distros will modify the scripts in /etc/grub.d to achieve a desired result.

Yes I now understand that these files are related to the disro and not the particular version of GRUB.

Thanks

ondoho · 04-02-2019, 01:27 AM

Quote:

Originally Posted by stemwinder

That's quite interesting, because all of these pieces are referred to in the GRUB manual I assumed that any version of GRUB included all these files in one big ball of wax. I should have known better when the os-prober version is 1.70ubuntu3.3 for example. That clears a few things up, different results because they're different programs and files.

you can run os-prober separately from grub. it's an interesting experience; try it with or without root privileges.
maybe you can use that to troubleshoot your problem.

btw, os-prober depends only on sh, so i guess it's just a shell script. human readable code.
the page i linked says it comes with various plugins - maybe you need to disable one or two - but it doesn't say how to do that.

share some actual troubleshooting output with us, we can help more.

stemwinder · 04-02-2019, 01:39 PM

Quote:

Originally Posted by ondoho

you can run os-prober separately from grub. it's an interesting experience; try it with or without root privileges.
maybe you can use that to troubleshoot your problem.

btw, os-prober depends only on sh, so i guess it's just a shell script. human readable code.
the page i linked says it comes with various plugins - maybe you need to disable one or two - but it doesn't say how to do that.

share some actual troubleshooting output with us, we can help more.

That's handy, I ran os-prober from root it runs many times faster than update-grub. However when I ran it from user I got this

don@T ~ $ os-prober
unshare failed: Operation not permitted
rm: cannot remove '/var/lib/os-prober/labels': Permission denied

If the update-grub problem appeared again what troubleshooting output would I provide.

Interesting links at the bottom of your post.

Thanks

stemwinder · 04-04-2019, 12:11 AM

Regarding my question #4 I found this patch here:

link https://askubuntu.com/questions/4746...u-in-grub-menu

Ok the trick is to edit the /etc/grub.d/10_linux file remove the right template entries that generates that option in grub menu:

Code:

# TRANSLATORS: %s is replaced with an OS name
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%    echo "submenu '$(gettext_printf "Advanced options for %s" "${OS}" | grub_quote)' \$menuentry_id_option 'gnulinux-advanced-$boot_device_id' {" 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
is_top_level=false
  fi

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%linux_entry "${OS}" "${version}" advanced \
%              "${GRUB_CMDLINE_LINUX} ${GRUB_CMDLINE_LINUX_DEFAULT}"
%  for supported_init in ${SUPPORTED_INITS}; do
%    init_path="${supported_init#*:}"
%    if [ -x "${init_path}" ] && [ "$(readlink -f /sbin/init)" != "${init_path}" ]; then
%      linux_entry "${OS}" "${version}" "init-${supported_init%%:*}" \
%         "${GRUB_CMDLINE_LINUX} ${GRUB_CMDLINE_LINUX_DEFAULT} init=${init_path}"
%    fi
%  done
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
  if [ "x${GRUB_DISABLE_RECOVERY}" != "xtrue" ]; then
    linux_entry "${OS}" "${version}" recovery \
                "${GRUB_CMDLINE_LINUX_RECOVERY} ${GRUB_CMDLINE_LINUX}"
  fi

  list=`echo $list | tr ' ' '\n' | fgrep -vx "$linux" | tr '\n' ' '`
done
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%# If at least one kernel was found, then we need to       
%# add a closing '}' for the submenu command.              
%if [ x"$is_top_level" != xtrue ]; then                    
%  echo '}'                                                   
%fi                                                        
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
echo "$title_correction_code"

Now note the % characters used, I used them to surround the lines that should be removed from that file to get the Advanced options for Ubuntu grub menu not to display, they aren't part of the original file. These lines exists from 345 down that file. Again all within that enclosure should be removed, compare to your own 10_linux file!

After that edit open the /etc/default/grub file and add the following line to it:

GRUB_DISABLE_RECOVERY=true

Now run sudo update-grub

I applied this to the Linux Mint 9.1 that was controlling GRUB and it worked for that one entry.
I then applied the same patch to Linux Mint Cinnamon 8.3 in sda10 and it worked for that one entry if sda10 was controlling GRUB but I couldn't get it to apply to a distro not controlling GRUB. Is this patch only meant for the distro controlling GRUB.