An Interesting GRUB Issue

vtel57 · 04-18-2011, 04:17 PM

Here's a little background for everyone...

I am running Slackware as my primary operating system and Arch as my secondary (backup) operating system on my main computer (ericsbane05). I also have installations of MS Windows XP, CentOS, and Debian on a separate hard drive.

SATA looks like this on my system:

SATA0 channel = /dev/sda, /dev/sdb
SATA1 channel = /dev/sdc, /dev/sr1
SATA2 channel = vacant, vacant

Originally, these drives were in ericsbane03 and 04 with one exception. I only had two SATA drives and one EIDE drive then. The EIDE drive was removed when building this new system and replaced with the drive /dev/sdc that you see above on the SATA1 channel (/dev/sr1 is a DVD R/W, by the way).

Arch Linux's GRUB controls /dev/sda's MBR and is the main bootloader for all operating systems on this system. Here's what my menu.lst looks like:

Code:

<snip># IMPORTANT --> Arch GRUB sees /dev/sda as hd0, but /dev/sdb as hd2 (should be hd1).
#
# (1) Slackware64-13.37
title  Slackware Primary
root   (hd0,0)
kernel /boot/vmlinuz root=/dev/sda1 ro
initrd /boot/initrd.gz
#
# (2) Arch64
title  Arch Linux Secondary
root   (hd0,4)
kernel /boot/vmlinuz26 root=/dev/sda5 ro
initrd /boot/kernel26.img

# (3) MS Windows XP/SP3
title MS WIN XP Pro
root (hd2,0)
map (hd0) (hd2)
map (hd2) (hd0)
chainloader +1

# (4) CentOS64 5.6
title	CentOS Tester01
root	(hd2,5)
kernel	/boot/vmlinuz-2.6.18-238.9.1.el5 root=/dev/sdb6 ro
initrd	/boot/initrd-2.6.18-238.9.1.el5.img<snip>

You can see that Arch's GRUB thinks that /dev/sdb is (hd2), when in fact, it should be (hd1). This was like this originally because of the hybrid EIDE/SATA setup that I had on previous systems. Now that I'm running pure SATA, I'd like to get this fixed. Here's what I've done so far:

1) I manually edited /boot/grub/device.map to look like this:

Code:

(fd0)	/dev/fd0
(hd0)	/dev/sda
(hd1)	/dev/sdb
(hd2)	/dev/sdc

It was originally incorrect, but this didn't fix the issue. GRUB was still seeing the drives incorrectly.

I then deleted all files from the /boot/grub directory and used pacman to uninstall GRUB completely. I then reinstalled GRUB and created a new menu.lst with the correct /dev to (hd) conversions. It looks like this now:

Code:

<snip>
# (1) Slackware64-13.37
title  Slackware Primary
root   (hd0,0)
kernel /boot/vmlinuz root=/dev/sda1 ro
initrd /boot/initrd.gz
#
# (2) Arch64
title  Arch Linux Secondary
root   (hd0,4)
kernel /boot/vmlinuz26 root=/dev/sda5 ro
initrd /boot/kernel26.img

# (3) MS Windows XP/SP3
title MS WIN XP Pro
root (hd1,0)
map (hd0) (hd1)
map (hd1) (hd0)
chainloader +1

# (4) CentOS64 5.6
title	CentOS Tester01
root	(hd1,5)
kernel	/boot/vmlinuz-2.6.18-238.9.1.el5 root=/dev/sdb6 ro
initrd	/boot/initrd-2.6.18-238.9.1.el5.img<snip>

The entries for Windows and CentOS reflect the new (and correct) device.map in /boot/grub. After all this, I reinstalled GRUB to the MBR like this:

Code:

# grub-install /dev/sda

Unfortunately, GRUB is still seeing /dev/sdb as (hd2). I can get all my operating systems to boot just fine by using the wrong (hd) strings in menu.lst, but I'd really like to get GRUB to work the way it's supposed to. I'm sure there is something simple here that I'm missing. Any help would be appreciated.

Thanks,

~Eric

jeff_sadowski · 04-18-2011, 05:06 PM

It sounds like you are saying
/dev/sda is hd0
/dev/sdb is hd2
/dev/sdc is hd1
is that what you are saying?

maybe it is a udev problem
look in /etc/udev/rules.d or where your udev rules are
it sounds like udev might have permanently mapped specific drives to specific sd[x] drives

correct me if I'm wrong but back in the day dos use to see the disks as follows
1st disk on bus1 was C
1st disk on bus2 was D
2nd disk on bus1 was E
2nd disk on bus2 was F
I'm pretty sure that the bios was the actual cause of the order.

and I'm thinking sata might be giving the disks in that order. It could be that grub is correct and your OS is mixing up the dive letters
(/dev/sd[x])
You may also want to look at your bios to see if you can change how it presents the order of the disks.

It could also be that linux has a different ordering of disks that grub maybe grub sees the disks the way that old dos does

vtel57 · 04-18-2011, 05:16 PM

Nah, Jeff... I'm not saying that. That is what GRUB sees the drives as. The correct map is sda = hd0, sdb = hd1, sdc = hd2. I know this because I can boot with a recovery CD (RIP, SuperGRUBDisk, etc.) using the proper identifications for the partitions and I can boot anything on any drive with it. It's only Arch's GRUB that seems to be confused. That being said, though... udev, that might not be a bad idea to check that. Thanks for that tip!

I'll be back...

syg00 · 04-18-2011, 07:50 PM

grub doesn't often get confused. Grub users get confused. Sometimes "users" include distro devs.

The init scripts determine what populates /dev, so udev is a good start. Remember grub-install runs after the init scripts, so you are seeing the "Arch" view of things, not necessarily the BIOS view of things.
From the boot menu, look at the geometry of each disk - should get you the real ("correct" to use your terminology) view of things.

vtel57 · 04-19-2011, 01:05 PM

syg00... you are absolutely correct on your assessment that I'm getting the "Arch view of things". I was trying to explain that earlier today to someone on another forum. It's when GRUB is installed/initialized within the distribution that it takes on whatever characteristics or flaws. I'm sure this is an Arch issue because for many years Debian was my secondary OS/bootloader on all my systems, and it never had this issue with pure SATA or hybrid setups.

I didn't see anything that jumped out at me when looking through Arch's udev rules or config. I'll keep checking in there, though. Anymore hints to lead me in the right direction would not go unappreciated.

Thanks,

~Eric

P.S. Here's a copy of my fdisk -l with notations. It might also be helpful:

Code:

Disk /dev/sdb: 250.0 GB, 250000000000 bytes
255 heads, 63 sectors/track, 30394 cylinders, total 488281250 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0009e140

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1    *      16384    51216383    25600000    7  HPFS/NTFS/exFAT <-- MS Windows XP C partition
/dev/sdb2        51232768   102432767    25600000    7  HPFS/NTFS/exFAT
/dev/sdb3       102453181   488270159   192908489+   5  Extended <-- swap
/dev/sdb5       102453183   106550639     2048728+  82  Linux swap
/dev/sdb6       106565823   137289599    15361888+  83  Linux <-- CentOS /
/dev/sdb7       137304783   188501039    25598128+  83  Linux
/dev/sdb8       188516223   219239999    15361888+  83  Linux <-- Debian /
/dev/sdb9       219255183   270451439    25598128+  83  Linux
/dev/sdb10      270466623   301190399    15361888+  83  Linux <-- tester /
/dev/sdb11      301205583   352401839    25598128+  83  Linux
/dev/sdb12      352417023   383140799    15361888+  83  Linux <-- tester /
/dev/sdb13      383155983   434352239    25598128+  83  Linux
/dev/sdb14      434367423   488270159    26951368+  83  Linux <-- storage

Disk /dev/sdc: 250.1 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders, total 488397168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0002cd1d

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1           16065   102414374    51199155   83  Linux <-- Slackware backups
/dev/sdc2       102430440   204828749    51199155   83  Linux <-- Arch backups
/dev/sdc3       204844815   488392064   141773625    5  Extended
/dev/sdc5       204860880   307259189    51199155    b  W95 FAT32 <-- MS Windows backups
/dev/sdc6       307275318   409673564    51199123+  83  Linux <-- tester backups
/dev/sdc7       409689693   488392064    39351186    b  W95 FAT32 <-- common storage

Disk /dev/sda: 250.1 GB, 250058268160 bytes
255 heads, 63 sectors/track, 30401 cylinders, total 488395055 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x24812481

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *       16065    51215219    25599577+  83  Linux <-- Slackware primary operating system /
/dev/sda2        51247350   153645659    51199155   83  Linux
/dev/sda3       153661725   488392064   167365170    5  Extended
/dev/sda5       153661788   204860879    25599546   83  Linux <-- Arch secondary operating system /
/dev/sda6       204877008   307275254    51199123+  83  Linux
/dev/sda7       307307448   311403959     2048256   82  Linux swap <-- swap
/dev/sda8       311420088   488392064    88485988+  83  Linux <-- archives

syg00 · 04-19-2011, 05:53 PM

Just a matter of legwork - you'll have to work through it.
Interestingly I've had similar where Ubuntu couldn't manage (E)IDE and SATA mixed. But Arch could. I kept all Debian based distros off that machine, and use Arch to boot it. Now has F14 on it as well - no problems with any of it; or Vista for that matter.

Haven't seen any issues on SATA only. Did you check the geometries - from the grub boot menu, not from grub command ?.

vtel57 · 04-19-2011, 06:43 PM

Not yet, syg00, but I'm headed into the GRUB menu in a bit. I'll ckeck and see.

Thanks.

vtel57 · 04-19-2011, 08:10 PM

Very interesting.

I had previously checked GRUB geometry using the grub command from within Arch. All was well:

Code:

grub> geometry (hd0)
drive 0x80: C/H/S = 30401/255/63, The number of sectors = 488395055, /dev/sda
   Partition num: 0,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 1,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 4,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 5,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 6,  Filesystem type unknown, partition type 0x82
   Partition num: 7,  Filesystem type is ext2fs, partition type 0x83

grub> geometry (hd1)
drive 0x81: C/H/S = 30394/255/63, The number of sectors = 488281250, /dev/sdb
   Partition num: 0,  Filesystem type unknown, partition type 0x7
   Partition num: 1,  Filesystem type unknown, partition type 0x7
   Partition num: 4,  Filesystem type unknown, partition type 0x82
   Partition num: 5,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 6,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 7,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 8,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 9,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 10,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 11,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 12,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 13,  Filesystem type is ext2fs, partition type 0x83

grub> geometry (hd2)
drive 0x82: C/H/S = 30401/255/63, The number of sectors = 488397168, /dev/sdc
   Partition num: 0,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 1,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 4,  Filesystem type is fat, partition type 0xb
   Partition num: 5,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 6,  Filesystem type is fat, partition type 0xb

The above is accurate as per my device.map:

(fd0) /dev/fd0
(hd0) /dev/sda
(hd1) /dev/sdb
(hd2) /dev/sdc

Now here is what geometry looked like from the grub command at bootup (menu):

Code:

grub> geometry (hd0)
drive 0x80: C/H/S = 30401/255/63, The number of sectors = 488395055, /dev/sda
   Partition num: 0,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 1,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 4,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 5,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 6,  Filesystem type unknown, partition type 0x82
   Partition num: 7,  Filesystem type is ext2fs, partition type 0x83

grub> geometry (hd1)
drive 0x82: C/H/S = 30401/255/63, The number of sectors = 488397168, /dev/sdb
   Partition num: 0,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 1,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 4,  Filesystem type is fat, partition type 0xb
   Partition num: 5,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 6,  Filesystem type is fat, partition type 0xb

grub> geometry (hd2)
drive 0x81: C/H/S = 30394/255/63, The number of sectors = 488281250, /dev/sdc
   Partition num: 0,  Filesystem type unknown, partition type 0x7
   Partition num: 1,  Filesystem type unknown, partition type 0x7
   Partition num: 4,  Filesystem type unknown, partition type 0x82
   Partition num: 5,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 6,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 7,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 8,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 9,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 10,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 11,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 12,  Filesystem type is ext2fs, partition type 0x83
   Partition num: 13,  Filesystem type is ext2fs, partition type 0x83

See the diff? GRUB geometry from the menu command line has /dev/sdb and /dev/sdc swapped. Now that I know for sure that this is a mapping issue, I just need to know what to do to fix it.

Thanks again for the tip, syg00.

Now what?

vtel57 · 04-20-2011, 08:13 PM

Well, after spending two solid days on this, I have a headache. I know how to work around this issue, so I think I'll just leave it alone for a while.

Thanks for the tips, folks. I still come here everyday and I have email notices set up... just in case someone has an idea to toss in here.

Later...

~Eric

rokytnji · 04-20-2011, 09:13 PM

Not sure if my post will be relevant. If not just ignore me. I did not have a problem with grub device mapping because I use grub4dos in mbr to boot into whatever distros grub is using in root folder of installed distro. grub4dos just points to wherever grub pbs is loaded in /

I did have a problem with conflicting device mapping though in /etc/fstab between 2 different kernels I have installed in the same Distro AntiX 11 testing. The 2.6.32 kernel would show in /etc/fstab all /dev as /dev/hdx. My 2.6.38 liqourix kernel would show in /etc/fstab all /dev as /dev/sdx. Same for fdisk -l depending on what kernel I was in.

I only noticed this as I have multiple distros and wanted their partitions to automount in thunar file manager and rox file manager at boot and folders in /mnt were hd1,hd6,etc...

So in 2.6.38 kernel. Of course the boot messages (I run no splash screen) would be warning /dev/hdx does not exist. And of course in Thunar. /mnt/hd1 folder would be empty. It was confusing the heck out of me.

I got around all this by going with UUID instead of /dev/hd or /dev/sd. My /etc/fstab as my example.

Code:

# Pluggable devices are handled by uDev, they are not in fstab
UUID=3fe7d362-f455-43a0-9eb6-d60231f24be2 / ext3 defaults,noatime 1 1
proc /proc proc defaults 0 0
devpts /dev/pts devpts mode=0622 0 0
# Dynamic entries below
UUID=1A20A02E20A01335 /mnt/hda1 ntfs-3g auto,users,gid=users,dmask=002,fmask=113,relatime 0 0
UUID=38d34375-a4c6-4fda-bcc7-736eeb487061 swap swap sw,pri=1 0 0
UUID=93317189-8b9c-440a-9224-0799fa567f9b /mnt/hda6 ext3 auto,users,exec,relatime 0 0
UUID=4B5F-B945 /mnt/hda7 vfat auto,users,gid=users,dmask=002,fmask=113,relatime 0 0
/dev/cdrom /media/cdrom udf,iso9660 noauto,users,exec,ro 0 0
/dev/hdc /media/cdrom udf,iso9660 noauto,users,exec,ro 0 0

Like I said. May not be relevant to your situation. But I thought I'd throw it out there.
It fixed my automount problems and read and write access to those partitions. I use UUID in the kernel lines in AntiX /boot/grub/menu.lst also.

Code:

title		Debian GNU/Linux, kernel 2.6.38-3.dmz.1-liquorix-686
root		(hd0,2)
kernel		/boot/vmlinuz-2.6.38-3.dmz.1-liquorix-686 root=UUID=3fe7d362-f455-43a0-9eb6-d60231f24be2 ro nosplash quiet  
initrd		/boot/initrd.img-2.6.38-3.dmz.1-liquorix-686

title		Debian GNU/Linux, kernel 2.6.38-3.dmz.1-liquorix-686 (init-3)
root		(hd0,2)
kernel		/boot/vmlinuz-2.6.38-3.dmz.1-liquorix-686 root=UUID=3fe7d362-f455-43a0-9eb6-d60231f24be2 ro nosplash quiet 3
initrd		/boot/initrd.img-2.6.38-3.dmz.1-liquorix-686

title		Debian GNU/Linux, kernel 2.6.32-1-mepis-smp
root		(hd0,2)
kernel		/boot/vmlinuz-2.6.32-1-mepis-smp root=UUID=3fe7d362-f455-43a0-9eb6-d60231f24be2 ro nosplash quiet 
initrd		/boot/initrd.img-2.6.32-1-mepis-smp

title		Debian GNU/Linux, kernel 2.6.32-1-mepis-smp (init-3)
root		(hd0,2)
kernel		/boot/vmlinuz-2.6.32-1-mepis-smp root=UUID=3fe7d362-f455-43a0-9eb6-d60231f24be2 ro nosplash quiet 3
initrd		/boot/initrd.img-2.6.32-1-mepis-smp

Hope I didn't waste your time.

vtel57 · 04-21-2011, 02:26 PM

Hi rokytnji!

Of course, you did NOT waste my time. The information you provided might not help me with my issue, but it may help others. And about the diff in the fstab naming of devices after the 2.6.32.x kernel, yes... I was definitely aware of that, although I don't think it's an issue with my troubles.

That all being said, though... I could probably resolve (or more accurately, work around) this annoying trival issue I'm dealing with my just using UUID. I've been avoiding it. Us older folks don't like change too much. The /dev/sdx identifiers were always fine for me. Why'd things have to change? Progress. That's why.

Thanks for you input.

~Eric