[SOLVED] FAIL: Migrating Chainloaded MBR to GPT on Intel dg965wh mb? Will not boot :(
Linux - DesktopThis forum is for the discussion of all Linux Software used in a desktop context.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
The bootloader does appear to point to the correct OS on sdc4 now, if I can go by matching the uuid of the os partition to the bootloader message at the beginning of bootinfo.
The BIOS seems to be asking me to insert a bootable device. So the mbr and bios-grub partitions are pointing to a known OS again, but the BIOS still cannot boot into anything resembling grub prompt? WHY? I should at least get a busybox.
I already rotated the boot order in the BIOS through all the hard drives just in case it was somehow finding a different disk to boot than I thought. It should boot. BIOS problem?
Or is there some other basic issue I am not seeing?
Several of the partitions seem to have their own 'boot sector' with old information about what sector the partition actually starts at. Some have old bootloaders, including the two I tried to boot. Why does the system not boot?
After finding that not only will it not boot, I also confused the old and new OS partitions, I am wondering if I should just start over from the beginning and copy the partitions in with Nautilus one at a time from the old drives.
Has anyone got any bright ideas yet? I seem to be churning on this.
I am out of ideas now. I fixed one mistake I made and now I am back against the wall again.
I wanted to upgrade anyway. Maybe if I fresh install 14.04 and get it working, I can use its grub-install to boot the 12.04 OS and get that thing working again. Then once the system configuration is stable I can go ahead and run the backup on mysql and finish the upgrade for mythtv...
Probably what I should have done in the first place, or something similar.
Your comments? I am about ready to give up on the current copy. Thx
ps it looks like the raid did start from bootinfo but the mount points have no filesystem because the raid is all encrypted.
xz: (stdin): Compressed data is corrupt
mdadm: /dev/md/0 has been started with 2 drives.
mdadm: /dev/md/3 has been started with 2 drives.
mdadm: /dev/md/1 has been started with 4 drives.
mdadm: /dev/md/2 has been started with 4 drives.
mdadm: cannot open /dev/md/0: No such file or directory
mdadm: cannot open /dev/md/3: No such file or directory
mdadm: cannot open /dev/md/1: No such file or directory
mdadm: cannot open /dev/md/2: No such file or directory
Last edited by CherylJosie; 08-26-2014 at 03:53 PM.
Reason: mistake
My next debug will be to install 14.04 over the 11.10 partition. I only use that for OS backups anyway and I would like to upgrade, so the move makes sense in any case, and who know, maybe it will also boot the 12.04 install. If 14.04 will not boot, then either the GPT is hosed or the BIOS has issues. Cross fingers, hold breath...
I plugged the drive into an ABIT board. It hung at grub-rescue so I know the bootloader is working, but something else is wrong. I guessed the GPT conversion must have corrupted something, since I tried everything else.
I wiped the disk and re-installed 12.04 on the Intel board. Still no boot. The bios is asking for bootable medium.
I used the 'something else' and told it to write a 1MiB bios-grub bootloader partition as well as a 1GiB swap partition and 100GiB root partition.
This time the ABIT board hangs at a blank line after trying to boot from CD. Maybe it is looking for a graphics driver now. Maybe. Usually GRUB at least gets to a menu or a prompt though, right? It is after all in text mode, is my understanding?
but I do not understand the issue. I already tried setting the boot flag. However I did not try setting the boot flag with the bios-grub partition on the disk at the same time, and I do not know which utility actually sets the boot flag correctly. Maybe I need both, and maybe the boot flag needs to be set on the bios-grub partition or the root partition or any partition (depending on the bios), maybe I need an 'ms-dos' partition type (linux and dos used to share the same, correct?). I am just totally confused as to the full nature of the problem and not interested in booting with block lists (yes I tried that too and it failed as well).
I am re-installing on the ABIT board now to see if that one sets itself up properly or not.
It looks like booting from GPT on a legacy system is not as easy as commonly believed.
OK people, anyone got a suggested protocol that does NOT involve a new motherboard or a dedicated boot drive MBR?
Do I need to use a particular partition or file system?
I assume the bios-grub partion is a requirement.
Do I need to set a boot flag? Which partition should I set it on? Which tool do I need to use to set it?
I tried to put everything I learned in here, just in case someone else is confused about what to do, and why, when trying to boot a legacy BIOS with GPT. I think I found out just about everything there is to know about it now...
Note that I found so many conflicting forum posts that I got totally confused on how to systematically approach such problems, so I thought I would write my own guide here, for those in the dark like I used to be.
I needed to have BOTH a 1MB bios_grub partition (in this case, /dev/sda1) and set the boot flag on '/dev/sda1/' using fdisk in order for the dg965wh Intel board to boot from GPT.
I did NOT need to set the CHS parameters in the protective MBR to something 'valid' (according to the BIOS) using gdisk, although I might have to do that to get it booting on my ABIT board since the 'bios_grub' plus 'boot flag' fix did NOT work on that board!
I did NOT need to create a 'hybrid MBR-GPT' (fortunately), although again I might have to do that to get it booting on my ABIT board, since I have not got root cause on that one yet.
I definitely would have needed 'hybrid MBR-GPT' if I wanted to keep chainloading with GRUB version 0.99 (incompatible with GPT). Refer to prior Rod's gdisk link and http://www.wensley.org.uk/gpt for more details. Although written for EFI, the concept at wensley is identical (I think?), except that for non-EFI the second stage bootloader goes in a bios_grub unformatted partition rather than using an EFI partition/filesystem to hold it (and EFI/UEFI BIOS to interpret the EFI partition).
The ONLY tool I could find that would set that boot flag in the protective MBR was fdisk. That flag does NOT point to the bios_grub partition on /dev/sda1 in the GPT, or to the ext4 partition on /dev/sda3 with Ubuntu in the GPT. It refers to /dev/sda1 as the GPT (minus protective MBR) because the GPT is the first and only 'primary partition' on the disk, according to the original meaning of the MBR specification, and fdisk reports it as such rather than as a partition table in its own right. According to the MBR specification definition of the 'boot sector', GPT is actually one of four possible primary partitions 'living' within its own protective MBR. Although GRUB2 sees the GPT as a sort of 'extended' partition (like in an MBR partition table) the BIOS does not, and cannot boot from it.
The reason for this need of a boot flag in addition to the bios_grub partition is that some older BIOS require at least one partition marked as bootable in order to recognize a valid system disk at all. Other BIOS require that the boot flag must be set on only one partition. This is actually considered to be a 'bug' by some, but it seems to me that it is in accord with the very first definition of MBR back in the days of DOS when its bootloader looked at these flags to see which partition it is supposed to boot to (I think). I recall changing that boot flag to change which 'disk' I booted, as this was the original multiboot technique. Boards that ignore these flags, and the CHS constraint as well, are probably in violation of the original MBR specification.
I saw some comments that the GPT spec should be changed to ensure that both the boot flag and the CHS are set according to the old MBR specification, even in the GPT protective MBR. I agree, but then I do not profit from selling new hardware to crusty old Linux nerds so I will just have to play this game of 'violate the spec' in order to save myself the cost of an upgrade to an EFI/UEFI system or the hassle of finding one that, unlike Intel, ignored this part of the MBR spec.
Note that setting the boot flag in the protective MBR is a violation of the GPT specification, but it must be set anyway to boot GPT from this Intel board. A bios_grub partition to hold the second-stage bootloader is also necessary.
My ABIT machine would not boot even with both the bios_grub partition and the boot flag set. It must be one of those really stubborn ones. Perhaps setting the CHS disk parameters in the protective MBR to something legal according to the BIOS (also a violation of the GPT specification) would fix the problem with booting from that BIOS, or maybe not. If not, a hybrid MBR-GPT would be in order to try next. If I get bored some day...
Note that my impression of hybrid MBR-GPT is highly risky. Apparently some partitioning tools zero out important information that synchronizes the two parts of the hybrid, and can require a re-installation of the bootloader every time the partition table is modified or something like that. I do not advise this approach, nor GRUB blocklists, as they can result in unintended 'spaghetti' side effects when changing the system configuration.
So now I have both machines successfully booting GPT finally. They both needed the boot flag set on the GPT 'primary partition' using fdisk. Not sure why the ABIT gave me issues the first time but it is working now. Seems I had to clear the CMOS because somehow it started throwing a DMI (desktop management interface) error during POST. At one point the SATA cable was sort of dangling halfway off the boot drive so maybe the BIOS got confused by bad disk reads when it saved the CMOS memory.
I put the original Intel server MBR drives that are still functioning into a larger case with the ABIT board (along with its original drives for a total of 7 SATA hard disks in that system), and will wipe and re-install Ubuntu on that machine with GPT when the data from all the drives in that ABIT machine is transferred back to the Intel machine. I should get another 6TB total (4TB usable RAID5) off that ABIT machine but all those drives are old and one already failed. It might not be the most reliable system in the world.
Big thank you to all those dedicated Linux programmers and sysadmins working for free and publishing everything on the Internet. This is an example of socialism functioning well alongside capitalism, like it should be. I can keep my old hardware in service without having to constantly buy crapware Windows or Solaris or something equally unworkable.
Now I am swimming in storage space instead of choking in a closet. How long to fill up 20TB of available storage with blu-ray? Hopefully not in this lifetime. Not looking forward to another hardware upgrade.
I decided to go with RAID6 in the new server because 6x4TB drives seems way too risky to try to rebuild after a single drive failure when statistically all the drives are likely to wear out and fail close together not randomly. It takes that system 3 days to wipe the drives with frandom and it also took it 3 days to finish the initial RAID resynch. Given that this upgrade was instigated by a failing 1.5TB drive as much as by space restriction, I chose not to risk losing 20TB of data just to get 4TB extra with RAID5 on that server. All the partitions except the OS and bios-grub are now RAID6 over 6 drives to maximize both space and reliability. Obviously with all this storage, backups are out of the question so I am relying entirely on the RAID6, careful management, and luck to preserve my data, plus I will still have the original media on hand if I need to reload any of it. Not so much of a comfort where all the vinyl LP and VHS analog stuff is concerned since it all had to be played at normal speed to digitize it, but I might be able to save a copy of that on each server since it is less 'spacey' than the HD material is. Any other data like personal documents etc. will be backed up to an external drive and stored in a safety deposit box I guess.
This is a big job. I knew it was big going in, but the part that took the longest was learning to understand it. I am still working on a reasonable encryption scheme that preserves data integrity plus security in this home environment and I have not even started on Kerberos or LDAP. It could take me another year to get this up and running as in a professional setting. Hopefully by that time I will finally have some useful sysadmin skills too. I might need them someday.
Based on my experience it seems that just about every legacy BIOS will require both the system reserved boot partition to hold the bootloader (OK to use gparted to create the reserved partition and then set the bios-grub partition type but remember that gparted will not let you do both operations in one 'apply' since the partition type is written immediately so you have to do that after the partition is created) and the boot flag set on the GPT with fdisk. It cannot hurt anything to set the boot flag since that bit is not used by GPT for anything anyway so I recommend it be done routinely for legacy BIOS booting on GPT just in case.
After investigating what is required to safely convert MBR to a properly aligned GPT after transferring the entire disk content from 512B to 4K sector disks using dd, I decided to avoid the issue entirely and transfer just the data via NFS instead. I probably could have done the conversion and alignment safely in-place if I had shrunk the file system inside the last logical partition of the extended partition with gparted first to make sure I moved all important data away from the second copy of the final resulting GPT after conversion, but the amount of time and work involved seemed a waste since I had done things wrong the first time already.
Also I have encrypted RAID partitions at the end of 4 of those drives and shrinking the filesystems to make room is a pain. Maybe I could have increased the extended partition size on the new larger drives first using gparted and then the conversion could happen safely. Dunno exactly how this works, does the second copy of the partition table go inside of, or after, the GPT partition? I think maybe the second copy goes after the GPT 'primary partition' so with data transferred to larger physical disk maybe it is OK to just convert to GPT in place? How does this work exactly? gdisk is not a user-friendly tool, it is more of a Swiss army knife. One needs knowledge, skill, and patience to do this sort of in-place conversion.
It was also especially difficult to move the partitions around on the new drives given that it takes gparted forever to do such operations and it was like solving a brain teaser to get from point A to point B with larger disks, larger partitions, more partitions, more physical disks in each RAID, etc. Then I still would have to resize all the partitions to fill the extra space and for encrypted RAID that is a chore, needing to do it in gparted to move/resize the partition first, then in mdadm for the raid volume, then in cryptsetup for the encryption, then in resize2fs on open and decrypted volume for the ext4 filesystem. That makes 4 complicated operations total and the original RAID was installed on an older OS with (probably) less than optimal stride/stripe for the new configuration. Very complicated to do safely and properly. I still am not sure if gparted even knows how to move encrypted RAID partitions. Does it preserve the data? It does not even recognize the RAID or the encryption or the filesystem. I used dd to move the actual data inside every time just to make sure I did not corrupt the RAID. Was that the right way to handle it? Who knows? I abandoned that method.
Finally, while configuring the new GPT disks I also discovered that there are alignment issues introduced at several levels. First the partitions inside the GPT must be aligned so they start on a 4K sector boundary. Second, the filesystems inside the GPT partitions must ALSO be aligned so that they too align on 4K sector boundaries! Gparted handles these issues automatically so I ended up using it to create all the partitions and all the non-RAID, non-encrypted file systems. I used mkfs on the RAID partitions and just hope it aligned them OK and set the stride/stripe to something reasonable (256/512 is what it picked). I really have no way of checking on encrypted RAID alignment but it seems to be working OK. The file transfers are slow over NFS with Nautilus invoked as root, especially on the original root partition that has 500K individual files with some 'special system files' that Nautilus refuses to copy. That is one reason why I ended up using dd to transfer the OS partition directly from disk to disk so I still had to physically install one old and one new drive into the same system temporarily to do that transfer.
Anyway I did figure out that using dd to transfer just the OS filesystem/partition content to a larger partition on the new GPT drive worked fine. All I had to do was, increase the size of the root filesystem with resize2fs to fill the new larger partition, then a simple and straightforward chroot and grub-install to put grub2 bootloader on the system reserved partition. Booted right up into my old environment, except for the chainloading with GRUB legacy is now gone. Maybe I will just put a dedicated boot partition on the system anyway eventually since I want to encrypt the OS too some day, just for fun. Not fun? True, but how else will I learn how to do it?
It has been real, people, a real PITA but at least I understand it better now. It looks like copying entire MBR disks with multiple partitions and encrypted RAID to larger disks with dd, then converting to GPT, resizing and re-arranging partitions, adding more physical RAID partitions to expand the volumes, and booting the resulting GPT on a legacy BIOS is all possible but I do not recommend it if there is any way to just transfer the data between two working systems instead. Fortunately I got there without any additional expenditures because I already had some old hardware lying around.
Thanks for the help. Bootinfo helped me to understand my mistake in trying to chroot to the wrong copy of the OS the first time, and to see that everything was apparently configured properly after I fixed that, pointing to other issues such as the boot flag that must be set with fdisk that modifies the protective MBR of the partition table, not gparted etc. that inhibit that action per the GPT specification. Successful assistance.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
