I'm running Debian/Lenny (testing) which has an up to date kernel, etc. I'm setting up my desktop to use software RAID 5 (replacing a 250G / and 500G /home drive with 3 x 500G drives). The new configuration will have / on a small RAID 5 array with /home on a larger RAID 5 array and 3 swap partitions (one on each physical drive).

At this point I ran into a stumbling block. I'd previously set up RAID 1 on systems and there was no real issue about /boot - it could be read on one drive or the other. From what I've been reading, it looks like Linux can't boot from a software RAID 5 array. However, I haven't seen this issue raised in any recent HOWTOs - just on old ones.

It seems cleaner to boot from a RAID 5 array. While I could set it up as a RAID 1 array, I'd only have the boot information on 2 drives, with some wasted space on the other. This could cause problems if a drive fails and the wrong drive ends up being the boot drive. Setting up /boot on each drive separately and rsync-ing them regularly also seems clumsy.

On the other hand, with RAID 5, I don't need a separate boot partition and all drives are equal when it comes to booting. However, I can't figure out to do a grub setup on a RAID 5 array. Grub seems to want to deal with physical disks that have a readable /boot on them.

Is there a way to boot from a software RAID 5 array?

I've booted from hardware RAID5 easily, and grub had no problems with it. Never tried it with software, and not even sure you can do it, since you'd have to have some parts of the system up and loaded first, before you could bring the array online.

I'd spring for a hw RAID controller.

Hardware RAID is a different kettle of fish. The controller makes the array look like a hard drive, so it's basically the same as booting from a hard drive. Software RAID conceptually could be the same, providing that the RAID driver can be loaded. However, I'm not sure if it's do-able.

The problem with hardware RAID is the expense. RAID 5 controllers seem to start at a couple of hundred dollars and go up from there, without giving any better performance.

You would need at least some way of loading the software for the array. I don't know if any of the "fakeraid" boards (or motherboard raid) do RAID 5; you can boot off of RAID 0 with those boards because of the BIOS support and they are a lot more affordable.

Yes, but RAID 0 and RAID 1 aren't RAID 5. With RAID 1 to RAID 5 you get s * (n - 1) amount of space (i.e. one less than the number of drives you allocate to the array provide storage space). With RAID 1, n is 2 so you lose half your space. With RAID 5 in a three drive array, you only lose a third of your space. I get 1,000G of usable storage with redundancy for the price of three smaller drives.

However, that extra drive means that you need to look at two drives to get the data.

As I said, there is no reason in principle why software RAID can't handle this. All it needs is to be able to load enough software to read the array. This is not substantially different from needing a driver to read the boot disk - you're just dealing with a different, more complex, device. Hey, you can boot from a USB key, can't you?

Sorry that I wasn't clear. The reason I mentioned RAID 0, is that, like RAID 5, you have to read 2 disks to get your OS and array up. Unlike RAID 1, where you only need one disk. I am aware of different RAID levels.

The only way you can boot off RAID 5 or 0 or more than one device is with the help of the BIOS or another device supported by the BIOS. The USB key you mention is a good example. You can't boot from one unless either your BIOS supports that or you use a CD/floppy/harddisk to hold the boot manager to reference it. Fake RAID boards and the onboard versions on some motherboards provide BIOS extensions to allow booting off of RAID 0 devices. They are not true hardware RAID. I do not know if these relatively cheap devices ever support RAID 5, but there is no reason why they could not. However, mine does not. Mdadm can autobuild an array on the fly, but you have to be able to read the kernel first or read an initrd off the disk, which means you need the help of the BIOS or another boot device which can be read at boot time and is supported by the BIOS.

I suppose you could write a patch to grub to incorporate RAID 5 and have it installed on the MBR of all the disks, but as far as I know, no one has gotten that ambitious. There is a grub patch for dmraid for support of fakeraid, but, again, the fakeraid would have to support the RAID level you want.

Anyone else know any different?

The cheap boards don't support RAID 5. That's why they're cheap.

re. booting: I think the BIOS just has to let the boot loader get the first sector of the device. Most RAID boards, and even SATA and IDE controllers, need a driver to work.

Interestingly, I've even run across a case where I could start an install from a CD but not complete it because at some point in the install process, Linux stops using the BIOS and tries to load its own driver.

I guess I'm just going to have to add a non-RAID boot partition and use rsync to keep all three drives current (so any drive could be used to boot the system).

Per the Gentoo docs no..

http://gentoo-wiki.com/Talk:HOWTO_Ge..._5_and_.2Fboot

We did this with a Dell having 5 disks. We had root and boot mirrored on disk 0 and 1. Then, the remainder of these drives and an equal portion of the remaining 3 disks as software RAID5. The "pieces" unused on disks 2, 3, and 4 were used for swap.

Farslayer: That's an interesting suggestion they make - about setting up the /boot as RAID 1 across all the disks. I'd only heard of 2-disk RAID 1 before, but I suppose there is no reason why a 3-disk RAID 1 can't be done. It eliminates the need to sync the disks.

Thanks.

TwoFeathers: I'd thought about doing that too, but with a three disk RAID 5, it would have left me with just one swap partition. I've set mine up with the swap partitions separate (non-RAID), although I've seen a warning that this could lead to a problem if a swap partition becomes unavailable.

I'm not familiar enough with how Linux handles swapping to decide if it's a real problem or not. Have you any ideas on it?

Just to let people know how things worked out:

I have a 3-disk system currently running as follows:
/dev/md0 - 100M RAID 1 array for /boot
/dev/md1 - 820G RAID 5 array for /home
/dev/md2 - 90G RAID 5 array for /

plus 3 x 2G non-RAID partitions for swap.

Using 3 disks in a RAID 1 array for /boot was something I'd not considered, but it seems to work OK. Each drive has grub installed, so any of them can boot if needed.

Thanks for the update !! Glad to see you got it working