Hello All

I'm helping my brother setup a server he has. The server has a 3Ware 9690SA Hardware RAID card, and it has 4 x 2Tb 5400 RPM Samsung Harddrives.

When I boot into the 3Ware BIOS, it recognizes all 4 drives, no problem. What he wants to do is install them in RAID 10 for a 4 Tb mirrored volume for performance and redundancy. When I try to create a RAID array with the 4 drives, and set it to use RAID 10, it only allows me to utilize 2079 Gb (1/4 x 8 Tb), which is half of what I would expect for this RAID config.

Assuming I go ahead and allow it to build the array with this volume, we can go on to install Fedora 14 with no issue, but still only 2 Tb are available.

This is our first time setting up a RAID array like this, so I'd appreciate any advice anyone can offer.

I've also tried enabling Auto-carving to insure that I have support for physical volumes larger than 2 Tb, but to no avail. With the version of Linux we're running, and the RAID card he's got, it should support greater than 2 Tb PVs either way though.

Thanks!

Use a live cd (gparted one would be a good idea for this) and set the partition table type to gtp for each drive. I'm fairly certain the default msdos table will only support partitions up to about 2 TB in size. Since that is about the size you are getting, that's probably your issue. I know this would be an issue when using mdadm (software raid), but am not positive how using a hardware raid controller would change things. I would imagine that you would still run into that limitation though.
WARNING: This will destroy anything you may have on the drives, so make sure you backup whatever may be on them.

Thanks for the response, Brian

After tooling around with it some more, I finally decided to just try installing Fedora on the array when the 3ware bios indicated only 2 Tb of storage for the RAID 10 setup across all drives.

What I found after using the disk tool (forget the name atm), is that there is a 1.8 Tb partition with Fedora on it, about 520 Mb for a swap file, and then a 2.2 Tb section of "Free space" which I am unable to partition (or at least unable to partition using the tool I was using). It gives some kind of error that I'm also forgetting...

Does this information offer any further explanation?

Thanks again

Yep, partition table size limitation. This will limit you to about 2 TB partitions. You need to create a GPT (ignore the misspelling in my first post) partition table on each of the disks. Then install Fedora, you should have a 4 TB array. The installer may not allow you to change the partition table type of the disks, I haven't installed Fedora since about 10 or 11 and don't remember if it did or not. Use the gparted live cd, change the partition table types and be done with it.

Thanks Brian. Forgive my ignorance here, but how do I setup a GPT partition on the drives before having installed Fedora? Are you suggesting that I boot to the live DVD, then format the drives to GPT, then reboot and setup the RAID array, and that the format will still hold even after I install Fedora?

yeah, use a live cd or something and do it from there. If the fedora live dvd has gparted, it should work. Open gparted, select drive 1, delete any current partitions, the go to device -> create partition table. Repeat for each remaining disk. Obviously, everything is gonna be deleted, to backup anything you want to keep thats on the disk now.

Hi Brian

I've been tinkering with the server. I think I have partially succeeded in what you suggested that I do; here's the steps I took.

Boot into the 3ware bios and configure a RAID 10 array with all four drives. On exiting the 3ware BIOS shows it has a capacity of 2 Tb. After that, I booted into the live CD (a Fed 12 disk I had laying around) and installed gparted and ran it on both drives, formatting them to GPT (gparted only showed sda and sdb, since I have four drives in a RAID 10 configuration, but it does recognize each PV as 2 Tb).

This seemed to work, so I rebooted and installed Fedora 14 from my DVD. Before doing this, I checked the 3ware BIOS to confirm that the GPT had worked, and saw that the RAID 10 array now shows a capacity of 3.8 Tb or thereabouts (success!)

Fedora 14 installed successfully, I put it onto sdb, which showed a capacity of around 1.7 Tb (I also used the "basic storage" devices method of installation), and left the sda drive alone for now (figured I can partition later).

The problem is that when I boot up, the system hangs on a black screen with a single cursor in the top left corner. I figured I would check to just insure that it installed OK, and ran the installer again to confirm the boot sector was there, and it was, it recognized everything just fine.

What do you think could be causing the system to not boot after a good install?

Good to see the array is showing the size it should. As to why it won't boot, according to this install guide, in section 9.1.1, it says the /boot partition must not be on a raid. Find another drive and install Fedora on that one, using the array for storage. An old 10 GB drive would probably work fine.

I installed a 2.5" laptop HDD, the least expensive one that I could find @ $40 from Fry's and used this for the OS drive. I then used gparted to format each file system to GPT, which allows for filesystems larger than 2 Tb.

Thank you so much for your help! Also, the link you posted to the install guide is a bit of a hidden gem, I never knew such a thorough guide was out there. That site is really neat.

My next question is how to use the space I've just created. I went ahead and made one large partition for each drive--do I now just need to mount it to make it work so that I can store files on it?

yep, add something like this to your fstab file and you should be set:
adjust accordingly for where you want to mount it and the file system type
mine is all 1 big partition, seemed fine at the time

Success! thanks for your help!