Issues Creating 16 TB Storage on CENTOS 6.2 with Ext4
 

Good Day,

I have searched your forums however I have been unable to find an answer close to what I am looking for. My hope is that I have posted it in the correct forum and that I have included enough information for the smart folks to help out.

We are attempting to build a Linux server on an ESX virtual machine with 16+ TB of storage that will be deployed as a Google Earth (GE) server. We are creating a raid 0 as the SAN is preconfigured with raid 6 (for redundancy) and for GE we must have contiguous space. We have been unable to concatenate the 8 x 2TB drives as raid0 and actually failed at 5 drives.

Our configurations are as follows:

- CENTOS 6.2 configured as desktop
- 2GB ram
- 1 CPU
- Disk 1: 100GB / OS drive
- Disk 2 -> 9: qty 8 x 2TB / to be created combined for raid 0

We have attempted a number of settings even going down a version of the OS and have had limited success but are unable to come close to our 16TB of storage. We have taken the defaults as follows

- Block Size = 4096
- Fragment Size = 4096
- Stride = 128 Blocks, Stripe width=640 blocks

We have had success with the following format:

- Format qty 1 x 2TB drives and format as ext4
- Format qty 3 x 2TB drives configured as raid 0 (total 6.4TB) formatted as ext4

We have attempted 5 x 2TB drives (11TB) and the format failed at approx 70% of the format. Errors in the logs are listed on each drive

Any assistance with this issue would be greatly appreciated

Cheers

Did you try to use LVM?

The easy idea of using LVM is to make raid 1 (mirroring) if you require data redundancy. Then make LVM and add all raids. LVM is useful instead of raid/stripping.
Make a partition on the LVM.

example:
md0 ... sda mirros sdb (2 TB of 2x2TB)
md1 ... sdc mirros sdc (2 TB of 2x2TB)
md2 ... sdd mirros sde (2 TB of 2x2TB)
md3 ... sdf mirros sdg (2 TB of 2x2TB)
etc.

LVM = md0 + md1 + md2 + md3 (8 TB)

or
LVM = sda + sdb + sdc + sdd + sde + sdf + sdg (16 TB) if you needn't redundant data, even thought there is a really high risk of data losing without any recovery possibilities.

I guess CentOS supports LVM.

Hi, few minutes after I sent the message I accidentally read that Fedora 17 is going to break the limitation of 16 TB filesystem size of ext4, exatly 2^32 fs blocks. Probably it should be the same case in CentOS and increasing the block size could help.

the source http://fedoraproject.org/wiki/Features/F17Ext4Above16T

Martin

Thanks for the response your tip on LVM. The LVM application enabled us to add the 8 x 2TB drives and create our 16TB drive.

Cheers