Over 8TB Raid 5 ext3

freehood4all · 12-11-2009, 10:50 PM

I'm looking for some help regarding my raid 5 array.

I expanded my raid recently from 6 to 7 1.5TB hdd's I now have a 9TB volume that would have ~8.2TB available after partitioning

Code:

# fdisk -l

Disk /dev/sda: 8999.9 GB, 8999998586880 bytes
255 heads, 63 sectors/track, 1094187 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/sda doesn't contain a valid partition table

I am running Debian Lenny 2.6.26-2-amd64 (stock kernel) with an Areca 1220 PCI-E 8 port Raid card

My current file system is ext3

Code:

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/Debian-root
                      6.5G  3.1G  3.1G  50% /
tmpfs                 4.0G     0  4.0G   0% /lib/init/rw
udev                   10M  860K  9.2M   9% /dev
tmpfs                 4.0G     0  4.0G   0% /dev/shm
/dev/sdb1             228M   18M  199M   9% /boot
/dev/mapper/Debian-home
                      276G  221M  261G   1% /home
/dev/sda              6.8T  6.4T  2.7G 100% /mnt/raid5

Is it possible to partition the array so I can use all available space? if so how?

Would it be feasible to convert such a large array to ext4 to break the 8TB limit?

*Also note, I do not have a way to backup the data on the array*

If someone could walk me through my options it would be greatly appreciated

EDIT: ext3 Wiki states:

Code:

Block size 	Max file size 	Max filesystem size
1 KiB 	16 GiB 	2 TiB
2 KiB 	256 GiB 	8 TiB
4 KiB 	2 TiB 	16 TiB

My Block size:

Code:

# tune2fs -l /dev/sda | grep -i 'Block Size'
Block size:               4096

So the limit is 16TB... but I still need some advice on how to repartition it

Gparted Live CD didn't work and to my understanding fdisk is useless >2TB

saavik · 12-12-2009, 08:31 AM

Think about the time for fsck!
We use ext3 for a 3TB Volume and the fsck is horible slow!
We also have an ext4 running where we love to make fsck! Takes less than 10% of the time.

freehood4all · 12-12-2009, 05:53 PM

I should be able to resize the partition but...

Gparted is giving me a warning message stating that it could not find a valid filesystem superblock

I located this via google http://forum.hddguru.com/ext3-filesy...ck-t12415.html

Code:

# dumpe2fs /dev/sda | grep -i superblock
dumpe2fs 1.41.3 (12-Oct-2008)
  Primary superblock at 0, Group descriptors at 1-437
  Backup superblock at 32768, Group descriptors at 32769-33205
  Backup superblock at 98304, Group descriptors at 98305-98741
  Backup superblock at 163840, Group descriptors at 163841-164277
  Backup superblock at 229376, Group descriptors at 229377-229813
  Backup superblock at 294912, Group descriptors at 294913-295349
  Backup superblock at 819200, Group descriptors at 819201-819637
  Backup superblock at 884736, Group descriptors at 884737-885173
  Backup superblock at 1605632, Group descriptors at 1605633-1606069
  Backup superblock at 2654208, Group descriptors at 2654209-2654645
  Backup superblock at 4096000, Group descriptors at 4096001-4096437
  Backup superblock at 7962624, Group descriptors at 7962625-7963061
  Backup superblock at 11239424, Group descriptors at 11239425-11239861
  Backup superblock at 20480000, Group descriptors at 20480001-20480437
  Backup superblock at 23887872, Group descriptors at 23887873-23888309
  Backup superblock at 71663616, Group descriptors at 71663617-71664053
  Backup superblock at 78675968, Group descriptors at 78675969-78676405
  Backup superblock at 102400000, Group descriptors at 102400001-102400437
  Backup superblock at 214990848, Group descriptors at 214990849-214991285
  Backup superblock at 512000000, Group descriptors at 512000001-512000437
  Backup superblock at 550731776, Group descriptors at 550731777-550732213
  Backup superblock at 644972544, Group descriptors at 644972545-644972981

I'm thinking about giving this a go, but would like some input from the guru's here

Code:

e2fsck -b 32768 /dev/sda

If I need to provide further info, I will happily do so

Thanks

adam999 · 12-13-2009, 06:10 AM

what happens when you run

Code:

resize2fs /dev/sda 8999G

thanks

freehood4all · 12-13-2009, 10:37 AM

Code:

# resize2fs /dev/sda 8999G
resize2fs 1.41.3 (12-Oct-2008)
The containing partition (or device) is only 2197265280 (4k) blocks.
You requested a new size of 2359033856 blocks.

so I then ran

Code:

resize2fs /dev/sda 8381G
resize2fs 1.41.3 (12-Oct-2008)
Please run 'e2fsck -f /dev/sda' first.

So should I run the e2fsck even though Gparted is complaining about bad filesystem superblock?

I'm being much more cautious than I usually would be. This is a whole lot of data

adam999 · 12-13-2009, 04:10 PM

If you're tinkering around with the partitions, its always good practice to make sure you have good backups if this data is critical.

running the journal should be ok (and is normally scheduled periodically as part of ext3)

freehood4all · 12-13-2009, 10:51 PM

Thanks for replying Adam.

I'm making progress

Code:

# e2fsck -f /dev/sda
e2fsck 1.41.3 (12-Oct-2008)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
raid5: 177333/457768960 files (23.1% non-contiguous), 1738807722/1831054400 blocks
Debian:~# resize2fs /dev/sda 8381G
resize2fs 1.41.3 (12-Oct-2008)
Resizing the filesystem on /dev/sda to 2197028864 (4k) blocks.

EDIT: and it's finished

Code:

# mount /dev/sda /mnt/raid5
Debian:~# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/Debian-root
                      6.5G  3.2G  3.0G  51% /
udev                   10M  860K  9.2M   9% /dev
/dev/mapper/Debian-home
                      276G  232M  261G   1% /home
/dev/sda              8.1T  6.4T  1.3T  84% /mnt/raid5

jag1506 · 03-12-2011, 10:21 AM

I know this post is old but I think the information is still needed for a lot of people, and a better way on how to solve this and how I did it:

bash-4.1# parted /dev/sdb
(parted) mklabel gpt
(parted) mkpart primary 0 100%
(parted) set 1 lvm on

mkfs.ext3 -F -b 4096 /dev/vg0/lvol0 >yes I'm using LVM but I think if you use /dev/sdb1 should work just fine.
if you are not using lvm should be something similar just read the parted man page:

# parted /dev/sdc
(parted) mklabel
New disk label type? gpt
(parted)

You can check more info in my website or ask me at any time.

http://www.nixheiser.org/LargerDisks