LinuxQuestions.org - RAID5: impossible to create raid properly on partitions

- Linux - Hardware (https://www.linuxquestions.org/questions/linux-hardware-18/)

- - RAID5: impossible to create raid properly on partitions (https://www.linuxquestions.org/questions/linux-hardware-18/raid5-impossible-to-create-raid-properly-on-partitions-841577/)

RAID5: impossible to create raid properly on partitions

Hi,

I've tried to create a raid 5 array for the past few days and it seems impossible to create it properly on the right partitions.
I'm using ubuntu 10.04 server, 3x 2TB Samsung F4EG (with 4k sectors that need to be properly aligned).

Here's how I proceded:
-I create a Linux raid partition on each drive with Gdisk (fdisk supporting GPT), aligned on sector 2048
-Launch mdadm to create array with:
sudo mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdd1
-I wait until the whole array is created
-I format md0 with:
mkfs.ext4 -b 4096 -E stride=16,stripe-width=32 -O extent -m 0 -v /dev/md0

The problem is that I end up with an array on the drives; not on the partitions:
my /proc/mdstat

Code:

ersonalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 

md0 : active raid5 sdd[2] sdb[1] sda[0]

      3907026944 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

      

unused devices: <none>

and mdadm --detail

Code:

/dev/md0:

        Version : 00.90

  Creation Time : Sat Oct 30 18:11:20 2010

    Raid Level : raid5

    Array Size : 3907026944 (3726.03 GiB 4000.80 GB)

  Used Dev Size : 1953513472 (1863.02 GiB 2000.40 GB)

  Raid Devices : 3

  Total Devices : 3

Preferred Minor : 0

    Persistence : Superblock is persistent



    Update Time : Sun Oct 31 16:16:40 2010

          State : clean

 Active Devices : 3

Working Devices : 3

 Failed Devices : 0

  Spare Devices : 0



        Layout : left-symmetric

    Chunk Size : 64K



          UUID : 0551e0d3:44e7a9f1:46479b19:e8b0c6ac (local to host miki-server)

        Events : 0.36



    Number  Major  Minor  RaidDevice State

      0      8        0        0      active sync  /dev/sda

      1      8      16        1      active sync  /dev/sdb

      2      8      48        2      active sync  /dev/sdd

I am going crazy, I've destroyed the raid 3 times (by erasing superblocks) to start over and every time I end up with a raid on the disks and not on the specified partitions.
Any help is welcome!

First of all, I'm not all that clued up on software RAID, I'm more used to HP's SmartArray hardware stuff.

Quote:

every time I end up with a raid on the disks and not on the specified partitions.

Maybe I'm wrong but I thought the unit of storage would be the disk and not a partition. RAID means Redundant Array of Inexpensive (or Individual) Disks, not partitions. I've never heard of RAIDed partitions. What would there be to stop you creating three partitions on a single disk and then RAIDing them? Maybe you can, I dunno, but what's the point? :scratch: Disks in a hardware RAID have a separate system partition which holds the RAID configuration in case the controller has to be replaced,is there a similar sort of thing in a software RAID to assist in rebuilding swapped out disks? Hmm...

Nope, I think I'll leave it to someone smarter than me to help you.

Play Bonny! :hattip:

Hi,

The array by itself looks okay. Are you using GPT or MBR partition table? I've never seen such a behavior before.

@Soadyheid: Geekmiki is right, in Linux software RAID the RAID-disks are partitions on physical devices.

Quote:

@Soadyheid: Geekmiki is right, in Linux software RAID the RAID-disks are partitions on physical devices.

Cheers mesiol, thanks for the clarification.

Play Bonny! :hattip:

This is driving me crazy!
@mesiol: I'm using a GPT partition table (created with GPT Fdisk)

4th time I create the array from scratch and 4th time this happens... I might be doing something wrong, but I'm following tutorials just as described.
The raid creation itself goes fine and once finished I do have an array from /dev/sd[abd]1. When I reboot and check /proc/mdstat everything has moved to /dev/sd[abd] from the orginal partitions.

Code:

personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] 

md0 : active raid5 sdd[2] sdb[1] sda[0]

      3907026944 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU]

      

unused devices: <none>

I'm about to give up...

This my last try, the array is recovering on the right partitions:

Code:



Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]

md0 : active raid5 sdd1[3] sdb1[1] sda1[0]

      3907026944 blocks level 5, 64k chunk, algorithm 2 [3/2] [UU_]

      [=>...................]  recovery =  6.8% (134665600/1953513472) finish=358.6min speed=84518K/sec



unused devices: <none>

Hi,

possibly a problem with GPT style layout. Have you tried normal fdisk to create MBR style partitions? Did this change anything?

Actually for the first array creation attempt, I used fdisk and MBR partitions (also starting at sector 2048), but the behaviour was the same. After first reboot the array didn't stay on the partitions.
That's when I decided to try GPT fdisk...

After you make the partitions, did you use "hdparm -z" for each hard drive before you add those partitions to the array. Every time you boot or reboot and the partitions that you created are not written, you will lose everything although the data could be there.

I think you have issues of the first sector of the new or 4 kilobytes and the old 512 bytes. You may want to zero out those sectors to have a clean drive to start over...again.

You can do a test to see if Linux recognizes the drive is using 4 kilobyte sector by just using one drive. If it does not, you will have to update the kernel and the disk utilities.

I have not setup RAID or software RAID, but what I know is is best to use a separate none software RAID drive to store the boot data. You can rely on the BIOS on what it knows best and then wait until the kernel loads which will give your computer more support for GPT. I suggest make sure the kernel and mdadm utilities are updated with the versions that support GPT.

Just a note, a 2 terabyte hard drive will take a long time to re-construct, so you will have a higher chance of losing data when any drives fails during re-constructing. You have to hope that the drives does not fail during about 15 hour that it takes for your setup to re-construct.

Just tested with one of the drives alone:
-created a GPT partition with gdisk
-mkfs.ext4
-mounted
-copied a file
-added to fstab
-rebooted

Works like a charm...

Hi,

so i am a little at the end. What is about viewing the mailing list of the programmers?

Problem solved by installing the package mdadm 3.1.14 from Debian sid to replace the 2.6 something included in Ubuntu 10.04!

Thanks all for your help!