mdadm raid10 with 6 drives or more

thirdbird · 02-09-2017, 03:19 PM

I'm experimenting a lot with mdadm lately and finding that regardless of how many drives I use, e.g. if I create a --level=10 with 6 or even 8 drives, it will only create 2 mirrored sets like it does with 4 drives where that's the only possible option. Example follows.

Command used:

Code:

mdadm --create /dev/md1 --level=10 --raid-devices=8 /dev/{sdc,sdd,sde,sdf,sdg,sdh,sdi,sdj}

Result:

Code:

Number   Major    Minor   RaidDevice  State
       0       8       32        0      active sync set-A   /dev/sdc
       1       8       48        1      active sync set-B   /dev/sdd
       2       8       64        2      active sync set-A   /dev/sde
       3       8       80        3      active sync set-B   /dev/sdf
       4       8       96        4      active sync set-A   /dev/sdg
       5       8      112        5      active sync set-B   /dev/sdh
       6       8      128        6      active sync set-A   /dev/sdi
       7       8      144        7      active sync set-B   /dev/sdj

Do I need to use nesting to achieve a stripe of 2x mirrors? Am I reading the result wrong - are they separate sets? (which would be very confusing and unclear). I'm trying to figure out if I'm missing a parameter or if this is expected behavior unless going nested.

Thanks.

syg00 · 02-09-2017, 07:01 PM

That is what RAID10 is - RAID1 (a stripe set) + 0 (mirroring). Anything using/mirroring less than the full stripe set will not be (point-in-time) synching all the data.

I don't know what you mean by "nested" - alter the above printout to elucidate me pls.

thirdbird · 02-09-2017, 07:13 PM

This would be nested 1+0 where you can actually define how many drives are in each mirrored set that's being striped.

Quote:

# mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/{sda,sdb}
# mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/{sdc,sdd}
# mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/{sde,sdf}
# mdadm --create /dev/md3 --run --level=0 --raid-devices=3 /dev/{md0,md1,md2}

Which would look like this:

Code:

===========================================
M	M | S | M	M | S | M	M |
===========================================

Opposed to the previous command in my OP which gives 2 mirrored sets regardless of raid devices defined, which would look like this if there was 6 drives:

Code:

===========================================
M	M	M | S | M	M	M |
===========================================

So my question remains. Am I missing an obvious parameter or something to customize the amount of mirrored sets when using --level=10 or do you need to nest arrays to achieve the desired set ranges.

compo · 05-24-2022, 02:06 PM

My answer comes a bit late. May be interesting for others:

I agree. The RAID10-Level of mdadm is not the same as nested RAID10 aka RAID1+0. Mdadm RAID10 is some kind of special RAID10 (called at SUSE also 'Complex RAID 10' (link)), described also at Arch-Wiki (link). If you use the default layout (=n2) and you have an even number of disks, it seems to be almost like nested 1+0. But not the same in every case (e.g. adding one more disk). So if you want to be safe which disks will be used by one mirror (as me), I would use the nested RAID, because you can fix, which disks are used in every single array.

The Suse-link is very helpful for understandig this.

Regards, compo

computersavvy · 05-24-2022, 04:27 PM

Raid level 10 is striped + mirrored.
One needs to have a minimum of 4 drives to create the smallest raid 10 array.
The striped array is created on 2 devices which is then mirrored on 2 additional devices. Expanding that to 6 drives gives 3 striped which is mirrored to 3 drives, etc.

Your post shows 2 sets A & B. Set A is the striped set, and set B is the mirror of set A.