mdadm problem

D-a-a-n · 03-30-2009, 03:49 AM

Hi,

I have 8 hard disks in a raid (sd[abcdefgh]).
When booting my openfiler system I get this error:
mdadm: /dev/md1 assembled from 5 drives - not enough to start the array.
Error: /bin/mdadm exited abnormally with value 1!

When I do an mdadm --examine /dev/sda3

I see that sdb3, sdc3, sdd3 are "faulty removed".
What does this mean, and how can I resolve this?

Greetz

eco · 03-30-2009, 04:45 AM

It looks like your drives are considered faulty. The following is a software RAID test I'd done to see if the RAID worked. You might use -a to try and add a disk back into the array.

I hope you have a good backup. You might want to rad up on the error you get before doing any of this.

Simulate a failure to the disk... expect an email:

Code:

# mdadm --manage --set-faulty /dev/md3 /dev/sda3

Now check the status of the RAID

Code:

# mdadm --detail /dev/md3

Remove the failed partition

Code:

# mdadm /dev/md3 -r /dev/sda3

Add it back into the raid

Code:

# mdadm /dev/md3 -a /dev/sda3

Monitor the RAID's health

Code:

# mdadm --detail /dev/md3

Hope this helps.

D-a-a-n · 03-30-2009, 05:11 AM

Thx for your reply,
I booted from a live cd.
When I do a: mdadm --manage --set-faulty /dev/md3 /dev/sda3
I get this: mdadm: error opening /dev/md3: No such file or directory.
That is because the array could not be assembled, so there isn't any /dev/md*

This is what I already tried:
mdadm -A /dev/md0 /dev/sd[abcdefgh]1 >> worked, I can mount /dev/md0 and can see the contents of the first partition

mdadm -A /dev/md0 /dev/sd[abcdefgh]3 >> doesn't work, I get this: mdadm : /dev/md0 assembled from 5 drives - not enough to start the array.

Greetz

eco · 03-30-2009, 06:30 AM

Hi again,

I think you need to let us know exactly what your RAID setup is.

Do you have 3 RAIDs (md0, md1, md2)?

Quote:

When I do a: mdadm --manage --set-faulty /dev/md3 /dev/sda3
I get this: mdadm: error opening /dev/md3: No such file or directory.
That is because the array could not be assembled, so there isn't any /dev/md*

Nop. That's because you did a cut and paste from my example without changing the devices

Quote:

mdadm -A /dev/md0 /dev/sd[abcdefgh]1 >> worked, I can mount /dev/md0 and can see the contents of the first partition

mdadm -A /dev/md0 /dev/sd[abcdefgh]3 >> doesn't work, I get this: mdadm : /dev/md0 assembled from 5 drives - not enough to start the array.

Here you seem to be mounting sdX1 to md0. Then you go and do the same by mounting sdX3 to md0. Should it not be md3?

Please send us the exact config you have and I'll do my best to try and help.

Do you have a backup of your data?

D-a-a-n · 03-31-2009, 04:32 AM

Whoa! Problem solved.
I had a raid6 (md1 = /dev/sd[abcdefgh]3 ) for my root directory. 3 of these partitions were set as faulty removed.
That's one too much, so I erased the superblock from 1 hard drive, and reassembled the array afterwards and it worked!

mdadm --zero-superblock /dev/sdb3
mdadm --create /dev/md0 --verbose --level=6 --raid-devices=8 /dev/sd[abcdefgh]3

I did this because I couldn't believe the drive was broken and so the raid info wasn't correct.

Greetz Daan