Recover mdadm Raid5

felix2k · 05-10-2011, 04:25 AM

Hi everyone,

i know this probably has been there before, but as you know every raid configuration differs a bit from the others so I hope you could help me recover mine.

I use Debian Squeeze and have a Raid5 with 6 hard drives, which I created with mdadm. Now comes the stupid part. I wanted to boot the array on another pc but failed doing so. I then succeeded starting it with mdadm on another computer but it wanted to only assemble 5 of 6 devices. i confirmed and readded the 6th disk, but then while rebuilding it messed up and I had to shutdown the system.

Now it says:

Code:

mdadm: /dev/md0 assembled from 1 drive and 1 spare - not enough to start the array.

I had no spare on my original raid, they were all active. The superblocks seem to exist, which gives me hope to recover it. But I didn't want to do anything of which I don't know what will happen, so I ask you what to do.

The superblocks look like that:

Code:

mdadm -E /dev/sdc
/dev/sdc:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : b138271a:da74d36b:1c25a485:e093d2ae (local to host fitpc)
  Creation Time : Thu Oct 28 16:35:38 2010
     Raid Level : raid5
  Used Dev Size : 1953512384 (1863.01 GiB 2000.40 GB)
     Array Size : 9767561920 (9315.07 GiB 10001.98 GB)
   Raid Devices : 6
  Total Devices : 6
Preferred Minor : 0

    Update Time : Tue May 10 07:58:53 2011
          State : clean
 Active Devices : 1
Working Devices : 2
 Failed Devices : 4
  Spare Devices : 1
       Checksum : 4112532d - correct
         Events : 4094528

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     6       8      112        6      spare   /dev/sdh

   0     0       0        0        0      removed
   1     1       8      128        1      active sync
   2     2       0        0        2      faulty removed
   3     3       0        0        3      faulty removed
   4     4       0        0        4      faulty removed
   5     5       0        0        5      faulty removed
   6     6       8      112        6      spare   /dev/sdh

Quote:

mdadm -E /dev/sdd
/dev/sdd:
Magic : a92b4efc
Version : 0.90.00
UUID : b138271a:da74d36b:1c25a485:e093d2ae (local to host fitpc)
Creation Time : Thu Oct 28 16:35:38 2010
Raid Level : raid5
Used Dev Size : 1953512384 (1863.01 GiB 2000.40 GB)
Array Size : 9767561920 (9315.07 GiB 10001.98 GB)
Raid Devices : 6
Total Devices : 6
Preferred Minor : 0

Update Time : Tue May 10 07:44:40 2011
State : clean
Active Devices : 5
Working Devices : 6
Failed Devices : 0
Spare Devices : 1
Checksum : 41124f5c - correct
Events : 4094515

Layout : left-symmetric
Chunk Size : 64K

Number Major Minor RaidDevice State
this 2 8 96 2 active sync /dev/sdg

0 0 0 0 0 removed
1 1 8 128 1 active sync
2 2 8 96 2 active sync /dev/sdg
3 3 8 64 3 active sync /dev/sde
4 4 8 80 4 active sync /dev/sdf
5 5 8 48 5 active sync /dev/sdd
6 6 8 112 6 spare /dev/sdh

Code:

mdadm -E /dev/sde
/dev/sde:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : b138271a:da74d36b:1c25a485:e093d2ae (local to host fitpc)
  Creation Time : Thu Oct 28 16:35:38 2010
     Raid Level : raid5
  Used Dev Size : 1953512384 (1863.01 GiB 2000.40 GB)
     Array Size : 9767561920 (9315.07 GiB 10001.98 GB)
   Raid Devices : 6
  Total Devices : 6
Preferred Minor : 0

    Update Time : Tue May 10 07:44:40 2011
          State : clean
 Active Devices : 5
Working Devices : 6
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 41124f32 - correct
         Events : 4094515

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     5       8       48        5      active sync   /dev/sdd

   0     0       0        0        0      removed
   1     1       8      128        1      active sync
   2     2       8       96        2      active sync   /dev/sdg
   3     3       8       64        3      active sync   /dev/sde
   4     4       8       80        4      active sync   /dev/sdf
   5     5       8       48        5      active sync   /dev/sdd
   6     6       8      112        6      spare   /dev/sdh

Code:

mdadm -E /dev/sdf
/dev/sdf:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : b138271a:da74d36b:1c25a485:e093d2ae (local to host fitpc)
  Creation Time : Thu Oct 28 16:35:38 2010
     Raid Level : raid5
  Used Dev Size : 1953512384 (1863.01 GiB 2000.40 GB)
     Array Size : 9767561920 (9315.07 GiB 10001.98 GB)
   Raid Devices : 6
  Total Devices : 6
Preferred Minor : 0

    Update Time : Tue May 10 07:58:53 2011
          State : clean
 Active Devices : 1
Working Devices : 2
 Failed Devices : 4
  Spare Devices : 1
       Checksum : 41125339 - correct
         Events : 4094528

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     1       8      128        1      active sync

   0     0       0        0        0      removed
   1     1       8      128        1      active sync
   2     2       0        0        2      faulty removed
   3     3       0        0        3      faulty removed
   4     4       0        0        4      faulty removed
   5     5       0        0        5      faulty removed
   6     6       8      112        6      spare   /dev/sdh

Code:

mdadm -E /dev/sdg
/dev/sdg:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : b138271a:da74d36b:1c25a485:e093d2ae (local to host fitpc)
  Creation Time : Thu Oct 28 16:35:38 2010
     Raid Level : raid5
  Used Dev Size : 1953512384 (1863.01 GiB 2000.40 GB)
     Array Size : 9767561920 (9315.07 GiB 10001.98 GB)
   Raid Devices : 6
  Total Devices : 6
Preferred Minor : 0

    Update Time : Tue May 10 07:44:40 2011
          State : clean
 Active Devices : 5
Working Devices : 6
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 41124f50 - correct
         Events : 4094515

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     4       8       80        4      active sync   /dev/sdf

   0     0       0        0        0      removed
   1     1       8      128        1      active sync
   2     2       8       96        2      active sync   /dev/sdg
   3     3       8       64        3      active sync   /dev/sde
   4     4       8       80        4      active sync   /dev/sdf
   5     5       8       48        5      active sync   /dev/sdd
   6     6       8      112        6      spare   /dev/sdh

Code:

mdadm -E /dev/sdh
/dev/sdh:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : b138271a:da74d36b:1c25a485:e093d2ae (local to host fitpc)
  Creation Time : Thu Oct 28 16:35:38 2010
     Raid Level : raid5
  Used Dev Size : 1953512384 (1863.01 GiB 2000.40 GB)
     Array Size : 9767561920 (9315.07 GiB 10001.98 GB)
   Raid Devices : 6
  Total Devices : 6
Preferred Minor : 0

    Update Time : Tue May 10 07:44:40 2011
          State : clean
 Active Devices : 5
Working Devices : 6
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 41124f3e - correct
         Events : 4094515

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     3       8       64        3      active sync   /dev/sde

   0     0       0        0        0      removed
   1     1       8      128        1      active sync
   2     2       8       96        2      active sync   /dev/sdg
   3     3       8       64        3      active sync   /dev/sde
   4     4       8       80        4      active sync   /dev/sdf
   5     5       8       48        5      active sync   /dev/sdd
   6     6       8      112        6      spare   /dev/sdh

As I said one has been readded after assembling with 5 disks so the data should be gone from one (I know that was stupid). The others just seem to have trouble finding the other ones they knew some time ago.

I'd be very glad if someone could help me

Regards

peter1234 · 05-12-2011, 07:02 AM

Hi,

Before I start, I am NOT an expert on raid, so don't blindly do what I did.

Since no one replied to your post for 2 days, I'll see if I can at least point you in write direction.

I have a 2 disk (mdadm) raid1 array (md0), one day for some reason one of the disk in the array was not detected (by bios).
My pc booted with the other disk in the array. I did,

Code:

dmesg |grep -r md
and got some error about one disk missing in md0

I restarted the pc, this time both disks were detected (by bios), did a

Code:

dmesg |grep -r md
and to the same error as above.

Thought of just adding the disk to the array using mdadm (where is fun in that)

,
so I booted using my install cd (zenwalk 6.4) exist the install went to cli,
had a look to to see if both hdd are there (they were) I fdisk the “failed” hdd (sda) repartitioned identical to (sdb) and used the following to add “sda” to “md0”

Code:

mdadm -a /dev/md0 /dev/sda1

and after that it did syncing for about an hour or so, did

Code:

dmesg |grep -r md
and saw “raid1: raid set md0 active with 2 out of 2 mirrors”

Just to make sure “sdb” was copied to “sda” I forced mdadm to remove “sdb” so when I rebooted it would bood from “sda” (and I added “sdb” back to “md0”).

Anyway In your case see adding the drive that did not join the array is an option,
Again, I am NOT an expert on raid, so don't blindly do what I did! Especially if you have important
data in the array.

Here are some threads in LQ regarding raid.
http://www.linuxquestions.org/questi...-mdadm-551732/
http://www.linuxquestions.org/questi...semble-783687/
http://www.linuxquestions.org/questi...ess-up-584149/

hope this helps.....

.