Recover mdadm RAID5

gorankav · 02-04-2010, 11:26 PM

Tried to look for similar cases but can't find anything similar.

I had a system with 4 disks:
OS debian:
- /dev/sda

RAID:
- /dev/sdb
- /dev/sdc
- /dev/sdd

The old system disk crashed, so I installed a new one and retried to resync the array.
It took two hours to resync the RAID, but I'm unable to mount it.

After reboot I get this:

Code:

hoster:~# mdadm --assemble  --scan -vv

mdadm: looking for devices for further assembly
mdadm: no RAID superblock on /dev/sdd
mdadm: no RAID superblock on /dev/sdc1
mdadm: no RAID superblock on /dev/sdc
mdadm: no RAID superblock on /dev/sdb
mdadm: cannot open device /dev/root: Device or resource busy
mdadm: no RAID superblock on /dev/sda12
mdadm: no RAID superblock on /dev/sda11
mdadm: no RAID superblock on /dev/sda10
mdadm: no RAID superblock on /dev/sda9
mdadm: no RAID superblock on /dev/sda8
mdadm: no RAID superblock on /dev/sda7
mdadm: no RAID superblock on /dev/sda6
mdadm: cannot open device /dev/sda5: Device or resource busy
mdadm: no RAID superblock on /dev/sda3
mdadm: no RAID superblock on /dev/sda2
mdadm: no RAID superblock on /dev/sda1
mdadm: cannot open device /dev/sda: Device or resource busy
mdadm: /dev/sdd1 is identified as a member of /dev/md/0_0, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md/0_0
mdadm: no uptodate device for slot 2 of /dev/md/0_0
mdadm: added /dev/sdd1 to /dev/md/0_0 as 0
mdadm: /dev/md/0_0 assembled from 1 drive - not enough to start the array.
mdadm: looking for devices for further assembly
mdadm: no RAID superblock on /dev/sdc1
mdadm: no RAID superblock on /dev/sda12
mdadm: no RAID superblock on /dev/sda11
mdadm: no RAID superblock on /dev/sda10
mdadm: no RAID superblock on /dev/sda9
mdadm: no RAID superblock on /dev/sda8
mdadm: no RAID superblock on /dev/sda7
mdadm: no RAID superblock on /dev/sda6
mdadm: no RAID superblock on /dev/sda3
mdadm: no RAID superblock on /dev/sda2
mdadm: no RAID superblock on /dev/sda1
mdadm: /dev/sdd is identified as a member of /dev/md/imsm0, slot -1.
mdadm: /dev/sdc is identified as a member of /dev/md/imsm0, slot -1.
mdadm: /dev/sdb is identified as a member of /dev/md/imsm0, slot -1.
mdadm: added /dev/sdc to /dev/md/imsm0 as -1
mdadm: added /dev/sdb to /dev/md/imsm0 as -1
mdadm: added /dev/sdd to /dev/md/imsm0 as -1
mdadm: Container /dev/md/imsm0 has been assembled with 3 drives
mdadm: looking for devices for further assembly
mdadm: cannot open device /dev/sdc1: No such file or directory
mdadm: no recogniseable superblock on /dev/sda12
mdadm: no recogniseable superblock on /dev/sda11
mdadm: no recogniseable superblock on /dev/sda10
mdadm: no recogniseable superblock on /dev/sda9
mdadm: no recogniseable superblock on /dev/sda8
mdadm: no recogniseable superblock on /dev/sda7
mdadm: no recogniseable superblock on /dev/sda6
mdadm: no recogniseable superblock on /dev/sda3
mdadm: no recogniseable superblock on /dev/sda2
mdadm: no recogniseable superblock on /dev/sda1
mdadm: looking for devices for further assembly
mdadm: cannot open device /dev/sdd: Device or resource busy
mdadm: cannot open device /dev/sdc: Device or resource busy
mdadm: cannot open device /dev/sdb: Device or resource busy
mdadm: cannot open device /dev/root: Device or resource busy
mdadm: no recogniseable superblock on /dev/sda12
mdadm: no recogniseable superblock on /dev/sda11
mdadm: no recogniseable superblock on /dev/sda10
mdadm: no recogniseable superblock on /dev/sda9
mdadm: no recogniseable superblock on /dev/sda8
mdadm: no recogniseable superblock on /dev/sda7
mdadm: no recogniseable superblock on /dev/sda6
mdadm: cannot open device /dev/sda5: Device or resource busy
mdadm: no recogniseable superblock on /dev/sda3
mdadm: no recogniseable superblock on /dev/sda2
mdadm: no recogniseable superblock on /dev/sda1
mdadm: cannot open device /dev/sda: Device or resource busy
mdadm: looking in container /dev/md127
mdadm: found match on member /md127/0 in /dev/md127
mdadm: Started /dev/md/raid5_0 with 3 devices
mdadm: looking for devices for further assembly
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on 259:0
mdadm: no recogniseable superblock on /dev/md/raid5_0
mdadm: cannot open device /dev/sdd: Device or resource busy
mdadm: cannot open device /dev/sdc: Device or resource busy
mdadm: cannot open device /dev/sdb: Device or resource busy
mdadm: cannot open device /dev/root: Device or resource busy
mdadm: no recogniseable superblock on /dev/sda12
mdadm: no recogniseable superblock on /dev/sda11
mdadm: no recogniseable superblock on /dev/sda10
mdadm: no recogniseable superblock on /dev/sda9
mdadm: no recogniseable superblock on /dev/sda8
mdadm: no recogniseable superblock on /dev/sda7
mdadm: no recogniseable superblock on /dev/sda6
mdadm: cannot open device /dev/sda5: Device or resource busy
mdadm: no recogniseable superblock on /dev/sda3
mdadm: no recogniseable superblock on /dev/sda2
mdadm: no recogniseable superblock on /dev/sda1
mdadm: cannot open device /dev/sda: Device or resource busy
mdadm: looking in container /dev/md127
mdadm: member /md127/0 in /dev/md127 is already assembled

After this is done the md status is like:

Code:

hoster:~# cat /proc/mdstat

Personalities : [raid6] [raid5] [raid4]
md126 : active (auto-read-only) raid5 sdd[2] sdc[1] sdb[0]
      976768000 blocks super external:/md127/0 level 5, 64k chunk, algorithm 0 [3/3] [UUU]

md127 : inactive sdd[2](S) sdb[1](S) sdc[0](S)
      6771 blocks super external:imsm

unused devices: <none>

Any help is welcome.

gorankav · 02-04-2010, 11:38 PM

If it helps I add this:

There are 3x 500GiB disks (supposingly) in the RAID.

Code:

hoster:~# mdadm -E /dev/md/imsm0

/dev/md/imsm0:
          Magic : Intel Raid ISM Cfg Sig.
        Version : 1.2.02
    Orig Family : cc99b183
         Family : cc99b183
     Generation : 00000de2
           UUID : e1b6e899:59ffeefb:9d4cd6b1:a5fcc3bd
       Checksum : 28d495b8 correct
    MPB Sectors : 2
          Disks : 3
   RAID Devices : 1

  Disk02 Serial : WD-WCAS85849430
          State : active
             Id : 00020000
    Usable Size : 976768654 (465.76 GiB 500.11 GB)

[raid5]:
           UUID : 66772f25:7de1759c:f409d4ec:08887f08
     RAID Level : 5
        Members : 3
      This Slot : 2
     Array Size : 1953536000 (931.52 GiB 1000.21 GB)
   Per Dev Size : 976768264 (465.76 GiB 500.11 GB)
  Sector Offset : 0
    Num Stripes : 7631000
     Chunk Size : 64 KiB
       Reserved : 0
  Migrate State : idle
      Map State : normal
    Dirty State : clean

  Disk00 Serial : WD-WCAS85810025
          State : active
             Id : 00040000
    Usable Size : 976768654 (465.76 GiB 500.11 GB)

  Disk01 Serial : WD-WCAS85209004
          State : active
             Id : 00030000
    Usable Size : 976768654 (465.76 GiB 500.11 GB)

/dev/md/raid5_0 seemd not to be even close to a RAID:

Code:

hoster:~# mdadm -E /dev/md/raid5_0

mdadm: No md superblock detected on /dev/md/raid5_0.

The content of /etc/mdadm/mdadm.conf (currently renamed) is:

Code:

DEVICE partitions
CREATE owner=root group=disk mode=0660 auto=yes
HOMEHOST <system>
MAILADDR goran@indea.si

# definitions of existing MD arrays
ARRAY metadata=imsm UUID=e1b6e899:59ffeefb:9d4cd6b1:a5fcc3bd
ARRAY /dev/md/raid5_0 level=raid5 num-devices=3 container=e1b6e899:59ffeefb:9d4cd6b1:a5fcc3bd member=0 UUID=66772f25:7de1759c:f409d4ec:08887f08

gorankav · 02-04-2010, 11:45 PM

Some more info...

Code:

hoster:~# mdadm -D --scan -vvv

/dev/md/imsm0:
        Version : imsm
     Raid Level : container
  Total Devices : 3

Working Devices : 3


           UUID : e1b6e899:59ffeefb:9d4cd6b1:a5fcc3bd
  Member Arrays :

    Number   Major   Minor   RaidDevice

       0       8       32        -        /dev/sdc
       1       8       16        -        /dev/sdb
       2       8       48        -        /dev/sdd
/dev/md/raid5_0:
      Container : /dev/md/imsm0, member 0
     Raid Level : raid5
     Array Size : 976768000 (931.52 GiB 1000.21 GB)
  Used Dev Size : 488384128 (465.76 GiB 500.11 GB)
   Raid Devices : 3
  Total Devices : 3

          State : clean
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-asymmetric
     Chunk Size : 64K


           UUID : 66772f25:7de1759c:f409d4ec:08887f08
    Number   Major   Minor   RaidDevice State
       2       8       48        0      active sync   /dev/sdd
       1       8       32        1      active sync   /dev/sdc
       0       8       16        2      active sync   /dev/sdb