LinuxQuestions.org - mdadm fails to assemble reconnected drives (debian, raid0)

- Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)

- - mdadm fails to assemble reconnected drives (debian, raid0) (https://www.linuxquestions.org/questions/linux-software-2/mdadm-fails-to-assemble-reconnected-drives-debian-raid0-674327/)

mdadm fails to assemble reconnected drives (debian, raid0)

Hello,

I've got some trouble with a raid0 created with mdadm.

At first time, I created a raid0 from an incomplete tutorial resulting a fonctionnal raid0 with no superblock and not created in a ext partition.

After cold-unplugging disks, reboot, cold-replugging disks, reboot, mdadm displays the following message:

System startup:

Code:

mdadm: no devices found for /dev/md1

Manual assemble:

Code:

# mdadm --assemble --force --verbose /dev/md1 /dev/sdb /dev/sdc

mdadm: looking for devices for /dev/md1

mdadm: no recogniseable superblock on /dev/sdb

mdadm: /dev/sdb has no superblock - assembly aborted

Here are some extra infos before crash (note that md0 is ok):

Code:

# mdadm --detail --scan /dev/md1

/dev/md1:

        Version : 00.90

  Creation Time : Mon Mar 10 20:08:51 2008

    Raid Level : raid0

    Array Size : 976772992 (931.52 GiB 1000.22 GB)

  Raid Devices : 2

  Total Devices : 2

Preferred Minor : 1

    Persistence : Superblock is persistent



    Update Time : Sat Aug 30 09:32:57 2008

          State : active

 Active Devices : 2

Working Devices : 2

 Failed Devices : 0

  Spare Devices : 0



    Chunk Size : 64K



          UUID : 2da77535:1e27906e:01dac8d0:789f5d74 (local to host Takhisis)

        Events : 0.7



    Number  Major  Minor  RaidDevice State

      0      8      16        0      active sync  /dev/sdb

      1      8      32        1      active sync  /dev/sdc

# mdadm --detail --scan /dev/md0

/dev/md0:

        Version : 00.90

  Creation Time : Mon Feb 11 12:06:30 2008

    Raid Level : raid0

    Array Size : 976767872 (931.52 GiB 1000.21 GB)

  Raid Devices : 2

  Total Devices : 2

Preferred Minor : 0

    Persistence : Superblock is persistent



    Update Time : Sat Aug 30 09:32:55 2008

          State : active

 Active Devices : 2

Working Devices : 2

 Failed Devices : 0

  Spare Devices : 0



    Chunk Size : 64K



          UUID : 2f910242:c95948d3:56ea1cb5:0bb3cffa

        Events : 0.15



    Number  Major  Minor  RaidDevice State

      0      8      49        0      active sync  /dev/sdd1

      1      8      65        1      active sync  /dev/sde1

root@Takhisis:/mnt/r1/Backups/computer.Asus-A6JA-Q078.SandRock/2008-02-23# mdadm --detail --scan

ARRAY /dev/md0 level=raid0 num-devices=2 metadata=00.90 UUID=2f910242:c95948d3:56ea1cb5:0bb3cffa

ARRAY /dev/md1 level=raid0 num-devices=2 metadata=00.90 UUID=2da77535:1e27906e:01dac8d0:789f5d74



# cat /proc/mdstat

Personalities : [raid0]

md1 : active raid0 sdb[0] sdc[1]

      976772992 blocks 64k chunks



md0 : active raid0 sdd1[0] sde1[1]

      976767872 blocks 64k chunks

Here are some extra infos after crash:

Code:

# fdisk -l

Disk /dev/sdb: 500.1 GB, 500106780160 bytes

255 heads, 63 sectors/track, 60801 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Disk identifier: 0x08040000



Disk /dev/sdb doesn't contain a valid partition table



Disk /dev/sdc: 500.1 GB, 500106780160 bytes

255 heads, 63 sectors/track, 60801 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Disk identifier: 0xeac0fc89



Disk /dev/sdc doesn't contain a valid partition table

I have 800GB data on this array, can someone help me assemble it?

Using
Debian lenny 2.6.26-1-amd64
mdadm v2.6.7
All disks sd[b-e] are SAMSUNG Spinpoint 500GB HD501LJ

Is there any command to check integrity of the drives?

Quote:

At first time, I created a raid0 from an incomplete tutorial resulting a fonctionnal raid0 with no superblock and not created in a ext partition.

Did you intentionally use /dev/sdb and /dev/sdc instead of /dev/sdb1 and /dev/sdc1? Maybe fdisk is telling you there's no partition table because you used the whole device, and there's no partition table!

Might help if you're clear about what you're trying to do.

Quote:

Originally Posted by mostlyharmless (Post 3304413)

I'm just trying to assemble a raid which don't assmble any more. I use sdX instead of sdX1 because I created the array this way:

Code:

mdadm --create /dev/md1 -v --level=0 --raid-devices=2 /dev/sdb /dev/sdc

whole disk vs partition

You would probably be better off giving the disks a partition table with fdisk and then building the array out of the partitions. That way the rest of the system (fdisk included) will know what the disk array is for.

mdadm may not be able to get the devices to assemble because at boot time the system recognized the disks as unformatted, or, worse, something else, and is not making them available, or using them as swap, or something.

If you want to use them as whole disk for some reason, then I'd look at the messages in dmesg to see how the disks are being recognized and whether the system is doing something else with them.

Hope that helps.

Quote:

Originally Posted by mostlyharmless (Post 3306234)

Yeah, a patition table would be great but I have some data (800GB) on this raid! I would like to recover them... A partition table will overwrite data on the disk, will not?

Quote:

Originally Posted by mostlyharmless (Post 3306234)

mdadm may not be able to get the devices to assemble because at boot time the system recognized the disks as unformatted, or, worse, something else, and is not making them available, or using them as swap, or something.

This worked before unpluging/pluging-in... I don't really know how but...

Quote:

Originally Posted by mostlyharmless (Post 3306234)

If you want to use them as whole disk for some reason, then I'd look at the messages in dmesg to see how the disks are being recognized and whether the system is doing something else with them.

Code:

[  11.030798] md: raid0 personality registered for level 0

[  11.035903] md: md0 stopped.

[  11.057620] md: bind<sde1>

[  11.059912] md: bind<sdd1>

[  11.063928] md0: setting max_sectors to 128, segment boundary to 32767

[  11.063928] raid0: looking at sdd1

[  11.063928] raid0:  comparing sdd1(488383936) with sdd1(488383936)

[  11.063928] raid0:  END

[  11.063928] raid0:  ==> UNIQUE

[  11.063928] raid0: 1 zones

[  11.063928] raid0: looking at sde1

[  11.063928] raid0:  comparing sde1(488383936) with sdd1(488383936)

[  11.063928] raid0:  EQUAL

[  11.063928] raid0: FINAL 1 zones

[  11.063928] raid0: done.

[  11.063928] raid0 : md_size is 976767872 blocks.

[  11.063928] raid0 : conf->hash_spacing is 976767872 blocks.

[  11.063928] raid0 : nb_zone is 1.

[  11.063928] raid0 : Allocating 8 bytes for hash.

[  11.063983] md: md1 stopped.

[  11.093563] device-mapper: uevent: version 1.0.3

[  11.097563] device-mapper: ioctl: 4.13.0-ioctl (2007-10-18) initialised: dm-devel@redhat.com

[  11.933139] ieee1394: Host added: ID:BUS[0-00:1023]  GUID[00c4f9d200001a4d]

md0 starts. md1 not (there are no more details in dmesg) but on the screen, this message appears: mdadm: no devices found for /dev/md1.

My goal is to assemble this raid just one more time to backup everything. I've got 5 new 1TB hdd where I want to create a raid5. All data from md0 and md1 will go there. But at this time, only md0 is working.

Quote:

Originally Posted by mostlyharmless (Post 3306234)

Hope that helps.

Thanks, I hope I could recover those data one day :/

Quote:

Yeah, a patition table would be great but I have some data (800GB) on this raid! I would like to recover them... A partition table will overwrite data on the disk, will not?

Well, sorry, didn't know you already had 800 G of data on board; I wouldn't think you'd want to fdisk it all away.

Quote:

This worked before unpluging/pluging-in... I don't really know how but...

If I understand the situation, when you setup up the array, you hadn't booted and the drives were recognized, now you've rebooted and they are not, because you've wiped the partition table with mdadm.

dmesg says nothing about the drives? Not the mdadm part, but earlier, something like (e.g.):

Code:

Probing IDE interface ide3...

hdg: WDC WD1600AAJB-00PVA0, ATA DISK drive

hdg: host max PIO4 wanted PIO255(auto-tune) selected PIO4

hdg: UDMA/100 mode selected

ide3 at 0xa400-0xa407,0xa802 on irq 16

hda: max request size: 512KiB

hda: 625142448 sectors (320072 MB) w/16384KiB Cache, CHS=38913/255/63

And is there anything in /dev/disk to suggest how the system sees the disks?

Does fdisk /dev/sdb find the device? You can use it without writing.

Quote:

Originally Posted by mostlyharmless (Post 3307799)

Well, sorry, didn't know you already had 800 G of data on board; I wouldn't think you'd want to fdisk it all away.

If I understand the situation, when you setup up the array, you hadn't booted and the drives were recognized, now you've rebooted and they are not, because you've wiped the partition table with mdadm.

Not really
- for ~10 months, all was okay, md0 (sdd & sde) and md1 (sdb & sdc) were mounting at boot-time
- ~1 month ago, I needed to temporarily connect some hdd, so I unplugged sdb and sdc (hdd of md1) and plugged some other hdd. After using the others hdd, I replugged sdb and sdc. Since this moment, md1 does not assemble anymore.

Quote:

Originally Posted by mostlyharmless (Post 3307799)

dmesg says nothing about the drives? Not the mdadm part, but earlier, something like (e.g.):

Code:

[    3.702092] ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)

[    3.702083] ata1.00: HPA detected: current 488395055, native 488397168

[    3.702083] ata1.00: ATA-7: ST3250620NS, 3.AEK, max UDMA/133

[    3.702083] ata1.00: 488395055 sectors, multi 0: LBA48 NCQ (depth 31/32)

[    3.804420] ata1.00: configured for UDMA/133

[    4.288156] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300)

[    4.308592] ata2.00: HPA detected: current 976771055, native 976773168

[    4.308592] ata2.00: ATA-8: SAMSUNG HD501LJ, CR100-10, max UDMA7

[    4.308592] ata2.00: 976771055 sectors, multi 0: LBA48 NCQ (depth 31/32)

[    4.312134] ata2.00: configured for UDMA/133

[    5.023704] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)

[    5.028150] ata3.00: HPA detected: current 976771055, native 976773168

[    5.028150] ata3.00: ATA-8: SAMSUNG HD501LJ, CR100-10, max UDMA7

[    5.028150] ata3.00: 976771055 sectors, multi 0: LBA48 NCQ (depth 31/32)

[    5.034564] ata3.00: configured for UDMA/133

[    5.644151] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)

[    5.657951] ata4.00: ATA-8: SAMSUNG HD501LJ, CR100-12, max UDMA7

[    5.657951] ata4.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)

[    5.960140] ata4.00: configured for UDMA/133

[    6.439946] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)

[    6.440406] ata5.00: ATA-8: SAMSUNG HD501LJ, CR100-11, max UDMA7

[    6.440406] ata5.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)

[    6.445790] ata5.00: configured for UDMA/133

[    6.900041] ata6: SATA link down (SStatus 0 SControl 300)

[    6.900121] scsi 0:0:0:0: Direct-Access    ATA      ST3250620NS      3.AE PQ: 0 ANSI: 5

[    6.900245] scsi 1:0:0:0: Direct-Access    ATA      SAMSUNG HD501LJ  CR10 PQ: 0 ANSI: 5

[    6.900331] scsi 2:0:0:0: Direct-Access    ATA      SAMSUNG HD501LJ  CR10 PQ: 0 ANSI: 5

[    6.900402] scsi 3:0:0:0: Direct-Access    ATA      SAMSUNG HD501LJ  CR10 PQ: 0 ANSI: 5

[    6.900474] scsi 4:0:0:0: Direct-Access    ATA      SAMSUNG HD501LJ  CR10 PQ: 0 ANSI: 5

...

[  10.633691] Driver 'sd' needs updating - please use bus_type methods

[  10.633691] sd 0:0:0:0: [sda] 488395055 512-byte hardware sectors (250058 MB)

[  10.633691] sd 0:0:0:0: [sda] Write Protect is off

[  10.633691] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00

[  10.633691] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

[  10.633691] sd 0:0:0:0: [sda] 488395055 512-byte hardware sectors (250058 MB)

[  10.633691] sd 0:0:0:0: [sda] Write Protect is off

[  10.633691] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00

[  10.633691] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

[  10.633691]  sda: sda1 sda2 sda3 sda4

[  10.649699] sd 0:0:0:0: [sda] Attached SCSI disk

[  10.649699] sd 1:0:0:0: [sdb] 976771055 512-byte hardware sectors (500107 MB)

[  10.649699] sd 1:0:0:0: [sdb] Write Protect is off

[  10.649699] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00

[  10.649699] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

[  10.649699] sd 1:0:0:0: [sdb] 976771055 512-byte hardware sectors (500107 MB)

[  10.649699] sd 1:0:0:0: [sdb] Write Protect is off

[  10.649699] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00

[  10.649699] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

[  10.649699]  sdb: unknown partition table

[  10.716114] sd 1:0:0:0: [sdb] Attached SCSI disk

[  10.716114] sd 2:0:0:0: [sdc] 976771055 512-byte hardware sectors (500107 MB)

[  10.716114] sd 2:0:0:0: [sdc] Write Protect is off

[  10.716114] sd 2:0:0:0: [sdc] Mode Sense: 00 3a 00 00

[  10.716114] sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

[  10.716114] sd 2:0:0:0: [sdc] 976771055 512-byte hardware sectors (500107 MB)

[  10.716114] sd 2:0:0:0: [sdc] Write Protect is off

[  10.716114] sd 2:0:0:0: [sdc] Mode Sense: 00 3a 00 00

[  10.716114] sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

[  10.716114]  sdc: unknown partition table

[  10.736108] sd 2:0:0:0: [sdc] Attached SCSI disk

[  10.736108] sd 3:0:0:0: [sdd] 976773168 512-byte hardware sectors (500108 MB)

[  10.736108] sd 3:0:0:0: [sdd] Write Protect is off

[  10.736108] sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00

[  10.736108] sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

[  10.736108] sd 3:0:0:0: [sdd] 976773168 512-byte hardware sectors (500108 MB)

[  10.736108] sd 3:0:0:0: [sdd] Write Protect is off

[  10.736108] sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00

[  10.736108] sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

[  10.736108]  sdd: sdd1

[  10.763286] sd 3:0:0:0: [sdd] Attached SCSI disk

[  10.763336] sd 4:0:0:0: [sde] 976773168 512-byte hardware sectors (500108 MB)

[  10.763349] sd 4:0:0:0: [sde] Write Protect is off

[  10.763351] sd 4:0:0:0: [sde] Mode Sense: 00 3a 00 00

[  10.763372] sd 4:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

[  10.763412] sd 4:0:0:0: [sde] 976773168 512-byte hardware sectors (500108 MB)

[  10.763424] sd 4:0:0:0: [sde] Write Protect is off

[  10.763426] sd 4:0:0:0: [sde] Mode Sense: 00 3a 00 00

[  10.763447] sd 4:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

[  10.763450]  sde: sde1

[  10.810088] sd 4:0:0:0: [sde] Attached SCSI disk

[  11.030798] md: raid0 personality registered for level 0

[  11.035903] md: md0 stopped.

[  11.057620] md: bind<sde1>

[  11.059912] md: bind<sdd1>

[  11.063928] md0: setting max_sectors to 128, segment boundary to 32767

[  11.063928] raid0: looking at sdd1

[  11.063928] raid0:  comparing sdd1(488383936) with sdd1(488383936)

[  11.063928] raid0:  END

[  11.063928] raid0:  ==> UNIQUE

[  11.063928] raid0: 1 zones

[  11.063928] raid0: looking at sde1

[  11.063928] raid0:  comparing sde1(488383936) with sdd1(488383936)

[  11.063928] raid0:  EQUAL

[  11.063928] raid0: FINAL 1 zones

[  11.063928] raid0: done.

[  11.063928] raid0 : md_size is 976767872 blocks.

[  11.063928] raid0 : conf->hash_spacing is 976767872 blocks.

[  11.063928] raid0 : nb_zone is 1.

[  11.063928] raid0 : Allocating 8 bytes for hash.

[  11.063983] md: md1 stopped.

Here is everything related to disks from dmesg. Everything seems ok.

Quote:

Originally Posted by mostlyharmless (Post 3307799)

And is there anything in /dev/disk to suggest how the system sees the disks?

Hmm, the disks seems ok (ls -l /dev/disk/by-id/):

Code:

lrwxrwxrwx 1 root root  9 2008-10-08 21:10 scsi-SATA_ST3250620NS_5QE4VRPE -> ../../sda

lrwxrwxrwx 1 root root 10 2008-10-08 21:10 scsi-SATA_ST3250620NS_5QE4VRPE-part1 -> ../../sda1

lrwxrwxrwx 1 root root 10 2008-10-08 21:10 scsi-SATA_ST3250620NS_5QE4VRPE-part2 -> ../../sda2

lrwxrwxrwx 1 root root 10 2008-10-08 21:10 scsi-SATA_ST3250620NS_5QE4VRPE-part3 -> ../../sda3

lrwxrwxrwx 1 root root 10 2008-10-08 21:10 scsi-SATA_ST3250620NS_5QE4VRPE-part4 -> ../../sda4

lrwxrwxrwx 1 root root  9 2008-10-08 21:10 scsi-SATA_SAMSUNG_HD501LJS0MUJ1KP803815 -> ../../sdb

lrwxrwxrwx 1 root root  9 2008-10-08 21:10 scsi-SATA_SAMSUNG_HD501LJS0MUJ1KP803816 -> ../../sdc

lrwxrwxrwx 1 root root  9 2008-10-08 21:10 scsi-SATA_SAMSUNG_HD501LJS0MUJ1FPB84229 -> ../../sdd

lrwxrwxrwx 1 root root 10 2008-10-08 21:10 scsi-SATA_SAMSUNG_HD501LJS0MUJ1FPB84229-part1 -> ../../sdd1

lrwxrwxrwx 1 root root  9 2008-10-08 21:10 scsi-SATA_SAMSUNG_HD501LJS0MUJ1EP926433 -> ../../sde

lrwxrwxrwx 1 root root 10 2008-10-08 21:10 scsi-SATA_SAMSUNG_HD501LJS0MUJ1EP926433-part1 -> ../../sde1

Quote:

Originally Posted by mostlyharmless (Post 3307799)

Does fdisk /dev/sdb find the device? You can use it without writing.

fdisk found it. What info could I get with that? I already posted a fdisk -l

PS: I don't think it changes anything but when assembled, both array md0 & md1 are LUKS partitions... (sda[2-4] too)

Quote:

- for ~10 months, all was okay, md0 (sdd & sde) and md1 (sdb & sdc) were mounting at boot-time
- ~1 month ago, I needed to temporarily connect some hdd, so I unplugged sdb and sdc (hdd of md1) and plugged some other hdd. After using the others hdd, I replugged sdb and sdc. Since this moment, md1 does not assemble anymore.

Well, ok, that's extra information. Here's a couple of probably dumb questions.

(1) Is it at all possible that the two drives got switched when replugged?
(2) Did you change anything else in the configuration when you had the other drives plugged in? I presume they must have been recognized as sdb and sdc and not mounted by mdadm and that you didn't change anything else.

Quote:

Originally Posted by mostlyharmless (Post 3310007)

(1) Is it at all possible that the two drives got switched when replugged?
2) Did you change anything else in the configuration when you had the other drives plugged in? I presume they must have been recognized as sdb and sdc and not mounted by mdadm and that you didn't change anything else.

Just checked everything, no change...

build or assemble?

Well, I'm running out of ideas, but here's another one: is it possible that when you had the other harddrives plugged in that mdadm.conf got modified? Are these still the last lines?

Quote:

ARRAY /dev/md0 level=raid0 num-devices=2 metadata=00.90 UUID=2f910242:c95948d3:56ea1cb5:0bb3cffa
ARRAY /dev/md1 level=raid0 num-devices=2 metadata=00.90 UUID=2da77535:1e27906e:01dac8d0:789f5d74

The other thought I had that you could try, in case mdadm is confused about md1 for some inexplicable reason, is to try to build sdb and sdc into a new array, call it md2, like this:

mdadm --build /dev/md2 --chunk=64 --level=0 --raid-devices=2 /dev/sdb /dev/sdc

I suggested "build" instead of "assemble" because I noticed something odd. You mentioned

Quote:

I created a raid0 from an incomplete tutorial resulting a fonctionnal raid0 with no superblock

Yet your first post also says

Quote:

/dev/md1:
Version : 00.90
Creation Time : Mon Mar 10 20:08:51 2008
Raid Level : raid0
Array Size : 976772992 (931.52 GiB 1000.22 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1
Persistence : Superblock is persistent

Let me know if it works...

Quote:

Originally Posted by mostlyharmless (Post 3323147)

I suggested "build" instead of "assemble" because I noticed something odd. You mentioned

Yet your first post also says

Let me know if it works...

OMG THIS WORKS!
I love you man :D
Really thousand of thanks :D

Code:

# mdadm --build /dev/md3 --chunk=64 --level=0 --raid-devices=2 /dev/sdf /dev/sde

mdadm: array /dev/md3 built and started.

# cryptsetup luksOpen /dev/md3 md3_crypt

Enter LUKS passphrase:

key slot 0 unlocked.

Command successful.

# mount -t ext3 /dev/mapper/md3_crypt /mnt/r1

And it's not a dream, it's true! :O
Great :D