Raid1 ext3 partitions lost with every reboot

xushi · 03-08-2008, 03:12 AM

Hi,

I'm moving all my 5 sets of RAID1 mirrored hard drives from Gentoo to Debian, however, everytime I add a new set, the old ones lose their partitions for some reason.. And the new ones appear as if they don't have partitions on them.

Currently i have md0, md1, md2, and was about to add md3. md0 and md2 are always fine. md1 always gives me a problem when i add new hard drives, and the new to come md4 too.

As an example, I add one set (md1), sdc and sdd. When booting, my machine sees the partition on sda (ext3 as FD) as sdd1, but fails to see the partition on sdc. Not a big problem, I just recreate the partition as FD for sdc, now having sdc1, then i use mdadm to mirror the working sdd1 raid1 disk into sdc1.

Code:

mdadm /dev/md1 -a /dev/sdc1

wait a few hours, and all is well. Reboot, it's still there.. Just to check, in /dev/ i see /dev/sda, /dev/sda1, /dev/sdb, /dev/sdb1 /dev/sdc /dev/sdc1 /dev/sdd /dev/sdd1.

Good!

The problem is, I add another RAID set (sdg and sdh) that were partitioned and working on the other machine, but now after booting, it no longer sees sdc's partition sdc1, and same with the new sdg and sdh.. It won't see sdg1 or sdh1.

when searching in /dev/, all i see is /dev/sda, /dev/sda1, /dev/sdb, /dev/sdb1 /dev/sdc, /dev/sdd /dev/sdd1 /dev/sde /dev/sde1 /dev/sdf /dev/sdf1 /dev/sdg /dev/sdh . (missing /dev/sdc1 /dev/sdg1 /dev/sdh1)

If i try to mount /dev/sdc /mnt/tmp, i can see the data there.. But i'm not sure if this is normal on Debian or not.. If i should just create the RAID sets with /dev/sdc /dev/sdd instead of /dev/sdc1 /dev/sdd1 . I'm confused to why the partitions keep getting lost..

This is tedious, as i still have another 7 sets to re-plug in..

Here is an output of dmesg. HDA is the IDE Debian OS. The current 3 raid sets are SATA, and supposed to be md0 (sda1 sdb1), md1 (sdc1 sdd1), and md2 (sde1 sdf1). The new RAID set i just installed was supposed to be md3 (sdg1 sdh1).

NOTE: Contrary to the dmesg output saying it detected /dev/sdc1, I see no evidence of that..

Code:

sempron:/usr/src# fdisk -l /dev/sdc

Disk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1       60801   488384001   fd  Linux raid autodetect
sempron:/usr/src# 



sempron:/usr/src# fdisk -l /dev/sdg

Disk /dev/sdg: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdg doesn't contain a valid partition table
sempron:/usr/src# fdisk -l /dev/sdh

Disk /dev/sdh: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdh doesn't contain a valid partition table
sempron:/usr/src#

Notice above, it says sdc has a partition, but i cannot see sdc1 in /dev. As for the other two, i'm 100% sure they have working partitions, but they're gone... heck i can still mount them as /dev/sdg /mnt/tmp ! ... anyway, on to dmesg

Code:

Probing IDE interface ide0...
ata1: SATA max UDMA/100 cmd 0xFFFFC20000024C80 ctl 0xFFFFC20000024C8A bmdma 0xFFFFC20000024C00 irq 201
ata2: SATA max UDMA/100 cmd 0xFFFFC20000024CC0 ctl 0xFFFFC20000024CCA bmdma 0xFFFFC20000024C08 irq 201
ata3: SATA max UDMA/100 cmd 0xFFFFC20000024E80 ctl 0xFFFFC20000024E8A bmdma 0xFFFFC20000024E00 irq 201
ata4: SATA max UDMA/100 cmd 0xFFFFC20000024EC0 ctl 0xFFFFC20000024ECA bmdma 0xFFFFC20000024E08 irq 201
scsi0 : sata_sil
hda: WDC WD200AB-00CDB0, ATA DISK drive

ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata1.00: ATA-7, max UDMA/133, 976773168 sectors: LBA48 NCQ (depth 0/32)
ata1.00: ata1: dev 0 multi count 16
ata1.00: configured for UDMA/100
scsi1 : sata_sil

ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata2.00: ATA-7, max UDMA/133, 976773168 sectors: LBA48 NCQ (depth 0/32)
ata2.00: ata2: dev 0 multi count 16
ata2.00: configured for UDMA/100
scsi2 : sata_sil

ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
Probing IDE interface ide1...
ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata3.00: ATA-7, max UDMA/133, 976773168 sectors: LBA48 NCQ (depth 0/32)
ata3.00: ata3: dev 0 multi count 16
ata3.00: configured for UDMA/100
scsi3 : sata_sil

ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata4.00: ATA-7, max UDMA/133, 976773168 sectors: LBA48 NCQ (depth 0/32)
ata4.00: ata4: dev 0 multi count 16
ata4.00: configured for UDMA/100
  Vendor: ATA       Model: WDC WD5000KS-00M  Rev: 07.0
  Type:   Direct-Access                      ANSI SCSI revision: 05
  Vendor: ATA       Model: WDC WD5000KS-00M  Rev: 07.0
  Type:   Direct-Access                      ANSI SCSI revision: 05
  Vendor: ATA       Model: WDC WD5000KS-00M  Rev: 07.0
  Type:   Direct-Access                      ANSI SCSI revision: 05
  Vendor: ATA       Model: ST3500630AS       Rev: 3.AA
  Type:   Direct-Access                      ANSI SCSI revision: 05
ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 18
GSI 20 sharing vector 0xD1 and IRQ 20
ACPI: PCI Interrupt 0000:02:08.0[A] -> Link [LNKC] -> GSI 18 (level, low) -> IRQ 209
ata5: SATA max UDMA/100 cmd 0xFFFFC20000026880 ctl 0xFFFFC2000002688A bmdma 0xFFFFC20000026800 irq 209
ata6: SATA max UDMA/100 cmd 0xFFFFC200000268C0 ctl 0xFFFFC200000268CA bmdma 0xFFFFC20000026808 irq 209
ata7: SATA max UDMA/100 cmd 0xFFFFC20000026A80 ctl 0xFFFFC20000026A8A bmdma 0xFFFFC20000026A00 irq 209
ata8: SATA max UDMA/100 cmd 0xFFFFC20000026AC0 ctl 0xFFFFC20000026ACA bmdma 0xFFFFC20000026A08 irq 209
scsi4 : sata_sil

ata5: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata5.00: ATA-7, max UDMA/133, 976773168 sectors: LBA48 NCQ (depth 0/32)
ata5.00: ata5: dev 0 multi count 16
ata5.00: configured for UDMA/100
scsi5 : sata_sil

ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata6.00: ATA-7, max UDMA/133, 976773168 sectors: LBA48 NCQ (depth 0/32)
ata6.00: ata6: dev 0 multi count 16
ata6.00: configured for UDMA/100
scsi6 : sata_sil

ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata7.00: ATA-8, max UDMA7, 976773168 sectors: LBA48 NCQ (depth 0/32)
ata7.00: ata7: dev 0 multi count 16
ata7.00: configured for UDMA/100
scsi7 : sata_sil

ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata8.00: ATA-8, max UDMA7, 976773168 sectors: LBA48 NCQ (depth 0/32)
ata8.00: ata8: dev 0 multi count 16
ata8.00: configured for UDMA/100
  Vendor: ATA       Model: ST3500630AS       Rev: 3.AA
  Type:   Direct-Access                      ANSI SCSI revision: 05
  Vendor: ATA       Model: ST3500630AS       Rev: 3.AA
  Type:   Direct-Access                      ANSI SCSI revision: 05
  Vendor: ATA       Model: SAMSUNG HD501LJ   Rev: CR10
  Type:   Direct-Access                      ANSI SCSI revision: 05
  Vendor: ATA       Model: SAMSUNG HD501LJ   Rev: CR10
  Type:   Direct-Access                      ANSI SCSI revision: 05
sata_nv 0000:00:0a.0: version 2.0
ACPI: PCI Interrupt Link [LTID] BIOS reported IRQ 0, using IRQ 21
ACPI: PCI Interrupt Link [LTID] enabled at IRQ 21
ACPI: PCI Interrupt 0000:00:0a.0[A] -> Link [LTID] -> GSI 21 (level, low) -> IRQ 185
PCI: Setting latency timer of device 0000:00:0a.0 to 64
ata9: SATA max UDMA/133 cmd 0x9F0 ctl 0xBF2 bmdma 0xC800 irq 185
ata10: SATA max UDMA/133 cmd 0x970 ctl 0xB72 bmdma 0xC808 irq 185
scsi8 : sata_nv

SCSI device sda: 976773168 512-byte hdwr sectors (500108 MB)
sda: Write Protect is off
sda: Mode Sense: 00 3a 00 00
SCSI device sda: drive cache: write back
SCSI device sda: 976773168 512-byte hdwr sectors (500108 MB)
sda: Write Protect is off
sda: Mode Sense: 00 3a 00 00
SCSI device sda: drive cache: write back
 sda: sda1
sd 0:0:0:0: Attached scsi disk sda

SCSI device sdb: 976773168 512-byte hdwr sectors (500108 MB)
sdb: Write Protect is off
sdb: Mode Sense: 00 3a 00 00
SCSI device sdb: drive cache: write back
SCSI device sdb: 976773168 512-byte hdwr sectors (500108 MB)
sdb: Write Protect is off
sdb: Mode Sense: 00 3a 00 00
SCSI device sdb: drive cache: write back
 sdb: sdb1
sd 1:0:0:0: Attached scsi disk sdb

SCSI device sdc: 976773168 512-byte hdwr sectors (500108 MB)
sdc: Write Protect is off
sdc: Mode Sense: 00 3a 00 00
SCSI device sdc: drive cache: write back
SCSI device sdc: 976773168 512-byte hdwr sectors (500108 MB)
sdc: Write Protect is off
sdc: Mode Sense: 00 3a 00 00
SCSI device sdc: drive cache: write back
 sdc: sdc1
sd 2:0:0:0: Attached scsi disk sdc

SCSI device sdd: 976773168 512-byte hdwr sectors (500108 MB)
sdd: Write Protect is off
sdd: Mode Sense: 00 3a 00 00
SCSI device sdd: drive cache: write back
SCSI device sdd: 976773168 512-byte hdwr sectors (500108 MB)
sdd: Write Protect is off
sdd: Mode Sense: 00 3a 00 00
SCSI device sdd: drive cache: write back
 sdd: sdd1
sd 3:0:0:0: Attached scsi disk sdd

SCSI device sde: 976773168 512-byte hdwr sectors (500108 MB)
sde: Write Protect is off
sde: Mode Sense: 00 3a 00 00
SCSI device sde: drive cache: write back
SCSI device sde: 976773168 512-byte hdwr sectors (500108 MB)
sde: Write Protect is off
sde: Mode Sense: 00 3a 00 00
SCSI device sde: drive cache: write back
 sde: sde1
sd 4:0:0:0: Attached scsi disk sde

SCSI device sdf: 976773168 512-byte hdwr sectors (500108 MB)
sdf: Write Protect is off
sdf: Mode Sense: 00 3a 00 00
SCSI device sdf: drive cache: write back
SCSI device sdf: 976773168 512-byte hdwr sectors (500108 MB)
sdf: Write Protect is off
sdf: Mode Sense: 00 3a 00 00
SCSI device sdf: drive cache: write back
 sdf: sdf1
sd 5:0:0:0: Attached scsi disk sdf

SCSI device sdg: 976773168 512-byte hdwr sectors (500108 MB)
sdg: Write Protect is off
sdg: Mode Sense: 00 3a 00 00
SCSI device sdg: drive cache: write back
SCSI device sdg: 976773168 512-byte hdwr sectors (500108 MB)
sdg: Write Protect is off
sdg: Mode Sense: 00 3a 00 00
SCSI device sdg: drive cache: write back
 sdg: unknown partition table
sd 6:0:0:0: Attached scsi disk sdg

SCSI device sdh: 976773168 512-byte hdwr sectors (500108 MB)
sdh: Write Protect is off
sdh: Mode Sense: 00 3a 00 00
SCSI device sdh: drive cache: write back
SCSI device sdh: 976773168 512-byte hdwr sectors (500108 MB)
sdh: Write Protect is off
sdh: Mode Sense: 00 3a 00 00
SCSI device sdh: drive cache: write back
 sdh:<6>ata9: SATA link down (SStatus 0 SControl 300)
scsi9 : sata_nv
 unknown partition table
sd 7:0:0:0: Attached scsi disk sdh

ata10: SATA link down (SStatus 0 SControl 300)
hda: max request size: 128KiB
hda: 39102336 sectors (20020 MB) w/2048KiB Cache, CHS=38792/16/63, UDMA(100)
hda: cache flushes not supported
 hda: hda1 hda2 < hda5 >

md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
md: linear personality registered for level -1
md: multipath personality registered for level -4
md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
raid5: automatically using best checksumming function: generic_sse
   generic_sse:  4923.000 MB/sec
raid5: using function: generic_sse (4923.000 MB/sec)
raid6: int64x1   1462 MB/s
raid6: int64x2   1920 MB/s
raid6: int64x4   1799 MB/s
raid6: int64x8   1285 MB/s
raid6: sse2x1    1985 MB/s
raid6: sse2x2    2681 MB/s
raid6: sse2x4    2917 MB/s
raid6: using algorithm sse2x4 (2917 MB/s)
md: raid6 personality registered for level 6
md: raid5 personality registered for level 5
md: raid4 personality registered for level 4
md: raid10 personality registered for level 10

md: bind<sdf1>
md: bind<sde1>
raid1: raid set md2 active with 2 out of 2 mirrors
md: could not open unknown-block(8,33).
md: md_import_device returned -6
md: bind<sdd1>
raid1: raid set md1 active with 1 out of 2 mirrors
md: bind<sda1>
md: bind<sdb1>
raid1: raid set md0 active with 2 out of 2 mirrors

Attempting manual resume
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.


Adding 859436k swap on /dev/hda5.  Priority:-1 extents:1 across:859436k
EXT3 FS on hda1, internal journal
loop: loaded (max 8 devices)

EXT3 FS on md0, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds

EXT3 FS on md1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds

EXT3 FS on md2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting.  Commit interval 5 seconds

EXT3 FS on sdg, internal journal
EXT3-fs: recovery complete.
EXT3-fs: mounted filesystem with ordered data mode.

I appreciate the help. Thanks.

tovis · 03-08-2008, 07:05 AM

I had similar experience when somewhat goes wrong with drives hw settings jumpers or cable selected.
Only if I understand well you have problems alway with the same pair of hdd? - for you at first I would remove them, and follow with others. Also check the behavior with a live CD - is it see those partitions?

xushi · 03-08-2008, 07:25 AM

At first I thought it was from the same set, but now i'm not sure if it's true, or maybe the controller cards.. Now when i tried to shutdown, it kept saying things about ata.5 not responding to shutdown, and when rebooting it says the same thing about being too slow or unresponsive.. But this is the third controller card i use now.. And also replaced the cables 3 times

At one point, i'm suspecting it might be my motherboard acting up..

I'm so tired! it's been ongoing for 2 months now.. Every day i find something new so i cant point my finger at it.. I think i'm going to build another computer from scratch, with only the hard drives there, and see if the problem persists.. if not, then slowly move things over (RAM, Controller cards, SCSI, etc..).