raid 5 out of sync and I think I just hosed myself

lenster · 04-26-2010, 11:56 PM

As per the subject, I think I just hosed myself. I'm well aware of what I did wrong. I'm just hoping there is a way to recover.

I had a raid 5 that was slightly out of sync and wouldn't come back online when I reassembled it. I tried the --force option with assemble but forgot to put my /dev/md2 device as the first device so I hosed my superblock on the first device. This is what I did

Code:

mdadm --assemble --force /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1

I then had a brain fart and based on the output I ran the same command without /dev/sdc1 like this

Code:

mdadm --assemble --force /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 missing /dev/sdc1

So I just hosed the superblocks for 2 of my devices. Is there anyway to recover from this? I'm confident the data is there but slightly out of sync and with two superblocks hosed.

To give you an idea of how out of sync things were before I hosed it, here's the abbreviated output of mdadm -E. I have the full output as well if that helps.

Code:

[root@heisenberg]# mdadm -E /dev/sdc1 |grep Events
         Events : 11685
[root@heisenberg]# mdadm -E /dev/sdd1 |grep Events
         Events : 11671
[root@heisenberg]# mdadm -E /dev/sde1 |grep Events
         Events : 11682
[root@heisenberg]# mdadm -E /dev/sdf1 |grep Events
         Events : 11683
[root@heisenberg]# mdadm -E /dev/sdg1 |grep Events
         Events : 11683

I'm guessing I need to go to backups but I'd love to try anything folks have to suggest.

lenster · 04-27-2010, 12:19 AM

Turns out that I didn't hose myself as badly as I thought. I just blew away the /dev/sdc1 and /dev/sdd1 device nodes. A simple reboot fixed my problem. I'm back at the point where I have 5 drives that are slightly out of sync.

I'm trying to determine if I should continue down the path I was going which as to force an assemble or should I look at another path?

lenster · 04-27-2010, 09:27 PM

Quote:

Originally Posted by lenster

Turns out that I didn't hose myself as badly as I thought. I just blew away the /dev/sdc1 and /dev/sdd1 device nodes. A simple reboot fixed my problem. I'm back at the point where I have 5 drives that are slightly out of sync.

I'm trying to determine if I should continue down the path I was going which as to force an assemble or should I look at another path?

Forcing the assemble worked. One drive was considered failed so I simply rebuilt it by adding it. Aftewards I did a filesystem repair and so far so good.

Code:

mdadm -assemble --force /dev/md2 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1