LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 07-26-2007, 09:31 PM   #1
norwolf
LQ Newbie
 
Registered: Jul 2007
Posts: 5

Rep: Reputation: 0
Major problem with software raid (mdadm) and disk failure


Ok, this is the case:

I've got two raid-5 arrays with software raid, both with three disks.

Setup:
md0 has hdb2, hdd1 and sda1
md1 has hdb5, hdd3 and sda3

Tonight, the system lost power due to a power spike. The result was a reboot where it attempted to fix the raid, but it didn't exactly work. I have now booted a live CD and using utilities there.

It seems the checksum value is slightly off for all disks. And the update time is the same for some disks, but not all.

When I try mdadm --assemble /dev/md1 (or md0) with various options like force, scan, auto, etc - I get error messages like only 1 drive found, not enough.. no disks found and no mention of md0/1 found in config file.

What should I do? Which partitions (two in each array) do I pick as "these two are the one I should rebuild with"? And how do I actually FORCE mdadm to start an array with those two set devices?

See files in ... for output from mdadm.

Assistance deeply appreciated...
 
Old 07-26-2007, 09:32 PM   #2
norwolf
LQ Newbie
 
Registered: Jul 2007
Posts: 5

Original Poster
Rep: Reputation: 0
ooops... the URL for the files: http://gandalf.n4f.no/~aj/md/
 
Old 07-26-2007, 10:33 PM   #3
rupertwh
Member
 
Registered: Sep 2006
Location: Munich, Germany
Distribution: Debian / Ubuntu
Posts: 298

Rep: Reputation: 49
Hi norwolf,

md1 should be startable from just hdb5 and sda3, as those two still match. Once it is up, let it rebuild hdd3.

md0 will be trickier, because none of the disks match. hdd1 is the worst (oldest) so that's the one you want to start without and rebuild later, once you have the array going with just hdb2 and sda1.
I don't know if mdadm will let you assemble the array with hdb2 and sda1, maybe with some force/run persuasion. Worst case would be to actually re-create the array from those two disks.
 
Old 07-26-2007, 11:23 PM   #4
norwolf
LQ Newbie
 
Registered: Jul 2007
Posts: 5

Original Poster
Rep: Reputation: 0
well, I haven't found a command that will do that yet. I've tried mdadm --assemble /dev/md1 --force, mdadm --assemble /dev/md1 --scan, etc. Any clues?
 
Old 07-26-2007, 11:39 PM   #5
rupertwh
Member
 
Registered: Sep 2006
Location: Munich, Germany
Distribution: Debian / Ubuntu
Posts: 298

Rep: Reputation: 49
Have you tried explicitly giving the device names of the partitions you want to add?
I can't test it right now, but something like
Code:
mdadm --assemble /dev/md1 /dev/hdb5 /dev/sda3
Then add --force and --run as needed. ;o)
 
Old 07-27-2007, 12:33 AM   #6
rupertwh
Member
 
Registered: Sep 2006
Location: Munich, Germany
Distribution: Debian / Ubuntu
Posts: 298

Rep: Reputation: 49
Good news after a quick test: it should work painlessly as suggested.
For md1:
Code:
mdadm --assemble /dev/md1 /dev/hdb5 /dev/sda3
mdadm /dev/md1 --add /dev/hdd3
For md0:
Code:
mdadm --assemble /dev/md0 /dev/hdb2 /dev/sda1 --force
mdadm /dev/md0 --add /dev/hdd1
(And obviously do a fsck afterwards, especially on md0)
 
Old 07-27-2007, 05:27 AM   #7
norwolf
LQ Newbie
 
Registered: Jul 2007
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by rupertwh
Have you tried explicitly giving the device names of the partitions you want to add?
I can't test it right now, but something like
Code:
mdadm --assemble /dev/md1 /dev/hdb5 /dev/sda3
Then add --force and --run as needed. ;o)
Tried the following:
# mdadm --assemble /dev/md1 /dev/hdb5 /dev/sda3
no devices found for /dev/md1
# mdadm --assemble --force /dev/md1 /dev/hdb5 /dev/sda3
no devices found for /dev/md1
# mdadm --assemble --run /dev/md1 /dev/hdb5 /dev/sda3
no devices found for /dev/md1
# mdadm --assemble --force --run /dev/md1 /dev/hdb5 /dev/sda3
no devices found for /dev/md1

Although - the devices ARE there.
 
Old 07-27-2007, 06:45 AM   #8
rupertwh
Member
 
Registered: Sep 2006
Location: Munich, Germany
Distribution: Debian / Ubuntu
Posts: 298

Rep: Reputation: 49
hm, intersting.
Try adding a '-v' switch on the assemble, maybe that'll give some useful info.

PS: Is there a /etc/mdadm/mdadm.conf on the system you are trying to assemble from (i.e. the live cd)? If so, please post the contents.

Last edited by rupertwh; 07-27-2007 at 06:53 AM.
 
Old 07-27-2007, 07:14 AM   #9
norwolf
LQ Newbie
 
Registered: Jul 2007
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by rupertwh
hm, intersting.
Try adding a '-v' switch on the assemble, maybe that'll give some useful info.

PS: Is there a /etc/mdadm/mdadm.conf on the system you are trying to assemble from (i.e. the live cd)? If so, please post the contents.
-v didn't give any more info.

No, there isn't an /etc/mdadm.conf or /etc/mdadm/mdadm.conf. The entire system is offline, we've got it booting from a rescue CD.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Software Raid 5 (md) recovery using mdadm fakeroot Linux - Server 24 04-15-2013 10:19 AM
Why can't I mount this md0 raid? (mdadm and software raid) cruiserparts Linux - Software 35 01-05-2013 04:35 PM
Debian sarge netinstall software raid major problem araldit Linux - Newbie 0 07-22-2004 06:58 PM
Software RAID with mdadm problems... robbow52 Debian 2 04-19-2004 08:06 PM
Software Raid Setup Ok - Reboot fails on disk failure test ikke Linux - General 2 05-11-2003 07:42 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 02:41 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration