LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 10-15-2008, 01:25 PM   #1
ufmale
Member
 
Registered: Feb 2007
Posts: 386

Rep: Reputation: 30
mdadm RAID 5, 6 disks failed. recovery possible?


I have a RAID 5 linux machine setup using mdam. The RAID has 10 disks and 6 of them failed. Is there a way i can still recover the data?

In addition, I have learned in the past that mdadm may report that the disk failed, but the disk itself may still work if reformated. Is that true?
 
Old 10-15-2008, 02:07 PM   #2
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora
Posts: 3,935
Blog Entries: 5

Rep: Reputation: Disabled
Let's just talk RAID 5 theory for starters. You're supposed to be able to lose one drive and continue operating normally.

You've lost six of ten?? (I can't speak to the other questions.)
 
Old 10-15-2008, 02:46 PM   #3
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,187

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
Was that a simultaneous failure of six drives? If so, were they on the same controller, cable, or power supply? In other words, are you sure you have multiple drive failures rather than a single drive infrastructure failure?
 
Old 10-15-2008, 02:47 PM   #4
ufmale
Member
 
Registered: Feb 2007
Posts: 386

Original Poster
Rep: Reputation: 30
That is right..
When i tried to start the RAID with mdadm

$ mdadm --assemble /dev/md0 /dev/sd[a-j]1

I got
mdadm: /dev/md0 assembled from 4 drives - not enough to start the array.

Any suggestion? Should i start reformating these disks?
 
Old 10-15-2008, 11:45 PM   #5
thetakan
LQ Newbie
 
Registered: Oct 2008
Distribution: SuSe
Posts: 7

Rep: Reputation: 0
Can u check cable connection for two drives those are failing?

I haven't heard of 6 drive failure in my entire IT career.

Thanks though...
 
Old 10-16-2008, 09:29 AM   #6
ufmale
Member
 
Registered: Feb 2007
Posts: 386

Original Poster
Rep: Reputation: 30
Quote:
Originally Posted by thetakan View Post
Can u check cable connection for two drives those are failing?

I haven't heard of 6 drive failure in my entire IT career.

Thanks though...
I puzzle that myself too. It might be a power surge since it does not connect to any UPS. The cable looks fine. I can see all 10 disks from "fdisk -l", but the mdadm does not want to build RAID with those 6 drives. Only 4 drives are good. I am thinking about reformating the drives, but post here in case someone has a good suggestion how to recover the data.

The disk enclosure is packed with 10 drives, There are 2 sata cables from this enclosure to a 2-port interface card to a Gentoo machine.
 
Old 10-16-2008, 12:03 PM   #7
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,187

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
Can you replace the 2-port interface card, at least temporary, to see if that makes a difference?

Since fdisk can see the drives, perhaps the problem is in the mdadmin control file. Have a look at it to see if it looks correct. Maybe a fsck -f on the drive containing the administrative file might be in order.
 
Old 10-18-2008, 09:16 AM   #8
ufmale
Member
 
Registered: Feb 2007
Posts: 386

Original Poster
Rep: Reputation: 30
Quote:
Originally Posted by PTrenholme View Post
Can you replace the 2-port interface card, at least temporary, to see if that makes a difference?

Since fdisk can see the drives, perhaps the problem is in the mdadmin control file. Have a look at it to see if it looks correct. Maybe a fsck -f on the drive containing the administrative file might be in order.


I change the card, but i got the same problem.
fsck -f does not work since the drive is ext3. What information i would get from that command?

What is mdadmin[/b] control file?
 
Old 10-18-2008, 10:24 AM   #9
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,187

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
Quote:
Originally Posted by ufmale View Post
I change the card, but i got the same problem.
fsck -f does not work since the drive is ext3. What information i would get from that command?

What is mdadmin control file?
O.K., first fsck is a script that identifies the Linux file system on a partition, and then runs the check program specific for that type of system. It works quite well on ext type file systems. (Note that you, of course, can only run the checking program if the partition is not mounted.) The information you would get would be a report of any problems found and what was done to correct those problems.

As to the mdadmin control file, according to man mdadm it's /etc/mdadm.conf, so I was suggesting, in effect, that you boot from a rescue disk and run fsck on your boot drive. I'm suggesting this because the simultaneous failure od six different hard disks is so implausible that some other cause seems more likely. Especially since fdisk seems to think the drives are in good shape.

By the way, man mdadm discusses several options for attempting to recover from RAID failures.

Oh, another thought: md is sometimes used to set up a RAID system using partitions of physical disks. Is your RAID composed of 10 different physical disk drives or is it made of a smaller number of physical drives using different partitions on those drives?

Last edited by PTrenholme; 10-18-2008 at 10:26 AM.
 
Old 10-19-2008, 10:19 AM   #10
Yaniv-Fer
Member
 
Registered: Feb 2006
Distribution: Fedora,Solaris
Posts: 46

Rep: Reputation: 15
i found here some more details about software raid... maybe it will help you
http://www.linuxhomenetworking.com/w..._Software_RAID


here is an example of the mdadm.conf file

[root@bigboy tmp]# mdadm --detail --scan --verbose
ARRAY /dev/md0 level=raid5 num-devices=4
UUID=77b695c4:32e5dd46:63dd7d16:17696e09
devices=/dev/hde1,/dev/hdf2,/dev/hdg1
[root@bigboy tmp]#




again this is copy paste from the other site... DO NOT PASTE THIS IN YOUR FILE
this should give you a idea how this file should be looking...



good luck
Yaniv Ferszt
 
Old 10-20-2008, 08:24 AM   #11
slackman
Member
 
Registered: Mar 2003
Distribution: Slack 9.0
Posts: 123

Rep: Reputation: 15
6 disk failed? ehh that's a bit strange. are they visible by controller? if not check power cables. if they are check each HD independently with mdadm -E /dev/sdX. run badblocks on failed devices.

could it be the enclosure sata controllers? do u have spare sil controllers that u can hook up all 10 HDs and force reassembly? just a thought
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Software Raid 5 (md) recovery using mdadm fakeroot Linux - Server 24 04-15-2013 09:19 AM
Creating RAID 5 with 2 disks using mdadm amitpardesi Linux - Software 6 10-12-2008 11:02 PM
Using mdadm - Failed RAID-5 Array but individual disks check out ok JRFrogman Linux - Server 0 06-05-2008 02:46 PM
RAID 6 failure - 3 disks failed but 2 are ok, recovery possible? auroraglacialis Linux - Server 4 11-10-2007 11:25 AM
mdadm reports no superblock trying to rebuild failed RAID 5 hotcut23 Linux - Hardware 0 08-18-2007 01:39 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 06:09 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration