LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 12-17-2011, 02:06 PM   #1
jhon614
LQ Newbie
 
Registered: Dec 2011
Posts: 3

Rep: Reputation: Disabled
Software (mdadm) Raid 6 rebuild problems


I have a Raid6 array built in mdadm, currently running under Ubuntu 11.04, using 7 2tb Sata2 drives. Everything had been running fine for quite a long time. Then one day, a drive reported it had failed, so I changed it with a new one and started to rebuild. I hit a bad sector on a different drive, and it removed it from the array. Some time later in the rebuild it happened again, leaving the array with 4 discs online, 2 removed and 1 partially synced. This is what I'm left with.

johnny@Ubuntu:~$ sudo mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Mon Jan 31 23:19:09 2011
Raid Level : raid6
Used Dev Size : 1953466624 (1862.97 GiB 2000.35 GB)
Raid Devices : 7
Total Devices : 5
Persistence : Superblock is persistent

Update Time : Wed Sep 14 12:39:46 2011
State : active, FAILED, Not Started
Active Devices : 4
Working Devices : 5
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 64K

Name : :Raid6
UUID : f4d0374a:0aaa0563:a6fed9f2:9931a37f
Events : 288197

Number Major Minor RaidDevice State
5 8 81 0 spare rebuilding
1 8 65 1 active sync /dev/sde1
8 8 33 2 active sync /dev/sdc1
3 0 0 3 removed
4 0 0 4 removed
6 8 49 5 active sync /dev/sdd1
7 8 129 6 active sync /dev/sdi1

Notice the 2 discs that are marked as "removed" and the 1 marked "spare rebuilding".

I would understand mdadm removing a disc with bad sectors during a Raid5 rebuild because this would render the array broken, since 1 failed disc and some bad sectors on another disc means you have lost data. But in Raid6 this isn't necessarily the case due to the double parity. Is there a way to keep discs with bad sectors from being removed during a rebuild, forcing the array to rebuild using the remaining parity until the resyncing drive has finished? If so, I could finish rebuilding the new drive and then replace the other two with bad sectors afterward.

1. Can I force the removed discs back into the array?

2. Can I force the removed discs with bad sectors to remain in the array and successfully resync the spare?

Thank you very much ahead of time for any help. Please be detailed in your responses since I'm a novice in all this.
 
Old 12-20-2011, 08:02 PM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,359

Rep: Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751
You may find this useful
Quote:
RAID 6 (block-level striping with double distributed parity) provides fault tolerance of two drive failures; the array continues to operate with up to two failed drives. This makes larger RAID groups more practical, especially for high-availability systems. This becomes increasingly important as large-capacity drives lengthen the time needed to recover from the failure of a single drive. Single-parity RAID levels are as vulnerable to data loss as a RAID 0 array until the failed drive is replaced and its data rebuilt; the larger the drive, the longer the rebuild takes. Double parity gives additional time to rebuild the array without the data being at risk if a single additional drive fails before the rebuild is complete.
https://en.wikipedia.org/wiki/RAID

By the looks of your output there, you have 3 non-active disks ...
 
Old 12-31-2011, 01:59 PM   #3
jhon614
LQ Newbie
 
Registered: Dec 2011
Posts: 3

Original Poster
Rep: Reputation: Disabled
There are indeed 3 inactive disks, however I don't believe these disks were removed due to failure and I should be able to re-add them somehow. My best guess is that my RAID cards were on the fritz after a big thunderstorm we had. Maybe that's why the drives were dropped so close together. My original questions are still what I need to know please.

1. Can I force the removed discs back into the array?

2. Can I force the removed discs with bad sectors to remain in the array and successfully resync the spare?

Also if it helps, I know that I hadn't added any new data to the array in quite a long time. This array is basically a torrent media archive with about 5 years worth of downloads on it.

Maybe this is unorthodox and I hope it's not against the rules, but if someone out there is an expert in mdadm, I'd be willing to pay a small fee through paypal if you can walk me through it. PM me if you're interested in helping me through this over the phone or through email. The person who helps me get it back up will get $50. How does that sound? I just really don't want to lose everything.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
mdadm: Rebuild a software raid from drives with existing partitions Boeby Linux - Server 3 04-26-2011 01:32 PM
mdadm: how to avoid complete rebuild of RAID 6 array (6/8 active devices) pbwtortilla Linux - Server 4 12-25-2009 08:06 PM
replace failure disk and rebuild RAID with mdadm ufmale Linux - Software 0 11-15-2007 02:24 PM
mdadm reports no superblock trying to rebuild failed RAID 5 hotcut23 Linux - Hardware 0 08-18-2007 01:39 AM
Software RAID with mdadm problems... robbow52 Debian 2 04-19-2004 07:06 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 05:26 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration