LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 03-27-2007, 09:50 AM   #1
mackdav
Member
 
Registered: Feb 2005
Distribution: RHEL, CentOS
Posts: 113

Rep: Reputation: 16
Can't stop mdadm resync!


So I have this system with two sata disks, sda and sdb. sdb dies, so last night I replaced it with a new disk. I resync'd the array, and everything looks good.

Overnight I started to get unrecoverable read errors on sda. The array's response to these errors is to restart the sync. It's been doing this constantly ever since.

So I say OK, clearly sda is bad too. (First I checked to make sure I'd really pulled the dead drive and not the survivor. I conclude that since I have log files on the device between ). This resync is never going to finish, and I don't want to prematurely kill my new drive with this constant activity. So I'd like to kill the resync.

Only I can't.

I try to fail, then remove the array member like so:

# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sdb1[2] sda1[0]
77071680 blocks [2/1] [U_]
[=>...................] recovery = 9.2% (7165440/77071680) finish=25.2min speed=46124K/sec
# mdadm /dev/md1 -f /dev/sda1 -r /dev/sda1
mdadm: set /dev/sda1 faulty in /dev/md1
mdadm: hot remove failed for /dev/sda1: Device or resource busy

The resync restarts immediately after the device is marked faulty.

Anyone know how I might get myself out of this loop? (Ideally without having to reboot into single user mode or anything like that -- I'm doing this remotely.) I have two new disks on order and they should get here today or tomorrow, and I do have tape backups which are good, but I still don't want to burn out this new disk if I don't have to.

This is the text in dmesg related to the disk in distress:

ata1: status=0x51 { DriveReady SeekComplete Error }
ata1: error=0x40 { UncorrectableError }
scsi0: ERROR on channel 0, id 0, lun 0, CDB: Read (10) 00 09 30 06 3b 00 00 04 00
Current sda: sense key Medium Error
Additional sense: Unrecovered read error - auto reallocate failed
end_request: I/O error, dev sda, sector 154142267
 
Old 03-27-2007, 10:12 AM   #2
dgar
Member
 
Registered: Jun 2005
Location: Candia, NH
Distribution: Ubuntu, FC, RHE3, RHE4, CentOS
Posts: 121

Rep: Reputation: 15
Doesn't look promising. Try turning off DMA? Kinda hard to do with SATA.
 
Old 03-27-2007, 10:13 AM   #3
dgar
Member
 
Registered: Jun 2005
Location: Candia, NH
Distribution: Ubuntu, FC, RHE3, RHE4, CentOS
Posts: 121

Rep: Reputation: 15
Another possibility: Set the jumper on the drive to read-only.
 
Old 03-27-2007, 12:57 PM   #4
mackdav
Member
 
Registered: Feb 2005
Distribution: RHEL, CentOS
Posts: 113

Original Poster
Rep: Reputation: 16
Actually the problem turned out to be I was screwed: the initial sync to sdb never completed, so the sync operation from sda -> sdb kept failing, and the RAID software was trying to recover the only way it knew how. The hint is when looking at the array, the alleged "good" disk is labeled "spare", and the "failed" disks is labeled "active sync".

I've replaced both disks (fortunately the read errors were in unused sectors) and everything looks good again.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Resync mysql dlublink Linux - Software 1 06-21-2006 10:37 AM
resync in Partitionable RAID 5. FinalFantasy Linux - General 4 10-15-2005 09:02 AM
do you know mdadm? Coldburn Linux - Newbie 1 08-31-2004 02:22 PM
RAID resync thedeud Linux - General 6 03-14-2003 05:22 PM
Resync every reboot u4113072 Linux - Software 0 10-15-2002 02:06 AM


All times are GMT -5. The time now is 03:31 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration