LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices

Reply
 
Search this Thread
Old 10-10-2008, 06:10 AM   #1
theodm
LQ Newbie
 
Registered: Jul 2008
Posts: 6

Rep: Reputation: 0
Software raid failure every day at the exact same time


Hi,

I have a Debian server which has always been a bit unstable. I think it may be a design flaw in one of the hardware components, because another server with the exact same hardware config has the same problem. In the beginning it often had raid crashes that made the filesystem go into readonly, until at some point it was put in a data center and it ran fine for 1.5 year. I then had to add some memory after which it started crashing again once every 1-2 weeks. Very annoying but without alternatives nothing to do about it.

In the past week, the raid has crashed every single day though, and always at exactly the same time, 6:28 in the morning give or take half a minute. In most cases the same sector. It's a web server and there's not that many users at that time. No crons or anything scheduled at that time either. Logs show nothing that helps, as always. If there's a bad sector, I just don't understand why it always dies at that same time. And since that other server with the same hardware config has the same kind of crashes (not sure if at the same time) I still doubt it's really because of bad sectors. Resyncing after a failure always goes without any errors, so shouldn't that give an error as well when there's bad sectors?

Here's a part of the syslog:
Oct 10 06:28:21 kernel: scsi0: ERROR on channel 0, id 2, lun 0, CDB: Read (10) 00 01 a1 c5 b7 00 00 60 00
Oct 10 06:28:21 kernel: Info fld=0x1a1c5d7, Current sda: sense key Medium Error
Oct 10 06:28:21 kernel: Additional sense: Unrecovered read error
Oct 10 06:28:21 kernel: end_request: I/O error, dev sda, sector 27379127
Oct 10 06:28:21 kernel: raid1: Disk failure on sda2, disabling device.
Oct 10 06:28:21 kernel: ^IOperation continuing on 1 devices
Oct 10 06:28:21 kernel: raid1: sda2: rescheduling sector 27379064
Oct 10 06:28:21 kernel: raid1: sdb2: redirecting sector 27379064 to another mirror

The reschedule and redirect goes on forever. When I remove and add the failing disk the server dies and needs an apc reboot.

Any suggestions what could cause this? Thanks!
 
Old 10-11-2008, 01:18 AM   #2
checkmate3001
Member
 
Registered: Sep 2007
Location: Folsom, California
Distribution: Debian 4.0 (Etch), Debian 5.0 (Lenny), Ubuntu 8.04
Posts: 301

Rep: Reputation: 32
I would see if the mobo has the latest firmware. But that doesn't explain why the raid is failing at the same time every day. It really doesn't make any sense. Have you tried running SMART on the drive(s) in question to see if the drive(s) have any issues?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
recover dirty, degraded software raid 1 after power failure Rascale Linux - Server 3 07-31-2008 12:00 PM
Software Raid 1 Array Failure - Detecting and Repairing sloik2000 Linux - Hardware 5 04-05-2008 12:28 AM
Major problem with software raid (mdadm) and disk failure norwolf Linux - Server 8 07-27-2007 06:14 AM
Software RAID Failure? carlosinfl Linux - General 3 07-13-2007 10:06 PM
Linux Software RAID failure - NEED HELP! tkconn Linux - Hardware 1 01-07-2006 06:01 PM


All times are GMT -5. The time now is 11:18 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration