LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 12-12-2005, 08:01 PM   #1
edkhosting
LQ Newbie
 
Registered: Sep 2003
Posts: 10

Rep: Reputation: 0
Had a crash and need help and info, running a fsck and clearing millions of inodes


I have a db server its a dual xeon with 2gb ram and a Adaptec 2230SLP with 4 Fujitsu 74GB running in a raid 10 setup and a Western Digital Raptor for a backup drive.


The main os is on the raid array(everything is). I didn't make it so \var\lib\mysql was on the raid array and mount it and have the os on another drive. Now this db server has 2 large databases on it totaling in over 3 million tables and 38gb of space. The server locked up the other night and it was rebooted and so I went to run a check on it when it came up and opened up a screen session and ran "mysqlcheck -rof --all-databases" which I have done many times before nothing unusual. The database was running correctly and the forum was working fine. It usually takes about 6-8 hours to do this and about 2 hours into it I noticed the server was somewhat down but before usually I wouldn't be able to ssh in but I was able to this time. I went and checked the screen session and every table it was checking was saying that it was read only and it was going through all the tables. Going onto the forums that the databases are used for it was throwing up an error saying the sessions table was read only and couldn't write to it. Well I tried to do a shutdown -r now and reboot the server but shutdown just hung I went into top and tried to kill it and couldn't so the server was hard rebooted. It came up and said that there were errors on the drive/raid array and to run a check. Well this has been 17 hours ago that it started clearing inodes after running fsck and it has currently cleared around 6 million inodes as of 12 hours ago and counting.

Now after all this my question is do you think my databases are gone? I would hope not since the os was able to load and the server is running its just clearing inodes which means its fixing the file system. I cannot ssh in it is still running now clearing inodes.
 
Old 12-12-2005, 08:42 PM   #2
Matir
LQ Guru
 
Registered: Nov 2004
Location: San Jose, CA
Distribution: Ubuntu
Posts: 8,507

Rep: Reputation: 125Reputation: 125
You may well have lost data. This sounds a lot like a possible hard drive failure. I would perform some SMART tests on the individual drives to be sure.
 
Old 12-12-2005, 09:55 PM   #3
edkhosting
LQ Newbie
 
Registered: Sep 2003
Posts: 10

Original Poster
Rep: Reputation: 0
Well its a raid 10 setup and if a hard drive failed the other array would have taken over. All the equipment is less then 2 months old. It said that the files were read only it didn't say they didn't exist or that the data is gone. Its a raid 10 array it there incase a hard drive failed and the other would take over.

Last edited by edkhosting; 12-12-2005 at 09:56 PM.
 
Old 12-13-2005, 10:12 AM   #4
edkhosting
LQ Newbie
 
Registered: Sep 2003
Posts: 10

Original Poster
Rep: Reputation: 0
can anyone else help me, it is now 1 day and 6 hours after the sever has come back online and it has cleared 6 million 19 thousand and something.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to fsck / while system running LinuxGeek Linux - Software 2 07-19-2005 10:43 AM
Running fsck clinton Linux - Software 2 01-25-2005 07:12 PM
mount -r while fsck is running? aaccomazzi Linux - Hardware 1 10-20-2004 04:00 PM
Is there any point in running fsck.ext3? Rundi Linux - General 11 03-12-2004 09:28 AM
running fsck rbrooks Linux - Newbie 1 05-16-2002 04:30 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 04:40 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration