LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 01-10-2007, 02:36 PM   #1
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 5,995
Blog Entries: 5

Rep: Reputation: 782Reputation: 782Reputation: 782Reputation: 782Reputation: 782Reputation: 782Reputation: 782
slocate updatedb locked files?


Some time back (mid-October) we noticed a dramatic increase in CPU load during a weekend. On researching this we were able to trace it back to a file on an ext3 filesystem that would cause any process that attempted to access it to hang (thereby adding to the runq). Since the slocate cron job ran every day as did an incremental or full backup they too would get hung and add to the runq. This was the first time such an issue had been noted in over a year of running this RH AS 3 system. The issue was solved by rebooting the server. The file in question was easily accessible after the reboot.

Overnight our incremental backup failed and I see once again that slocate/updatedb has hung on a file but it is not the same file nor even the same filesystem as the prior one though it is an ext3 filesystem.

We did a reboot to clear the problem but Im wondering if this is being caused by slocate/updatedb or is it just the first thing that finds it. If the latter; what is causing the initial file lock?

P.S. Before anyone suggests it of course we had tried doing progressive kills on all processes referencing the file the kills including kill -9 do not work.
 
Old 01-10-2007, 05:37 PM   #2
stress_junkie
Senior Member
 
Registered: Dec 2005
Location: Massachusetts, USA
Distribution: Ubuntu 10.04 and CentOS 5.5
Posts: 3,873

Rep: Reputation: 331Reputation: 331Reputation: 331Reputation: 331
I had similar problems. I appeared to be having increasing problems with disk accesses. I wasted a lot of time exercising the disks in numerous different ways. I changed the disks around, changed the jumpers, etc. etc. When I put the apparently defective disks on a different computer they worked okay. It turned out that my motherboard was malfunctioning. New motherboard -> problem fixed.

Last edited by stress_junkie; 01-10-2007 at 05:39 PM.
 
Old 01-11-2007, 08:49 AM   #3
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 5,995
Blog Entries: 5

Original Poster
Rep: Reputation: 782Reputation: 782Reputation: 782Reputation: 782Reputation: 782Reputation: 782Reputation: 782
Thanks for your reply.

This is on a Dell PowerEdge rather than a build my own system. Doesn't mean the motherboard can't be a problem but since it has a SCSI (PERC) adapter for the drives and they're mirrored it doesn't seem likely. Maybe it's glitches in the PERC. I just wanted to see if there were any known issues with slocate/updatedb causing this kind of thing on occasion.
 
Old 04-15-2010, 01:57 PM   #4
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 5,995
Blog Entries: 5

Original Poster
Rep: Reputation: 782Reputation: 782Reputation: 782Reputation: 782Reputation: 782Reputation: 782Reputation: 782
Resolved - sort of

What I eventually figured out was that this was a heat issue. We saw this on other systems as well. All of them were in the same rack which was fairly full and in the center of the data center. Often we would see battery issues reported on the system LED for the PERC cards.

Although the motherboards had monitors for heat the PERC card doesn't. Dell wouldn't admit the PERC was more heat sensitive than the motherboard but we were able to prove it to ourselves by observation. Simply by opening the door of the rack we were able to make the battery message go away and by closing the door we were able to make it come back. As time has gone on every time there has been a heat event in the data center we saw the same file locking. There was never a time I saw it that I wasn't able to trace it back to increased heat. (In one event I found someone had turned off the fan in the top of the rack - for no apparent reason.) We have mostly mitigated this by adding additional fans to the back of the rack door itself.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to create automatically the /var/lib/slocate/slocate.db irhernan Linux - Newbie 3 09-12-2006 02:29 PM
No slocate.db file. Can't updatedb hottdogg Slackware 10 06-05-2006 05:12 PM
updatedb and slocate runs at 5am, how can I change this? Archeantus Linux - Software 4 02-12-2005 05:06 PM
What's the difference between 'updatedb' and 'updatedb&' ?? DreameR-X Linux - Newbie 4 12-18-2004 04:36 PM
Quick slocate/updatedb question? King of Japan Linux - Newbie 4 08-29-2004 01:35 PM


All times are GMT -5. The time now is 04:02 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration