LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 12-29-2011, 12:59 AM   #1
ray63
LQ Newbie
 
Registered: Nov 2010
Posts: 2

Rep: Reputation: 0
/etc/cron.daily/slocate.cron - multiple /usr/bin/updatedb degrades performance


I run openfiler as a NAS and recently noticed significant performance degradation of my rynsc backups.

I have 2x1.5TB drives in RAID-1, backed up to a 2TB drive using rsync, on the same server. The 2TB drive is 96% full, and the raid is 83% full

Whilst I have been able to get 40MB/s throughput (samba to windows) to both the RAID and backup drive, I recently noticed that my backups were taking longer and longer each day, and that although a reboot restores the backup speed, perforamnce starts to degrade again day by day. SAMBA throughput has now dropped to 1MB/s on the 2TB drive.

After trying to umount the 2TB drive, I discovered that there are multiple 'updatedb' processes running, and I assume this might be because cron starts another job, before the prior job had finished. At the moment, I have 11 updatedb jobs running.


Here's the slocate.cron file
Code:
#!/bin/sh
renice +19 -p $$ >/dev/null 2>&1
/usr/bin/updatedb -f "nfs,smbfs,ncpfs,proc,devpts" -e "/tmp,/var/tmp,/usr/tmp,/afs,/net"

Has anyone got any suggestions as to how to solve the problem (other than making more space available on the 2TB drive)?

Should I run updatedb weekly instead? Or not at all? I have never really bother using 'locate' before, so does it matter if I run updatedb?
Should I test to see if an updatedb process is already running, before starting updatedb? My shell scripting writing ability is limited, so hints are welcome.
How long does updatedb take to update 2TB of files?
 
Old 12-29-2011, 04:24 AM   #2
TenTenths
Senior Member
 
Registered: Aug 2011
Location: Dublin
Distribution: Centos 5 / 6
Posts: 1,599

Rep: Reputation: 483Reputation: 483Reputation: 483Reputation: 483Reputation: 483
Check to see if any other scripts make use of "locate", if there are none then you can remove that script from the cron or, as you suggest, run it once a week.

Think about timings and see if you can schedule it to run when there it least load on the system and preferably when you don't have an rsync working.

Of course you could always try something as simple as turning it off and seeing what breaks!
 
Old 01-05-2012, 06:04 PM   #3
ray63
LQ Newbie
 
Registered: Nov 2010
Posts: 2

Original Poster
Rep: Reputation: 0
Thanks TenTenths

Just to close this one out, I ended up re-arranging my rysnc backups and removing numerous backups. Rather than save every daily backup (over 18 months worth to date), I got around to saving monthly and weekly copies and deleted all the daily backups, except for the last month. This got rid of around 150 GB of data, but more importantly, millions of links. I started deleting files in a script in the morning, and files/directories were still being deleted during the night - so this took a long time.

I then recreated the slocate database (480 MB), and this still took about six hours, so I've put the cron job back into cron.daily, to see how this goes. This job is indexing three volumes (including 1 raid) - total of 2.7 TB, with around 30 million files/links

My backups run at 02:00, so they should normally be finished by 04:00, when cron.daily jobs are run.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
cron - how to give priority to a task running in cron.daily? MeeLee Linux - Newbie 3 11-09-2010 09:41 AM
adding a perl script to cron.daily / cron.d to setup a cron job CrontabNewBIE Linux - Software 6 01-14-2008 09:16 AM
cron hourly, daily, cron.d jobs don't execute eggsmartha Linux - General 3 09-17-2007 07:37 PM
Can any one plz explain why/what for cron.d, cron.daily, cron.weekly etc are there. mavinashbabu Linux - Newbie 4 09-21-2006 02:50 PM
updatedb daily cron job TranceDude Linux - Software 12 08-25-2005 01:09 PM


All times are GMT -5. The time now is 04:27 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration