Download your favorite Linux distribution at LQ ISO.
Go Back > Forums > Linux Forums > Linux - Software
User Name
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.


  Search this Thread
Old 04-07-2009, 02:28 PM   #1
Registered: Dec 2003
Location: Calgary, AB
Distribution: Any!
Posts: 146

Rep: Reputation: 18
Limiting Cron by Load Average


Is it possible with vixie cron to limit cron's activities by the current load average? Basically I'm looking for the same functionality that's in fcron, but I'm restricted to use the vixie cron with CentOS 5.3.

If not, does anyone have any recommendations on limiting the impact that cron has when it runs 20-30 user-specified crontabs at once?

Old 04-08-2009, 02:47 AM   #2
Senior Member
Registered: Apr 2004
Location: Potchefstroom, South Africa
Distribution: Fedora 17 - 3.3.4-5.fc17.x86_64
Posts: 1,506

Rep: Reputation: 100Reputation: 100
Well Hetzner (a German hosting provider) uses something called "processwatch" to regulate crontabs (and all processes on their server). They don't seem to control the number of cron processes, but if something runs for too long (not sure how "too long" is defined...) it is killed automatically.

I mostly got around this by staggering my cronjobs, i. e.

2 0 * * * /usr/home/fwfunn/public_html/ >/dev/null 2>&1
3 0 * * * /home/httpd/cgi-bin/php5 /usr/home/fwfunn/public_html/daily_cron.php >/dev/null 2>&1
4 0 1 * * /home/httpd/cgi-bin/php5 /usr/home/fwfunn/public_html/monthly_cron.php >/dev/null 2>&1
05 00 * * * /home/http/cgi-bin/php5 /usr/home/fwfunn/public_html/fix_funeral_dep_group_type_mismatches.php >/dev/null 2>&1
I. e. run at 00:02 each day, then run daily_cron.php at 00:03 each day, etc.

Maybe you can also enforce staggering? Hetzner does this by limiting their web-interface for configuring your domain-wide cronjob by making it impossible to specify cronjobs "tighter" than once every ten minutes. They thus ensure that you won't be able to overload the server with cronjobs that run too quickly, one after the other.

(I have my own server with them, so I have SSH access, I manually wrote the above crontab, not through their interface - i. e. that is how I could run stuff each minute, not just every ten minutes as their KonsoleH web interface enforces).

Last edited by rylan76; 04-08-2009 at 02:48 AM.
Old 04-08-2009, 08:11 AM   #3
Registered: Dec 2003
Location: Calgary, AB
Distribution: Any!
Posts: 146

Original Poster
Rep: Reputation: 18
Hi Stefan,

Thanks for your reply and the information! I will look into something like processwatch and see if I can utilize it.

Staggering cron jobs does work and it works great. We had a server that was constantly hitting a load of 80 or higher and we couldn't figure out why. Finally we noticed that there were several hundred cron jobs running during the 5,10,30 minute intervals of the hour. I staggered all the jobs by 1-2 minutes and the load has not gone above 8 now.

So, it works great, but we have 40 servers to maintain and need to find a more universal solution. That's why I was looking into a queuing feature or I will even give processwatch a try and just kill the jobs.

I would love to move to fcron with it's queuing and load average support, but from what I read it is not 100% compatible with vixie cron (no cron.d support) and so I don't want to break our standard CentOS setup.
Old 04-09-2009, 05:39 AM   #4
Senior Member
Registered: Apr 2004
Location: Potchefstroom, South Africa
Distribution: Fedora 17 - 3.3.4-5.fc17.x86_64
Posts: 1,506

Rep: Reputation: 100Reputation: 100
Ok well, glad to be of help.

Just a word about processwatch. I'm not sure (of course) what your setup or service level agreements with your customers / users are, but processwatch can be a nasty surprise to someone who might not be aware it is present. I. e. I never even -knew- it was running on Hetzner servers, until I one day scheduled a hugely important, reasonably "heavy" cronjob and it never completed - leaving a database halfway archived (in my own semantic view of the data) and causing chaos for some of my customers. I had a real storm of excrement to deal with when I got to the office that morning, and only THEN did I find out that processwatch was running on Hetzner servers, and I would NOT be able to execute the cronjob as planned.

I. e. just warn or publicize the fact (if you do use it or something like it) that you will be deploying this kind of load-limiting device for cronjobs, if you decide to do it, don't just quietly deploy it so it might catch some unwary developers - like it caught me!

Old 09-10-2010, 03:40 PM   #5
LQ Newbie
Registered: Sep 2010
Posts: 1

Rep: Reputation: 0
may be this helps...

To prevent cron jobs from run during high system load I wrote some script. It checks the load and if it too high sleeps another period of time.
When loadavg become low it run its command-line.
For the cron I modified etc/crontab (F10 distro):

# run-parts
01 * * * * root loadlimit run-parts /etc/cron.hourly
02 4 * * * root loadlimit run-parts /etc/cron.daily
22 4 * * 0 root loadlimit run-parts /etc/cron.weekly
42 4 1 * * root loadlimit run-parts /etc/cron.monthly
where loadlimit is the name of my script:


# Script to limit cron jobs by high system loadavg.
# (Cron don't honor loadavg at all)
# Some ideas are from:
# http: //
# Author: A.V.Varlashkin (varlash at newmail dot ru)

# Define Threshold. This value will be compared with the current load average.
# Set the value as per your wish. Scale: 100 = 1.00

# Retrieve the load average of the past 15 minute (change $5 to $3 or $4 for other loadavg)
Load_AVG=`uptime | cut -d'l' -f2 | awk '{print $5}' | sed "s/\.//"`
# Compare the current load average with the Threshold and
[ $Load_AVG -gt $LIMIT ]
# wait until it become acceptable
        sleep $TIME2WAIT

# Load is low enough, run the command
# echo $@
exec $@
Beware since script is only lightly tested, so
#include <discard.all>
Good luck!


average, centos, cron, load

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Please Help me in my load average black-code Linux - Server 3 04-08-2009 01:17 PM
Limiting cron mail i_nomad Linux - Software 4 01-12-2009 06:15 AM
load average? ampex189 Linux - Newbie 2 03-06-2005 07:17 PM
Load average 1.0, 1.0, 1.0 ? belated Linux - Newbie 4 11-30-2003 03:49 PM
Average load Cyth Linux - General 1 01-22-2002 03:33 PM > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 12:19 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration