LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 05-01-2014, 07:54 AM   #1
jimbo1954
Member
 
Registered: Oct 2006
Location: High Wycombe, Bucks, UK.
Distribution: Debian and Fedora Core in equal measure
Posts: 264

Rep: Reputation: 33
Randomising Cron Job runtimes


I will be building and deploying a number of devices that will be requesting updates once a week from a central server.

I want to be able to build the devices as standard clones, but I don't want them all to try to grab updates at the same time, so I need to randomise the running of the job that grabs the updates.

I see there being two possibilities, one being to actually randomise the time the cron job runs, so it runs once a week on a random day, or the other, potentially better way which is that the job is run at bootup, then at exactly one-weekly intervals after that. The chances of many of the devices being booted at the exact same time are minimal, and the interval would not vary, being just 1 week this way, so it looks like a better way forward.

My problem is that I'm not sure how to achieve this. I've seen a whole bunch of stuff on Google that describes creating jobs that cron runs and that in turn run the update job, but they are all fairly old and very convoluted (=prone to failure!) so I'm asking really to see if there is a simple, elegant way of achieving my aims.

Thanks for any suggestions
 
Old 05-01-2014, 10:08 AM   #2
jyoung4
LQ Newbie
 
Registered: Apr 2006
Location: Minneapolis, Minnesota, USA
Posts: 16

Rep: Reputation: 1
One suggestion for how to randomize cron jobs

What I would do is create a file somewhere containing the day you want this particular server to do it's update. For example:
echo Thu > /myupdate.day
Then set your cron job to run every day and check to see if today is the same day of the week as /myupdate.day. If it is, do the update, otherwise quietly go away. Since I'm a big fan of log files, I'd probably write a log entry listing the date and a short message indicating that it either did run the update or didn't.
 
Old 05-01-2014, 10:55 AM   #3
jimbo1954
Member
 
Registered: Oct 2006
Location: High Wycombe, Bucks, UK.
Distribution: Debian and Fedora Core in equal measure
Posts: 264

Original Poster
Rep: Reputation: 33
Hi Jim,
That will get the job to run on a certain day of the week, then the cron job will run only on that day. That will mean (if I understand your proposal correctly) that since all the systems are clones of each other, the problem will remain that they all do the update at the same time, which will flood the server with requests...or did I misunderstand? (Highly possible)
 
Old 05-01-2014, 11:09 AM   #4
YankeePride13
Member
 
Registered: Aug 2012
Distribution: Ubuntu 10.04, CentOS 6.3, Windows 7
Posts: 262

Rep: Reputation: 55
What about writing a script that will create AT jobs at random intervals?
 
Old 05-01-2014, 11:22 AM   #5
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 15,643

Rep: Reputation: 5127Reputation: 5127Reputation: 5127Reputation: 5127Reputation: 5127Reputation: 5127Reputation: 5127Reputation: 5127Reputation: 5127Reputation: 5127Reputation: 5127
I would rather randomize the list of hosts (probably containing dummy elements) and fire up the jobs on a regular base taking always the next element of that list.
 
Old 05-02-2014, 08:00 AM   #6
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,866

Rep: Reputation: 1487Reputation: 1487Reputation: 1487Reputation: 1487Reputation: 1487Reputation: 1487Reputation: 1487Reputation: 1487Reputation: 1487Reputation: 1487
You still don't really want "random".

One way to handle things is to have all of them START the cron job at the same time.

Then each one uses "sleep n" where n is determined from either a lookup table, or just use a random number of seconds to delay.

One reason a random delay is not exactly desired:

Since all the systems are identical, the updates to the systems will also be identical. If they all start at the same time, the server will load the data to be sent to each system in the system buffers - once.

After that they all get data from the same buffer, reducing the I/O by a factor of the number of systems being updated.

I observed this in a lab - all the systems were being installed, but the instructor said they all needed to be started simultaneously (within a second or two), or it would take many hours due to the disk thrashing caused by rereading data that had already been read and sent out. As it was, the reinstall only took 20 minutes....
 
Old 05-02-2014, 10:31 AM   #7
jyoung4
LQ Newbie
 
Registered: Apr 2006
Location: Minneapolis, Minnesota, USA
Posts: 16

Rep: Reputation: 1
Quote:
Originally Posted by jimbo1954 View Post
Hi Jim,
That will get the job to run on a certain day of the week, then the cron job will run only on that day. That will mean (if I understand your proposal correctly) that since all the systems are clones of each other, the problem will remain that they all do the update at the same time, which will flood the server with requests...or did I misunderstand? (Highly possible)
Yeah - there is a misunderstanding. My thought was that the contents of /myupdate.day on each machine would be different. So /myupdate.day on host1 would have "Sun" in it, the one on host2 would have "Mon" and so on.

If that's not possible, then will each host will have a different host name? If so, you could create a set of files, with the set containing one file for each host - /myupdate.day.host1, /myupdate.day.host2, /myupdate.day.host3, ... where host1, host2, host3, ... are the host names given to each system. All the files for the set would be present on all the systems and all the sets would be identical. Put a different day in each file in the set (/myupdate.day.host1 has "Sun", /myupdate.day.host2 has "Mon", ...) and have the cron job check what host it's running on and then check the file for that host to determine if it should run the update today or not.
 
Old 05-08-2014, 03:30 AM   #8
jimbo1954
Member
 
Registered: Oct 2006
Location: High Wycombe, Bucks, UK.
Distribution: Debian and Fedora Core in equal measure
Posts: 264

Original Poster
Rep: Reputation: 33
Thanks for the replies and suggestions: it looks like my search for a simple "silver bullet" is not going to deliver, and I will have to get up off my tail and do some work/thinking...
 
  


Reply

Tags
cron random


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Debian daily cron job won't run, but does run in cron.hourly. sandersch Linux - General 7 05-24-2012 02:50 AM
how to abort cron if the previous cron job not yet finished? Winanjaya Linux - Newbie 2 05-22-2012 07:44 PM
[SOLVED] Python: equivalent programs with different runtimes ColInvictus Programming 4 12-01-2011 03:04 PM
linux cron job duplicate job question cpthk Linux - Newbie 4 09-11-2009 09:52 PM
adding a perl script to cron.daily / cron.d to setup a cron job CrontabNewBIE Linux - Software 6 01-14-2008 09:16 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 12:34 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration