Randomising Cron Job runtimes
 

I will be building and deploying a number of devices that will be requesting updates once a week from a central server.

I want to be able to build the devices as standard clones, but I don't want them all to try to grab updates at the same time, so I need to randomise the running of the job that grabs the updates.

I see there being two possibilities, one being to actually randomise the time the cron job runs, so it runs once a week on a random day, or the other, potentially better way which is that the job is run at bootup, then at exactly one-weekly intervals after that. The chances of many of the devices being booted at the exact same time are minimal, and the interval would not vary, being just 1 week this way, so it looks like a better way forward.

My problem is that I'm not sure how to achieve this. I've seen a whole bunch of stuff on Google that describes creating jobs that cron runs and that in turn run the update job, but they are all fairly old and very convoluted (=prone to failure!) so I'm asking really to see if there is a simple, elegant way of achieving my aims.

Thanks for any suggestions

One suggestion for how to randomize cron jobs
 

What I would do is create a file somewhere containing the day you want this particular server to do it's update. For example:

echo Thu > /myupdate.day

Then set your cron job to run every day and check to see if today is the same day of the week as /myupdate.day. If it is, do the update, otherwise quietly go away. Since I'm a big fan of log files, I'd probably write a log entry listing the date and a short message indicating that it either did run the update or didn't.

Hi Jim,
That will get the job to run on a certain day of the week, then the cron job will run only on that day. That will mean (if I understand your proposal correctly) that since all the systems are clones of each other, the problem will remain that they all do the update at the same time, which will flood the server with requests...or did I misunderstand? (Highly possible:))

What about writing a script that will create AT jobs at random intervals?

I would rather randomize the list of hosts (probably containing dummy elements) and fire up the jobs on a regular base taking always the next element of that list.

You still don't really want "random".

One way to handle things is to have all of them START the cron job at the same time.

Then each one uses "sleep n" where n is determined from either a lookup table, or just use a random number of seconds to delay.

One reason a random delay is not exactly desired:

Since all the systems are identical, the updates to the systems will also be identical. If they all start at the same time, the server will load the data to be sent to each system in the system buffers - once.

After that they all get data from the same buffer, reducing the I/O by a factor of the number of systems being updated.

I observed this in a lab - all the systems were being installed, but the instructor said they all needed to be started simultaneously (within a second or two), or it would take many hours due to the disk thrashing caused by rereading data that had already been read and sent out. As it was, the reinstall only took 20 minutes....

Yeah - there is a misunderstanding. My thought was that the contents of /myupdate.day on each machine would be different. So /myupdate.day on host1 would have "Sun" in it, the one on host2 would have "Mon" and so on.

If that's not possible, then will each host will have a different host name? If so, you could create a set of files, with the set containing one file for each host - /myupdate.day.host1, /myupdate.day.host2, /myupdate.day.host3, ... where host1, host2, host3, ... are the host names given to each system. All the files for the set would be present on all the systems and all the sets would be identical. Put a different day in each file in the set (/myupdate.day.host1 has "Sun", /myupdate.day.host2 has "Mon", ...) and have the cron job check what host it's running on and then check the file for that host to determine if it should run the update today or not.

Thanks for the replies and suggestions: it looks like my search for a simple "silver bullet" is not going to deliver, and I will have to get up off my tail and do some work/thinking...:(