Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Distribution: Debian and Fedora Core in equal measure
Posts: 264
Rep:
Randomising Cron Job runtimes
I will be building and deploying a number of devices that will be requesting updates once a week from a central server.
I want to be able to build the devices as standard clones, but I don't want them all to try to grab updates at the same time, so I need to randomise the running of the job that grabs the updates.
I see there being two possibilities, one being to actually randomise the time the cron job runs, so it runs once a week on a random day, or the other, potentially better way which is that the job is run at bootup, then at exactly one-weekly intervals after that. The chances of many of the devices being booted at the exact same time are minimal, and the interval would not vary, being just 1 week this way, so it looks like a better way forward.
My problem is that I'm not sure how to achieve this. I've seen a whole bunch of stuff on Google that describes creating jobs that cron runs and that in turn run the update job, but they are all fairly old and very convoluted (=prone to failure!) so I'm asking really to see if there is a simple, elegant way of achieving my aims.
What I would do is create a file somewhere containing the day you want this particular server to do it's update. For example:
echo Thu > /myupdate.day
Then set your cron job to run every day and check to see if today is the same day of the week as /myupdate.day. If it is, do the update, otherwise quietly go away. Since I'm a big fan of log files, I'd probably write a log entry listing the date and a short message indicating that it either did run the update or didn't.
Distribution: Debian and Fedora Core in equal measure
Posts: 264
Original Poster
Rep:
Hi Jim,
That will get the job to run on a certain day of the week, then the cron job will run only on that day. That will mean (if I understand your proposal correctly) that since all the systems are clones of each other, the problem will remain that they all do the update at the same time, which will flood the server with requests...or did I misunderstand? (Highly possible)
I would rather randomize the list of hosts (probably containing dummy elements) and fire up the jobs on a regular base taking always the next element of that list.
One way to handle things is to have all of them START the cron job at the same time.
Then each one uses "sleep n" where n is determined from either a lookup table, or just use a random number of seconds to delay.
One reason a random delay is not exactly desired:
Since all the systems are identical, the updates to the systems will also be identical. If they all start at the same time, the server will load the data to be sent to each system in the system buffers - once.
After that they all get data from the same buffer, reducing the I/O by a factor of the number of systems being updated.
I observed this in a lab - all the systems were being installed, but the instructor said they all needed to be started simultaneously (within a second or two), or it would take many hours due to the disk thrashing caused by rereading data that had already been read and sent out. As it was, the reinstall only took 20 minutes....
Hi Jim,
That will get the job to run on a certain day of the week, then the cron job will run only on that day. That will mean (if I understand your proposal correctly) that since all the systems are clones of each other, the problem will remain that they all do the update at the same time, which will flood the server with requests...or did I misunderstand? (Highly possible)
Yeah - there is a misunderstanding. My thought was that the contents of /myupdate.day on each machine would be different. So /myupdate.day on host1 would have "Sun" in it, the one on host2 would have "Mon" and so on.
If that's not possible, then will each host will have a different host name? If so, you could create a set of files, with the set containing one file for each host - /myupdate.day.host1, /myupdate.day.host2, /myupdate.day.host3, ... where host1, host2, host3, ... are the host names given to each system. All the files for the set would be present on all the systems and all the sets would be identical. Put a different day in each file in the set (/myupdate.day.host1 has "Sun", /myupdate.day.host2 has "Mon", ...) and have the cron job check what host it's running on and then check the file for that host to determine if it should run the update today or not.
Distribution: Debian and Fedora Core in equal measure
Posts: 264
Original Poster
Rep:
Thanks for the replies and suggestions: it looks like my search for a simple "silver bullet" is not going to deliver, and I will have to get up off my tail and do some work/thinking...
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.