LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 06-22-2012, 02:40 PM   #1
fantasygoat
Member
 
Registered: Sep 2009
Posts: 119

Rep: Reputation: 17
Cron Clustering / Clustered Job Scheduling


I have a need to manage cron jobs across a bunch of servers - setting the schedule, and having jobs automatically migrate to other servers if the server fails.

I've not had any luck finding something simple to handle this - mostly it seems to be large, complicated installs to handle 1000+ servers in job control. I just need a basic web interface and to handle a few dozen machines, max.

Can anyone recommend something before I bake my own?
 
Old 06-22-2012, 03:38 PM   #2
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora
Posts: 3,935
Blog Entries: 5

Rep: Reputation: Disabled
I have a similar situation -- a small group of DB servers (for which many of the "enterprise" solutions are massive overkill).

I use Webmin's cluster modules for this. I can copy files easily, run shell commands simultaneously, or schedule cronjobs across the defined cluster.

http://www.webmin.com/cgi-bin/search...tegory=Cluster

I like. YMMV.
 
Old 06-22-2012, 03:39 PM   #3
rch
Member
 
Registered: Feb 2003
Location: Santa Clara,CA
Distribution: Mandriva
Posts: 909

Rep: Reputation: 48
Maybe something like puppet?
http://puppetlabs.com/
 
Old 06-23-2012, 05:27 AM   #4
Reuti
Senior Member
 
Registered: Dec 2004
Location: Marburg, Germany
Distribution: openSUSE 15.2
Posts: 1,339

Rep: Reputation: 260Reputation: 260Reputation: 260
Maybe the OP can define “manage cron jobs across a bunch of servers” more precisely. The Webmin module seems to distribute the same cron schedule to all servers, and puppet is more to install machines - none of them have the ability to schedule (batch) jobs across a bunch of machines AFAICS (I never used any of them though).

To schedule (batch) jobs across a cluster there are solutions like PBS/Torque or GridEngine (SGE/OGE) which might be an overkill, and in fact don’t have the ability for recurring cron jobs.

I would suggest to check GNUbatch (commercial support at XiSoft), one can define jobs which should run every day or in whatever interval (which is a cron job in the end, and GNUbatch is more like a replacement for cron than a full blow queuingsystem, as it lacks any prioritization of jobs per user), reschedule them if a run was lost, skip weekends, run the next instance on a remote machine... To avoid oversubscription of a machine, one could use a counter per machine to keep the (at the same time) running jobs under a certain limit.
 
Old 06-23-2012, 07:31 AM   #5
rch
Member
 
Registered: Feb 2003
Location: Santa Clara,CA
Distribution: Mandriva
Posts: 909

Rep: Reputation: 48
Quote:
Originally Posted by Reuti View Post
Maybe the OP can define “manage cron jobs across a bunch of servers” more precisely. The Webmin module seems to distribute the same cron schedule to all servers, and puppet is more to install machines - none of them have the ability to schedule (batch) jobs across a bunch of machines AFAICS (I never used any of them though).
Well as you pointed, you have never used Puppet. Here's a link which was the fourth result from a Google search.
 
Old 06-23-2012, 08:04 AM   #6
Reuti
Senior Member
 
Registered: Dec 2004
Location: Marburg, Germany
Distribution: openSUSE 15.2
Posts: 1,339

Rep: Reputation: 260Reputation: 260Reputation: 260
Quote:
Originally Posted by rch View Post
Well as you pointed, you have never used Puppet. Here's a link which was the fourth result from a Google search.
Thanks for the pointer, if I get it right it creates a crontab (or like in the pointed link: individual files) on all or certain nodes for which this class is included. But how can Puppet handle: run this job every 15 minutes on any (i.e. just one) of my 20 nodes in the cluster. It could be the same node like the last time, or any other which is currently free?

As the OP mentioned “having jobs automatically migrate to other servers if the server fails” it would mean to recreate crontab entries on other machines if one machine fails where certain jobs were defined on.
 
Old 06-23-2012, 08:20 AM   #7
fantasygoat
Member
 
Registered: Sep 2009
Posts: 119

Original Poster
Rep: Reputation: 17
GnuBatch looks like it might do what I want, but I can't get it to compile on CentOS.
 
Old 06-23-2012, 09:39 AM   #8
Reuti
Senior Member
 
Registered: Dec 2004
Location: Marburg, Germany
Distribution: openSUSE 15.2
Posts: 1,339

Rep: Reputation: 260Reputation: 260Reputation: 260
There is a mailing list for GNUBatch too. What errors do get? I remember that I had to install flex, bison, m4 and some other stuff to compile it on openSUSE.
 
Old 06-23-2012, 10:12 AM   #9
fantasygoat
Member
 
Registered: Sep 2009
Posts: 119

Original Poster
Rep: Reputation: 17
Adding flex, bison and m4 gets it to compile. Unfortunately it wants GTK installed as well. But thanks for the tip, I'll see if this does what I want.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Scheduling a CRON job to run on the first Sunday of the month New2Linux2 Linux - Software 10 11-08-2016 12:25 PM
[SOLVED] Cron job not scheduling smilemukul Linux - Newbie 16 01-20-2011 10:09 AM
Scheduling Cron Job chnlinux Linux - Newbie 5 10-16-2007 04:51 AM
LXer: Linux tip: Job scheduling with cron and at LXer Syndicated Linux News 0 07-21-2007 03:16 AM
scheduling monthly job in cron from debian sarge 3.1 sridhar11 Debian 2 10-28-2005 09:03 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 09:52 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration