consecutive jobs, different users, high cpu load
 

Hello everybody

I am new here, this is my first post. I know something about linux, although not too much. I have a problem, looking for solutions.
My sistem is a Centos 5.4, 64 bit, multiCPU (intel xeons). There is a program installed on it, that will execute high cpu consuming tasks (all cpu's at once), for different amounts of time (optimizations, that is..).
I need to make it available for different users to somehow queue this tasks, so that only one at a time is executed.
Tasks are given by ssh (like "command >& logfile &").
I only need to queue this commands (from different users) so that the commands are executed one at a time.
I know commands like cron, at, and batch, but they does not satisfy my needs (or I don't know yet how to do it).
Any help is appreciated. Thank you.

Greetingz and Welcome to LQ!

It sounds like SQS is something that could help you out.

However, if you're adamant about keeping things as close to "as-is" in your environment, then you might need to create a custom solution. Perhaps something like this;

Directory Structure
/path/to/jobs
If your users are sloppy, this should be on a filesystem separate from the important ones.
/path/to/jobs/old
Because keeping the old jobs might prove useful.
/path/to/jobs/new
Where as-of-yet un-run jobs go.
/path/to/jobs/running
Where currently running job(s) go.

Create a "batch_schedule.sh" script that does the following;
Take input file "batch_job.in" and copy it's contents to "/path/to/jobs/new/`seconds-since-the-epoch`.job".

Create a "batch_run.sh" script and have it called by cron every minute.
This script would check "/path/to/jobs/running", if empty (or if less-than-$MAX jobs running), it would move a job from "/path/to/jobs/new" and then start running that job.
If there was a *.job file in /path/to/jobs/running (or $MAX jobs currently running), then do nothing (maybe log that).

NOTE: I'm not sure how your users would adept to change, but if experience has taught me anything, it's that non-technical people loath learning a new process. So test it, document it, put that document on an Intranet website, then email a bookmark to everyone.
Oh, and only change things once a year or so.

EDIT: If you need something with more "oomph", take a look at TORQUE.