Job launcher and manager

Gamma_User · 05-05-2012, 10:35 AM

The problem:
I have a main program and several input files and I need to run the main program for each input file in a cluster. This cluster may be in use by other people, so I need to care about the number of jobs per machine. Launching each job manually is tedious and error prone (since I may run the same input file twice, for instance). Besides I do not have I fine control about which job has been finished and when to launch the remainder.

The question:
There exist any free software that launches and manages the jobs?

KenJackson · 05-05-2012, 12:54 PM

Are you concerned about speed? If not, you could just run them all sequentially from one script and one cron job. Maybe use a script like this for cron to execute.

Code:

#/bin/sh
INPUT_FILES=(
file1.txt
"file 2.txt"
file3.txt
)

for f in "${INPUT_FILES[@]}"; do
    /path/to/program "$f"
done

Gamma_User · 05-06-2012, 08:40 AM

Yes, I am concerned about the speed, because each job runs a computationally expensive simulation. So, the script solves partially my problem. Thanks anyway...

Quote:

Originally Posted by KenJackson

Are you concerned about speed? If not, you could just run them all sequentially from one script and one cron job. Maybe use a script like this for cron to execute.

Code:

#/bin/sh
INPUT_FILES=(
file1.txt
"file 2.txt"
file3.txt
)

for f in "${INPUT_FILES[@]}"; do
    /path/to/program "$f"
done