Hi,
There's a blade server, I have to ssh to it every time to run my program. And the program got run on one cpu. If I want to run the program N times in parallel, I have to open N ssh terminal, which is painful. Is there anyway I can force the server to run N times parallely by script?

(I don't mind rewrite my program in java code if that helps. I tried multithreading but it seemed not working, still only one cpu got running.)

Hi,

here is an example onliner assuming you want to run 16 jobs and are using bash.

Evo2.

The question is somewhat vague ... a simple way would be to background the
process (that is, if it doesn't require user input while it runs).


Cheers,
Tink

thanks for reply evo2 and Tinkster.
To be specific, the way I normally run my program, say, for 2 times, with 2 input sets is

SSH to server then
SSH to server then
so how do I achieve that? tks

Hi,

If your progam produces output on stdout it would be best to pipe it to a log file.

Eg.
Evo2.

1 members found this post helpful.

tks evo2, work like a champ. However I'm still curious, the programs run, but my prompt is still at my machine (like me@mymachine rather than me@server)?

Presumably that's because ssh runs the command (essentially everything in between '') on the remote machine and then quits. You could of course put those commands in a script, log in and run that script.

1 members found this post helpful.

You can control how many instances of a process are running, but you can't control which CPU(s) it may run on at any particular time.

(I'm assuming that you're not dabbling with "CPU affinity ..." That is Very Advanced Magick.)

I would highly recommend that you install the program "screen".

This is a very nifty little tool for doing exactly the kind of thing you want to do. You ssh in once, and then start as many screen sessions as you like to run your different instances of the program. Additionally, if your connection gets interrupted, screen will simply detach and keep running, instead of killing your program.

Screen's man page has the complete set of options for creating and controlling screen sessions. I recommend using the -S option to give your session a sensible name so that it's easy to keep track of which is which.


This assumes, of course that you're permitted to install things on the server. However it's a fairly trivial program to install and may even be installed already. If it's not, then whoever's managing the server may well be interested in such a thing as well, so asking for it's probably not too much of a long-shot.

A example:

shell script: run a batch of N commands in parallel, wait for all to finish, run next N
Task: run blocks consisting of 3-5 commands (in parallel/background). Example block:
dd if=/dev/urandom of=/mnt/1/x bs=1024 count=1024000000 &
dd if=/dev/urandom of=/mnt/2/x bs=1024 count=1024000000 &
dd if=/dev/urandom of=/mnt/3/x bs=1024 count=1024000000 &
When it's done, next block should run. I suppose, this can be done via lock files:
task1.sh:
real_task1 real_param1 ; rm /var/lock/myscript/task1.lock

task2.sh:
real_task2 real_param1 ; rm /var/lock/myscript/task2.lock

taskgen.sh:
# loop
# while directory isn't empty - wait...
gen_tasks.pl # build task files from some queue
for i in 1 2 3; do touch /var/lock/myscript/task$i.lock ; done
./task1.sh &
./task2.sh &
./task3.sh &
# if task1.sh doesn't exits then exit, else loop waits for files to be deleted
A number of methods to check if the directory is empty can be found here, don't sure which to use;

try the bash 'wait' cmd
http://linux.die.net/man/1/bash