LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 06-01-2011, 12:08 AM   #1
choconlangthang
LQ Newbie
 
Registered: Dec 2010
Posts: 8

Rep: Reputation: 0
forcing running multiple CPU on server


Hi,
There's a blade server, I have to ssh to it every time to run my program. And the program got run on one cpu. If I want to run the program N times in parallel, I have to open N ssh terminal, which is painful. Is there anyway I can force the server to run N times parallely by script?

(I don't mind rewrite my program in java code if that helps. I tried multithreading but it seemed not working, still only one cpu got running.)
 
Old 06-01-2011, 12:37 AM   #2
evo2
Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,527

Rep: Reputation: 1218Reputation: 1218Reputation: 1218Reputation: 1218Reputation: 1218Reputation: 1218Reputation: 1218Reputation: 1218Reputation: 1218
Hi,

here is an example onliner assuming you want to run 16 jobs and are using bash.

Code:
ssh somemachine 'for i in {1..16} ; do someprogram >& out$i.log & done'
Evo2.
 
Old 06-01-2011, 12:38 AM   #3
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,000
Blog Entries: 11

Rep: Reputation: 893Reputation: 893Reputation: 893Reputation: 893Reputation: 893Reputation: 893Reputation: 893
The question is somewhat vague ... a simple way would be to background the
process (that is, if it doesn't require user input while it runs).

Code:
#!/bin/sh
prog&
prog&
prog&
prog&
...

Cheers,
Tink
 
Old 06-01-2011, 12:59 AM   #4
choconlangthang
LQ Newbie
 
Registered: Dec 2010
Posts: 8

Original Poster
Rep: Reputation: 0
thanks for reply evo2 and Tinkster.
To be specific, the way I normally run my program, say, for 2 times, with 2 input sets is

SSH to server then
Code:
java myPro input1.xml
SSH to server then
Code:
java myPro input2.xml
so how do I achieve that? tks
 
Old 06-01-2011, 01:06 AM   #5
evo2
Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,527

Rep: Reputation: 1218Reputation: 1218Reputation: 1218Reputation: 1218Reputation: 1218Reputation: 1218Reputation: 1218Reputation: 1218Reputation: 1218
Hi,

Code:
ssh somemachine 'for i in {1..2} ; do java myPro input$i.xml & done'
If your progam produces output on stdout it would be best to pipe it to a log file.

Eg.
Code:
ssh somemachine 'for i in {1..2} ; do java myPro input$i.xml >& output$i.log & done'
Evo2.
 
1 members found this post helpful.
Old 06-01-2011, 01:23 AM   #6
choconlangthang
LQ Newbie
 
Registered: Dec 2010
Posts: 8

Original Poster
Rep: Reputation: 0
tks evo2, work like a champ. However I'm still curious, the programs run, but my prompt is still at my machine (like me@mymachine rather than me@server)?
 
Old 06-01-2011, 01:26 AM   #7
Nylex
LQ Addict
 
Registered: Jul 2003
Location: London, UK
Distribution: Slackware
Posts: 7,464

Rep: Reputation: Disabled
Presumably that's because ssh runs the command (essentially everything in between '') on the remote machine and then quits. You could of course put those commands in a script, log in and run that script.
 
1 members found this post helpful.
Old 06-01-2011, 10:33 AM   #8
sundialsvcs
Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 5,425

Rep: Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159
You can control how many instances of a process are running, but you can't control which CPU(s) it may run on at any particular time.

(I'm assuming that you're not dabbling with "CPU affinity ..." That is Very Advanced Magick.)
 
Old 07-11-2011, 03:44 PM   #9
tlhonmey
LQ Newbie
 
Registered: Nov 2008
Posts: 14

Rep: Reputation: 4
I would highly recommend that you install the program "screen".

This is a very nifty little tool for doing exactly the kind of thing you want to do. You ssh in once, and then start as many screen sessions as you like to run your different instances of the program. Additionally, if your connection gets interrupted, screen will simply detach and keep running, instead of killing your program.

Screen's man page has the complete set of options for creating and controlling screen sessions. I recommend using the -S option to give your session a sensible name so that it's easy to keep track of which is which.


This assumes, of course that you're permitted to install things on the server. However it's a fairly trivial program to install and may even be installed already. If it's not, then whoever's managing the server may well be interested in such a thing as well, so asking for it's probably not too much of a long-shot.

Last edited by tlhonmey; 07-11-2011 at 03:45 PM.
 
Old 07-11-2011, 04:39 PM   #10
baxzius
Member
 
Registered: Jan 2010
Location: India
Distribution: ubuntu
Posts: 134
Blog Entries: 1

Rep: Reputation: 21
A example:

shell script: run a batch of N commands in parallel, wait for all to finish, run next N
Task: run blocks consisting of 3-5 commands (in parallel/background). Example block:
dd if=/dev/urandom of=/mnt/1/x bs=1024 count=1024000000 &
dd if=/dev/urandom of=/mnt/2/x bs=1024 count=1024000000 &
dd if=/dev/urandom of=/mnt/3/x bs=1024 count=1024000000 &

When it's done, next block should run. I suppose, this can be done via lock files:
task1.sh:
real_task1 real_param1 ; rm /var/lock/myscript/task1.lock

task2.sh:
real_task2 real_param1 ; rm /var/lock/myscript/task2.lock

taskgen.sh:
# loop
# while directory isn't empty - wait...
gen_tasks.pl # build task files from some queue
for i in 1 2 3; do touch /var/lock/myscript/task$i.lock ; done
./task1.sh &
./task2.sh &
./task3.sh &
# if task1.sh doesn't exits then exit, else loop waits for files to be deleted

A number of methods to check if the directory is empty can be found here, don't sure which to use;
 
Old 07-12-2011, 02:12 AM   #11
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,311

Rep: Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040Reputation: 2040
try the bash 'wait' cmd
Quote:
wait [n ...]
Wait for each specified process and return its termination status. Each n may be a process ID or a job specification; if a job spec is given, all processes in that job's pipeline are waited for. If n is not given, all currently active child processes are waited for, and the return status is zero. If n specifies a non-existent process or job, the return status is 127. Otherwise, the return status is the exit status of the last process or job waited for.
http://linux.die.net/man/1/bash
 
  


Reply

Tags
cpu, multiple, server


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Running Multiple MySQL Server Instances in Parallel on a Linux Server LXer Syndicated Linux News 0 10-11-2010 01:00 PM
Forcing SSH Process to continue running sploit Linux - Networking 7 04-27-2009 12:23 PM
Server aburptly stops running High CPU usage PERL script after about 5 hours bpmee Linux - Server 3 05-18-2007 12:24 PM
Running multiple SSL websites on one server tictocdoc Linux - Networking 1 07-07-2004 04:26 PM


All times are GMT -5. The time now is 08:50 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration