LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (https://www.linuxquestions.org/questions/programming-9/)
-   -   Parallel Shell scripts to execute using LAM?? (https://www.linuxquestions.org/questions/programming-9/parallel-shell-scripts-to-execute-using-lam-700115/)

vallme2003 01-27-2009 12:09 AM
Parallel Shell scripts to execute using LAM??
 
Guys i want to know whether we can write a shell script which can utilize 2 CPU's of a cluster...I heard its possible through C Shell ( Ref : http://www.gnu-darwin.org/gdc/ccp4/parallelization/ ) But im familiar with bourne shell only....So is there a possibility to write Parallel Scripts..Expecting ur Answer.....:newbie:

colucix 01-27-2009 05:00 PM
You can be interested in "PPSS - Parallel Processing Shell Script". It is a single bash script spawning multiple threads on multiple cores.

vallme2003 01-28-2009 01:11 AM
My Case
 
I have a doubt again...I already heard about PPSH script..Guess it was written to utilize the multicores...In my case i have build a beowulf cluster which is originally 2 Cpu's but virtually a single CPU...Plus i use a live distro to setup my cluster..So not sure how the script will detect it? (May be as Single Cluster CPU or as Single Node)...Ur Suggestions Guys?

colucix 01-28-2009 01:41 AM
Indeed, looking at the source code, the script ppss.sh has a weak method to retrieve the number of CPUs:
Code:
cat /proc/cpuinfo | grep "cpu cores" | cut -d ":" -f 2 | uniq | sed -e s/\ //g
on a system with multiple CPUs this will detetct only the number of cores for each CPU, resulting in a number of available parallel threads which is less than the total sum of the cores. I tested it on a 2 XEON QUAD CORE machine, and obviously it detects only 4 cores instead of the actual 8. You can try to modify the code to reveal the actual number of cores using a method specific to your beowulf build.

Louwrentius 03-09-2009 06:50 PM
Quote:
Originally Posted by colucix (Post 3423697)
Indeed, looking at the source code, the script ppss.sh has a weak method to retrieve the number of CPUs:
Code:
cat /proc/cpuinfo | grep "cpu cores" | cut -d ":" -f 2 | uniq | sed -e s/\ //g
on a system with multiple CPUs this will detetct only the number of cores for each CPU, resulting in a number of available parallel threads which is less than the total sum of the cores. I tested it on a 2 XEON QUAD CORE machine, and obviously it detects only 4 cores instead of the actual 8. You can try to modify the code to reveal the actual number of cores using a method specific to your beowulf build.
Hi,

I'm the author of PPSS and I'd like to inform you that this bug has been fixed, in my opinion in the new (distributed) version of PPSS.

If you find such bugs in the future, it would be a great help if an issue is created at the website of ppss.

http://code.google.com/p/ppss

With regards.

colucix 03-09-2009 07:02 PM
Nice to meet you, Louwrentius. Indeed I was too focused on replying to this thread and did not think about reporting a bug to the developer's site. I will accomplish in the future. Thank you.

Louwrentius 03-12-2009 05:49 PM
Quote:
Originally Posted by colucix (Post 3470232)
Nice to meet you, Louwrentius. Indeed I was too focused on replying to this thread and did not think about reporting a bug to the developer's site. I will accomplish in the future. Thank you.
Never mind and thanks anyway ;)


All times are GMT -5. The time now is 04:37 PM.