Welcome to the most active Linux Forum on the web.
Go Back > Forums > Linux Forums > Linux - Newbie
User Name
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!


  Search this Thread
Old 03-19-2015, 10:33 AM   #1
LQ Newbie
Registered: Sep 2013
Posts: 21

Rep: Reputation: Disabled
Querying up jobs in a multi-core environment, via a shell script


I am running a script per nohup ./
Do.stuff -some_parameters &
Do.stuff -some_parameters

I have 64 cores at my disposal, so i start a script with 60 lines (ignoring 4 cores to be free for allday-jobs). I have to do this for ~800 jobs. A job is done within 20 hours, so each morning i come in, start a script with 60 lines, and come back the other morning.

Today i was thinking...would it be possible to query my jobs? So, whenever a core is free, start the next job from the list given in the script?

I know, i am expected to have already tried things out. But seriously, i cant. This is way beyond my skill level. I am just asking to avoid coming to the lab this weekend

Last edited by nouse; 03-19-2015 at 11:19 AM.
Old 03-19-2015, 11:36 AM   #2
LQ Guru
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,527

Rep: Reputation: 2108Reputation: 2108Reputation: 2108Reputation: 2108Reputation: 2108Reputation: 2108Reputation: 2108Reputation: 2108Reputation: 2108Reputation: 2108Reputation: 2108
I think you'd be much better off using a scheduling system like torque/maui/moab. All jobs get submitted to the scheduler, and the scheduler then automatically farms them out based on memory requirements, system load, etc.

At a bare minimum, you could check the load by parsing the output of "uptime". If it's less than 64 it means you have an available processor. Just make sure you give it time to update, since it's a load average it takes some time to react to load changes.

Last edited by suicidaleggroll; 03-19-2015 at 11:38 AM.
Old 03-19-2015, 11:59 AM   #3
LQ Newbie
Registered: Sep 2013
Posts: 21

Original Poster
Rep: Reputation: Disabled
Thanks! Ill ask my system admin if he can install such a thing. Seems like a must-have-tool anyway (especially when many coworkers are asking for cpu-time).


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Script to run XBMC in a multi-monitor environment refuses to work correctly Big-Blue Linux - Software 0 07-25-2013 10:45 PM
Shell script for rDNS with for loop querying each line of a text file wolfsden3 Programming 3 06-11-2012 03:20 PM
how to write a shell script for submitting multiple jobs sequentially cliffyao Programming 4 05-27-2010 02:44 PM
Multi Core Environment - process migration chamila1986 Programming 5 10-04-2009 07:19 AM
ps/jobs in shell script paraiso Linux - Newbie 3 04-22-2005 09:04 AM > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 12:07 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration