LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-17-2014, 12:48 AM   #1
hoi
LQ Newbie
 
Registered: Jun 2013
Posts: 29

Rep: Reputation: Disabled
execution many job in one node


Lets, there are four nones and each node has 32 processors. If I submit 4 job with 8 processors per job then it distributes to each 4 node. So, the uses of node is 100% but the uses of processor is 50%. Is it possible to distribute 4 job in two nodes, that means uses of node is 50% and also the uses of processor is 50% ??
 
Old 05-17-2014, 06:45 PM   #2
btmiller
Senior Member
 
Registered: May 2004
Location: In the DC 'burbs
Distribution: Arch, Scientific Linux, Debian, Ubuntu
Posts: 4,284

Rep: Reputation: 371Reputation: 371Reputation: 371Reputation: 371
What software are you using to distribute the jobs to nodes. There are many batch scheduler available such as TORQUE, Open Grid Engine, SLURM, LSF, etc. The way that job distribution is configured depends heavily on which one (if any) is in use. You need to provide more details about how you're launching and distributing jobs.
 
Old 05-19-2014, 08:15 AM   #3
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,688

Rep: Reputation: 1259Reputation: 1259Reputation: 1259Reputation: 1259Reputation: 1259Reputation: 1259Reputation: 1259Reputation: 1259Reputation: 1259
In addition to the type batch system, it can also depend on the configuration of that batch system.

Most have ways to designate queues that define the number of CPUs available (say 32), and only allow one queue on a server - thus multiple jobs on one compute server are not possible. This is usually done to prevent overloading a particular compute resource. If the queue was configured for two concurrent jobs, then the CPUs would also tend to be restricted to 16 CPUs each... This would allow the fastest throughput.

Now on a low priority configuration NOT configured for high throughput (for testing jobs for example) the configuration COULD allow for two concurrent jobs, on a compute service that has only 32 CPUs available with jobs that use 32 CPUs - at the cost of potential thrashing.

The users desire may conflict with site policy, and available queue structures.

Where I worked, we had high priority queues and low priority queues - and a "whatever is left over" queue. In our case, the 16 CPUs (it was a Cray mainframe) had queues for 4 cpus (high priority, only one job possible), a queue for 4 more CPUS, but with two jobs possible (a medium priority), a production queue of 8 CPUs.... and for very high priority, a queue with 16 CPUs, but it was idled (jobs could be submitted, but they wouldn't run - and it required special permission to use it as operations had to stop all other queues before enabling this one to run).

The "whatever was left" was a low priority queue (it only ran if none of the higher priority queues had a job), and was permitted only 2 CPUs (but 4 concurrent jobs). Its purpose was just for testing.
 
  


Reply

Tags
linux, linux client, shell script, unix


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] issue with crontab job execution INDy2k Linux - Server 7 04-24-2014 01:38 AM
how to know the node names of a Linux server so I can install a job scheduler Zope Linux - General 1 01-27-2013 09:32 PM
How to set the SGE job execution order cliffyao Linux - Newbie 1 10-16-2010 10:46 PM
[SOLVED] error while installing execution node in sun grid me_spearhead Linux - General 1 05-27-2009 07:30 AM
guaranteed scheduled job execution Xstack Programming 4 02-15-2005 10:30 PM


All times are GMT -5. The time now is 07:19 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration