Hi!

I am running a script per nohup ./script.sh:
#!/bin/bash
Do.stuff -some_parameters &
Do.stuff -some_parameters

I have 64 cores at my disposal, so i start a script with 60 lines (ignoring 4 cores to be free for allday-jobs). I have to do this for ~800 jobs. A job is done within 20 hours, so each morning i come in, start a script with 60 lines, and come back the other morning.

Today i was thinking...would it be possible to query my jobs? So, whenever a core is free, start the next job from the list given in the script?




I know, i am expected to have already tried things out. But seriously, i cant. This is way beyond my skill level. I am just asking to avoid coming to the lab this weekend

I think you'd be much better off using a scheduling system like torque/maui/moab. All jobs get submitted to the scheduler, and the scheduler then automatically farms them out based on memory requirements, system load, etc.

At a bare minimum, you could check the load by parsing the output of "uptime". If it's less than 64 it means you have an available processor. Just make sure you give it time to update, since it's a load average it takes some time to react to load changes.

Thanks! Ill ask my system admin if he can install such a thing. Seems like a must-have-tool anyway (especially when many coworkers are asking for cpu-time).