If a script is already running 3 times wait till one finishes before starting another

Techno Guy · 04-17-2009, 02:39 AM

Hi, it's me again

I'v been googleing for about the last hour and a bit with little success.

What I want:
This post is using similar code to my other post: http://www.linuxquestions.org/questi...lders.-714468/

Ok so with that other post, all was working fine, until I realized that my server can't really handle any more then 2 or 3 of the "forloop.sh" script running at any one time, and when I had something like 5-7 of them running at once my server pretty much froze

(yea I know, I have a pretty crappy server but it gets the job done... eventually

)

So I thought, well why not just add a queue in so that say only 3 of them can work at a time, and once one of the 3 have finished (if there are any waiting) start it.

From what I found out from google it seems like the "wait" command would be the way to go?

Thoughts:
Current code:

Code:

inotifywait --monitor --format %f -e moved_to -e create /demo/ | while read file; do echo "New files moved in $file";

/forloop.sh $file &

#something like this??
$!
wait if ($! > 3)....Or something.. I'm not sure how it would go :(


done

Looking forward to some awesome help

unSpawn · 04-17-2009, 03:10 AM

# Find processes by commandname, PIDs in array:
PROCS=($(pgrep /forloop.sh));
# If array has more than 2 elements then holdThis, else doSomething
[ ${#PROCS} -gt 2 ] && holdThis || doSomething
...maybe something like that?

BTW, you also might want to post the contents of your forloop script. Optimising things might not help a lot but it just might. For instance for some stuff I'll use a shell that is way quicker starting up compared to Bash like Ash.

colucix · 04-17-2009, 03:14 AM

Nope. That's not the correct way to use the wait statements. You have to do something like

Code:

wait $!

where $! is the PID of last job run in background. Since you want to run 3 jobs at once, you can store the PID of each job in a variable, then check the number of running processes and every time it is equal to 3, just wait for the first of the three to complete. Here is an example:

Code:

#!/bin/bash
count=0
inotifywait --monitor --format %f -e moved_to -e create /demo/ | while read file
do
  # Increment the count by one
  ((count++))

  echo "New files moved in $file";
  /forloop.sh $file &

  # Store the pid of last job in a variable (use indirect reference here)
  eval pid_$count=$!

  # Check the number of running processes
  if [ $(ps h -C forloop.sh | wc -l) -eq 3 ]
  then
    # build the name of the variable storing the pid of the first job
    pid=pid_$((count - 2))

    # wait for the job to complete
    wait ${!pid}
  fi
done

Techno Guy · 04-21-2009, 07:58 PM

Thanks colucix!

I gave it a go a few days ago, but it wasn't working proper, so today I had some time again and figured out why it wasn't working.

Turns out this line:

Code:

 if [ $(ps h -C forloop.sh | wc -l) -eq 3 ]

Should be like this (for my prepossess): I changed it so that it isn't equal to but greater then, because once I had more then 3 of that script running they would just start, but changing it to greater then fixed that

Code:

 if [ $(ps h -C forloop.sh | wc -l) -ge 3 ]

Thanks again for your help, I never thought it would be so few lines of code, I would of thought it to be about half a page, but you proved me wrong