Bash script: restart each instance after it has ended within a loop

Jajamd · 07-01-2012, 05:26 PM

Hello everyone,

I'm writing a little script to have several instances of curl download different things at the same time. This is what I have so far:

Code:

#!/bin/bash
while :
do
curl command1 &
curl command2 &
curl command3
sleep 1
done

The three instances of curl are running at the same time. This is what I wanted.
But all of them are only restarted after the last instance of curl (curl command 3) has finished. I need to find a way to restart each instance of curl after it has finished instead of having to wait for the last one to finish.

I hope it makes sense !

Thank you very much for your help.

pixellany · 07-01-2012, 05:34 PM

I'd take it one command at a time---for example, what is supposed to happen after "curl command1" finishes? If it's just supposed to run again, why not put it inside its own loop?

BUT, why would you want to keep running the same curl command repeatedly? On a lot of sites, if you do that, you'll probably be kicked off rather quickly.

In the global sense, what are you trying to accomplish?

Jajamd · 07-01-2012, 05:38 PM

It's not the same curl command repeatedly. I'm adding a variable each time.
I'm going to follow your suggestion and put every command inside its own loop. Thanks for the tip

dru8274 · 07-01-2012, 08:15 PM

I am sure there is more than one way to do this. But I've put some pseudo-code down here, as to how it might be done.

Basically, I've decided to store the pid of each backgrouded curl command in an array, and to periodically poll each process pid with a kill -0 command, to see if it still running. And if not, then the nextcurl() function will be called, to start up the next curl command.

Furthermore, I suspect that backgrounded processes do not always die when their parent is killed. So a trap function may be necessary here too.

Its all quite rough and untested, bad-habits and errors in there, no doubt.

Code:

# Run next curl command. $1 = the job number
nextcurl()  {
    foo=$1

    # Generate and run your next curl command. Pseudo-code. 
    curl command1 1>/dev/null 2>&1 &    # disown

    # Store the backgrounded curls pid number
    pidx[$foo]=$!
}

pidx=( 0 0 0 )

# Some kind of trap function may be needed here, to ensure that
# all children processes are killed at exit.

trap ' for numx in "${pidx[$@]}" ; do kill $numx ; done ; exit ; ' INT

# The pids of running curls are each tested, to see if they are still running. If not,
# then the nextcurl function is called, to restart the next curl command

while :
do
    for varx in {1..3}  do
        if  [[ ${pidx[$varx]} -eq 0 ]] || ! kill -0 ${pidx[$varx]} 1>/dev/null 2>&1
        then
            nextcurl $varx
        fi
    done
    sleep 1
done