Parallel Execution Capable Scripts?

irfanhab · 08-28-2004, 12:34 AM

Hi,

Can you make shell scripts which can perform to tasks at the same time! something like a parallel execution capable scripts in bash scripting, or do you have to make a program for that.

Dark_Helmet · 08-28-2004, 01:15 AM

Yes, you can, just like you can from a shell prompt. Inside your script, you can execute a command, and tell the script to execute it in the background with the '&' character at the end of the line. You can do that to spawn a specific program or to launch your own sub-scripts.

If the main script needs to wait until the spawned processes are finished, then you'll need to make some form of communication. It could be as simple as sitting in a loop waiting for a file to exist from each sub-process. Once they are all accounted for, then the sub-processes are finished, and the main script can continue.

irfanhab · 08-28-2004, 01:45 AM

Any sample script Dark_Helmet?

Dark_Helmet · 08-28-2004, 03:59 AM

Well.... pretty much like I said. Here's a very simple example.

Main script:

Code:

#!/bin/bash

# Spawn off two sub-scripts to perform some function.
# This script will wait until they both signal their job is complete
#   by creating an empty file named /var/run/sub_script1.done
#   and /var/run/sub_script2.done

# First, make sure the "done" files don't exist
# In case there was an abrupt end and we didn't
#   get a chance to cleanup the files
sub_script1_file="/var/run/sub_script1.done"
sub_script2_file="/var/run/sub_script2.done"

rm -f ${sub_script1_file} ${sub_script2_file}

# Launch the scripts
/path/to/directory/containing/sub_script1.bash ${sub_script1_file} &
/path/to/directory/containing/sub_script2.bash ${sub_script2_file} &

while [ ! -e ${sub_script1_file} -a ! -e ${sub_script2_file} ] ; do
  sleep 10
done

# Both sub-scripts are done, clean up the .done files
#   and continue processing
rm -f ${sub_script1_file} ${sub_script2_file}

# Do more stuff...

sub_script1.bash:

Code:

#!/bin/bash

# All this script does is write a list of all the files in the /etc
#   directory and all subdirectories
find /etc -type f > /var/run/sub_script1.data

# The find command is finished. Create an empty file to
#   signal we're done. We were given the path and the
#   name of the file to create as a command-line argument
touch $

sub_script2.bash:

Code:

#!/bin/bash

# This script grabs the slashdot.org homepage
cd /tmp
wget http://slashdot.org

# We're done, create the "done" file that was given as
#   a command-line argument
touch $1

jlliagre · 08-28-2004, 04:47 AM

Dark_Helmet, your example can be strongly simplified and enhanced by the use of the wait shell builtin command.

1) Remove the touch $1 in both subscripts (temporary file no more needed)

2) The main script is modified like this:

Code:

#!/bin/bash

# Spawn off two sub-scripts to perform some function.
# This script will wait until they both signal their job is complete

# Launch the scripts
/path/to/directory/containing/sub_script1.bash &
/path/to/directory/containing/sub_script2.bash &

# wait for the scripts to complete
wait

# Do more stuff...

Dark_Helmet · 08-28-2004, 06:04 AM

Yeah, that's hands-down better than mine. I should have looked up the documentation for the wait command. I'm used to putting "wait pid" that I didn't even think about it, and I figured grep'ing ps output for child process pid's would have been going overboard.

Thanks for pointing it out.