Bash Command Substitution

dakensta · 11-28-2006, 05:48 AM

I am having a few problems with scripting a loop using command substitution, specifically with string expansion.

I want to run a query through a number of mysql hosts, but, I think, building the command as a string means that the arguments are presented as a single string rather than multiple strings

Specifically:

Code:

user=username
pass=password
hosts=(host1 host2 host3 host4)
for h in ${hosts@}
do
    res=$(mysql -u ${user} -p${password} -h ${h} -e "SELECT blah FROM bleh")
    # Do stuff with ${res}
done

This is not the only time this has come up and regardless of the command the output is always the commands help produced in case of incorrect arguments.

What I want to know is how to correctly build commands as strings and then pass them on for command substitution in a shell script. I have tried all sort of combinations but to no avail.

jschiwal · 11-28-2006, 06:00 AM

I looks like you have a few errors in the start of the script.

Code:

user=$username            # maybe you want an argument instead as in "user=$1" and "pass=$2"
pass=$password
hosts=(host1 host2 host3 host4)
for h in ${hosts[@]}

You are putting a string of hosts into the variable "hosts" and then converting it back into a string of hosts. You might want to do this if you will use ${hosts} again, or if you want the list of hosts near the top of a longer script so you can find and edit it easier. But for a 3 line script, you could just use:
for host in host1 host2 host3 host4; do
...
done

You assigned $password to $pass but used $password in the sql line.

You can also build the entire command in a variable, let's call it mysqlcmd. Then a line like:
${mysqlcmd}
will execute the cmd.

dakensta · 11-29-2006, 03:46 AM

The script was only an off the cuff example, I know it is not right, just trying to illustrate the bit I don't understand.

Quote:

You can also build the entire command in a variable, let's call it mysqlcmd. Then a line like:
${mysqlcmd}
will execute the cmd.

But this doesn't store the output in a variable, which is why I am using command substitution and furthermore, still does not correctly execute the command

Here is a complete example:

Code:

for i in 1 2
do
  daysAgo="'${i} days ago'" 
  echo ${daysAgo}                  # 1. Output is: '1 days ago'
  cmd="date --date="${daysAgo}" +%d/%m/%Y"
  echo ${cmd}                        # 2. Output is: date --date='1 days ago' +%Y-%m-%d
  ${cmd}                                # 3. Output is: date: too many non-option arguments: ago'
  $(${cmd})                            # 4. Output is: date: too many non-option arguments: ago'
done

Copy and pasting the output at 2. (from echo ${cmd}) into the command line directly successfully executes the command (i.e. the string is correctly built), giving 28/11/2006

fvu · 11-29-2006, 11:57 AM

eval does the trick:

Code:

#!/bin/bash
for i in 1 2; do
    cmd="date --date=\"$i days ago\""
    eval $cmd
done

See also: Uwe Waldmann - A Guide to Unix Shell Quoting - Common misconceptions
Before the shell executes a command, it performs the following operations (check the manual for details):

1. Syntax analysis (Parsing)
2. Brace expansion
3. Tilde expansion
4. Parameter and variable expansion
5. Command substitution
6. Arithmetic expansion
7. Word splitting
8. Filename expansion
9. Quote removal

It is important to realize that parsing takes place before parameter and command substitution. The result of a step n is subject to the next steps (n+), but the preceding steps (n-) are not re-executed. In other words: after for example variable expansion (step 4) the result is not re-parsed (step 1).

In the example of the command:

$cmd # cmd="date --date=\"1 days ago\""

the command date --date=\"$i days ago\" is the result of variable expansion (step 4). Quote removal (step 9) removes the backslashes and turns the command into date --date="$i days ago". But since parsing (step 1) already has taken place, "1 days ago" is not seen as 1 argument but as 3 separate arguments: "1, days and ago", which gives you the error message:

date: extra operand `ago"'
Try `date --help' for more information.

To force one more run through the parsing/expansion procedure, use:

eval $cmd

Now first $cmd is expanded (steps 1-9) to date --date="1 days ago". Then eval causes the expanded string to be re-parsed again (steps 1-9), resulting in the correct command:

date --date="1 days ago"
Wed Nov 29 08:11:46 CET 2006

Freddy Vulto
http://fvue.nl/wiki/Bash

dakensta · 11-30-2006, 10:53 AM

Thank you very much.

An excellent answer that has significantly helped fill gaps in my understanding.

gnashley · 11-30-2006, 03:10 PM

Try this:
hosts="host1 host2 host3 host4"
for h in $hosts