Meaning of backslash in bash within here document and command substitution

moraxu · 04-02-2013, 11:50 AM

Hello,

I've questions regarding somewhat obscure rules on the meaning of backslash in bash shell in different contexts.

The following snippet is taken from here (p. 54):

Code:

The behavior of \ depends on whether it is escaped, strong-quoted, weak-quoted, or appearing within
command substitution or a here document.

# Simple escaping and quoting
echo \z # z
echo \\z # \z
echo '\z' # \z
echo '\\z' # \\z
echo "\z" # \z
echo "\\z" # \z

# Command substitution
echo `echo \z` # z
echo `echo \\z` # z
echo `echo \\\z` # \z
echo `echo \\\\z` # \z
echo `echo \\\\\\z` # \z
echo `echo \\\\\\\z` # \\z
echo `echo "\z"` # \z
echo `echo "\\z"` # \z

# Here document
cat <<EOF
\z
EOF # \z
cat <<EOF
\\z
EOF # \z

Simple escaping is rather obvious to me, but I have a question regarding old-syntax command substitution and here document.

1) From the bash manual:

Quote:

When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by ‘$’, ‘`’, or ‘\’. The first backquote not preceded by a backslash terminates the command substitution.

When backslash retains its literal meaning, it should appear as itself, right? Then why in this case:

Code:

echo `echo \z`

the output is z instead of \z? Same here:

Code:

$ var=test
$ echo `echo \$var`

echo outputs test when it supposed to write out the variable name var. Why is this happening?

2) Again, from the bash manual:

Quote:

Here Documents
If word [EOF in the example below] is unquoted, all lines of the here-document are subjected to parameter expansion, command substitution, and arithmetic expansion. In the latter case, the character sequence \newline is ignored, and ‘\’ must be used to quote the characters ‘\’, ‘$’, and ‘`’.

Since backslash must be used to quote another one, then why here:

Code:

cat <<EOF
\z
EOF

cat outputs \z instead of just z?

grail · 04-02-2013, 12:45 PM

You need to have a look at the commands you are issuing.

Code:

$ echo \z
z
$ echo "\z"
\z

Since the output of the echo is what is dealing with the backslash, passing it as output of a substitution means it is already removed.

Your second example with the variable follows the same rule:

Code:

$ var=test
$ echo \$var
$var

As '$var' is now passed to the next echo it merely outputs the value of the variable.

As for the here document, I am not sure what your issue is? None of, parameter expansion, command substitution, and arithmetic expansion, are relevant to '\z', so it is output as is.

David the H. · 04-02-2013, 12:51 PM

In regular, unquoted situations, every character with a backslash preceding it has it's backslash processed, whether it needs it or not.

But inside quotes or a heredoc, only the few listed characters are treated specially. Backslashes preceding all other regular characters will remain literal.

command + # output + # explanation:

Code:

echo \z     # z         #backslash interpreted.
echo \\z    # \z        #2 backlashes become one.
echo "\z"   # \z        #backslash not intepreted, not preceding a special character
echo "\\z"  # \z        #2 backslashes become one. "\" precedes a special character, another '\'.

A heredoc does not process quotemarks, but it does process "$" and "`", so substitutions still happen. It also processes backslashes if they are in front of the reserved characters, allowing you to escape them if needed. All other characters are ignored just as in the quoted situation above.

Code:

foo=bar

cat <<EOF
\z
\\z
"\z"
"\\z"
echo $foo
echo \$foo
$( echo $foo )
$( echo \$foo )
EOF

#output:
\z
\z
"\z"
"\z"
echo bar
echo $foo
bar
$foo

Note that if the delimiting word (the opening EOF) is quoted or backslashed, it won't process anything. The entire text will become literal.

As for this:

Code:

$ var=test
$ echo `echo \$var`
test

The way `..` processes backslashes appears to be rather confusing overall. See here. It appears to be doing some kind of double-expansion, in fact.

$(..), on the other hand, simply processes everything it contains as a regular command subshell. Just another reason not to use backticks!