[SOLVED] bash: how to un-escape a string (printf %q and %b?)

catkin · 11-23-2013, 03:40 AM

How to un-escape a string like a\ b\ c which is a value in a bash variable?

The value was created using ...

Code:

printf -v dir_path %q "$dir_path"

... so it can be used to test if a directory exists on a remote system using

Code:

buf=$(ssh "$remote_host" stat --format=%F "$dir_path" 2>&1)

In case there is an error the directory path has to be shown to humans. I had thought it was a simple matter of using printf with the %b format string to invert the printf %q conversion. But no:

Code:

c@CW8:~$ dir_path='/tmp/a b c'            
c@CW8:~$ printf -v dir_path %q "$dir_path"
c@CW8:~$ echo "$dir_path"                 
/tmp/a\ b\ c
c@CW8:~$ printf -v dir_path %b "$dir_path"
c@CW8:~$ echo "$dir_path"                 
/tmp/a\ b\ c

How best to invert the printf %q conversion? And why doesn't printf %b do it (a cursory reading of all the available documentation suggests it might do it).

EDIT:

I also tried echo piped into read with unexpected (by me) results:

Code:

c@CW8:~$ echo "$dir_path" | read; echo $REPLY

c@CW8:~$ echo "$dir_path" | read -r; echo $REPLY

c@CW8:~$ echo "$dir_path"                       
/tmp/a\ b\ c
c@CW8:~$ type read
read is a shell builtin
c@CW8:~$ type echo
echo is a shell builtin

EDIT 2:
There's a list of what printf %b converts in O'reilly's bash cookbook, found at http://books.google.co.in/books?id=Q...25b%22&f=false. It does not convert "\ " to spaces.

NevemTeve · 11-23-2013, 03:52 AM

Code:

$ VAR=$(printf "%q" "a b")
$ echo "$VAR"
a\ b

$ X=$(eval "echo $VAR")
$ echo $X
a b

catkin · 11-23-2013, 03:58 AM

Thanks NevemTeve

Unfortunately dir_path can contain arbitrary strings so using it in eval arguments would be insecure

NevemTeve · 11-23-2013, 04:10 AM

Another option would be keeping the original value, so that you shouldn't have to recreate it...

Code:

dir_path='a b c'
quoted_dir_path=$(printf '%q' "$dir_path")
...
echo "dir_path is still $dir_path"

ntubski · 11-23-2013, 09:08 AM

Quote:

Originally Posted by catkin

I also tried echo piped into read with unexpected (by me) results:

That's because piping creates a subshell so the REPLY variable is unset in the parent shell. You can use a Here String instead:

Code:

~$ dir_path='/tmp/a b c'
~$ printf -v dir_path %q "$dir_path"
~$ echo "$dir_path"
/tmp/a\ b\ c
~$ read <<<"$dir_path" ; echo "$REPLY"
/tmp/a b c

catkin · 11-25-2013, 12:31 AM

Thanks ntubski, that works perfectly

And thanks NevemTeve; a nice bit of lateral thinking

dejayc · 12-04-2015, 01:40 PM

Quote:

Originally Posted by catkin

Unfortunately dir_path can contain arbitrary strings so using it in eval arguments would be insecure

The whole point of quoting a value with printf '%q' is to make it safe for shell commands, including eval. Any character that would, in other circumstances, cause code to be executed, gets escaped into a safe version that cannot be used to execute code. Note that of course, once you escape the string, the resulting unescaped string can no longer be considered safe for eval.

If you ever run into as situation where eval can be broken by a particular sequence of characters from printf '%q', please let me know, because that would be an interesting scenario.

Another alternative is to use declare -a to auto-parse arguments to achieve the same thing:

Code:

declare NL=$'\n'
printf -v VAL '%q' "hello${NL}there"
declare -a VAR="( ${VAL} )"
echo "VAR: [${VAR[0]}]"

> VAR: [hello
there]

Note that the double-quotes surround the parenthesis, to indicate that the whole string "( $'hello\nthere' )" should be parsed into an array by declare -a. And since %q escapes characters into a single resulting escaped string, the array will only contain one item, despite the presence of any special characters within the string.

Note that you can use this technique to effectively return arrays from a function:

Code:

function get123()
{
    declare -a NUMS=( '1=one' '2=two' '3=three' )
    printf '%q ' "${NUMS[@]}"
}

declare -a RESULTS="( $( get123 ) )"
printf 'RESULT: [%s]\n' "${RESULTS[@]}"

> RESULT: [1=one]
> RESULT: [2=two]
> RESULT: [3=three]

It only took me fifteen years of using bash to come up with this usage.

aitte · 01-10-2022, 04:49 AM

Quote:

Originally Posted by NevemTeve

Code:

$ VAR=$(printf "%q" "a b")
$ echo "$VAR"
a\ b

$ X=$(eval "echo $VAR")
$ echo $X
a b

Hey! I just made an account on this website to mention a better way to do that.

Your method is close to perfect but you are creating subshells, and your use of "eval echo" is very dubious.

The following is the best escape and unescape method. No subshells involved.

To escape: Use printf with -v to write directly to a variable.
To unescape: Evaluate the assignment of the quoted (%q) safe data to a new variable.

Code:

$ printf -v FOO "%q" "a; echo hello"$'\n'"world \"guy\"! it's a great day!"

$ echo "$FOO"
$'a; echo hello\nworld "guy"! it\'s a great day!'

$ eval BAR=$FOO

$ echo "$BAR"
a; echo hello
world "guy"! it's a great day!

You may notice the lack of quotes around `eval BAR=$FOO`. That is intentional, because quotes are harmful since the $FOO value is already escaped, and adding quotes would make us treat the escaped data as literal, which is bad.

Also notice that eval doesn't run any of the code we put in the variable (such as echo and ; semicolon). It's all perfectly safe.

This is what people should be doing.

I registered an account just to say it since this is a high result when searching for unescaping "printf" %q solutions.