[SOLVED] exec in bash prefixes the specified command with the home directory

Reuti · 12-06-2011, 12:45 PM

Dear all,

I streamlined down the behavior to the following lines as starter.sh:

Code:

#!/bin/sh
echo Starting...
echo "==> $@"
echo exec $@
exec $@

and the called script dummy.sh:

Code:

#!/bin/sh
echo Yes

The output is:

Code:

$ ./starter.sh '"/home/reuti/dummy.sh"'
Starting...
==> "/home/reuti/dummy.sh"
exec "/home/reuti/dummy.sh"
./starter.sh: line 5: /home/reuti/"/home/reuti/dummy.sh": No such file or directory
./starter.sh: line 5: exec: /home/reuti/"/home/reuti/dummy.sh": cannot execute: No such file or directory

(The double quotation marks are originally added by MPICH2 while calling Hydra, I just mimic the behavior by putting all in single quotation marks.) Why does the exec builtin add the home directory of the user in front of the specified command? It seems like it’s checking whether there is any charcater between the start of the specified command and the first slash inside:

Code:

$ ./starter.sh '"foobar/dummy.sh"'
Starting...
==> "foobar/dummy.sh"
exec "foobar/dummy.sh"
./starter.sh: line 5: /home/reuti/"foobar/dummy.sh": No such file or directory
./starter.sh: line 5: exec: /home/reuti/"foobar/dummy.sh": cannot execute: No such file or directory

but:

Code:

$ ./starter.sh '"dummy.sh"'
Starting...
==> "dummy.sh"
exec "dummy.sh"
./starter.sh: line 5: exec: "dummy.sh": not found

The behavior can be fixed by adding eval to the script like:

Code:

#!/bin/sh
echo Starting...
echo "==> $@"
echo exec $@
eval exec $@

But the question remains: where is this behavior defined (and the error messages are different between the two cases)? I can’t spot it in the bash’s manpage, and it happens in version 3.2.25 and 4.1.10, so it might also in some why depend on the underlying exec kernel function. But as /home/reuti/ is not in the set PATH, I would wonder why execlp or alike adds it.

David the H. · 12-06-2011, 03:48 PM

I don't know offhand why it's adding the extra path info, something to do with the way the shell interprets quotes and slashes I expect. It appears to be adding the PWD to the string. But the problem is definitely with the double-layered quotemarks. Double-quotes are escaped and literal when inside single quotes, and vice-versa, so they are being treated as if they were a literal part of the path, causing the error. Remove one set or the other and the thing works as expected.

By the way, $@ is designed to be used with double-quotes. Quoting it has the effect of treating each expanded value as if it were quoted separately. So you really should be using exec "$@" in your script, to avoid any word-splitting on the input values before execution.

In addition, while not so important here, when you run a script with #!/bin/sh at the top, the script is parsed in posixly-compliant mode, and many of the extended bash features are disabled. Not to mention that on some systems bash isn't even the default shell for parsing sh scripts. dash is often used as a substitute, for example. Be sure to use #!/bin/bash as your shebang any time the script contains any bashisms.

PS: Please don't use eval in your scripts unless you know exactly what the executed command will do. It's a nasty security risk. Rather, have your script process the incoming arguments first to remove the redundant quotes, if you can't figure out any other way around it.

Reuti · 12-06-2011, 04:12 PM

Thx for the reply. Yes, I’m aware of the difference of $@ and $* inside quotation marks. I removed them to streamline it. Even if I add them again to starter.sh:

Code:

#!/bin/bash
echo Starting...
echo "==> $@"
echo exec $@
exec "$@"

and dummy.sh:

Code:

#!/bin/bash
echo Yes

nothing changes:

Code:

$ ./starter.sh '"/home/reuti/dummy.sh"'
Starting...
==> "/home/reuti/dummy.sh"
exec "/home/reuti/dummy.sh"
./starter.sh: line 5: /home/reuti/"/home/reuti/dummy.sh": No such file or directory
./starter.sh: line 5: exec: /home/reuti/"/home/reuti/dummy.sh": cannot execute: No such file or directory

NB: This is not artificial chess puzzle, I faced it with a custom starter_method for GridEngine and got:

Code:

$ cat test_openmpi.sh.o3589
Starting...
==> /var/spool/sge/node01/job_scripts/3589
exec /var/spool/sge/node01/job_scripts/3589
Starting...
==> "/home/reuti/local/mpich2-1.4/bin/hydra_pmi_proxy" --control-port node01:53990 --demux poll --pgid 0 --retries 10 --proxy-id 1
exec "/home/reuti/local/mpich2-1.4/bin/hydra_pmi_proxy" --control-port node01:53990 --demux poll --pgid 0 --retries 10 --proxy-id 1
$ cat test_openmpi.sh.e3589
/home/reuti/starter.sh: line 5: /home/reuti/"/home/reuti/local/mpich2-1.4/bin/hydra_pmi_proxy": No such file or directory
/home/reuti/starter.sh: line 5: exec: /home/reuti/"/home/reuti/local/mpich2-1.4/bin/hydra_pmi_proxy": cannot execute: No such file or directory

But it’s not a MPICH2 or GridEngine issue as I can reproduce it without them being involved. Who is adding /home/reuti/ to the path and why?

jthill · 12-06-2011, 05:48 PM

strace says that weird prefix is actually being passed to execve, and this experiment from the command line says it has nothing to do with any variable:

Code:

~$ exec \"bin/bar
bash: /home/jthill/"bin/bar: No such file or directory
bash: exec: /home/jthill/"bin/bar: cannot execute: No such file or directory
~$

It happens only in bash, not dash or ksh. I think you've hit a bash bug.

(wrong, just bash explicitly prefixing $PWD)

Nominal Animal · 12-06-2011, 06:04 PM

Quote:

Originally Posted by Reuti

Code:

$ ./starter.sh '"/home/reuti/dummy.sh"'

This specifies parameter

Code:

"/home/reuti/dummy.sh"

The double quotes are part of the parameter. The single quotes tell the shell you use, that the value of the parameter contains the double quotes.

Because your starter.sh script then uses

Code:

exec $@

you effectively run

Code:

exec \"/home/reuti/dummy.sh\"

Again, the double quotes are part of the parameter! Note that you're not streamlining anything by omitting proper quoting, you are just making sure any file names that contain spaces will never work for your script. You certainly cannot fix that by adding extra quotes when calling the script. I recommend you learn about proper quoting and escaping using Bash and POSIX shells instead. Check out Uwe Waldmann's Quoting Guide, for example.

In particular, if you omit double quotes around $@ the command will invariably fail with filenames and paths containing spaces or newlines. (Using correct quoting, i.e. exec "$@" , does not fix your original problem, because you introduced the error before the script is even executed, by supplying it with an invalid script name.)

The error message,

Quote:

./starter.sh: line 5: /home/reuti/"/home/reuti/dummy.sh": No such file or directory
./starter.sh: line 5: exec: /home/reuti/"/home/reuti/dummy.sh": cannot execute: No such file or directory

occurs, because the first parameter to exec is a relative path:

If it begins with a slash, it is an absolute path (starting from root).
If it contains a slash in it, it is a relative path (starting from current directory).
This feature is not supported by all shells, but it is by Bash.
If there are no slashes in it, it is a filename, and Bash will look for it in the directories specified in PATH.

In other words, because the to-be-executed starts with a double quote, and contains a slash, Bash has to look for it in the current directory. The error message makes perfect sense to me.

Using eval to fix the situation is ... I'm sorry, I cannot find the words to describe my horror. It is extremely unadvisable to do something like that. For example, consider what happens if you were unlucky enough to run something like

Code:

./starter.sh '</dev/null ; rm -rf "$HOME"'

If you used eval in starter.sh, the exec would just redirect standard input from /dev/null, then it would proceed with removing the user's home directory!
If you do not use eval , then nothing bad happens. exec will just try to find a script with a very peculiar name to execute. The semicolon is just a part of the file name (or a separate parameter if you alter the quoting in the original command); without eval it will never act as an end marker for the exec command.

jthill · 12-06-2011, 06:12 PM

Nope, not a bug.

You're passing a file name that doesn't begin with a slash (it begins with a doublequote) and does contain a slash. That makes it an explicit path relative to $PWD. bash is merely prefixing $PWD explicitly. cd to anywhere and retry, you'll see the effect, or change the doublequote to xyz, or whatever. all hail irc.

(edit: erm, yup -- what n.a. said)

Reuti · 12-07-2011, 06:00 AM

Quote:

Originally Posted by jthill

You're passing a file name that doesn't begin with a slash (it begins with a doublequote) and does contain a slash. That makes it an explicit path relative to $PWD. bash is merely prefixing $PWD explicitly.

Aha, this is a straight forwards explanation. Thx. This means, that the bash is already scanning the parameter name (i.e. filename) and not execve (what I would have expected). Only puzzle left is, why there are one time two error messages, and in the second case only one.

@Nominal Animal: I appreciate the hint about the eval flaws.

Reuti · 12-07-2011, 10:25 AM

Quote:

Originally Posted by David the H.

PS: Please don't use eval in your scripts unless you know exactly what the executed command will do. It's a nasty security risk.

Although we are going away from the original issue: this is a quite nice link. But I don’t get, why in the example eval FILES=($FILES) is used at all, the assignment works without the eval command too.

David the H. · 12-07-2011, 02:53 PM

Thanks to N.A. for providing a better, more detailed description of what I had tried to explain. My comment on $@ was intended to be a side point about script syntax, and not directly related to the original problem. But apparently it just confused the issue.

For the last question, read what N.A. wrote again about how bash recognizes files and paths. In the first case, since the argument you passed contains a slash, it's treated as a relative path. But since the path is corrupt, you get a "No such file or directory" error from the shell. Then exec also errors on the bad file name.

In the second case, there is no slash in the name, so bash treats it as a simple filename in the current directory, and doesn't throw a path error. But there is still no file by that name, so exec again chokes on it.

re eval: That site is a repository of tips and examples taken from their IRC help channel, so that's probably an actual script someone posted.

The author was trying to protect filenames with spaces. So he used find's -printf option to output them surrounded by quotes, and then tried to use eval to remove them again when setting the array. Kind of similar to the way you tried to use it here, in fact. Nobody said the logic was correct, only that the script did generally work as written, but in an entirely unsafe manner.