Help interpreting shell script

NotionCommotion · 05-04-2017, 07:04 AM

Hello again!

Still having a little trouble with the following. My interpretation is:

As $RUNAS, run $SCRIPT, and send the output to file descriptor #3, and echo the spawned ID.
Then as previous user, send file descriptor #3 to $LOGFILE, and send the echo'd ID to $PIDFILE.

But as seen, I get a syntax error.

Thanks

Code:

RUNAS=Michael
PIDFILE=/var/run/test_soap.pid
LOGFILE=/var/log/test_soap.log

start() {
  if [ -f "$PIDFILE" ] && kill -0 $(cat "$PIDFILE"); then
    echo 'Service already running' >&2
    return 1
  fi
  echo 'Starting service…' >&2
  #local CMD="$SCRIPT &> "$LOGFILE" & echo \$!"
  #su -c "$CMD" $RUNAS > "$PIDFILE"
  # Change file descriptors and move the output redirection outside the su command:
  # Opens $LOGFILE on file descriptor 3 before invoking su, then redirects the command's stdout and stderr to that opened fd3.
  # The redirection onto "$PIDFILE" is already outside the su command.
  local CMD="$SCRIPT &> &3 & echo \$!"
  su -c "$CMD" $RUNAS  3>"$LOGFILE" >"$PIDFILE"
  echo 'Service started' >&2
}

Code:

[Michael@devserver ~]$ sudo /etc/init.d/test_soap start
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]
Starting service…
bash: -c: line 0: syntax error near unexpected token `&'
bash: -c: line 0: `/usr/local/bin/test_soap &> &3 & echo $!'
Service started
[Michael@devserver ~]$

pan64 · 05-04-2017, 07:07 AM

probably you wanted && instead of & before that echo

NotionCommotion · 05-04-2017, 07:28 AM

Quote:

Originally Posted by pan64

probably you wanted && instead of & before that echo

I don't think so. Not looking for Boolean, but to run in the background (and also acts as a separator?) This local CMD="$SCRIPT &> "$LOGFILE" & echo \$!" works fine, but requires that $RUNAS have write permission to the log which I didn't want.

pan64 · 05-04-2017, 07:39 AM

yes, the assignment works fine, but this $CMD is syntactically incorrect, therefore cannot be executed.

Code:

&> filename 
# and
>&[fd]
# are valid redirection, but
&>&[fd]
# is incorrect

http://www.tldp.org/LDP/abs/html/io-redirection.html

NotionCommotion · 05-04-2017, 07:54 AM

Thanks pan64. Why not?

Code:

# Redirect both stdout and stderr to file "filename."
&>filename

# Redirect whatever happened previously to file descriptor #fd?
>&[fd]

# But not redirect stdout and stderr to file descriptor #fd??
&>&[fd]

rknichols · 05-04-2017, 09:25 AM

I have no idea why "&> &3" does not work, but this does work:

Code:

local CMD="$SCRIPT >&3 2>&1 & echo \$!"
su -c "$CMD" $RUNAS  3>"$LOGFILE" >"$PIDFILE"

pan64 · 05-04-2017, 09:27 AM

it is not implemented, not supported, but I do not have any idea why. Probably not really unambiguous (how can you decide if &>&5 is really &> into a file named &5 or &>& into file descriptor 5).

rknichols · 05-04-2017, 10:21 AM

Quote:

Originally Posted by pan64

it is not implemented, not supported, but I do not have any idea why. Probably not really unambiguous (how can you decide if &>&5 is really &> into a file named &5 or &>& into file descriptor 5).

One would hope it would be the same way you would handle ">&5" or "> &5", both of which refer to file descriptor 5 and not a file named "&5".

Parsing of shell command lines is a nasty business, and it's not surprising that there are corner cases where it breaks down. Consider that

Code:

CMD="$SCRIPT >&3 2>&1 & echo \$!"

is legal in a script, but fails while trying to parse the !" when typed at the command prompt.

NotionCommotion · 05-04-2017, 10:48 AM

Thank you both,

Code:

local CMD="$SCRIPT >&3 2>&1 & echo \$!"
su -c "$CMD" $RUNAS  3>"$LOGFILE" >"$PIDFILE"

Should I be thinking that each process has its own stdIn, stdOut, stdErr? For instance, for my original example, both /etc/init.d/test_soap and /usr/local/bin/test_soap ($SCRIPT) create a process, so both have their own individual I/O?

So, $SCRIPT's stdOut is /etc/init.d/test_soap's stdIn?

The ampersand isn't implying it is one or the other's std in/out/err, is it? What is purpose, and why wouldn't one write 2>1 instead of 2>&1?

Also, is $SCRIPT >&3 the same as $SCRIPT 1>&3 or maybe it should be $SCRIPT 0>&3, but not shown because it is implied?

So.... Take $SCRIPT's stdOut and redirected to &3, and take $SCRIPT's stdErr and redirect its stdOut, which in turn gets redirect to &3, so now when I later redirect to $LOGFILE, I get both?

rknichols · 05-04-2017, 11:25 AM

Quote:

Originally Posted by NotionCommotion

Code:

local CMD="$SCRIPT >&3 2>&1 & echo \$!"
su -c "$CMD" $RUNAS  3>"$LOGFILE" >"$PIDFILE"

Should I be thinking that each process has its own stdIn, stdOut, stdErr? For instance, for my original example, both /etc/init.d/test_soap and /usr/local/bin/test_soap ($SCRIPT) create a process, so both have their own individual I/O?

Each process has its own stdin, stdout, stderr, and other file descriptors, which are inherited from the parent unless you tell the shell to redirect them. (You can mark a file descriptor "close on exec" so that it will not be inherited by a child process, but that is not being done here, and I don't know if you even can do that in a shell script.)

Quote:

So, $SCRIPT's stdOut is /etc/init.d/test_soap's stdIn?

No, $SCRIPT's stdout has been redirected to $LOGFILE. When that script executes test_soap, that test_soap process will inherit that same stdout. You haven't done anything with stdin or set up any pipelines, so each process will inherit the terminal as its stdin.

Quote:

The ampersand isn't implying it is one or the other's std in/out/err, is it? What is purpose, and why wouldn't one write 2>1 instead of 2>&1?

That ampersand says to duplicate the file descriptor number that follows. Without the ampersand, you would be creating a file named "1" in the current directory.

Quote:

Also, is $SCRIPT >&3 the same as $SCRIPT 1>&3 or maybe it should be $SCRIPT 0>&3, but not shown because it is implied?

On an output redirection, omitting the descriptor number implies "1" (stdout). On an input redirection, omitting the descriptor number implies "0" (stdin).

Quote:

So.... Take $SCRIPT's stdOut and redirected to &3, and take $SCRIPT's stdErr and redirect its stdOut, which in turn gets redirect to &3, so now when I later redirect to $LOGFILE, I get both?

Pretty much. Within a command, those redirections are processed left-to-right. First, stdout is redirected to fd3. Then, stderr is redirected to wherever stdout is now going (i.e., fd3). Note that you are not redirecting stderr to stdout. You are redirecting stderr to the same place that stdout is directed at that moment. If you later change the redirection of stdout, stderr is not going to follow along. That's why the order is important. If you wrote "command 2>&1 1>&3" you would not be redirecting stderr to fd3.

The place it gets a bit confusing is when pipelines are involved.

Code:

command1 2>&1 | command2

It no longer looks like left-to-right, because the pipeline is set up first, redirecting the stdout from command 1 to the stdin of command2. Then the other redirection for command1 is processed, sending stderr to where stdout is now going (the pipeline).

NotionCommotion · 06-30-2017, 10:50 AM

I've been struggling on how to take the output from $script and use grep -v to remove lines I don't want logged. I thought I would have been able to pipe the output from either the script or file descriptor 3 through grep, but it gives me 3: 3: Bad file descriptor error.

Code:

local CMD="$SCRIPT | grep -v "remove me" >&3 2>&1 & echo \$!"
#local CMD="$SCRIPT >&3 2>&1 & echo \$!"
su -c "$CMD" $RUNAS  3>>"$LOGFILE" >"$PIDFILE"
# su -c "$CMD" $RUNAS  3 | grep -v "remove me" >>"$LOGFILE" >"$PIDFILE"

pan64 · 06-30-2017, 11:03 AM

I think that syntax is invalid. You can only pipe from stdout to stdin, cannot use fd 3 for that.
I would suggest you to create named pipes to do that.

Code:

mkfifo /tmp/fifo
local CMD="$SCRIPT >/tmp/fifo 2>&1 & echo \$!"
sudo -c "$CMD" $RUNAS >$PIDFILE
grep -v "remove me" /tmp/fifo >> $LOGFILE

not tested

rknichols · 06-30-2017, 01:05 PM

You can do it as long as you pay attention to the order in which the shell performs the redirections. I'm confused by the alternatives in your examples, and it's not clear what you wanted to do with the command's stderr, but

Code:

su -c "$CMD" $RUNAS 3>&1 2>stderr.out >"$PIDFILE" | grep -v "remove me" >>"$LOGFILE"

will do the following in order:

Redirect the command's stdout through the pipe to grep's stdin.
Redirect the command's fd3 to the place fd1 is now going (i.e., the pipe).
Redirect the command's stderr (fd2) to the file stderr.out
Redirect the command's stdout (fd1) to the "$PIDFILE" file.

Note that the redirection in step 4 does not affect what was done in step 2, so fd3 is left connected to the pipe.

If you wanted the command's stderr output to be combined with that on fd3 and sent to the grep command, then

Code:

su -c "$CMD" $RUNAS 3>&1 2>&1 >"$PIDFILE" | grep -v "remove me" >>"$LOGFILE"

should do it.