BASH comments

danielbmartin · 01-12-2012, 09:20 PM

Sometimes code readability is improved by breaking a pipe into individual commands, one per line.

Contrived example:

Code:

cat $InFile   \
|rev          \
|cut -d, -f7- \
|rev          \
> $Work3

It would be even better to comment every line...

Code:

cat $InFile   \  # Read input file
|rev          \  # Reverse (flip each line end-for-end)
|cut -d, -f7- \  # Toss fields 1-6
|rev          \  # Reverse, again 
> $Work3         # Write interim result to a work file

... but that doesn't work.

Is there a way to do this?

Daniel B. Martin

David the H. · 01-12-2012, 10:17 PM

No, I don't believe that's possible. the backslash is there to escape the non-printing newline character that normally terminates the line. There can be nothing else following it.

To the shell, the multiple lines appear to be a single line, so the comments can only come after the last one.

Telengard · 01-12-2012, 11:57 PM

If you put the pipe at the end of the line Bash will continue with the next line. No need for the backslash.

Code:

$ cat a
#! /bin/bash

echo "one two three" | #comment
cut -d ' ' -f 2
$ ./a
two
$

The comment following the pipe doesn't seem to interfere with the continuation.
I failed to find any support for this in the GNU Bash Reference Manual though, so caveat emptor.

Nominal Animal · 01-13-2012, 03:14 AM

Quote:

Originally Posted by Telengard

If you put the pipe at the end of the line Bash will continue with the next line. No need for the backslash.

The same also applies for the && and || operators. It also works in dash and tcsh shells too, so I do believe this is common behaviour to all shells.

Actually, it seems to be obliquely defined: all these operators require both sides to exist -- an empty command on either side makes absolutely no sense for these, and a comment or newline or empty line(s) do not produce any statement. The situation is complementary to 'do' or 'then' statements -- they need a preceding semicolon or a new line in Bash, Bourne shells and derivatives and POSIX shells -- so this behaviour is quite intuitive and useful.

I just wish it was explicitly documented somewhere.

danielbmartin · 01-13-2012, 09:25 AM

Quote:

Originally Posted by Telengard

If you put the pipe at the end of the line Bash will continue with the next line. No need for the backslash.

I like your idea but was unable to apply it. Please see if you can modify this contrived code snippet and make it work.

Code:

cat $InFile   \  # Read input file
|rev          \  # Reverse (flip each line end-for-end)
|rev          \  # Reverse, again 
> $Work3         # Write interim result to a work file

Daniel B. Martin

Telengard · 01-13-2012, 10:23 AM

Quote:

Originally Posted by danielbmartin

I like your idea but was unable to apply it. Please see if you can modify this contrived code snippet and make it work.

Code:

cat $InFile   \  # Read input file
|rev          \  # Reverse (flip each line end-for-end)
|rev          \  # Reverse, again 
> $Work3         # Write interim result to a work file

Daniel B. Martin

What did you try? What happened when you tried it? What error message, if any?

Code:

test$ ls -gGh                                                 
total 8.0K
-rwxr--r-- 1 243 2012-01-13 11:02 contrived
-rw-r--r-- 1  24 2012-01-13 10:45 file.in
test$ cat contrived 
#! /bin/bash

InFile="file.in"
Work3="file.out"

{
    cat $InFile  |  # Read input file
    rev          |  # Reverse (flip each line end-for-end)
    rev             # Reverse, again
} > $Work3          # Write interim result to a work file
test$ cat file.in 
one
two
three
four
five
test$ ./contrived 
test$ ls -gGh
total 12K
-rwxr--r-- 1 243 2012-01-13 11:02 contrived
-rw-r--r-- 1  24 2012-01-13 10:45 file.in
-rw-r--r-- 1  24 2012-01-13 11:04 file.out
test$ cat file.out 
one
two
three
four
five
test$

If this style of code is tricky to remember and write then it would be a mistake to adopt it.

Telengard · 01-13-2012, 10:46 AM

Quote:

Originally Posted by Nominal Animal

I just wish it was explicitly documented somewhere.

The fact that I found no documentation of the behavior makes it difficult to recommend.

H_TeXMeX_H · 01-13-2012, 11:04 AM

I don't see the point of so many comments. Just make a comment on what the pipe accomplishes. For example:

Code:

# cut off last 6 fields
cat $InFile | rev | cut -d, -f7- | rev > $Work3

danielbmartin · 01-13-2012, 03:02 PM

Quote:

Originally Posted by H_TeXMeX_H

I don't see the point of so many comments. Just make a comment on what the pipe accomplishes. For example:

Code:

# cut off last 6 fields
cat $InFile | rev | cut -d, -f7- | rev > $Work3

Let me remind you that this is a contrived example. My real pipes may be much longer. As a matter of personal style I like to comment every line. You might have a different style, and I respect that.

When writing a complicated piece of code (regardless of language) I like to express in words the logic I want to implement. Then, one piece at a time, I fill in the code. When done, my code is fully commented because I started with all comments.

Daniel B. Martin

H_TeXMeX_H · 01-13-2012, 03:18 PM

Well, I do write a lot of scripts, and usually if a command becomes too complicated I split it up, I don't do it all at once.

This helps not only because you can comment each part, but because it is easier to understand.

Oftentimes there are different and more readable ways to do things that don't involve huge commands or long pipes.

Either way Telengard posted what you want.

Nominal Animal · 01-13-2012, 11:27 PM

I myself like to use subshells when working with long pipes, i.e.

Code:

(
    # First part of the pipe ...
) | (
    # Second part of the pipe ...
) | (
    # and so on ...
)

For me, a very typical one is something like

Code:

( echo 'plot "datafile" u 1:2 t "data" w points, \'
  echo '     cos(x)*exp(x/2.5) t "expected" w lines'

  echo "press Enter to close the Gnuplot window" >&2
  while read LINE ; do
      [ -z "$LINE" ] && break
      echo "$LINE"
  done
) | gnuplot

which uses Gnuplot to plot some data, but stays interactive. An empty line will close Gnuplot and exit the compound command, but if I happen to think of an additional Gnuplot command -- say set logscale x; replot -- all I need is to type it and hit enter.

On embedded machines with a limited memory subsystem subshells may not be the best option, but for standard Intel/PowerPC et al. architectures, the subshell is forked from the parent using copy-on-write, using physically the same RAM for code and initial data structures, and therefore uses very little actual system resources. (Just a per-process kernel structure for each subshell, I believe.) This means that on a typical workstation or a server, there is no practical difference in resource use between plain pipe commands and piped subshells.

Telengard · 01-14-2012, 12:07 AM

Quote:

Originally Posted by Nominal Animal

On embedded machines with a limited memory subsystem subshells may not be the best option, but for standard Intel/PowerPC et al. architectures, the subshell is forked from the parent using copy-on-write, using physically the same RAM for code and initial data structures, and therefore uses very little actual system resources. (Just a per-process kernel structure for each subshell, I believe.) This means that on a typical workstation or a server, there is no practical difference in resource use between plain pipe commands and piped subshells.

Neat idea, and well reasoned, but I don't see the advantage over { command-list; }.

Nominal Animal · 01-14-2012, 02:41 AM

Quote:

Originally Posted by Telengard

Neat idea, and well reasoned, but I don't see the advantage over { command-list; }.

You're absolutely right.

I've just never really bothered to find out about the side effects when using command lists in a pipe (specifically, does shell state propagate or not, or if it is just inherited from the parent shell like subshells) -- and to be honest, I tend to always forget the required semicolon from the end of the command list. I've gravitated to using subshells, because I've felt them to be more intuitive.

To those that are unaware of the semicolon detail with command lists, the equivalent command list variant of piped subshells,

Code:

( echo foo ) | ( cat ; echo bar )

is

Code:

{ echo foo ; } | { cat ; echo bar ; }

Using command lists, the shell does not create unnecessary extra processes. Note the semicolons. If you try

Code:

{ echo foo } | { cat ; echo bar }

the shell does not recognize the braced expressions as command lists, and at least Bash 4.2.10 complains about a syntax error. It's very difficult to realize that the only problem is missing semicolons before closing braces. (Well, unless you remember that the command list syntax is, like Telengard stated above, {command(s)...;} and not just {command(s)...} .)

Telengard · 01-14-2012, 10:57 AM

Quote:

Originally Posted by Nominal Animal

I've gravitated to using subshells, because I've felt them to be more intuitive.

That makes sense as an advantage. Parenthesized command groups have simpler syntax and may be easier to type. Curly braces need spaces and the list must end with a control operator.

David the H. · 01-14-2012, 11:14 AM

I have no problem with command grouping. They're basically just an anonymous functions. And it only takes getting caught by the final semicolon thing a few times before you learn to watch out for it.