LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (http://www.linuxquestions.org/questions/linux-software-2/)
-   -   bash: concatenating the output of multiple commands without using temp files (http://www.linuxquestions.org/questions/linux-software-2/bash-concatenating-the-output-of-multiple-commands-without-using-temp-files-934604/)

twoprop 03-15-2012 10:45 AM

bash: concatenating the output of multiple commands without using temp files
 
I keep running into the problem of trying to concatenate the output of multiple commands in order to pipe them to the next filter. It seems as though it must be trivial, but I cannot seem to find a solution.

Here's a real-world example:


Code:

#!/bin/bash

TEMPFILE=/var/tmp/watchmcs.$$
COLUMNS=$(tput cols)
LINES=$(($(tput lines)-3));

tail -n 50 /home/minecraft/server.log \
|        sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[twopro\]\2/' \
|        cut --characters=1-$COLUMNS >$TEMPFILE
tail -n 50 /home/mcjosh/server.log \
|        sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[joshop\]\2/' \
|        cut --characters=1-$COLUMNS >>$TEMPFILE
tail -n 50 /home/joshop/server.log \
|        sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[longho\]\2/' \
|        cut --characters=1-$COLUMNS >>$TEMPFILE
tail -n 50 /home/survival/server.log \
|        sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[surviv\]\2/' \
|        cut --characters=1-$COLUMNS >>$TEMPFILE

sort <$TEMPFILE \
|        tail --lines=$LINES

I should be able to accomplish this without using a temporary file, which seems inefficient and inelegant. Ultimately, it seems to require something like:

Code:

/bin/firstcommand | cat - $(/bin/secondcommand) | /bin/nextfilter
Except that the above (unsurprisingly) causes cat to see the output of secondcommand as command line parameters instead of file content.

Any suggestions? Thanks...

--Ron

colucix 03-15-2012 11:15 AM

This kind of task is accomplished by means of process substitution. In this case the output of a command is seen as a file, since process substitution uses a file descriptor. Example:
Code:

#!/bin/bash

TEMPFILE=/var/tmp/watchmcs.$$
COLUMNS=$(tput cols)
LINES=$(($(tput lines)-3));


sort \
  <(tail -n 50 /home/minecraft/server.log |
  sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[twopro\]\2/' |
  cut --characters=1-$COLUMNS) \
  <(tail -n 50 /home/mcjosh/server.log |
  sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[joshop\]\2/' |
  cut --characters=1-$COLUMNS) \
  <(tail -n 50 /home/joshop/server.log |
  sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[longho\]\2/' |
  cut --characters=1-$COLUMNS) \
  <(tail -n 50 /home/survival/server.log |
  sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[surviv\]\2/' |
  cut --characters=1-$COLUMNS) |
  tail --lines=$LINES

Hope this helps.

David the H. 03-15-2012 03:59 PM

More generally, it's possible to use either a subshell or command grouping, and redirect the output of the whole group at once.

Code:

( command1 ; command2 ; command3 ) | cat

{ command1 ; command2 ; command3 ; } > outfile.txt

The main difference between the two is that the first one splits of a child process, while the second one operates in the context of the main shell. This can have consequences regarding the setting and use of variables and other environment settings, as well as performance.

Don't forget that the closing bracket in command grouping (and functions) must be separated from the contents by either a semicolon or a newline. This is because "}" is actually a command (keyword) of its own, and must be treated like one.

twoprop 03-16-2012 01:22 AM

Thank you, and thank you. Obvious now that I see it, but it has plagued me for years.


All times are GMT -5. The time now is 08:26 AM.