LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 03-15-2012, 09:45 AM   #1
twoprop
LQ Newbie
 
Registered: Jan 2011
Posts: 7

Rep: Reputation: 0
bash: concatenating the output of multiple commands without using temp files


I keep running into the problem of trying to concatenate the output of multiple commands in order to pipe them to the next filter. It seems as though it must be trivial, but I cannot seem to find a solution.

Here's a real-world example:


Code:
#!/bin/bash

TEMPFILE=/var/tmp/watchmcs.$$
COLUMNS=$(tput cols)
LINES=$(($(tput lines)-3)); 

tail -n 50 /home/minecraft/server.log \
|	sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[twopro\]\2/' \
|	cut --characters=1-$COLUMNS >$TEMPFILE
tail -n 50 /home/mcjosh/server.log \
|	sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[joshop\]\2/' \
|	cut --characters=1-$COLUMNS >>$TEMPFILE
tail -n 50 /home/joshop/server.log \
|	sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[longho\]\2/' \
|	cut --characters=1-$COLUMNS >>$TEMPFILE
tail -n 50 /home/survival/server.log \
|	sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[surviv\]\2/' \
|	cut --characters=1-$COLUMNS >>$TEMPFILE

sort <$TEMPFILE \
|	tail --lines=$LINES
I should be able to accomplish this without using a temporary file, which seems inefficient and inelegant. Ultimately, it seems to require something like:

Code:
/bin/firstcommand | cat - $(/bin/secondcommand) | /bin/nextfilter
Except that the above (unsurprisingly) causes cat to see the output of secondcommand as command line parameters instead of file content.

Any suggestions? Thanks...

--Ron
 
Old 03-15-2012, 10:15 AM   #2
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,489

Rep: Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956
This kind of task is accomplished by means of process substitution. In this case the output of a command is seen as a file, since process substitution uses a file descriptor. Example:
Code:
#!/bin/bash

TEMPFILE=/var/tmp/watchmcs.$$
COLUMNS=$(tput cols)
LINES=$(($(tput lines)-3)); 


sort \
  <(tail -n 50 /home/minecraft/server.log |
   sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[twopro\]\2/' |
   cut --characters=1-$COLUMNS) \
  <(tail -n 50 /home/mcjosh/server.log |
   sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[joshop\]\2/' |
   cut --characters=1-$COLUMNS) \
  <(tail -n 50 /home/joshop/server.log |
   sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[longho\]\2/' |
   cut --characters=1-$COLUMNS) \
  <(tail -n 50 /home/survival/server.log |
   sed 's/\(^....................\)\[[A-Z]*\]\(.*\)/\1\[surviv\]\2/' |
   cut --characters=1-$COLUMNS) |
  tail --lines=$LINES
Hope this helps.
 
1 members found this post helpful.
Old 03-15-2012, 02:59 PM   #3
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
More generally, it's possible to use either a subshell or command grouping, and redirect the output of the whole group at once.

Code:
( command1 ; command2 ; command3 ) | cat

{ command1 ; command2 ; command3 ; } > outfile.txt
The main difference between the two is that the first one splits of a child process, while the second one operates in the context of the main shell. This can have consequences regarding the setting and use of variables and other environment settings, as well as performance.

Don't forget that the closing bracket in command grouping (and functions) must be separated from the contents by either a semicolon or a newline. This is because "}" is actually a command (keyword) of its own, and must be treated like one.

Last edited by David the H.; 03-15-2012 at 03:01 PM.
 
1 members found this post helpful.
Old 03-16-2012, 12:22 AM   #4
twoprop
LQ Newbie
 
Registered: Jan 2011
Posts: 7

Original Poster
Rep: Reputation: 0
Thank you, and thank you. Obvious now that I see it, but it has plagued me for years.
 
  


Reply

Tags
bash scripting, cat


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Issue String concatenating in output file daredevil_1981 Linux - Newbie 4 10-14-2011 12:56 PM
[SOLVED] Bash Scripting - Output as Multiple Files zer0signal Programming 14 01-26-2011 06:43 PM
[SOLVED] Help with unpacking and concatenating multiple zip files SChipS Linux - Software 6 02-03-2010 05:13 AM
Question about bash history temp files WingnutOne Linux - General 4 05-16-2008 05:53 AM
Correlate output from multiple commands wendtbd Programming 4 07-18-2007 07:16 AM


All times are GMT -5. The time now is 07:24 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration