LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 01-17-2011, 09:49 AM   #1
figo
LQ Newbie
 
Registered: May 2009
Posts: 14

Rep: Reputation: 0
Question how to retain the eol (\n) in bash variable or ssh output


Hi, I have to save the result of ssh/grep into a file to keep the eol ("/n"):

ssh $SSH_OPTIONS $USER@$NODE "cd $LOG_DIR; grep -h '$pattern' log.*" > $file

So that when I grep on the local file again later, it can be printed out with original log lines.

Otherwise, the log lines will be dropped and lines becomes concatenated into a single line, e.g., if I rewrite the script in this way, echoing the $result is not a good idea..

result=`ssh $SSH_OPTIONS $USER@$NODES "cd $LOG_DIR; grep -h '$pattern' log.*"`

So question: is there some workaround that I can save it to a variable rather than file but still keep the eol? That will simplify my script and don't need to do all those I/Os!

Thanks in advance and happy MLK day!

Last edited by figo; 01-17-2011 at 09:50 AM.
 
Old 01-17-2011, 11:45 AM   #2
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
Using a temporary file is a pretty good way to go about this sort of thing. Putting log output into a variable is potentially problematic in shell scripting as there is a pretty small limit to the amount of data a variable can hold, and log output can get pretty big - especially when there are problems which you're trying to diagnose... at that time you really don't want to be debugging your shellscript!

Since most modern distros mount /tmp as a ramdisk, there's not really much overhead for something like this:

Code:
#!/bin/bash

# set up your variables...

tmp=$(mktemp) || exit 1

ssh $SSH_OPTIONS $USER@$NODES "cd $LOG_DIR; grep -h '$pattern' log.*" > "$tmp"

# now do whatever you like to your log data, for example:
error_count=$(grep -c ERROR "$tmp")
echo "I found $error_count errors for $USER@$NODES"

# don't forget to delete the tmp file
rm -f "$tmp"
*edit* If you really want to do a lot of manipulation of log data in memory, then shell scripting is probably not the best option. A perl, awk or python program might be a better way to go.

Last edited by matthewg42; 01-17-2011 at 11:46 AM. Reason: extra opinion!
 
Old 01-18-2011, 03:53 AM   #3
Reuti
Senior Member
 
Registered: Dec 2004
Location: Marburg, Germany
Distribution: openSUSE 11.4
Posts: 1,319

Rep: Reputation: 252Reputation: 252Reputation: 252
Another option might be a pipe, in case you want e.g. to parse the output for some special strings.
 
Old 01-18-2011, 04:35 AM   #4
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,498

Rep: Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956Reputation: 1956
Quote:
Originally Posted by figo View Post
Otherwise, the log lines will be dropped and lines becomes concatenated into a single line, e.g., if I rewrite the script in this way, echoing the $result is not a good idea..
This is the echo's normal behaviour: it prints out the args, separated by spaces, followed by a newline. If you use double quotes to embed the variable reference, the argument passed to echo is only one and it is printed out as is, with newlines preserved. Otherwise the text is splitted into tokens using IFS as separator (default to <space><tab><newline>).

Hence, you can simply do
Code:
echo "$result"
to preserve newlines. Anyway, I second the argumentation by matthewg42 about the possible large amount of data coming from log output.
 
  


Reply

Tags
bash, eol


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Bash - Passing variable to ssh zepphead5 Programming 8 05-09-2010 09:34 AM
how to retain changes made in a global variable by a child process after in exits sagsriv Linux - Newbie 3 03-12-2008 12:44 AM
Bash store last line from displayed text output in a variable carl0ski Programming 1 01-16-2007 03:38 AM
bash script: how to assign an output (not the result) to a variable? Singing Banzo Programming 8 10-01-2006 06:29 PM
retain a variable each time a script re-runs cachemonet Programming 4 08-01-2006 12:11 AM


All times are GMT -5. The time now is 11:01 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration