LinuxQuestions.org - how to retain the eol (“\n”) in bash variable or ssh output

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - how to retain the eol (“\n”) in bash variable or ssh output (https://www.linuxquestions.org/questions/programming-9/how-to-retain-the-eol-%93%5Cn%94-in-bash-variable-or-ssh-output-856840/)

how to retain the eol (“\n”) in bash variable or ssh output

Hi, I have to save the result of ssh/grep into a file to keep the eol ("/n"):

ssh $SSH_OPTIONS $USER@$NODE "cd $LOG_DIR; grep -h '$pattern' log.*" > $file

So that when I grep on the local file again later, it can be printed out with original log lines.

Otherwise, the log lines will be dropped and lines becomes concatenated into a single line, e.g., if I rewrite the script in this way, echoing the $result is not a good idea..

result=`ssh $SSH_OPTIONS $USER@$NODES "cd $LOG_DIR; grep -h '$pattern' log.*"`

So question: is there some workaround that I can save it to a variable rather than file but still keep the eol? That will simplify my script and don't need to do all those I/Os!

Thanks in advance and happy MLK day!

Using a temporary file is a pretty good way to go about this sort of thing. Putting log output into a variable is potentially problematic in shell scripting as there is a pretty small limit to the amount of data a variable can hold, and log output can get pretty big - especially when there are problems which you're trying to diagnose... at that time you really don't want to be debugging your shellscript!

Since most modern distros mount /tmp as a ramdisk, there's not really much overhead for something like this:

Code:

#!/bin/bash



# set up your variables...



tmp=$(mktemp) || exit 1



ssh $SSH_OPTIONS $USER@$NODES "cd $LOG_DIR; grep -h '$pattern' log.*" > "$tmp"



# now do whatever you like to your log data, for example:

error_count=$(grep -c ERROR "$tmp")

echo "I found $error_count errors for $USER@$NODES"



# don't forget to delete the tmp file

rm -f "$tmp"

*edit* If you really want to do a lot of manipulation of log data in memory, then shell scripting is probably not the best option. A perl, awk or python program might be a better way to go.

Another option might be a pipe, in case you want e.g. to parse the output for some special strings.

Quote:

Originally Posted by figo (Post 4227467)

Otherwise, the log lines will be dropped and lines becomes concatenated into a single line, e.g., if I rewrite the script in this way, echoing the $result is not a good idea..

This is the echo's normal behaviour: it prints out the args, separated by spaces, followed by a newline. If you use double quotes to embed the variable reference, the argument passed to echo is only one and it is printed out as is, with newlines preserved. Otherwise the text is splitted into tokens using IFS as separator (default to <space><tab><newline>).

Hence, you can simply do

Code:

echo "$result"

to preserve newlines. Anyway, I second the argumentation by matthewg42 about the possible large amount of data coming from log output.