bash_-_new line separator?

babag · 06-10-2008, 10:45 PM

having a problem with a bash script in which i'm reading
from a text file.

Code:

i=0
ii=61
IFS="
"
#IFS=$'\n'

pathis="path/to/file"

for line in `cat $pathis/ScriptVariables.txt`;
     do
          ScriptVariables[$i]="$line"
          let i++
     done

this file, ScriptVariables.txt, has many lines,
some of which are empty.

my problem is that, when the file is read into
my script, the empty lines are skipped over. i
need to have these lines read, even if the value
is null.

how do i get this script to include the empty
lines?

as you can see, i've tried two differing methods
for declaring the IFS but they both are behaving
the same way: skipping empty lines.

thanks,
BabaG

Hko · 06-11-2008, 02:48 AM

In general, this kind of loop is not a good idea:

Code:

for line in `cat $pathis/ScriptVariables.txt`;

. Because:

Uses more memory (more lines, more memory up to the point where bash even may choke in it)
slow (every extra line slows down more)
lines with spaces in will be seen as seperate lines. (OTOH if that is what you need..., but even then there are better ways)
empty lines are skipped (your problem).

Two ways fix the above problems:

Code:

#!/bin/bash

i=1
cat /tmp/lines.txt | while read LINE ; do
    if [ "$LINE" == "" ] ; then
        echo "===EMPTY LINE==="
    else
        echo "#$i : $LINE"
    fi
    let i++
done

or:

Code:

#!/bin/bash

i=1
while read LINE ; do
    if [ "$LINE" == "" ] ; then
        echo "===EMPTY LINE==="
    else
        echo "#$i : $LINE"
    fi
    let i++
done </tmp/lines.txt

osor · 06-11-2008, 11:55 AM

With the while read loops above, you will probably want to temporarily change IFS as well (otherwise, leading spaces or tabs will be swallowed up).

babag · 06-11-2008, 01:01 PM

thanks. this is getting me there. tried this:

Code:

i=0
ii=160
IFS="
"

pathis="/home/babag/Desktop/To_LOM"

cat $pathis/ScriptVariables.txt | while read LINE ; 
do
    if [ "$LINE" == "" ] ; then
        echo "===EMPTY LINE==="
        ScriptVariables[$i]="===EMPTY LINE==="
    else
        echo "#$i : $LINE"
        ScriptVariables[$i]="$LINE"
    fi
    let i++
done

echo ${ScriptVariables[112]}

just added the ScriptVariables[$i]="$LINE" stuff to try to get
values stored. when i try the

Code:

echo ${ScriptVariables[$112]}

bit at the end, though, i get nothing. did i muck up the syntax
on that or am i assigning the array wrongly?

thanks again,
BabaG

osor · 06-11-2008, 01:11 PM

Try using redirection instead of the pipe (otherwise a subshell is started)

babag · 06-11-2008, 01:16 PM

edit:

switched to second example from Hko and seem to be able
to get that one to work. presumably that's because it
doesn't use the pipe as you point out. would still love
to see an example of how to rewrite the piped line using
redirection, though.

thanks osor. kind of dumb here. example please?

we're talking about this line?

Code:

cat $pathis/ScriptVariables.txt | while read LINE ;

thanks again,
BabaG

ntubski · 06-11-2008, 03:26 PM

Hko's second example is the way to rewrite with redirection.

by the way, you could also do

Code:

IFS="
"
ScriptVariables=($(sed 's/^$/===EMPTY LINE===/' $pathis/ScriptVariables.txt))
echo ${ScriptVariables[112]}

This has the efficiency issues Hko mentioned, but if you're putting the whole file into an array anyways I assume it's small...

osor · 06-11-2008, 03:46 PM

Quote:

Originally Posted by babag

would still love
to see an example of how to rewrite the piped line using
redirection, though.

thanks osor. kind of dumb here. example please?

When you have a pipeline, each part of the pipeline is executed in a subshell. So any variables which are created or modified in that subshell go out of scope at the end of the subshell.

If you want pipe-like behavior with redirection, some of the newer shells (e.g., bash) allow process substitution to a temporary file (/dev/fd/number).

So in bash, when you type

Code:

<(process)

a temporary file is created (in memory) with the contents of the output of process. You can now redirect from this temporary file:

Code:

< <(process)

Btw, if you are putting all lines of a file or command into an array, see this thread.

babag · 06-12-2008, 02:11 PM

thanks osor! and for the link as well.

BabaG