bash shell script read file line by line.

Darren[UoW] · 01-20-2004, 02:06 PM

I am using the following code to read line by line from a file, however my problem is that $value=0 at the end of the loop possibly because bash has created a subshell for the loop or something similar. How can I solve this.

value=0;

while read line
do
value=`expr $value + 1`;
echo $value;
done < "myfile"

echo $value;

Note: This example just counts the number of lines, I actually desire to do more complex processing than this though, so 'wc' is not an alternative, nor is perl im afraid.

Thanks Darren.

TheOther1 · 01-20-2004, 02:27 PM

I think value may be a reserved word, maybe? I used a=$(($a+1)); instead of value=`expr $value + 1`; but it essentially the same. Replacing value with a seemed to work:

Code:

a=0
while read line
do a=$(($a+1));
echo $a;
done < "myfile"
echo "Final line count is: $a";

Hko · 01-20-2004, 03:43 PM

Strange, I copied/pasted the first script (Darren's) into emacs. Put #!/bin/bash at the first line. chmod +x the script. And made a dummy "myfile" containing 23 lines of rubish.

And it worked!

(Debian testing/sarge w/ bash 2.05b.0(1) )

Strike · 01-21-2004, 01:42 AM

Actually, if awk is available, that would be better

TheOther1 · 01-21-2004, 08:06 AM

Quote:

nor is perl im afraid.

Perl is actually excellent for text maniplulation and has sed, awk and many other native utilities capablilites built in. With modules from CPAN you do pretty close to anything with perl.

narendra_i · 01-21-2004, 11:58 PM

#!bin/bash

while read line
do
echo $line
done

paonethestar · 01-22-2004, 01:06 AM

Code:

 #!/bin/sh
  echo enter file name
  read fname
   
   exec<$fname
   value=0
   while read line
   do
            value=`expr $value + 1`;
            echo $value;
   done
   echo "****$value";

Above code worked fine in Redhat 9.0 . I don't know what is ur problem here ? If u have even some more doubts at this code , inform me. Bye.

ojeknyolot · 03-21-2004, 11:53 PM

Bash's sh emulation (that is if a script is called with /bin/sh instead of /bin/bash) does not spawn a subshell for that kind of while loop invloving IO redirection. Traditional sh as the one found in Solaris does.

A quick and dirty solution would be to save the result in a temporary file.

Code:

#!/bin/sh
TMPFILE="/tmp/test.$$.`whoami`"

n=0
while read curline; do
    n=`expr $n + 1`
    echo "$n" > $TMPFILE
done < "yourfile"
n=`cat $TMPFILE`
echo "Total: $n"
                                                                                                                             
rm -f $TMPFILE

Still I'd like to know if there are more elegant solutions.

Cheers,

Ronny

ldp · 09-22-2005, 02:25 PM

Hi, I don't know if anyone still reads this but when I try to read a file line by line, I use something like "head -n $x filename | tail -n 1" where $x loops from 1 through the amount of lines in the file which can easily be found using "wc -l filename"
Does this make any sense or isn't it what you're looking for.
cheers!

ldp · 09-22-2005, 03:20 PM

In fact, what's bothering me is that wc doesn't just return the number of lines in the file but also the name of the file which made it unusable in a conditional statement... (damn you! damn you, wc!)
I had to trick it like this:

**********************************
#!/bin/bash
# read a file line by line

file=/mnt/hdb/backupscripts/backup.conf

x=0
lns=`wc -l $file`
y=`expr "$lns" : '$[0-9]*$'`
while [ "$x" -lt "$y" ]
do
let x=x+1
head -n $x $file | tail -n 1
done

exit 0
***********************************

If you see a more elegant way, please let me know.

thanks.

jlliagre · 09-22-2005, 03:46 PM

Quote:

In fact, what's bothering me is that wc doesn't just return the number of lines in the file but also the name of the file which made it unusable in a conditional statement... (damn you! damn you, wc!)

what about:

Code:

while [ $x -lt $(cat $file | wc -l) ]
do
   ...

?

eddiebaby1023 · 09-25-2005, 10:32 AM

Quote:

Originally posted by jlliagre
what about:

Code:

while [ $x -lt $(cat $file | wc -l) ] do ...

?

The clever way to get rid of that pesky filename is not to give it!

Code:

while [ $x -lt $(wc -l <$file) ]
do
  ...

jlliagre · 09-25-2005, 10:41 AM

Yep, that one is better !

jtmoon · 06-14-2006, 04:08 PM

I am having a lot of trouble with this form of the read line script.

Code:

oneLineFile= 
for myFile in `dir`
do 
  while read myLine
  do
     oneLineFile="$oneLineFile$myLine"
  done < $myFile
  echo $oneLineFile
  echo ------------
done

For some reason while read myLine spits out the directory listing after it has read the entire file!

For example, a file with contents

Code:

hello from some file

in the directory with `dir` listing:

Code:

file1.txt file2.txt ... file99.txt

yields output:
hello from some file file1.txt file2.txt ... file99.txt
------------

My guess is read myLine is not returning the correct value that would exit the while loop. And so it just keeps reading on up the stack where the `dir` output is stored.

I am using
GNU bash, version 3.1.17(6)-release (i686-pc-cygwin)

-J Tom Moon

jtmoon · 06-14-2006, 04:38 PM

Nevermind!

After many stoooopid problems (too many to repeat) I finally went with this solution:

File rmLines.sh

Code:

for myFile in `dir` 
do 
  /helper.sh "$myFile"
done

File helper.sh

Code:

  echo 1 is $1
  cat $1 | while myLine=`line`
  do
     temp="$allFile"
     allFile="$temp $myLine" 
     echo "$allFile" > ../edit/$1
  done

Perhaps this has something to do with CR/CL line endings in Windows text files?
Or just a problem with cygwin bash?

I would *swear* that my first attempt would have worked under bash on my Fedore Core machine.

-J_Tom_Moon_79

UPDATE 2009/4/22 :
See my last post in this thread for a better solution:
http://www.linuxquestions.org/questi...ml#post3517917