LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (http://www.linuxquestions.org/questions/programming-9/)
-   -   bash shell script read file line by line. (http://www.linuxquestions.org/questions/programming-9/bash-shell-script-read-file-line-by-line-136784/)

Darren[UoW] 01-20-2004 02:06 PM

bash shell script read file line by line.
 
I am using the following code to read line by line from a file, however my problem is that $value=0 at the end of the loop possibly because bash has created a subshell for the loop or something similar. How can I solve this.


value=0;

while read line
do
value=`expr $value + 1`;
echo $value;
done < "myfile"

echo $value;


Note: This example just counts the number of lines, I actually desire to do more complex processing than this though, so 'wc' is not an alternative, nor is perl im afraid.


Thanks Darren.

TheOther1 01-20-2004 02:27 PM

I think value may be a reserved word, maybe? I used a=$(($a+1)); instead of value=`expr $value + 1`; but it essentially the same. Replacing value with a seemed to work:
Code:

a=0
while read line
do a=$(($a+1));
echo $a;
done < "myfile"
echo "Final line count is: $a";


Hko 01-20-2004 03:43 PM

Strange, I copied/pasted the first script (Darren's) into emacs. Put #!/bin/bash at the first line. chmod +x the script. And made a dummy "myfile" containing 23 lines of rubish.

And it worked!

(Debian testing/sarge w/ bash 2.05b.0(1) )

Strike 01-21-2004 01:42 AM

Actually, if awk is available, that would be better :)

TheOther1 01-21-2004 08:06 AM

Quote:

nor is perl im afraid.
Perl is actually excellent for text maniplulation and has sed, awk and many other native utilities capablilites built in. With modules from CPAN you do pretty close to anything with perl.

narendra_i 01-21-2004 11:58 PM

#!bin/bash

while read line
do
echo $line
done

paonethestar 01-22-2004 01:06 AM

Code:

#!/bin/sh
  echo enter file name
  read fname
 
  exec<$fname
  value=0
  while read line
  do
            value=`expr $value + 1`;
            echo $value;
  done
  echo "****$value";

Above code worked fine in Redhat 9.0 . I don't know what is ur problem here ? If u have even some more doubts at this code , inform me. Bye.

ojeknyolot 03-21-2004 11:53 PM

Bash's sh emulation (that is if a script is called with /bin/sh instead of /bin/bash) does not spawn a subshell for that kind of while loop invloving IO redirection. Traditional sh as the one found in Solaris does.

A quick and dirty solution would be to save the result in a temporary file.

Code:

#!/bin/sh
TMPFILE="/tmp/test.$$.`whoami`"

n=0
while read curline; do
    n=`expr $n + 1`
    echo "$n" > $TMPFILE
done < "yourfile"
n=`cat $TMPFILE`
echo "Total: $n"
                                                                                                                           
rm -f $TMPFILE

Still I'd like to know if there are more elegant solutions.

Cheers,

Ronny

ldp 09-22-2005 02:25 PM

Hi, I don't know if anyone still reads this but when I try to read a file line by line, I use something like "head -n $x filename | tail -n 1" where $x loops from 1 through the amount of lines in the file which can easily be found using "wc -l filename"
Does this make any sense or isn't it what you're looking for.
cheers!

ldp 09-22-2005 03:20 PM

In fact, what's bothering me is that wc doesn't just return the number of lines in the file but also the name of the file which made it unusable in a conditional statement... (damn you! damn you, wc!)
I had to trick it like this:

**********************************
#!/bin/bash
# read a file line by line

file=/mnt/hdb/backupscripts/backup.conf

x=0
lns=`wc -l $file`
y=`expr "$lns" : '\([0-9]*\)'`
while [ "$x" -lt "$y" ]
do
let x=x+1
head -n $x $file | tail -n 1
done

exit 0
***********************************

If you see a more elegant way, please let me know.

thanks.

jlliagre 09-22-2005 03:46 PM

Quote:

In fact, what's bothering me is that wc doesn't just return the number of lines in the file but also the name of the file which made it unusable in a conditional statement... (damn you! damn you, wc!)
what about:
Code:

while [ $x -lt $(cat $file | wc -l) ]
do
  ...

?

eddiebaby1023 09-25-2005 10:32 AM

Quote:

Originally posted by jlliagre
what about:
Code:

while [ $x -lt $(cat $file | wc -l) ]
do
  ...

?

The clever way to get rid of that pesky filename is not to give it!
Code:

while [ $x -lt $(wc -l <$file) ]
do
  ...


jlliagre 09-25-2005 10:41 AM

Yep, that one is better !

jtmoon 06-14-2006 04:08 PM

while read line giving giving the directory listing instead of just the file!?
 
I am having a lot of trouble with this form of the read line script.

Code:

oneLineFile=
for myFile in `dir`
do
  while read myLine
  do
    oneLineFile="$oneLineFile$myLine"
  done < $myFile
  echo $oneLineFile
  echo ------------
done

For some reason while read myLine spits out the directory listing after it has read the entire file! :confused:

For example, a file with contents
Code:

hello
from some 
file


in the directory with `dir` listing:
Code:

file1.txt
file2.txt
...
file99.txt


yields output:
hello from some file file1.txt file2.txt ... file99.txt
------------



My guess is read myLine is not returning the correct value that would exit the while loop. And so it just keeps reading on up the stack where the `dir` output is stored.

I am using
GNU bash, version 3.1.17(6)-release (i686-pc-cygwin)

-J Tom Moon

jtmoon 06-14-2006 04:38 PM

read line use `cat $file` instead of just $file
 
Nevermind! :mad:

After many stoooopid problems (too many to repeat) I finally went with this solution:

File rmLines.sh
Code:

for myFile in `dir`
do
  /helper.sh "$myFile"
done

File helper.sh
Code:

  echo 1 is $1
  cat $1 | while myLine=`line`
  do
    temp="$allFile"
    allFile="$temp $myLine"
    echo "$allFile" > ../edit/$1
  done

Perhaps this has something to do with CR/CL line endings in Windows text files?
Or just a problem with cygwin bash?

I would *swear* that my first attempt would have worked under bash on my Fedore Core machine.

-J_Tom_Moon_79

UPDATE 2009/4/22 :
See my last post in this thread for a better solution:
http://www.linuxquestions.org/questi...ml#post3517917


All times are GMT -5. The time now is 04:45 PM.