bash: variables inside while loop

J_Szucs · 07-19-2009, 05:56 AM

I am not quite unfamiliar with bash scripts, yet, I am faced with a rather lame problem.

I have a short bash script code that is something like this:

Code:

IFS="
"
cat $file | while read line; do
  len=${#line}
  let "total = $total + $len"
  echo "$total"
done
echo "$total"

My problem is that the value of variable "total" is always zero outside the loop. I.e. the final echo command always reports 0, whilst the echo command inside the while loop reports the correct total.

Are bash variables local to the while loop in this syntax? If yes, is there a workaround, to use their values outside the loop?

P.S.
The above code snippet is of course just a meaningless example to show the problem. The code would be part of a script that splits up a large text file into smaller chunks having a fixed maximum number of characters.
The "split" command has proven to be unsuitable for this purpose, as it happily breaks unicode characters in two.
Anyway, if there is an other code snipped for the purpose using bash/awk/sed, it would be welcome.

David the H. · 07-19-2009, 07:00 AM

The external call works if you use redirection instead of a pipe. I'm not sure why though. It probably has something to do with the loop running in a subshell.

Code:

IFS="
"

while read line; do
  len=${#line}
    let "total = $total + $len"
    echo "$total"
done  <$file

echo "$total"

Besides, redirection is a bash built-in function, so it's more efficient than having to call the external cat command.

Edit: The ABSG has this to say:

Quote:

A pipe runs as a child process, and therefore cannot alter script variables.

ghostdog74 · 07-19-2009, 07:04 AM

Code:

awk '{t+=length}END{print t}' file

catkin · 07-19-2009, 07:06 AM

Hello J_Szucs

cat $file | while sets up a pipeline with the while running in a subshell. Variable settings made in the subshell are lost when the subshell exits.

Using input redirect avoids the problem

Code:

while IFS='
' read line
do
  len=${#line}
  let "total = $total + $len"
  echo "$total"
done < input
echo "Loop done, total: $total"

I just found out in another thread that the above idiom saves you having to reset IFS after the loop (but you may want its usual value in the loop, in which case it's no good o)).

Edit: David the H said it while I was typing

Best

Charles

David the H. · 07-19-2009, 07:18 AM

PS: can you explain in more detail exactly what you're trying to accomplish in "splitting a large text file"? Please give us an example of the text you want to split, and the final form you want it to be in, so we can help you find the solution you need.

J_Szucs · 07-19-2009, 08:24 AM

Thanks for your help, redirection works like a charm!

P.S.
The code is part of a larger converter script that makes e-books (in the most common document formats, except lit) readable by iPod (in iPod Note format).
The e-book is first converted to an UTF8 text file, then split up to max. 4012-characters-long chunks (iPod Note), each having a html-like link at the end to the next chunk.