bash shell script read file line by line.

Ramurd · 11-06-2011, 02:04 AM

altered the script here like this:

Code:

#!/bin/bash

bylet()
{
	val=0
	iter=1000000
	for((i=0;i<iter;i++))
	do
		let val+=i	
	done

	echo ${val}
}

other()
{
	val=0
	iter=1000000
	for((i=0;i<iter;i++))
	do
		((val+=i))
	done
	echo ${val}
}
time bylet
time other

To reduce on the dereferences still got me the rather big difference of:
499999500000

real 0m10.552s
user 0m10.253s
sys 0m0.236s
499999500000

real 0m8.684s
user 0m8.413s
sys 0m0.234s

That's nearly 2 seconds; While the dereferencing saved 2 seconds; The difference got halved;
Changing the for loop to the old style reduced the time equally in both:

499999500000

real 0m7.705s
user 0m7.433s
sys 0m0.176s
499999500000

real 0m5.640s
user 0m5.542s
sys 0m0.089s

The difference stays 2 seconds in an iteration of 1 million simple calculations;
(( ))-style calculations are also built-in shell and in my belief generally a much stronger method.

Things get really weird when you add a little stuff:

Code:

#!/bin/bash

bylet()
{
	val=0
	iter=1000000
	for i in {0..999999}
	do
		let val+=i
		if [[ $val > 10000000 ]]
		then
			let val=5
		fi
	done

	echo ${val}
}

other()
{
	val=0
	iter=1000000
	for i in {0..999999}
	do
		((val+=i))
		if ((val > 10000000 )) 
		then 
			((val=5)) 
		fi
	done
	echo ${val}
}
time bylet
time other

MadeInGermany · 01-09-2012, 06:47 AM

Quote:

Originally Posted by Darren[UoW]

I am using the following code to read line by line from a file, however my problem is that $value=0 at the end of the loop possibly because bash has created a subshell for the loop or something similar. How can I solve this.

value=0;

while read line
do
value=`expr $value + 1`;
echo $value;
done < "myfile"

echo $value;

Note: This example just counts the number of lines, I actually desire to do more complex processing than this though, so 'wc' is not an alternative, nor is perl im afraid.

Thanks Darren.

Indeed a while loop is a block, that can be executed in a sub shell.
A sub shell is a fork of the main shell, so it inherits the variables, but there is no easy way to copy its variables back to the main shell.
It depends on the implementation:
Traditional Unix shells make it a sub shell when the block's input/output is redirected or piped.
Your work-around then is:

Code:

value=0
#save &0 in the unused &5
exec 5<&0
# read &0 from file
exec <"myfile"
while read line
do
value=...
...
done
#restore &0
exec <&5
echo $value

Posix shells, ksh, bash, zsh only make a sub shell when the block's input/output is piped (and they support the built-in $((numeric expression)).Darren's code works.

Ksh and zsh handle the last part of a pipe in the main shell, so even this one works:

Code:

value=0
cat "myfile" | while read line
do
value=...
...
done
echo $value

keirvt · Yesterday, 06:02 PM

#!/bin/ksh
# Using kshell so that the variable value is seen outside while loop
value=0;

while read line
do
value=`expr $value + 1`;
echo $value;
done < "myfile"

echo $value;

Ramurd · Yesterday, 11:13 PM

A few notes on that last post: I think the fipo was written by engraving letters in stone and somehow they've been able to digitize this.

Next thing: use code tags, like shown below.
Backticks are soooo 2011... use $() instead

That'd make your script look like this (changed a few more things - but essentially the same)

Code:

#!/bin/ksh
#Using Korn Shell to ease scoping of variables
value=0;

while read line
do
  ((value ++))
  printf "\r%05d" ${value}
done < "myfile"

printf "\nValue of \$value after the loop: %d\n" ${value}