how to add numbers in a formatted file via a bash script?

zero79 · 06-08-2004, 03:18 PM

Hi all,

I need help. I have a file which contains a list of names followed by a number. I want to sum the numbers and pass that sum to a different program. The file format is

Code:

<name1> <number1>
<name2> <number2>
   .        .
   .        .
   .        .

For example, given this number_file

Code:

Joe 2
Mamma 10
Gumby 1

I would like the script to echo to me 13 (2 + 10 + 1).

I have come up with a simple script to parse out the numbers next to the names, but I can't seem to figure out how to add them together. Here are the contents of my script:

Code:

#!/bin/bash
let sum=0
cat number_file | while read line; do
  num=`echo $line | awk -F\  '{print $NF}'`
  let sum=$sum+$num
done
echo $sum

I set up the awk command so that it retieves the field after it detects a space. So that stores the numbers in num, but I think that the values are character strings. How do I convert a character string containing just a number to a number that can be used in mathematical operations?

This script, as is, yells at me at the let sum=$sum+$num line. I also tried let sum=$((sum+num)) and let sum=sum+num, none of which worked.

Any thoughts would be greatly appreciated. Peace.

homey · 06-08-2004, 04:51 PM

You really don't need awk when using read from a file. That's because you can list the fields in the read statement and pick any or all of the fields.
Make sure you don't have any blank lines at the end of the file which gets read or you will get an error when it tries to calculate things.

For example.....

Code:

#!/bin/bash

file="/home/test/file.txt"
cat ${file} | \
while read name num
do
sum=$(($sum + $num ))
echo "$sum"
done

bahramH · 06-09-2004, 02:28 AM

OK, this is not what you asked for, but why not use C ? It can be integrated nicely to any script you write, and you have so much easy control :

main(int argc, char *argv[])
{
char name[256];
int number, sum=0;
FILE input;

input=fopen(argv[1],"r");

while( fscanf(input,"%s%d",name, &number) != EOF ) sum += number;
printf("%d\n",sum);
fclose(input);
}

compile it under the name "mycount" for example, and then run "mycount myfile" in your general script.

zero79 · 06-09-2004, 11:04 AM

thanks a bunch homey. parsing directly from cat simplifies things quite nicely.

i now want to pass $sum to a program call, but outside the loop, it appears that the value of sum becomes 0. When I run this code

Code:

#!/bin/bash

file="/home/test/file.txt"
cat ${file} | \
while read name num
do
sum=$(($sum + $num ))
echo "$sum"
done
echo "$sum"
mpirun -np $sum test

I get the sum properly computed inside the loop, but outside the loop, $sum is 0. is there a way to force the variable to be available outside the loop? thanks a lot for your help.

homey · 06-09-2004, 11:26 AM

There are most likely better ways to do this

but how about redirect the output to another file and use cat to access that file. Or pipe the echo statement to your other command.....

Code:

#!/bin/bash

file="/home/test/file.txt"
cat ${file} | \
while read name num
do
sum=$(($sum + $num ))
echo "$sum" > /home/test/file2.txt
done
cat /home/test/file2.txt


or

#!/bin/bash

file="/home/test/file.txt"
cat ${file} | \
while read name num
do
sum=$(($sum + $num ))
echo "$sum" | mpirun -np $sum test
done

zero79 · 06-09-2004, 11:46 AM

thanks again homey! you're a genious. your first suggestion does the trick...not the cleanest code in the world...but i can clean it up with an "rm file2.txt" at the end of the script.

RogerThat · 05-03-2008, 04:14 PM

Here's a one-liner:

mpirun -np `awk '{sum+=$2}END{print sum}' number_file` test

colucix · 05-03-2008, 05:02 PM

Quote:

Originally Posted by zero79

i now want to pass $sum to a program call, but outside the loop, it appears that the value of sum becomes 0.

FYI, this happens because the while loop is executed after a pipe and it runs in its own subshell, with its own local variables. After exiting the subshell (when the while loop ends) all the local variables are lost.

To avoid running in a subshell, you can do something like this (using process substitution instead of the pipe):

Code:

#!/bin/bash
while read name num
do
   sum=$(($sum + $num))
   echo "$sum"
done < <(cat testfile)
echo "$sum"

Have a look at Chapter 22 of the Advanced Bash Scripting Guide for details about process substitution.

Anyway, I'd opt for the awk one-liner suggested by RogerThat.

ericwerk · 12-24-2010, 05:48 PM

bit late to the thread, but this works nicely as well without having to use awk:

for line in $(cat $1)
do
sum=$(($sum + $line ))
done
echo $sum