Bash maths question

kinetik · 01-19-2009, 08:57 AM

Hi all

I find myself a bit stuck again.

I got a text file containing something along the lines of this:

Code:

I'm trying to write a simple bash script that will give me a total of all the values. The machine I'm trying to do this on doesn't have bc installed, so I'm afraid I'm limited to taking shortcuts in the script only.

Here's what I got so far:

Code:

tail -n 50 /path/to/file.log | awk {'print $3'} | sed -n '/Rsp/!p' | sed -n '/--/!p' | sed -n '/\//!p' | sed -e '/^$/d' > /path/to/values.log

while read num
do
sum=$(($sum + $num))
echo "$sum"
done < <(cat /path/to/values.log)
clear
echo "$sum"

Problem is it doesn't seem like the .[0-9] is liked very much by the while do statement.

Is there a way I can get this to work with [0-9].[0-9]* values?

Thanks in advance

EDIT: My first language isn't English, turns out .something is floating point math? And BASH can't do floating point math? That kind of sucks, especially since I do not have bc at my disposal...

H_TeXMeX_H · 01-19-2009, 09:29 AM

Does that machine have awk installed, if so, you can do it quite easily.

Code:

bash-3.1$ echo 12.344 123.344 8585.4848 | awk '{ printf("%0.5f\n",$1 + $2 + $3) }'
8721.17280

ErV · 01-19-2009, 09:42 AM

Quote:

Originally Posted by kinetik

And BASH can't do floating point math? That kind of sucks, especially since I do not have bc at my disposal...

You could use python (if you have it) - it is perfectly suitable for your task.

kinetik · 01-19-2009, 10:07 AM

First of all, thanks ErV and H_TeXMeX_H for the assists, much appreciated.

I'm first going to try to tackle this in another way and will keep you posted whether that worked or not. If it doesn't work (and I doubt it will), I'll definitely fire some more questions in here.

Thanks again for the help so far.

makyo · 01-19-2009, 10:33 AM

Hi.

And if it doesn't work, then perhaps you could tell us what you have available among the likely suspects such as awk, python, perl, ksh, zsh, etc ... cheers, makyo

PTrenholme · 01-19-2009, 11:14 AM

If your numbers are all of the form [0-9]*.[0-9]{0,4} you could change them to integers by concatenating "0000"to the input number, truncating the result four digits after the "decimal point" and then removing the "decimal point." Then the total would be 10000 * the desire total, and you could jam in a "decimal point" in the total if you need to do so.

Since you seem to have sed installed, implementing this "convert input to 10000*(input)" scheme should be fairly simple to implement.

With a little effort you could implement you whole "find the total"in sed, although it might be tedious. Look at the "Incrementing a number" example in the "Sample scripts" section of info sed for an example of how n <- n+1 can be done with sed. (Note: I don't recommend using sed, just saying it's possible.)

kinetik · 01-20-2009, 02:57 AM

Hi Makyo and PTrenholme

I'm glad to say I got it working (sort of) by using sed. I think the system does have Python, awk, ksh, zsh etc. installed, just not bc for some daft reason.

I doubt this is going to be of any use to anyone else in future, but here's how the script turned out:

Code:

## totala.sh
tail -n 50 /path/to/file.log | awk {'print $3'} | sed -n '/^[0-9]/p' | sed "s|0$||g" | sed -n '/\.$/!p' | sed -e '/^$/d' | sed "s|\(\.[0-9]$\)|\10|g" | sed "s|\(\.[0-9][0-9]$\)|\10|g" | sed "s|^0\.||g" | sed "s|^0||g" | sed "s|^0||g" | sed "s|^0||g" | sed "s|\.||g" > /path/to/values.log

while read num
do
sum=$(($sum + $num))
echo "$sum"
done < <(cat /path/to/values.log)
clear
echo "$sum" > /path/to/values1.log

OK, if I got this right, here's what the sed commands in the script does:

sed -n '/^[0-9]/p' <-- Only print lines beginning with numbers
sed "s|0$||g" <-- Replace any ending 0 with nothing
sed -n '/\.$/!p' <-- Don't print anything ending with a "." And this one doesn't look quite right now that I look at it
sed -e '/^$/d' <-- Take out any blank lines
sed "s|$\.[0-9]$$|\10|g" <-- Adds a 0 at the end of numbers like .4
sed "s|$\.[0-9][0-9]$$|\10|g" <-- Adds a 0 at the end of numbers like .44
sed "s|^0\.||g" <-- Removes "0." from all the numbers Doesn't look quite right either
sed "s|^0||g" <-- Removes any 0 at the beginning of the numbers
sed "s|^0||g" <-- Same as above, just does it again to be safe
sed "s|\.||g" <-- Removes the .

Anyhoo, I made a second script that will run the above script, let's call it totalb.sh:

Code:

/path/to/totala.sh > /dev/null 2>&1
cat /path/to/values1.log | sed ':a;s/\(^\|[^0-9.]\)\([0-9]\+\)\([0-9]\{3\}\)/\1\2,\3/g;ta'

I think the script works OK-ish for a file containing multiple values, but the problem comes in when it only contains things like 0.1 and 0.009 or something.

Thanks to everyone that helped me out (colucix, H_TeXMeX_H, ErV, makyo, PTrenholme), I really appreciate it. This chapter isn't over yet though, so if you don't mind I'm probably going to fire some Python and awk questions your way soon.

H_TeXMeX_H · 01-20-2009, 03:21 AM

Quote:

Originally Posted by kinetik

I'm trying to write a simple bash script that will give me a total of all the values. The machine I'm trying to do this on doesn't have bc installed, so I'm afraid I'm limited to taking shortcuts in the script only.

EDIT: My first language isn't English, turns out .something is floating point math? And BASH can't do floating point math? That kind of sucks, especially since I do not have bc at my disposal...

.....

Code:

/path/to/totala.sh > /dev/null 2>&1
cat /path/to/values1.log | sed ':a;s/\(^\|[^0-9.]\)\([0-9]\+\)\([0-9]\{3\}\)/\1\2,\3/g;ta'

Look, it's not that complicated, here's what I would do:

Code:

#!/bin/awk -f
BEGIN {
# initialize total
    total=0;
}
{
# this code is executed once for each line
# add field 1 to the total for each line
    total+=$1;
}
END {
# end, now output the total
    print "The total is: ", total;
}

I've checked it and it prints out the right total. If you want to format the output use printf instead of print.

I used a script from here to start off:
http://www.grymoire.com/Unix/Awk.html
If you need more help, check there.

kinetik · 01-20-2009, 04:08 AM

Quote:

Originally Posted by H_TeXMeX_H

Look, it's not that complicated, here's what I would do:

Code:

#!/bin/awk -f
BEGIN {
# initialize total
    total=0;
}
{
# this code is executed once for each line
# add field 1 to the total for each line
    total+=$1;
}
END {
# end, now output the total
    print "The total is: ", total;
}

I've checked it and it prints out the right total. If you want to format the output use printf instead of print.

I used a script from here to start off:
http://www.grymoire.com/Unix/Awk.html
If you need more help, check there.

You sir, I owe you a beer. Thanks H_TexMeX_H

H_TeXMeX_H · 01-20-2009, 04:43 AM

No problem. And remember that in most programming or scripting languages that are sane, when things start to get really complicated, obscure, and unreadable (as illustrated above) then perhaps you have taken a wrong turn somewhere, perhaps you are missing a simpler alternative.