LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Issues doing simple math in Bash script (https://www.linuxquestions.org/questions/linux-newbie-8/issues-doing-simple-math-in-bash-script-821098/)

buee 07-20-2010 03:59 PM

Issues doing simple math in Bash script
 
I've been trying to find the issue for the last couple of days now and I'm apparently not processing this very well. It seems as though everytime I get an error, I fix that one and another one pops up. The only thing I can think of is that there's something I'm overlooking that would set it all straight. As stupid as the script looks, it's slightly functional for me but it's also an educational experience. Please don't suggest python or perl as a fix. There has to be a way to do it within bash.

At the time of this writing, here is the script:
$downloadop is the output file from wget

Code:

cat $downloadop | awk '{print $8}' | grep M | tr -d [A-Z][a-z] > tmp.txt

##########################

zero=0
count=$(wc -l tmp.txt | cut -f1 -d' ')
((count++))

while [ $zero -lt $count ]; do
((zero++))
kilos=$(cat tmp.txt | head -$zero | tail -1 | bc -l)
kilos=$(( $kilos * 1000 ))
echo $kilos >> tmp1.txt
done

##########################

cat $downloadop | awk '{print $8}' | grep K | tr -d [A-Z][a-z] >> tmp1.txt

zero=0
count=$(wc -l tmp1.txt | cut -f1 -d' ')
((count++))
a=0

while [ $zero -lt $count ]; do
((zero++))
nextline=$(cat tmp1.txt | head -$zero | tail -1)
$nextline=$(( $nextline*1000 ))
a=$(( $a + $nextline ))
done

#rm tmp1.txt tmp.txt $downloadop

avgrate=$($a/$count)
echo "$avgrate KB/s"

kbps=$(( $avgrate*8 ))
echo "$kbps kbps"

exit 0

And these are the errors I get:
Code:

./Speedtest.sh: line 35: 600=600000: command not found
+ a=12923
+ '[' 22 -lt 13012 ']'
+ (( zero++ ))
++ cat tmp1.txt
++ head -23
++ tail -1
+ nextline=406
+ 406=406000
./Speedtest.sh: line 35: 406=406000: command not found

I'm trying to figure out why 406=406000 is even being displayed like that, let alone being processed as a command? It's not making sense.

Telengard 07-20-2010 04:07 PM

Quote:

Originally Posted by line 34 or so
avgrate=$($a/$count)

Looks problematic to me. The "$( blah )" syntax means to replace with the output of the command in parens. I'm not sure exactly what you're trying to do, but I think maybe you want this:

Code:

avgrate=$(($a/$count))
The "$(( blah ))" syntax expands to the result of the mathematical expression contained in double parens.

PTrenholme 07-20-2010 04:20 PM

You basic problem is that you don't need/want the $ in front of the left-hand variable in the = expression. Having it there causes bash to evaluate the variable and then try to evaluate the whole line. Since "408" is not a valid command, that fails.

buee 07-20-2010 04:20 PM

Quote:

Originally Posted by Telengard (Post 4039946)
Looks problematic to me. The "$( blah )" syntax means to replace with the output of the command in parens. I'm not sure exactly what you're trying to do, but I think maybe you want this:

Code:

avgrate=$(($a/$count))
The "$(( blah ))" syntax expands to the result of the mathematical expression contained in double parens.

Basically I'm trying to convert the speed output from each line of wget to kbps or mbps rather than KB/s or MB/s. The purpose of all the math is because wget will output 1006KB/s but it will also output 1.11MB/s as opposed to 1110KB/s. So there has to be conversion somewhere in there.

While your suggestion will undoubtedly save me some errors later, it didn't change the errors I'm currently getting where the line:

Code:

$nextline=$(( $nextline*1000 ))
is being interpreted as a command for some reason.

GrapefruiTgirl 07-20-2010 04:23 PM

$nextline=$(( $nextline*1000 ))

On a line like this, the leading $ on $nextline= is causing problems - remove that. Also, while I have not found it to seem to make a difference, the $ on variables inside $(( )) evaluations is not needed. So:

Code:

nextline=$(( nextline*1000 ))
will work better.

buee 07-20-2010 04:23 PM

Quote:

Originally Posted by PTrenholme (Post 4039956)
You basic problem is that you don't need/want the $ in front of the left-hand variable in the = expression. Having it there causes bash to evaluate the variable and then try to evaluate the whole line. Since "408" is not a valid command, that fails.

If I understood your suggestion correctly, I made the appropriate changes and now I'm getting the following errors:

Code:

+ cat /home/buee/Desktop/OutputFile.txt
+ awk '{print $8}'
+ grep M
+ tr -d '[A-Z][a-z]'
+ zero=0
++ wc -l tmp.txt
++ cut -f1 '-d '
+ count=864
+ (( count++ ))
./Speedtest.sh: line 21: syntax error near unexpected token `('
./Speedtest.sh: line 21: `kilos=(( $kilos * 1000 ))'
+ echo
./Speedtest.sh: line 23: syntax error near unexpected token `done'
./Speedtest.sh: line 23: `done'


Here's the code just to make sure I didn't understand correctly:

Code:

cat $downloadop | awk '{print $8}' | grep M | tr -d [A-Z][a-z] > tmp.txt

##########################

zero=0
count=$(wc -l tmp.txt | cut -f1 -d' ')
((count++))

while [ $zero -lt $count ]; do
((zero++))
kilos=$(cat tmp.txt | head -$zero | tail -1 | bc -l)
kilos=(( $kilos * 1000 ))
echo $kilos >> tmp1.txt
done

##########################

cat $downloadop | awk '{print $8}' | grep K | tr -d [A-Z][a-z] >> tmp1.txt

zero=0
count=$(wc -l tmp1.txt | cut -f1 -d' ')
((count++))
a=0

while [ $zero -lt $count ]; do
((zero++))
nextline=$(cat tmp1.txt | head -$zero | tail -1)
$nextline=(( $nextline*1000 ))
a=$(( $a + $nextline ))
done

#rm tmp1.txt tmp.txt $downloadop

avgrate=$(($a/$count))
echo "$avgrate KB/s"

kbps=$(( $avgrate*8 ))
echo "$kbps kbps"

exit 0


GrapefruiTgirl 07-20-2010 04:25 PM

Code:

kilos=(( $kilos * 1000 ))
Now the $ is missing from the front of $(( ... ))

So:
Code:

kilos=$(( kilos * 1000 ))

Telengard 07-20-2010 05:11 PM

Quote:

Originally Posted by GrapefruiTgirl (Post 4039960)
Also, while I have not found it to seem to make a difference, the $ on variables inside $(( )) evaluations is not needed.

Quite true, but Bash allows the "$" prefix to variable names within arithmetic expansion expressions. I prefer to use it for the sake of consistency since the "$" prefix to variable names is required in other instances such as "[ blah ]" conditional expressions.

Quote:

Originally Posted by `The GNU Bash Reference Manual' Version 3.2, section 6.5
Within an expression,
shell variables may also be referenced by name without using the
parameter expansion syntax.


buee 07-20-2010 05:58 PM

Quote:

Originally Posted by GrapefruiTgirl (Post 4039962)
Code:

kilos=(( $kilos * 1000 ))
Now the $ is missing from the front of $(( ... ))

So:
Code:

kilos=$(( kilos * 1000 ))

After some tweaking, I've gotten through the script without errors. However, there appears to be some mathematical issues somewhere within the script. The end result is telling me that I'm doing 776kbps or 97KB/s. This cannot be correct. Looking through the tmp files, I'm seeing an average (just ballparking, nothing set in stone) of about 900KB/s. Now, either my script is doing something screwy with the math (rounding produces a huge margin of error perhaps?) or wget doesn't know how to distinguish KB/s (kilobytes per second) from kbps (kilobits per second). When running an HTTP speedtest from the same connection to the same server, I usually range around 9mbps which should be a little over 1MB/s, so there's definitely a rather large error somewhere. Sounds alot like the virus Michael Bolton programmed in Office Space, doesn't it?

Anyway, here's the newest script:
Code:

#!/bin/bash

set -ax

downloadop=/home/buee/Desktop/OutputFile.txt
url=http://ubuntu-releases.cs.umn.edu/10.04/ubuntu-10.04-server-amd64.iso
start=$(date '+%k:%M')

wget -v --delete-after -o $downloadop $url

cat $downloadop | awk '{print $8}' | grep M | tr -d [A-Z][a-z] > tmp.txt

##########################

zero=0
count=$(wc -l tmp.txt | cut -f1 -d' ')
((count++))

while [ $zero -lt $count ]; do
((zero++))
kilos=$(cat tmp.txt | head -$zero | tail -1)
kilos=$(echo "scale=0; $kilos*1000" | bc -l | xargs printf "%1.0f")
echo $kilos >> tmp1.txt
done

##########################

cat $downloadop | awk '{print $8}' | grep K | tr -d [A-Z][a-z] >> tmp1.txt

zero=0
count=$(wc -l tmp1.txt | cut -f1 -d' ')
((count++))
a=0

while [ $zero -lt $count ]; do
((zero++))
nextline=$(cat tmp1.txt | head -$zero | tail -1)
#nextline=$(( $nextline*1000 ))
a=$(( $a + $nextline ))
done

#rm tmp1.txt tmp.txt $downloadop

avgrate=$(echo "scale=0; $a/$count" | bc -l | xargs printf "%1.0f")
echo "$avgrate KB/s"

kbps=$(( $avgrate*8 ))
echo "$kbps kbps"

finish=$(date '+%k:%M')

echo $a $count
echo $start
echo $finish

exit 0


jlinkels 07-20-2010 07:09 PM

Wget reports in KB/s, which is KBytes. I get this line running a similar speed test as you do:
Code:

2010-07-21 00:00:50 (470 KB/s) - `/home/jlinkels/reference.pdf' saved [23510720/23510720]
My nominal speed is 4 mbit/s

Run your script with sh -x yourscript, and much will become clear about where errors are made.

When I have to do calculations in Bash scripts which go beyond a simple addition, I use dc.
Example:
Code:

result=$("echo 12 14 * 3 / p" | dc)
(This is RPN notation)

Finally, I would propose you do not use a Linux site for speed tests. I have done so as well until I realized that I was just eating up bandwidth mostly from non-profit organizations. I use this link for speed testing: :)
http://download.microsoft.com/downlo...a/dotnetfx.exe

jlinkels

buee 07-20-2010 10:46 PM

Quote:

Originally Posted by jlinkels (Post 4040097)
Wget reports in KB/s, which is KBytes. I get this line running a similar speed test as you do:
Code:

2010-07-21 00:00:50 (470 KB/s) - `/home/jlinkels/reference.pdf' saved [23510720/23510720]
My nominal speed is 4 mbit/s

Run your script with sh -x yourscript, and much will become clear about where errors are made.

When I have to do calculations in Bash scripts which go beyond a simple addition, I use dc.
Example:
Code:

result=$("echo 12 14 * 3 / p" | dc)
(This is RPN notation)

Finally, I would propose you do not use a Linux site for speed tests. I have done so as well until I realized that I was just eating up bandwidth mostly from non-profit organizations. I use this link for speed testing: :)
http://download.microsoft.com/downlo...a/dotnetfx.exe

jlinkels

Ok, so I've tweaked it here and there and got the processing time down a pretty good amount.

Code:

#!/bin/bash
#set -ax

downloadop=/home/buee/Desktop/OutputFile.txt
url=http://download.microsoft.com/download/1/6/5/165b076b-aaa9-443d-84f0-73cf11fdcdf8/WindowsXP-KB835935-SP2-ENU.exe
start=$(date '+%k:%M')

wget -v --delete-after -o $downloadop $url

downloaddone=$(date '+%k:%M')

cat $downloadop | awk '{print $8}' | grep M | tr -d [A-Z] | rl -c 1500 > tmp.txt

##########################

zero=0
count=$(wc -l tmp.txt | cut -f1 -d' ')
((count++))

while [ $zero -lt $count ]; do
((zero++))
kilos=$(cat tmp.txt | head -$zero | tail -1)
kilos=$(echo "scale=0; $kilos*1000" | bc -l | xargs printf "%1.0f")
echo $kilos >> tmp1.txt
done

##########################

cat $downloadop | awk '{print $8}' | grep K | tr -d [A-Z] | rl -c 1500 >> tmp1.txt
zero=0
count=$(wc -l tmp1.txt | cut -f1 -d' ')
((count++))
a=0

while [ $zero -lt $count ]; do
((zero++))
nextline=$(cat tmp1.txt | head -$zero | tail -1)
a=$(echo "scale=0; $a+$nextline" | bc -l | xargs printf "%1.0f")
done

rm tmp1.txt
rm tmp.txt
rm $downloadop

avgrate=$(echo "scale=0; $a/$count" | bc -l | xargs printf "%1.0f")
echo "$avgrate KB/s"

kbps=$(( $avgrate*8 ))
echo "$kbps kbps"

finish=$(date '+%k:%M')

echo $a $count
echo $start
echo $downloaddone
echo $finish

exit 0

This is working pretty well and it's not sucking bandwidth from a good company. I decided that, rather than processing 13,000K lines and doing the math on those, I could grab a random number of lines and calculate those (using rl -c 1500 which is randomize-lines in the Ubu repos. In the interest of tracking this and on the off chance someone would want to use the script, here are the results from it:

The first few lines are self explanatory
The line with seemingly random numbers is the sum of all speeds added together and the number of lines in the file
The first time (military) is when the whole script starts
The second time " " is when the download finishes and the calculations begin
The third time " " is when the whole process completes

Code:

This is with Ubuntu Distro, all lines calculated

buee@buee-desktop:~/Desktop$ ./Speedtest.sh
697 KB/s
5576 kbps
9685541 13877
22:40
22:59
23:27

This is the XP SP2, all lines calculated

965 KB/s
7720 kbps
5262099 5449
15:14
15:19
15:35

This is the XP SP2, 1000 lines calculated

1077 KB/s
8616 kbps
1080124 1002
15:54
15:58
16:01

This is the XP SP2, all lines calculated (Same Download is interpreted as above)
1045 KB/s
8360 kbps
5694547 5449
16:06
16:06
16:23

This is the XP SP2, 2000 calculated (Same Download is interpreted as above)
1027 KB/s
8216 kbps
2056891 2002
16:25
16:25
16:33

This is the XP SP2, 3000 calculated (Same Download is interpreted as above)
1024 KB/s
8192 kbps
3074153 3002
17:06
17:06
17:20

This is the XP SP2, 3000 calculated (Same Download is interpreted as above)

1046 KB/s
8368 kbps
3142354 3002
17:31
17:31
17:40

This is the XP SP2, 3000 calculated (Same Download is interpreted as above)

1017 KB/s
8136 kbps
3055990 3002
18:09
18:09
18:18

The bottom 3 entries are all from the same download files so the first two times won't change and I wanted to test randomness and accuracy. As you can see, I trimmed down the whole script run time significantly. Just adjust the number of random lines per your machine's capability and accuracy requirements and this should be a pretty good gauge of legitimate internet speeds.


All times are GMT -5. The time now is 08:25 AM.