LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-20-2010, 04:59 PM   #1
buee
Member
 
Registered: May 2009
Posts: 81

Rep: Reputation: 16
Issues doing simple math in Bash script


I've been trying to find the issue for the last couple of days now and I'm apparently not processing this very well. It seems as though everytime I get an error, I fix that one and another one pops up. The only thing I can think of is that there's something I'm overlooking that would set it all straight. As stupid as the script looks, it's slightly functional for me but it's also an educational experience. Please don't suggest python or perl as a fix. There has to be a way to do it within bash.

At the time of this writing, here is the script:
$downloadop is the output file from wget

Code:
cat $downloadop | awk '{print $8}' | grep M | tr -d [A-Z][a-z] > tmp.txt

##########################

zero=0
count=$(wc -l tmp.txt | cut -f1 -d' ')
((count++))

while [ $zero -lt $count ]; do
((zero++))
kilos=$(cat tmp.txt | head -$zero | tail -1 | bc -l)
kilos=$(( $kilos * 1000 ))
echo $kilos >> tmp1.txt
done

##########################

cat $downloadop | awk '{print $8}' | grep K | tr -d [A-Z][a-z] >> tmp1.txt

zero=0
count=$(wc -l tmp1.txt | cut -f1 -d' ')
((count++))
a=0

while [ $zero -lt $count ]; do
((zero++))
nextline=$(cat tmp1.txt | head -$zero | tail -1)
$nextline=$(( $nextline*1000 ))
a=$(( $a + $nextline ))
done

#rm tmp1.txt tmp.txt $downloadop

avgrate=$($a/$count)
echo "$avgrate KB/s"

kbps=$(( $avgrate*8 ))
echo "$kbps kbps"

exit 0
And these are the errors I get:
Code:
./Speedtest.sh: line 35: 600=600000: command not found
+ a=12923
+ '[' 22 -lt 13012 ']'
+ (( zero++ ))
++ cat tmp1.txt
++ head -23
++ tail -1
+ nextline=406
+ 406=406000
./Speedtest.sh: line 35: 406=406000: command not found
I'm trying to figure out why 406=406000 is even being displayed like that, let alone being processed as a command? It's not making sense.
 
Old 07-20-2010, 05:07 PM   #2
Telengard
Member
 
Registered: Apr 2007
Location: USA
Distribution: Kubuntu 8.04
Posts: 579
Blog Entries: 8

Rep: Reputation: 147Reputation: 147
Quote:
Originally Posted by line 34 or so
avgrate=$($a/$count)
Looks problematic to me. The "$( blah )" syntax means to replace with the output of the command in parens. I'm not sure exactly what you're trying to do, but I think maybe you want this:

Code:
avgrate=$(($a/$count))
The "$(( blah ))" syntax expands to the result of the mathematical expression contained in double parens.
 
Old 07-20-2010, 05:20 PM   #3
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,186

Rep: Reputation: 346Reputation: 346Reputation: 346Reputation: 346
You basic problem is that you don't need/want the $ in front of the left-hand variable in the = expression. Having it there causes bash to evaluate the variable and then try to evaluate the whole line. Since "408" is not a valid command, that fails.
 
Old 07-20-2010, 05:20 PM   #4
buee
Member
 
Registered: May 2009
Posts: 81

Original Poster
Rep: Reputation: 16
Quote:
Originally Posted by Telengard View Post
Looks problematic to me. The "$( blah )" syntax means to replace with the output of the command in parens. I'm not sure exactly what you're trying to do, but I think maybe you want this:

Code:
avgrate=$(($a/$count))
The "$(( blah ))" syntax expands to the result of the mathematical expression contained in double parens.
Basically I'm trying to convert the speed output from each line of wget to kbps or mbps rather than KB/s or MB/s. The purpose of all the math is because wget will output 1006KB/s but it will also output 1.11MB/s as opposed to 1110KB/s. So there has to be conversion somewhere in there.

While your suggestion will undoubtedly save me some errors later, it didn't change the errors I'm currently getting where the line:

Code:
$nextline=$(( $nextline*1000 ))
is being interpreted as a command for some reason.
 
Old 07-20-2010, 05:23 PM   #5
GrapefruiTgirl
LQ Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551
$nextline=$(( $nextline*1000 ))

On a line like this, the leading $ on $nextline= is causing problems - remove that. Also, while I have not found it to seem to make a difference, the $ on variables inside $(( )) evaluations is not needed. So:

Code:
nextline=$(( nextline*1000 ))
will work better.
 
Old 07-20-2010, 05:23 PM   #6
buee
Member
 
Registered: May 2009
Posts: 81

Original Poster
Rep: Reputation: 16
Quote:
Originally Posted by PTrenholme View Post
You basic problem is that you don't need/want the $ in front of the left-hand variable in the = expression. Having it there causes bash to evaluate the variable and then try to evaluate the whole line. Since "408" is not a valid command, that fails.
If I understood your suggestion correctly, I made the appropriate changes and now I'm getting the following errors:

Code:
+ cat /home/buee/Desktop/OutputFile.txt
+ awk '{print $8}'
+ grep M
+ tr -d '[A-Z][a-z]'
+ zero=0
++ wc -l tmp.txt
++ cut -f1 '-d '
+ count=864
+ (( count++ ))
./Speedtest.sh: line 21: syntax error near unexpected token `('
./Speedtest.sh: line 21: `kilos=(( $kilos * 1000 ))'
+ echo
./Speedtest.sh: line 23: syntax error near unexpected token `done'
./Speedtest.sh: line 23: `done'

Here's the code just to make sure I didn't understand correctly:

Code:
cat $downloadop | awk '{print $8}' | grep M | tr -d [A-Z][a-z] > tmp.txt

##########################

zero=0
count=$(wc -l tmp.txt | cut -f1 -d' ')
((count++))

while [ $zero -lt $count ]; do
((zero++))
kilos=$(cat tmp.txt | head -$zero | tail -1 | bc -l)
kilos=(( $kilos * 1000 ))
echo $kilos >> tmp1.txt
done

##########################

cat $downloadop | awk '{print $8}' | grep K | tr -d [A-Z][a-z] >> tmp1.txt

zero=0
count=$(wc -l tmp1.txt | cut -f1 -d' ')
((count++))
a=0

while [ $zero -lt $count ]; do
((zero++))
nextline=$(cat tmp1.txt | head -$zero | tail -1)
$nextline=(( $nextline*1000 ))
a=$(( $a + $nextline ))
done

#rm tmp1.txt tmp.txt $downloadop

avgrate=$(($a/$count))
echo "$avgrate KB/s"

kbps=$(( $avgrate*8 ))
echo "$kbps kbps"

exit 0
 
Old 07-20-2010, 05:25 PM   #7
GrapefruiTgirl
LQ Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551
Code:
kilos=(( $kilos * 1000 ))
Now the $ is missing from the front of $(( ... ))

So:
Code:
kilos=$(( kilos * 1000 ))
 
Old 07-20-2010, 06:11 PM   #8
Telengard
Member
 
Registered: Apr 2007
Location: USA
Distribution: Kubuntu 8.04
Posts: 579
Blog Entries: 8

Rep: Reputation: 147Reputation: 147
Quote:
Originally Posted by GrapefruiTgirl View Post
Also, while I have not found it to seem to make a difference, the $ on variables inside $(( )) evaluations is not needed.
Quite true, but Bash allows the "$" prefix to variable names within arithmetic expansion expressions. I prefer to use it for the sake of consistency since the "$" prefix to variable names is required in other instances such as "[ blah ]" conditional expressions.

Quote:
Originally Posted by `The GNU Bash Reference Manual' Version 3.2, section 6.5
Within an expression,
shell variables may also be referenced by name without using the
parameter expansion syntax.
 
Old 07-20-2010, 06:58 PM   #9
buee
Member
 
Registered: May 2009
Posts: 81

Original Poster
Rep: Reputation: 16
Quote:
Originally Posted by GrapefruiTgirl View Post
Code:
kilos=(( $kilos * 1000 ))
Now the $ is missing from the front of $(( ... ))

So:
Code:
kilos=$(( kilos * 1000 ))
After some tweaking, I've gotten through the script without errors. However, there appears to be some mathematical issues somewhere within the script. The end result is telling me that I'm doing 776kbps or 97KB/s. This cannot be correct. Looking through the tmp files, I'm seeing an average (just ballparking, nothing set in stone) of about 900KB/s. Now, either my script is doing something screwy with the math (rounding produces a huge margin of error perhaps?) or wget doesn't know how to distinguish KB/s (kilobytes per second) from kbps (kilobits per second). When running an HTTP speedtest from the same connection to the same server, I usually range around 9mbps which should be a little over 1MB/s, so there's definitely a rather large error somewhere. Sounds alot like the virus Michael Bolton programmed in Office Space, doesn't it?

Anyway, here's the newest script:
Code:
#!/bin/bash

set -ax

downloadop=/home/buee/Desktop/OutputFile.txt
url=http://ubuntu-releases.cs.umn.edu/10.04/ubuntu-10.04-server-amd64.iso
start=$(date '+%k:%M')

wget -v --delete-after -o $downloadop $url

cat $downloadop | awk '{print $8}' | grep M | tr -d [A-Z][a-z] > tmp.txt

##########################

zero=0
count=$(wc -l tmp.txt | cut -f1 -d' ')
((count++))

while [ $zero -lt $count ]; do
((zero++))
kilos=$(cat tmp.txt | head -$zero | tail -1)
kilos=$(echo "scale=0; $kilos*1000" | bc -l | xargs printf "%1.0f")
echo $kilos >> tmp1.txt
done

##########################

cat $downloadop | awk '{print $8}' | grep K | tr -d [A-Z][a-z] >> tmp1.txt

zero=0
count=$(wc -l tmp1.txt | cut -f1 -d' ')
((count++))
a=0

while [ $zero -lt $count ]; do
((zero++))
nextline=$(cat tmp1.txt | head -$zero | tail -1)
#nextline=$(( $nextline*1000 ))
a=$(( $a + $nextline ))
done

#rm tmp1.txt tmp.txt $downloadop

avgrate=$(echo "scale=0; $a/$count" | bc -l | xargs printf "%1.0f")
echo "$avgrate KB/s"

kbps=$(( $avgrate*8 ))
echo "$kbps kbps"

finish=$(date '+%k:%M')

echo $a $count
echo $start
echo $finish

exit 0
 
Old 07-20-2010, 08:09 PM   #10
jlinkels
Senior Member
 
Registered: Oct 2003
Location: Bonaire
Distribution: Debian Wheezy/Jessie/Sid, Linux Mint DE
Posts: 4,493

Rep: Reputation: 635Reputation: 635Reputation: 635Reputation: 635Reputation: 635Reputation: 635
Wget reports in KB/s, which is KBytes. I get this line running a similar speed test as you do:
Code:
2010-07-21 00:00:50 (470 KB/s) - `/home/jlinkels/reference.pdf' saved [23510720/23510720]
My nominal speed is 4 mbit/s

Run your script with sh -x yourscript, and much will become clear about where errors are made.

When I have to do calculations in Bash scripts which go beyond a simple addition, I use dc.
Example:
Code:
result=$("echo 12 14 * 3 / p" | dc)
(This is RPN notation)

Finally, I would propose you do not use a Linux site for speed tests. I have done so as well until I realized that I was just eating up bandwidth mostly from non-profit organizations. I use this link for speed testing:
http://download.microsoft.com/downlo...a/dotnetfx.exe

jlinkels
 
Old 07-20-2010, 11:46 PM   #11
buee
Member
 
Registered: May 2009
Posts: 81

Original Poster
Rep: Reputation: 16
Quote:
Originally Posted by jlinkels View Post
Wget reports in KB/s, which is KBytes. I get this line running a similar speed test as you do:
Code:
2010-07-21 00:00:50 (470 KB/s) - `/home/jlinkels/reference.pdf' saved [23510720/23510720]
My nominal speed is 4 mbit/s

Run your script with sh -x yourscript, and much will become clear about where errors are made.

When I have to do calculations in Bash scripts which go beyond a simple addition, I use dc.
Example:
Code:
result=$("echo 12 14 * 3 / p" | dc)
(This is RPN notation)

Finally, I would propose you do not use a Linux site for speed tests. I have done so as well until I realized that I was just eating up bandwidth mostly from non-profit organizations. I use this link for speed testing:
http://download.microsoft.com/downlo...a/dotnetfx.exe

jlinkels
Ok, so I've tweaked it here and there and got the processing time down a pretty good amount.

Code:
#!/bin/bash
#set -ax

downloadop=/home/buee/Desktop/OutputFile.txt
url=http://download.microsoft.com/download/1/6/5/165b076b-aaa9-443d-84f0-73cf11fdcdf8/WindowsXP-KB835935-SP2-ENU.exe
start=$(date '+%k:%M')

wget -v --delete-after -o $downloadop $url

downloaddone=$(date '+%k:%M')

cat $downloadop | awk '{print $8}' | grep M | tr -d [A-Z] | rl -c 1500 > tmp.txt

##########################

zero=0
count=$(wc -l tmp.txt | cut -f1 -d' ')
((count++))

while [ $zero -lt $count ]; do
((zero++))
kilos=$(cat tmp.txt | head -$zero | tail -1)
kilos=$(echo "scale=0; $kilos*1000" | bc -l | xargs printf "%1.0f")
echo $kilos >> tmp1.txt
done

##########################

cat $downloadop | awk '{print $8}' | grep K | tr -d [A-Z] | rl -c 1500 >> tmp1.txt
zero=0
count=$(wc -l tmp1.txt | cut -f1 -d' ')
((count++))
a=0

while [ $zero -lt $count ]; do
((zero++))
nextline=$(cat tmp1.txt | head -$zero | tail -1)
a=$(echo "scale=0; $a+$nextline" | bc -l | xargs printf "%1.0f")
done

rm tmp1.txt
rm tmp.txt
rm $downloadop

avgrate=$(echo "scale=0; $a/$count" | bc -l | xargs printf "%1.0f")
echo "$avgrate KB/s"

kbps=$(( $avgrate*8 ))
echo "$kbps kbps"

finish=$(date '+%k:%M')

echo $a $count
echo $start
echo $downloaddone
echo $finish

exit 0
This is working pretty well and it's not sucking bandwidth from a good company. I decided that, rather than processing 13,000K lines and doing the math on those, I could grab a random number of lines and calculate those (using rl -c 1500 which is randomize-lines in the Ubu repos. In the interest of tracking this and on the off chance someone would want to use the script, here are the results from it:

The first few lines are self explanatory
The line with seemingly random numbers is the sum of all speeds added together and the number of lines in the file
The first time (military) is when the whole script starts
The second time " " is when the download finishes and the calculations begin
The third time " " is when the whole process completes

Code:
This is with Ubuntu Distro, all lines calculated

buee@buee-desktop:~/Desktop$ ./Speedtest.sh
697 KB/s
5576 kbps
9685541 13877
22:40
22:59
23:27

This is the XP SP2, all lines calculated

965 KB/s
7720 kbps
5262099 5449
15:14
15:19
15:35

This is the XP SP2, 1000 lines calculated

1077 KB/s
8616 kbps
1080124 1002
15:54
15:58
16:01

This is the XP SP2, all lines calculated (Same Download is interpreted as above)
1045 KB/s
8360 kbps
5694547 5449
16:06
16:06
16:23

This is the XP SP2, 2000 calculated (Same Download is interpreted as above)
1027 KB/s
8216 kbps
2056891 2002
16:25
16:25
16:33

This is the XP SP2, 3000 calculated (Same Download is interpreted as above)
1024 KB/s
8192 kbps
3074153 3002
17:06
17:06
17:20

This is the XP SP2, 3000 calculated (Same Download is interpreted as above)

1046 KB/s
8368 kbps
3142354 3002
17:31
17:31
17:40

This is the XP SP2, 3000 calculated (Same Download is interpreted as above)

1017 KB/s
8136 kbps
3055990 3002
18:09
18:09
18:18
The bottom 3 entries are all from the same download files so the first two times won't change and I wanted to test randomness and accuracy. As you can see, I trimmed down the whole script run time significantly. Just adjust the number of random lines per your machine's capability and accuracy requirements and this should be a pretty good gauge of legitimate internet speeds.

Last edited by buee; 07-22-2010 at 07:09 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to do simple math in shell script gidrow Programming 5 07-30-2009 04:46 PM
script Q : how do I force bash to perform the math? kevinyeandel Linux - Newbie 4 02-20-2009 03:35 AM
bash simple math (division) question babag Programming 3 04-03-2008 06:24 PM
Need simple bash script help rickenbacherus Linux - General 1 11-01-2006 03:44 AM
Syntax for performing bash simple math calculations kinetik Linux - General 3 03-31-2006 09:03 PM


All times are GMT -5. The time now is 10:52 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration