Monitoring internet connection speed

JoshConsulting · 07-30-2010, 11:48 PM

[POINTLESS BACKSTORY, SKIP THIS]

Quote:

I'm trying to find a good host for my site, and I've been trying to get one with a fast, reliable connection - I frequently use it as a proxy server for various areas, and since my connection is frequently a 50 Mb or 100 Mb line, I need a fast network connection so it doesn't slow down too much when I switch to the proxy server.

At the moment, I've narrowed it down to a couple of providers that are all within a few hops and <6ms away from my main location; it pretty much comes down to connection speed. One in particular that I'm trialing offers an metered 100\100 connection at a good price, so I'm hoping to go with them, but I want to make sure the connection is solid and doesn't drop to a much lower speed during the day if they are sharing it among many users.

I have a pretty simple speed test procedure - I use wget on a 1GB file hosted on a backbone with a 1GB connection, and see how fast it downloads. For now, it's sticking at a steady 11.2 MB\s or 90 Mb\s, which is fast enough for me, but I need to make sure it can maintain that speed even in times of heavy usage.

[/BACKSTORY]

Basically, I need a script that runs wget every half hour and logs the output and time in a reasonably readable format. It's probably something simple enough to do, but I'm just learning my way around the linux command shell, so some simple instructions on how to create and run such a script would be great [CentOS 5]. I have full root access to the server and I'm the only user on it if it matters.

r3sistance · 07-31-2010, 12:24 AM

I would point you at crontab for setting it up for every half an hour.

As far as I can think of...

wget http://domain.com/mytestfile &> /tmp/downloadtest.txt

would place the output into a single file, how to manipulate the data out of the data I am not sure how to do with bash scripting myself (I know other programming and scripting languages I myself could do it with but that entails going into much deeper coding, I am fairly sure a small bash script could easily handle this) but a substring command would likely be needed to pull out the actual download speed (given that it will be the characters before either MB/s or KB/s, potentially GB/s will be seen more in the future, as well as checking to ensure the download did not actually just fail).

After that a simple rm /tmp/downloadtest.txt to ensure it doesn't get overwritten.

What I might else wise suggest is there may already be a utility or service around that can do this for you, did you check for this possibility?

JoshConsulting · 07-31-2010, 02:31 AM

I just had a Eureka! moment:

Code:

wget domain.com/testfile -O /dev/null 2>&1 | grep saved | sed 's/\(s)\).*/\1/' >> /root/Desktop/speedtest.txt

Speedtest.txt output:

Quote:

00:53:44 (11.2 MB/s)

Each time it runs it adds a new line, and it's all on an easy to reach file on my desktop.

My crontab -e reads as follows:

Code:

*/5 * * * * wget domain.com/mytestfile -O /dev/null 2>&1 | grep saved | sed 's/\(s)\).*/\1/' >> /root/Desktop/speedtest.txt

I figured why not get good use out of that unmetered connection and run it every 5 minutes, after a few days it should provide an accurate picture of how fast my connection is over time.

Now, I ran into my usual problem with linux: Any time I solve a tough challenge, instead of stopping and congratulating myself, I think 'How can I make this even better?' Since I am already collecting accurate, easily formatted data showing a change or trend over time, the answer was obvious: Make it a graph! The best way to do this was to send it to excel by putting it into csv form with the required format. I wrote the following string to do so:

Code:

cat /root/Desktop/speedtest.txt | sed 's/:.. / /' | sed 's/ (/,/' |sed 's/ MB\/s)//' > /tmp/speedtest.tmp ; echo "Time:,Speed (MB/s):" | cat - /tmp/speedtest.tmp > /root/Desktop/speedtest.csv ; rm - f /tmp/speedtest.tmp

This changed this:

Quote:

01:15:10 (11.2 MB/s)
01:20:10 (11.2 MB/s)
01:25:10 (11.2 MB/s)
01:30:10 (11.2 MB/s)
01:35:10 (11.2 MB/s)
01:40:10 (11.1 MB/s)
01:45:10 (11.2 MB/s)
01:50:10 (11.2 MB/s)

Into this:

Quote:

Time:,Speed (MB/s):
01:15,11.2
01:20,10.2
01:25,11.2
01:30,11.2
01:35,11.2
01:40,11.1
01:45,11.2
01:50,11.2

With a few seconds worth of formatting in Excel, that turned into this beautiful graph:

http://i.imgur.com/6CPDZ.png

Lastly, I added the changes to my crontab file:

Code:

*/5 * * * * wget domain.com/testfile -O /dev/null 2>&1 | grep saved | sed 's/\(s)\).*/\1/' >> /root/Desktop/speedtest.txt ; cat /root/Desktop/speedtest.txt | sed 's/:.. / /' | sed 's/ (/,/' | sed 's/ MB\/s)//' > /tmp/speedtest.tmp ; echo "Time:,Speed (MB/s):" | cat - /tmp/speedtest.tmp &> /root/Desktop/speedtest.csv ; rm -f /tmp/

And I'm set! Whenever I want to check the speed logs for the past few days, I simply FTP it over and open it in Excel then make a graph out of it.

This illustrates my simultaneous love and hate for Linux. I love being able to do whatever I want in it; I hate getting sucked into it and emerging 2 hours later without any real work done on it. It's an awesome server\tweaking OS, but I can't see myself using something besides Windows for my work PC.

On the other hand, I'm quite proud of what I accomplished - I'm virtually clueless when it comes to the shell, and it took a LOT of googling and reading to, say, find out that the ; command waits for the current operation to finish before moving on to the next task. Basic stuff that linux gurus take for granted is painful to find out on demand!