LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > General
User Name
Password
General This forum is for non-technical general discussion which can include both Linux and non-Linux topics. Have fun!

Notices

Reply
 
Search this Thread
Old 08-31-2006, 06:29 PM   #1
vermaden
Member
 
Registered: Jan 2006
Location: pl_PL.lodz
Distribution: FreeBSD
Posts: 392

Rep: Reputation: 80
omit / break through network speed limit


internet connection from my ISP is 750kbps speed with 25g limit, if i download 25g of data the speed changes to 32kbps, i have found a way to break through network speed limit for a short period of time. if i start downloading a file, the speed for first 2-3 seconds is a lot higher then limited 4KB/s (90-250KB/s) which is even more then 750kbps, but after these 2-3 seconds it drops down to 750/32, so i wrote little dirty script to download files with fetch, here it is:

Code:
#!/bin/sh 

while true ;do 
  fetch -F -a -q -r -R ${1} & 
  sleep 3 
  pid=`pidof fetch | cut -d ' ' -f 1` 
  kill -9 "${pid}" 
  if [ $? -ne 0 ]; then 
    exit 
  fi 
  sleep 3 
done
maybe you got some tips how to improve it or maybe know other ways to break through the network speed limit, maybe some app, some aggressive download manager, anything.

any suggestions welcome
 
Old 08-31-2006, 06:33 PM   #2
tuxdev
Senior Member
 
Registered: Jul 2005
Distribution: Slackware
Posts: 2,011

Rep: Reputation: 110Reputation: 110
I believe this crack is against LQ rules
 
Old 08-31-2006, 07:31 PM   #3
Daws
Member
 
Registered: May 2006
Location: UK
Distribution: Debian, Linux From Scratch
Posts: 416
Blog Entries: 1

Rep: Reputation: 30
Quote:
if i start downloading a file, the speed for first 2-3 seconds is a lot higher then limited 4KB/s
Are you sure this fast speed at the beginning of the download is not just the buffer catching up? After you click on the download button, the download begins while you're typing in the file name. If you take a while to do that a considerable amount of data will be saved before the download speeds start registering. That's why it seems to be zipping along at first. But, after the buffer has caught up, it will then start showing your true download speed.

But then again maybe not...
 
Old 08-31-2006, 07:50 PM   #4
vermaden
Member
 
Registered: Jan 2006
Location: pl_PL.lodz
Distribution: FreeBSD
Posts: 392

Original Poster
Rep: Reputation: 80
Quote:
Originally Posted by Daws
(...) But, after the buffer has caught up, it will then start showing your true download speed.

But then again maybe not...
well it REALLY is downloading faster for the first 2-3 seconds, thx for response btw.
 
Old 08-31-2006, 09:49 PM   #5
primo
Member
 
Registered: Jun 2005
Posts: 542

Rep: Reputation: 34
Suppose the first 8KB arrive at 0.5 seconds... That's "16K/s". You can't trust on the initial count

Also, you lose a lot of time with the initial 3-way handshakes and resets plus those signals coming in maybe before the buffer is flushed out to the filesystem. You can't avoid a website banning your IP for abuse...

You may keep the traffic to a minimum by downloading iso's to a external machine you own at work, compress it using rzip (http://rzip.samba.org/) and updating old data using xdelta (http://sourceforge.net/projects/xdelta/) or bsdiff (http://www.daemonology.net/bsdiff/). KDE is known to provide xdeltas for new releases.

Last edited by primo; 08-31-2006 at 09:53 PM.
 
Old 08-31-2006, 09:57 PM   #6
MasterC
Guru
 
Registered: Mar 2002
Location: Salt Lake City, UT - USA
Distribution: Gentoo ; LFS ; Kubuntu
Posts: 12,612

Rep: Reputation: 64
Quote:
Originally Posted by vermaden
internet connection from my ISP is 750kbps speed with 25g limit, if i download 25g of data the speed changes to 32kbps, i have found a way to break through network speed limit for a short period of time. if i start downloading a file, the speed for first 2-3 seconds is a lot higher then limited 4KB/s (90-250KB/s) which is even more then 750kbps, but after these 2-3 seconds it drops down to 750/32, so i wrote little dirty script to download files with fetch, here it is:

Code:
#!/bin/sh 

while true ;do 
  fetch -F -a -q -r -R ${1} & 
  sleep 3 
  pid=`pidof fetch | cut -d ' ' -f 1` 
  kill -9 "${pid}" 
  if [ $? -ne 0 ]; then 
    exit 
  fi 
  sleep 3 
done
maybe you got some tips how to improve it or maybe know other ways to break through the network speed limit, maybe some app, some aggressive download manager, anything.

any suggestions welcome
What are you downloading each (month I'd guess is when it resets)? Do they offer "carry over" bandwidth? Meaning, if 11 months out of the year I download 10GB/month, can I take advantage of that remaining 15GB/month I didn't use in that last month?

I can't imagine downloading 25GB of 'stuff' regularly, but I imagine that when I am testing out a couple of new distros I could come close to hitting it, but just for a single month, not continually.

If it's legitimate use that you are using the 25GB for each month, maybe you should look at switching to a more lenient provider or to a better package. Something I've noticed from ISP's around here that cap is that they don't cap (and advertise they don't) in the evening (say from 10pm to 6am). Maybe your ISP has a similar policy, or possibly you can convince them to adopt one?

The biggest problem I see with doing what you are doing is that if I saw (and I think if Smoothwall sees) someone abusing my server like that, I would block your IP, and I think smoothwall would create a rule as well to drop your IP.

Cool
 
Old 08-31-2006, 10:55 PM   #7
vermaden
Member
 
Registered: Jan 2006
Location: pl_PL.lodz
Distribution: FreeBSD
Posts: 392

Original Poster
Rep: Reputation: 80
Quote:
Originally Posted by primo
Suppose the first 8KB arrive at 0.5 seconds... That's "16K/s". You can't trust on the initial count

Also, you lose a lot of time with the initial 3-way handshakes and resets plus those signals coming in maybe before the buffer is flushed out to the filesystem.
primo:well, maybe you think that i mised something and the speed is not a lot bigger at the first 2-3 seconds, yes a "loose" a little when i must restart fetch kill it and wait 3 secs, i tried diffrent values of sleeps, but these seems to work best. the speed increase is always diffrent, sometimes it is at ~250KB/s and sometimes only ~50KB/s, but that is a lot more then even a constant 4KB/s, so it is a lot faster, lets say that i download only for 3 secs, then do nothing for 3secs + (time to kill and restart fetch, lets say 1 second here), with constant 4KB/s rate i would get 4*7=28KB/7secs, and with my dirty script, i get somehere between ~100-500KB/7secs, so it is worth it and it works, at lest in my ISP network. i do not use it if i have 750kbps, havent even tested it with 750 and dont know if it goes faster with standart speed, but with 4KB/s it does the job.

Quote:
Originally Posted by primo
You may keep the traffic to a minimum by downloading iso's to a external machine you own at work, compress it using rzip (rzip.samba.org) and updating old data using xdelta (sourceforge.net/projects/xdelta) or bsdiff (daemonology.net/bsdiff).
primo:well it is not about iso,s i am not downloading tons of distros and trying everything out, i download movies and music, especially movies takes a lot of bundwitch, btw my upload is also limited, and downloading files through p2p donkey and torrent newtorks eats a lot of upload, i most of the time decrease it to 1KB/s so it helps a little. i will try apps you mentioned, maybe i will use them in the future.

MasterC:i am very supprised about these forums, favorably supprised, i posted identical thread in gentoo forums, and after several hours it has been closed because it was as they called it not legal, and here even moderators are giving me tips, great forum, keep it that way.

Quote:
Originally Posted by MasterC
What are you downloading each (month I'd guess is when it resets)? Do they offer "carry over" bandwidth? Meaning, if 11 months out of the year I download 10GB/month, can I take advantage of that remaining 15GB/month I didn't use in that last month?
no they dont unfortunately, well i live in poland and life is really fucked up here, everything is expensive comparing to how much money people earn here, government and bureaucracy sucks, everybosy is stealing and a lot of stupid people and cathlical fanatics, trust me dont come here. never.

Quote:
Originally Posted by MasterC
I can't imagine downloading 25GB of 'stuff' regularly, but I imagine that when I am testing out a couple of new distros I could come close to hitting it, but just for a single month, not continually.
well i mentioned upper the reasons why so much KB are flying through my nic

Quote:
Originally Posted by MasterC
If it's legitimate use that you are using the 25GB for each month, maybe you should look at switching to a more lenient provider or to a better package. Something I've noticed from ISP's around here that cap is that they don't cap (and advertise they don't) in the evening (say from 10pm to 6am). Maybe your ISP has a similar policy, or possibly you can convince them to adopt one?
dont have money for that, well they include info about the limits everywherem they are although good in this.

thx for valuable tips guys
 
Old 08-31-2006, 11:06 PM   #8
primo
Member
 
Registered: Jun 2005
Posts: 542

Rep: Reputation: 34
Quote:
Originally Posted by vermaden
primo:well, maybe you think that i mised something and the speed is not a lot bigger at the first 2-3 seconds, yes a "loose" a little when i must restart fetch kill it and wait 3 secs, i tried diffrent values of sleeps, but these seems to work best. the speed increase is always diffrent, sometimes it is at ~250KB/s and sometimes only ~50KB/s, but that is a lot more then even a constant 4KB/s, so it is a lot faster, lets say that i download only for 3 secs, then do nothing for 3secs + (time to kill and restart fetch, lets say 1 second here), with constant 4KB/s rate i would get 4*7=28KB/7secs, and with my dirty script, i get somehere between ~100-500KB/7secs, so it is worth it and it works, at lest in my ISP network. i do not use it if i have 750kbps, havent even tested it with 750 and dont know if it goes faster with standart speed, but with 4KB/s it does the job.
Man, the valid math is comparing overall time with and without this hack. If you get "16 K/s" with 8 KB coming at 0.5 seconds and wget/fetch/curl/whatever updates it every second, it's clear to me that the initial values don't count.
 
Old 09-02-2006, 03:07 AM   #9
vermaden
Member
 
Registered: Jan 2006
Location: pl_PL.lodz
Distribution: FreeBSD
Posts: 392

Original Poster
Rep: Reputation: 80
any other tips to improove the script?
 
Old 09-02-2006, 04:01 AM   #10
blackzone
Member
 
Registered: Jun 2004
Posts: 256

Rep: Reputation: 30
Not trying to be sarcastic. Not sure where yo live or the price, but can't you get a better connection?
 
Old 09-04-2006, 03:35 AM   #11
vermaden
Member
 
Registered: Jan 2006
Location: pl_PL.lodz
Distribution: FreeBSD
Posts: 392

Original Poster
Rep: Reputation: 80
i was talking about improving the script / method

if i was rich i would not have to use such script because i would have 100Mbit conneciton. tips like get better connection are just, silly.
 
Old 09-05-2006, 02:08 PM   #12
slantoflight
Member
 
Registered: Aug 2005
Distribution: Smoothwall
Posts: 283
Blog Entries: 3

Rep: Reputation: 35
Quote:
Originally Posted by MasterC

The biggest problem I see with doing what you are doing is that if I saw (and I think if Smoothwall sees) someone abusing my server like that, I would block your IP, and I think smoothwall would create a rule as well to drop your IP.

Cool

Hey, you use reactive firewall as well? Its awesome isn't it?
 
  


Reply

Tags
network, networking


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Upstream speed - How do I limit ? overproof Linux - Networking 1 04-23-2005 09:31 AM
How can i Limit FTP speed for any particular USER ??? saeed Linux - Networking 10 08-02-2004 05:13 AM
Limit Apache2's Speed NightShade737 Linux - Software 1 02-08-2004 03:42 PM
How to limit upload speed for IP? wild_beast Linux - Networking 4 11-18-2003 01:14 AM
any possible way to limit wget d/l speed? ziggie216 Linux - Software 3 10-13-2003 04:27 PM


All times are GMT -5. The time now is 05:43 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration