LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 06-28-2010, 09:41 AM   #1
fartashh
LQ Newbie
 
Registered: Jun 2010
Posts: 3

Rep: Reputation: 0
Optimize Internet bandwidth


Dear experts
I'm software engineering student. My thesis about a system that able to work with squid and optimized the Internet bandwidth in the network.
for example the needed bandwidth in different hours of day or in different day of week is changeable for different group of computer, this system try to find out the pattern of Internet usage and allocate the suitable bandwidth to each group.

I need to do some interview with some administrator to prove this system is required and it will able to optimized the Internet bandwidth.

I need your view point as network administrator and if you interested about this system please write you email, i will send you a interview form and you will be first person will be able to use this application inside your network
 
Old 06-28-2010, 10:50 AM   #2
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,617

Rep: Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963
Quote:
Originally Posted by fartashh View Post
Dear experts
I'm software engineering student. My thesis about a system that able to work with squid and optimized the Internet bandwidth in the network.
for example the needed bandwidth in different hours of day or in different day of week is changeable for different group of computer, this system try to find out the pattern of Internet usage and allocate the suitable bandwidth to each group.

I need to do some interview with some administrator to prove this system is required and it will able to optimized the Internet bandwidth.

I need your view point as network administrator and if you interested about this system please write you email, i will send you a interview form and you will be first person will be able to use this application inside your network
Well I, for one, will not be sending my email address out to someone I don't know. Aside from spamming and other threats, why don't you offer to put the questionnaire here? If it's for real, the moderators should have no problem with it.

And if you're really doing a thesis...you should already know there are LOTS of products out there that already do this. Bandwidth throttling is easily done with Squid, and most routers/switches do QOS. You can even get a little 4 port Linksys to do it. But what you're suggesting is a bit different.

In my opinion, though, it's not got many places where it would be of use. If you're a company, you're going to want speed for EVERYONE, not just one group. And even if the software 'learns' which group needs more at what time, and throttles back...all that's going to do is skew the results. For example, groupA gets a big share of bandwidth...slowing down groupB. So the software detects that groupB needs more, and takes it away from someone else. Repeat for all groups, with everyone experiencing slowdowns and speed-ups, depending on which minute they catch the software.

Accurate monitoring of bandwidth usage is what's needed. If you start to saturate the pipe, you add more bandwidth. Get a variable rate pipe, with burst capacity.
 
Old 06-29-2010, 01:16 AM   #3
fartashh
LQ Newbie
 
Registered: Jun 2010
Posts: 3

Original Poster
Rep: Reputation: 0
At the first thanks for your answer.
I'm able to write the question here but if you don't sign the ethic form I'm not able to use it in my document. because of that i need you email.
but you are free and I appreciate for your answer and suggestion.

but I think some miss understanding happen. 1-The academic view point of project is too important. 2- this project work with analysis the squid access log file with data mining technique o it will able to provide good pattern, I do lots of research but I can't find any application do this. 3- all of the available application set the fixed bandwidth for different group. 4- you can think about different countries that Internet is expensive, they can't add more bandwidth but the need good performance. 5- i want work with the SQUID Delay-pools that is use for control the bandwidth

i write some of the question bellow but if you interest to participate I need your email otherwise i can't use it in my Document but i will be useful for me.

Do you use SQUID proxy server inside the network?
Does it help to reduce traffic inside the network?
Do you use delay pool option inside your SQUID?
Do you think the required bandwidth for each group change during the day?
Do you have any policy to change the bandwidth in different time?
Do you interest to the system to find out the pattern of the traffic and set optimized bandwidth for each group?


Thanks for your answer.
I write my email for you
fartashh@gmail.com

Last edited by fartashh; 06-29-2010 at 01:18 AM.
 
Old 06-29-2010, 09:44 AM   #4
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,617

Rep: Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963
Quote:
Originally Posted by fartashh View Post
At the first thanks for your answer. I'm able to write the question here but if you don't sign the ethic form I'm not able to use it in my document. because of that i need you email. but you are free and I appreciate for your answer and suggestion.

but I think some miss understanding happen. 1-The academic view point of project is too important.
No, no misunderstanding at all. Doing this may be interesting, but serves no real purpose.
Quote:
2- this project work with analysis the squid access log file with data mining technique o it will able to provide good pattern, I do lots of research but I can't find any application do this.
Haven't looked at SARG, or any of the Squid monitoring tools that already exist, have you? Check the Squid website.
Quote:
3- all of the available application set the fixed bandwidth for different group.
See previous post, and below.
Quote:
4- you can think about different countries that Internet is expensive, they can't add more bandwidth but the need good performance.
See previous post and below.
Quote:
5- i want work with the SQUID Delay-pools that is use for control the bandwidth
Ok...so do it. They already exist in Squid

Quote:
i write some of the question bellow but if you interest to participate I need your email otherwise i can't use it in my Document but i will be useful for me.
Again, not sending you my email, and I doubt anyone here will either. And you've missed the point from my first post. If you do ANY dynamic bandwidth throttling based on group usage, it will AUTOMATICALLY skew the data, and render it invalid and useless. Assigning more to groupA from groupB will slow down groupB. Which will THEN make groupB (on a statistical level), need more bandwidth...which will dynamically be assigned BACK to groupB. Repeat.
Quote:
Do you use SQUID proxy server inside the network?
No one will, since internal LAN's don't benefit from a single choke point
Quote:
Does it help to reduce traffic inside the network?
How would a proxy server reduce traffic? It channels the traffic that's already THERE.
Quote:
Do you use delay pool option inside your SQUID?
Would depend on how it's set up. There are no hard and fast rules.
Quote:
Do you think the required bandwidth for each group change during the day?
No, they don't, and I can tell you that from years of experience. Squid sits between a LAN and the Internet. There will be times of day that have more traffic, but again, not per group. You may see INTERNAL LAN spikes per group, but using Squid on an internal network is pointless. The cheapest LAN equipment you can buy is Gigabit copper...putting Squid to funnel all your traffic through, only REDUCES internal LAN speed.
Quote:
Do you have any policy to change the bandwidth in different time?
Yes, it's call QOS, and it already exists. You buy circuits with burst capacity, and monitor them. If you saturate the pipe on a consistent basis, you add capacity. If you can't afford to add capacity, there is NOTHING you can do from a software standpoint. If you're using 99% of a pipe, you're using 99% of a pipe....letting groupA have more than groupB does NOTHING to increase performance. Bandwidth is expensive everywhere...again, if you're using all you've got, that's it. You either suffer with bad performance, or buy more, period. The only thing you could POSSIBLY do, would be to turn OFF group access during different times, and Squid already does that.
Quote:
Do you interest to the system to find out the pattern of the traffic and set optimized bandwidth for each group?
No, again, this is pointless. Internal traffic would slow down, and external traffic doesn't need it, only a close eye on the overall utilization of that link.

Last edited by TB0ne; 06-29-2010 at 09:51 AM.
 
Old 06-29-2010, 01:31 PM   #5
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897
Quote:
Originally Posted by fartashh View Post

but I think some miss understanding happen.
You can make this less likely by writing clearly and formatting clearly so that your contribution is easy to read and understand.

Quote:
  1. The academic view point of project is too important.
  2. this project work with analysis the squid access log file with data mining technique o it will able to provide good pattern, I do lots of research but I can't find any application do this.
  3. all of the available application set the fixed bandwidth for different group.
  4. you can think about different countries that Internet is expensive, they can't add more bandwidth but the need good performance.
  5. i want work with the SQUID Delay-pools that is use for control the bandwidth
The academic view point is too important for what? Please explain.


Quote:
  1. Do you use SQUID proxy server inside the network?
  2. Does it help to reduce traffic inside the network?
  3. Do you use delay pool option inside your SQUID?
  4. Do you think the required bandwidth for each group change during the day?
  5. Do you have any policy to change the bandwidth in different time?
  6. Do you interest to the system to find out the pattern of the traffic and set optimized bandwidth for each group?
Roughly
  1. Yes
  2. No, it can't
  3. No
  4. Don't know, don't care; this could be used as an artificial way to selectively make performance worse, but I don't have an interest in this.
  5. No, it would be useless.
  6. No


Quote:
I write my email for you
xxxxxxxx@xxxxx.xxx
Ah, presumably you are new at this. Posting your e-mail address in clear text is a good way of getting spam. Or maybe you like spam?
 
0 members found this post helpful.
Old 06-30-2010, 04:37 AM   #6
fartashh
LQ Newbie
 
Registered: Jun 2010
Posts: 3

Original Poster
Rep: Reputation: 0
Dear Mr.TBOne



According to "http://sarg.sourceforge.net/" Sarg - Squid Analysis Report Generator is a tool that allow you to view "where" your users are going to on the Internet.
Sarg provides many informations about Squid users activities: times, bytes, sites, etc...
I cant find anything that said SARG use data mining technique otherwise if it use data mining it don't generate any pattern for usage of Internet.

if group B need to their bandwidth system don't decrease the bandwidth but if they have overhead when group A under pressure system will able to allocate the overhead bandwidth to group A.

could you pleas explain "If you do ANY dynamic bandwidth throttling based on group usage, it will AUTOMATICALLY skew the data, and render it invalid and useless."
 
Old 06-30-2010, 09:05 AM   #7
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,617

Rep: Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963
Quote:
Originally Posted by fartashh View Post
Dear Mr.TBOne
According to "http://sarg.sourceforge.net/" Sarg - Squid Analysis Report Generator is a tool that allow you to view "where" your users are going to on the Internet.
Sarg provides many informations about Squid users activities: times, bytes, sites, etc...
I cant find anything that said SARG use data mining technique otherwise if it use data mining it don't generate any pattern for usage of Internet.

if group B need to their bandwidth system don't decrease the bandwidth but if they have overhead when group A under pressure system will able to allocate the overhead bandwidth to group A.

could you pleas explain "If you do ANY dynamic bandwidth throttling based on group usage, it will AUTOMATICALLY skew the data, and render it invalid and useless."
I did explain it, twice. Read what I posted.
 
Old 07-28-2010, 09:04 AM   #8
teknopaul
LQ Newbie
 
Registered: Apr 2006
Posts: 4

Rep: Reputation: 0
Slightly harsh responses, to a reasonable question IMHO

I think this is not really a bad question, and quite a harsh response, I'm looking at the same thing in a real bizniz situation.

Squid makes perfect sense in our situation. We have one site with a fast LAN but with a low bandwith VPN to a central site elsewhere. The small site does not have an Internet connection for security reasons. All Internet trafic goes via the VPN and the central site and it has internal Squid proxies as caches to the Internet and firewalls and content blockers. The smaller site will also use Squid between the local LAN and the central site's Squid proxies, so that two users hitting google in the smaller site do not use the network to the central site twice to download cachable content over the VPN.

1. Do you use SQUID proxy server inside the network? YES
2. Does it help to reduce traffic inside the network? YES
3. Do you use delay pool option inside your SQUID? NO
4. Do you think the required bandwidth for each group change during the day? YES
5. Do you have any policy to change the bandwidth in different time? NO
6. Do you interest to the system to find out the pattern of the traffic and set optimized bandwidth for each group? YES
 
Old 07-28-2010, 09:10 AM   #9
djsmiley2k
Member
 
Registered: Feb 2005
Location: Coventry, UK
Distribution: Home: Gentoo x86/amd64, Debian ppc. Work: Ubuntu, SuSe, CentOS
Posts: 343
Blog Entries: 1

Rep: Reputation: 72
What he wants is QoS,

What hes asking about is Squid.

Thats the problem.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to optimize Linux internet connection ahmed_as8 Linux - Newbie 1 01-10-2009 09:00 AM
How to check internet bandwidth jojojo Linux - Newbie 4 06-07-2006 06:33 PM
measuring the internet bandwidth supersucker Linux - Newbie 1 09-01-2005 07:59 AM
Test internet bandwidth tangle Linux - Networking 1 04-13-2005 04:10 PM
internet bandwidth monitor sami Linux - Networking 3 12-30-2002 01:50 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 11:42 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration