How to limit impact of apache mirror traffic on network
Linux - NetworkingThis forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
How to limit impact of apache mirror traffic on network
I have a small home network (1 windows box, 1 linux box w/a CentOS server vm,) and would like to start learning more about networking in the specific context of limiting the negative impact of the traffic brought by the CentOS vm on my ability to still surf, watch videos or play multiplayer games on the windows pc.
I've heard tc, mod_bw, setting up a Squid proxy or replacing my router's firmware with Linux (dd-rwt or the other one who's name I can't remember at the moment) are my main options. Are these all the options I have? Should I try more than one of these at the same time? My understanding of network optimization is quite primitive. I understand netmasks, subnets, packets, packet headers, bandwidth, etc., in general terms, but these possible solutions are all new to me.
I'd like to be able to have lowest latency for playing games and watching videos, surfing the web without having to just unplug the server from the network. I should think limiting the server's bandwidth during times when I'm awake and at home would be a good start.
What are my options for optimizing network traffic? Am I headed in the right direction with those mentioned above? Would Squid cache content and help as well with latency and bandwidth consumption, i.e., whatever destroys my ability to play online multiplayer games, watch youtube videos, surf?
My apache mirror gets about 15,000 unique visitors per month, about 250,000 DNS requests in August. I have a cable modem internet connection that runs at about 4 or 5 Mbps.
P.S.-- One of my other perhaps related self-appointed tasks is to either optimize my apache installation or repalce it with something leaner like nginx (which I read can also server as a reverse proxy.) If there is a thread of commonality here, can anyone point me to search terms or links to basic how-tos for learning about these things-- two birds-- server optimization/network optimization, one stone (squid, or nginx?) works well with such limited free time.
Anyone familiar with nginx, apache, squid, web server optimization and network optimization who could illuminate the finer points of consideration in choosing which direction to take this?
Last edited by mbvpixies78; 09-21-2013 at 02:42 PM.
If you're at the stage where you're serving web traffic from home and starting to have issues with traffic then perhaps it's time to consider out-hosting your website on a VPS somewhere.
But surely there is a relatively simple way to do this. I like having control over the server, doing it myself and having maximum learning potential hosting from home. That's the point after all-- to learn. I'll just pick one of the ideas I mentioned above and go with it, see what happens.
I like having control over the server, doing it myself and having maximum learning potential hosting from home.
Indeed, totally agree! My point was that if you've a website that getting to the traffic levels you mention then it would make sense to outhost that somewhere purely from an availability and bandwidth point of view. Keep your home server for learning and keep your busy website on your "production" servers. That's sort of what I do, I have a small VPS for fiddling around with and two dedicated servers (Germany and US) for the production site.
(Oh, and a RasPi for fiddling with and countless CentOS VMs for messing around with)
You could also look at mod_bandwidth for Apache, I THINK that would give you some control.
Indeed, totally agree! My point was that if you've a website that getting to the traffic levels you mention then it would make sense to outhost that somewhere purely from an availability and bandwidth point of view. Keep your home server for learning and keep your busy website on your "production" servers. That's sort of what I do, I have a small VPS for fiddling around with and two dedicated servers (Germany and US) for the production site.
Ok, I get where you're coming from now. Hmm. Last year I spent $45, This year I got it down to $15 (so far.) Will have to check prices on outsourcing (lol, funny to ever have a use for that word... makes me feel... unpatriotic?)
Quick question though-- not sure what constitutes a lot of traffic for a residential cable modem connection. Is a change to outsourced hosting going to decrease occurence of 404 errors? (Will mod_bandwidth or nginx affect this? Or are all websites bound to have a few 404's? By few-- going from memory here-- I calculated it to be something like 1% of traffic.)
Quote:
(Oh, and a RasPi for fiddling with and countless CentOS VMs for messing around with)
You could also look at mod_bandwidth for Apache, I THINK that would give you some control.
Have been eyeballing Raspberry Pi, thinking about getting one. What I might do is set up mod_bandwidth and then create another vm using nginx instead of apache for comparison. I would guess nginx will do better, but knowing apache is more useful as a marketable IT skill.
Last edited by mbvpixies78; 09-26-2013 at 11:30 AM.
Ok, I get where you're coming from now. Hmm. Last year I spent $45, This year I got it down to $15 (so far.) Will have to check prices on outsourcing (lol, funny to ever have a use for that word... makes me feel... unpatriotic?)
Why? Outsource to an in-country company I just happen to outsorce to where the best rates are (Germany and US, I'm in Ireland)
Quote:
Originally Posted by mbvpixies78
Is a change to outsourced hosting going to decrease occurence of 404 errors? (Will mod_bandwidth or nginx affect this? Or are all websites bound to have a few 404's? By few-- going from memory here-- I calculated it to be something like 1% of traffic.)
1% seems high, a lot depends on where exactly these 404's are being generated. Is the website dynamic content generated off of PHP scripts / database? If so then a faster server may help. Hard to say without an exact analysis.
Quote:
Originally Posted by mbvpixies78
Have been eyeballing Raspberry Pi, thinking about getting one.
Great for learning, but I'd never host my "production" site on one.
seems high, a lot depends on where exactly these 404's are being generated. Is the website dynamic content generated off of PHP scripts / database? If so then a faster server may help. Hard to say without an exact analysis.
All 404's are coming from the static apache mirror, updated twice daily (just occurred to me to see if 404's correspond at all to those twice-daily updates when bandwidth is definitely being challenged.)
For whatever reason Apache hasn't updated their list with my new domain name so for now this concern isn't even a concern. Another thing I might do is just mirror a restricted portion of the mirror, taking less bandwidth-intensive files.
I'm not quite sure what I'll do with a Raspberry Pi if I get one, not yet... maybe a Linux proxy server?
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.