LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 08-06-2008, 11:46 AM   #1
BigFunkyChief
LQ Newbie
 
Registered: Oct 2006
Posts: 15

Rep: Reputation: 0
Sizing a Linux web server


My company is building a website that they hope will get 1,000,000 hits a day (I think their estimation is high, but whatever).

I have been tasked to build the webserver and size it appropriately.

I am projecting between 10,000-100,000 hits a day to start off (first 3 months) I figured I could run just one webserver to start, and monitor it and add a clustered farm of servers if the load gets too high.

So, I'm trying to figure out how much one single webserver can handle. Box is a Pentium 4, 1.5 GB RAM and 250GB hard drive. Website content is static HTML (no MYSQL connects) and fairly low bandwidth, though this could change.

I've done some load testing on a similar box using the 'ab' tool. Benchmarking 100,000 connections, 50 at a time gives me descent results (ab -n 100000 -c 50 http://localhost/), with 99% of the requests being served in less than 63ms, which is quite acceptable.

Being new to webserver benchmarking, are the below results telling me one webserver can handle 50 concurrent connections and 100,000 hits in only a few minutes? And with a basic, low bandwidth website, does anyone have any experience as to what the webserver can handle?

I need to mention the site will be hosted at a data center, with around 20MBps of bandwidth available. Maybe that's more of my bottleneck?

Hope my questions is clear...trying to figure out how many webservers I need to start and how I'll know when to add another (besides just waiting and finding out when the load is too heavy!). Thanks in advance.


AB Results:

ab -n 100000 -c 50 http://localhost/
Requests per second: 1003.06 [#/sec] (mean)
Time per request: 49.847 [ms] (mean)
Time per request: 0.997 [ms] (mean, across all concurrent requests)
Transfer rate: 4031.38 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 4.5 0 52
Processing: 10 47 6.5 49 110
Waiting: 9 45 7.5 48 106
Total: 16 48 4.2 49 124

Percentage of the requests served within a certain time (ms)
50% 49
66% 50
75% 50
80% 50
90% 51
95% 53
98% 58
99% 63
100% 124 (longest request)
 
Old 08-06-2008, 11:51 AM   #2
trickykid
LQ Guru
 
Registered: Jan 2001
Posts: 24,149

Rep: Reputation: 256Reputation: 256Reputation: 256
Quote:
Originally Posted by BigFunkyChief View Post
My company is building a website that they hope will get 1,000,000 hits a day (I think their estimation is high, but whatever).
Are they going to compete with Google? Might want to set their expectations but if they insist, I'd convince them with that many hits per day, you need not only one server but a cluster of servers with some load balancing in there.
 
Old 08-06-2008, 11:56 AM   #3
BigFunkyChief
LQ Newbie
 
Registered: Oct 2006
Posts: 15

Original Poster
Rep: Reputation: 0
Hard to say exactly how many hits they get. Obviously they are shooting for 1,000,000 a day, but seems awful high.

In any case, they've asked me to basically get the minimum hardware necessary to make this thing work in the short term, and scale up as I see fit.

So, I guess a better question is for 10,000 hits a day, one webserver should be able to handle this? (assuming low bandwidth content and that not all 10,000 hits are coming at once).
 
Old 08-06-2008, 01:36 PM   #4
trickykid
LQ Guru
 
Registered: Jan 2001
Posts: 24,149

Rep: Reputation: 256Reputation: 256Reputation: 256
Quote:
Originally Posted by BigFunkyChief View Post
Hard to say exactly how many hits they get. Obviously they are shooting for 1,000,000 a day, but seems awful high.

In any case, they've asked me to basically get the minimum hardware necessary to make this thing work in the short term, and scale up as I see fit.

So, I guess a better question is for 10,000 hits a day, one webserver should be able to handle this? (assuming low bandwidth content and that not all 10,000 hits are coming at once).
Easily. I have two dual PIII 1.13 GHz servers with only 2GB of memory that probably sees about 10,000 hits a day across all the sites I serve with very little impact or resource usage, including a database and email running on them. 10k spread out shouldn't be a problem. Now if my server got slashdotted and 10k tried to hit within an hours time, that might be a different story.
 
Old 08-06-2008, 01:51 PM   #5
chort
Senior Member
 
Registered: Jul 2003
Location: Silicon Valley, USA
Distribution: OpenBSD 4.6, OS X 10.6.2, CentOS 4 & 5
Posts: 3,660

Rep: Reputation: 69
The key here is that it's static content. That's really, really easy to handle and your current box should have no problem with that. The trick is, if someone gets the bright idea to "revamp" the site with lots of flashy dynamic content, and they think it's "no big deal, cuz the webserver is fine with our traffic", then you're in for some problems...

Also, 20Mb/s is a huge amount of bandwidth. Is that your burstable max, or the sustained? Usually bandwidth has a soft cap that you're only allowed to exceed for a certain percentage of the time. If you're constantly using your burst limit, they will charge you $$$ for it.

If you go to a more dynamic website, you're going to need a lot more RAM (4GB at least), and as many CPU cores as you can throw at it. You also have to start worrying about the database design at that point, and how you spread the data group files across your storage spindles to keep I/O from being a bottleneck.
 
Old 08-07-2008, 02:47 AM   #6
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897
Quote:
Originally Posted by chort View Post
The key here is that it's static content...
Yes, but it does make the risks higher if someone makes "minor" alterations...

Depending on what you are doing, squid in http accelerator mode to cahe outgoing content might make sense, too.

It sounds as though what you need isn't so much an estimate of "how many hits = how big a server" but an expansion plan that takes you through several stages of "when this proves inadequate, we pay x and go up to the next stage" however many hits that copes with.
 
Old 08-07-2008, 05:23 PM   #7
BigFunkyChief
LQ Newbie
 
Registered: Oct 2006
Posts: 15

Original Poster
Rep: Reputation: 0
Thanks for all the replies. This has given me some good information I can bring forward, and like suggested I should come up with a plan to expand if my single server can't handle it.

On the bandwidth...I have a lucky situation where my friend runs an ISP and she's letting me drop a server right behind her outgoing pipe, so that's where the 20MBps comes from. I believe it's burstable up to 45MBps. Yes, a sweet deal!
 
Old 08-07-2008, 05:27 PM   #8
BigFunkyChief
LQ Newbie
 
Registered: Oct 2006
Posts: 15

Original Poster
Rep: Reputation: 0
One other quick question, what's the best way to really test the capacity of my server?

Seems like sendiing requests from the localhost with ab really only tests the connections, it doesn't really test the content being served up. Once I have some test content, is there something else I can use remotely to test the webserver is see how much content it can serve before getting too slow to serve?
 
Old 08-07-2008, 08:47 PM   #9
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,735

Rep: Reputation: 2523Reputation: 2523Reputation: 2523Reputation: 2523Reputation: 2523Reputation: 2523Reputation: 2523Reputation: 2523Reputation: 2523Reputation: 2523Reputation: 2523
Well, if you know any Perl, there are modules that can automate talking to websites. I tend to use WWW::Mechanize : http://search.cpan.org/~petdance/WWW...W/Mechanize.pm.
Just knock up a Perl prog to do a request, then wrap with eg a shell loop to run as many copies as fast as you want. Its easy.
 
Old 08-08-2008, 01:23 PM   #10
kenoshi
Member
 
Registered: Sep 2007
Location: SF Bay Area, CA
Distribution: CentOS, SLES 10+, RHEL 3+, Debian Sarge
Posts: 159

Rep: Reputation: 32
http://www.opensourcetesting.org/performance.php

I use OpenSTA and sometimes httperf for a quick test.
 
Old 08-12-2008, 04:48 PM   #11
rbrodine
LQ Newbie
 
Registered: Aug 2008
Posts: 2

Rep: Reputation: 0
Here's a different way to solve the server sizing problem.



You might be interested in "Mathematical Server Sizing" software, freely available on SourceForge. Here's a link.

http://sourceforge.net/search/?type_...+server+sizing

Be sure and check out the documentation with this project. It describes some sample sizings. For a formal description of the math involved, see IEEE "Computer", July 2006 issue.

Good luck!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Linux Mail server sizing night12pm Linux - Server 1 11-28-2007 07:37 AM
can we configure a Linux server with mail server,file server and web server kumarx Linux - Newbie 5 09-09-2004 06:21 AM
Hradware sizing for the open source messaging server umeshatwork Linux - Hardware 2 11-24-2003 11:35 PM
Setup a linux server, DNS, WEB, FTP, and Mail Server Help watermelon_lee Linux - Networking 1 08-26-2003 03:09 AM
Can't access Linux web server web pages from LAN client jaydave Linux - Networking 4 03-16-2003 02:38 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 07:51 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration