Apache Download Server Performance Issues
OK, lengthy issue so I'll try to frontload all of the details you'll need.
I run a large website, antsmarching.org, and through our membership system we offer downloads of nearly 600 live DMB shows. A generous fan was gracious enough to offer up his connection to the internet (which is massive since we works at a university) and some spare machines to host the shows. Users can download either via a webpage on our site (one song at a time), or by an application that we wrote that queues up an entire show and pulls it down for the user.
Here is how the network of download boxes is currently setup:
-450mhz G4 w/ 512MB ram - Mandrake PPC 8.2
This machine handles all requests (PHP is what our filesystem webscript is written in) for shows. Connected to this box are two NAS devices which actually house the shows. They are connected via NFS and mounted as a local drive. These devices house shows from the years 1990-2002. All downloads from these years are piped thru the G4. The G4 is responsive and holds up very well against high traffic and usage and continually dumps great rates.
We recently added a separate machine for 2003 alone. Since the summer tour is ongoing at the moment, 2003 shows are the highest in demand and thus we decided to try letting the year have its own server. The server specs:
900Mhz Athlon, 384MB ram, 120GB 7200RPM 8MB cache IDE drive. Mandrake 8.2. Now, there is an issue w/ the mobo because there are 3 256dimm sticks in there, but it only picks up half of the RAM. But the machine itself still runs. More on that later.
Now, the 2003 box will be both handling the PHP requests, as well as disk access, unlike the G4 setup, which has the g4 handling requests, and the two NAS devices handling disk access. I installed Apache 1.3.8 and PHP 4.3.2 on the 2003 box, and basically made identical adjustments to the httpd.conf file as they are on the g4. Upon starting up apache, and opening up the 2003 box on the website, after roughly 5-10 minutes, the load begins to climb, and the machine eventually is impossible to bring back - as logging in via SSH times out before it can actually prompt me for a login. When we shut off apache, all goes back to normal. Using "top" to monitor right after we started apache shows that our 2gb swap partition doesn't get touched, but the RAM stays at nearly 100% usage.
Ok, so I tried pulling off apache 1.3.8 and trying apache2, just for the heck of it. The difference? apache2 seemed to like using the swap, which was fine, and the machine was plugging along well until the entire 2gb of swap was used up, which then caused the load to once again skyrocket and brought the machine to its knees.
So here are my thoughts: apache2 correctly uses the swap space, but doesn't like to give it back to the system. Also, why isnt apache1.3.8 using it? Another thing might be - is the issue thats causing the system not to recognize all 768 MB of RAM (mobo most likely) also causing the system to perform poorly under any stress at all? a bus issue that would be resolved by replacing the mobo? (which not only would cause all the RAM to be recognized, but also fix whatever problem might be further degrading performance?). Also, might the 384MB of RAM be way too low? (The file sizes for each song can reach up into the 100-120MB range)
If anyone would be able to provide any insight into what might be causing these problems, I'd greatly appreciate it.
|