Reaching MaxClients, why ?
I've been experiencing an issue from time to time:
For no apparent reason ( no traffic spikes, no other scripting identifiable errors), I get an entry in the errog log that MaxClients has been reached and of course httpd stops responding.
There is no load on the server, only a sudden abrupt jump in tasks: there are usually 270~300 processes running but as I mentioned, from time to time, it suddenly jumps to 900+ and crashes apache.
I have configured Apache with a MaxCLients of 800 so far, I have 16 GB of RAM and running a dual Xeon Quadcore setup with CentOS 64 bit.
Could you please advise on some methodical approach to identify this issue ?
Thank you kindly for your time.
Try Apache's access_log & error_log.
Also /var/log/messages; see where the traffic is coming from.
If it's some kind of break-in attempt, try mod_throttle or fail2ban.
This is what I found there:
Oct 13 19:18:46 web kernel: possible SYN flooding on port 80. Sending cookies.
Oct 13 19:19:04 web kernel: TCPv6: Possible SYN flooding on port 80. Sending cookies.
Although It's defending against this possible SYN flood by sending cookies, apache gets thrown down. I'm reaching MaxClients and the server is unable to serve new requests.
Any help on how to continue ?
Another statistic that troubles me is this:
Running "netstat -s" I notice:
3403236 active connections openings
287134948 passive connection openings
3632799 failed connection attempts
3598067 connection resets received
There's a huge number of failed connection attempts.
Try fail2ban http://www.fail2ban.org/wiki/index.php/Main_Page.
You may want to disable IPv6 unless you need it (unlikely):
Edit: This on today's slashdot.org: help from google for webmasters who (may) have been compromised:
I have done basic securing of my Linux box. I admit, it could have been done better, but I do not see any signs of intrusions. SSH is secured/
Why did you make the assumption that the box might be hacked ? I can't see any causality between the SYN flood and possible intrusions. Could you please share your thoughts ?
SYN is the first TCP pkt sent when trying to connect to a system. Unless you've got some serious info eg, winning lottery nums for next week, why would you suddenly get massive floods like that. There's also a TCP SYN Flood DOS (Denial Of Service) attack: http://en.wikipedia.org/wiki/SYN_flood.
Note I only said 'If it's some kind of break-in attempt', 'if' being the key word there.
From your own logs, the kernel says it suspects the SYN Flood attack; that's why it says its 'sending cookies' as mentioned/recommended in the Wiki article.
|All times are GMT -5. The time now is 12:18 PM.|