Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
I have a webserver and just this morning it seems to have started to overload.
As of this morning, my server has been running at an abnormally high load due to what seems to be a whole lot of httpd connections. However, we are receiving no more traffic to our websites than normal, so I am sure it is not related to our website traffic. Also I have not changed any settings nor have I edited any of our website scripts. So none of these things could be causing this. We normally run at under 1% server load. When I examine our apache access-log file, i see that there are thousands of ""GET /whm-server-status HTTP/1.0" 200" messages. Could this be the culprit?
Also, the /var/log/secure file has thousands of lines of this:
Jan 21 09:30:12 ns3 xinetd: START: imap pid=4648 from=127.0.0.1
Jan 21 09:38:35 ns3 xinetd: START: imap pid=6216 from=127.0.0.1
Jan 21 09:46:58 ns3 xinetd: START: imap pid=9846 from=127.0.0.1
Jan 21 09:55:21 ns3 xinetd: START: imap pid=11350 from=127.0.0.1
Here is an export from a TOP i just ran:
18:50:56 up 56 min, 2 users, load average: 127.47, 106.16, 66.71
913 processes: 910 sleeping, 2 running, 1 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 13.1% 0.0% 10.4% 0.0% 0.4% 74.2% 1.6%
cpu00 16.9% 0.0% 6.0% 0.0% 0.8% 76.0% 0.0%
cpu01 13.0% 0.0% 10.1% 0.0% 0.0% 72.0% 4.8%
cpu02 10.7% 0.0% 8.9% 0.1% 0.7% 79.4% 0.0%
cpu03 11.9% 0.0% 16.7% 0.0% 0.0% 69.6% 1.6%
Mem: 2074544k av, 1982340k used, 92204k free, 0k shrd, 25936k buff
1612984k active, 226180k inactive
Swap: 2097136k av, 878380k used, 1218756k free 332448k cached
I am stumped and don't know what else to do. I've rebooted a couple of times. Restarted apache server...mysqld server...exim server....etc. I really need to get this fixed as there are a few high traffic websites on this server i need to have up. Pleasssseeee help guys!
Do you run anything that should check /whm-server-status?
Do the /whm-server-status queries come from one or multiple IP addresses?
What happens if you block access to /whm-server-status?
Is the box swamped in outbound traffic? To remote port TCP/25 by any chance?
Did you run a Chkrootkit / Rootkit Hunter check on the system (just in case)?
Otherwise checked for "weird" stuff in the system and daemon logs?
Otherwise checked for "weird" stuff in /tmp, /var/tmp and any other temp dirs apps are allowed to write?
I'm not an an Internet Service Provider guru, but I think I would:
1) Limit all services, such as IMAP, smtp, etc to the WAN, either by disabling or firewall.
2) Take several netstat snap shops, sort and diff them to see if you can see a incoming source.
3) Maybe use ethereal for a similar purpose.
4) Try the rate limiting features in the Linux firewall.
the whm check is a local check. Its something Cpanel does to make sure its still working.
I think I got this problem solved....or at least temporarily solved. Most of the HTTPD requests seemed to be coming from SSL connections. I started apache without SSL and that seemed to immediately solve the load problem. We have no need for SSL, so this is a temporary solution at least.
Thanks so much for your tips. I'll update this thread if the problems resume today.