High server load... might be this process: /usr/local/apache/bin/httpd -DSSL
 

Before I mention my issue, let me disclose that I know very little about Linux. :) I have just been learning as our server load has been spiking lately.

We run a pretty active site that uses Joomla as its CMS system. It is an active site, but nowhere near enough to result in server loads above 10-20. We have a dual processor with the following specs:

Command Used: more /proc/cpuinfo

vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Pentium(R) 4 CPU 3.06GHz
stepping : 9
cpu MHz : 3067.078
cache size : 1024 KB
physical id : 0

Command Used: dmesg | grep ^Memory:

Memory: 3110640k/3143552k available (2043k kernel code, 31712k reserved, 846k data, 232k init, 2226048k highmem)

Most of the time, it does fine during peak hours, but a few times out of the week, we see huge load issues.

There are two things that really stand out to me that might be causing the problem.

1) We have received some alerts about unreadable sectors (Device: /dev/sda, 1 Currently unreadable (pending) sectors). We have done some “SMART DATA SCANS” on the drive and it does say that we should backup the drive as it will fail in less than 24 hours. It has been saying this for a long time now, but it runs every day. We have ordered a new HD which will be replaced this weekend. So this might be what is causing the random high load issues.

2) When we run the command ps -aux we see the following processes over and over again:

nobody 29504 0.6 0.5 28588 17256 ? S 11:12 0:17 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29522 0.6 0.5 29748 18052 ? S 11:12 0:19 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29536 0.7 0.5 29032 17464 ? S 11:13 0:20 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29542 0.4 0.6 30736 19492 ? S 11:13 0:12 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29552 0.4 0.5 29824 18064 ? S 11:13 0:13 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29554 11.3 0.6 31176 19524 ? S 11:13 5:18 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29558 0.5 0.5 29468 17832 ? S 11:13 0:14 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29559 7.7 0.6 33772 19188 ? S 11:13 3:38 \_ /usr/local/apache/bin/httpd –DSSL
nobody 30050 0.4 0.5 28720 17484 ? S 11:21 0:10 \_ /usr/local/apache/bin/httpd –DSSL
nobody 31168 0.3 0.5 29908 18400 ? S 11:41 0:03 \_ /usr/local/apache/bin/httpd –DSSL

Right now you don’t see that they are only opened for a short time. The longest being 5:18. That is because we manually killed all of these processes 5:18 ago. They restart as soon as we kill them. Before we killed them, they have been open since the last time we killed them a few days ago.

Essentially these will keep running/open until they are manually killed. When the load is high and we are actually able to SSH in to the machine and run the TOP command, one of these HTTPD processes seems to be requiring enormous amounts of %memory. Regardless, these processes should be running forever.

So those are the two things we have been able to find out that might be causing the high load issues.

Does anyone out there recognize the /usr/local/apache/bin/httpd –DSSL processes?

We did locate an article online that referenced these processes as hacker programs. You can view that article here: http://www.webmasterworld.com/websit...gy/3415033.htm

Your help will be greatly appreciated!

The httpd process is the main Apache process (the process that servers your website). The -DSSL switch is used to start Apache with SSL support. It is quite normal to have multiple copies of this process. Have you looked at the Apache logs to see what the processes are doing at the time of the problem. The same theory applies to /var/log/messages.

Finally, I would tend to think your underling problem is the dodgy disk and would get this fixed before investigating too much.

High server load... might be this process: /usr/local/apache/bin/httpd -DSSL
 

Harry,

Thanks for the quick reply. Yes, we are changing out the drive this weekend. So hopefully that will take care of the issue.

Quick question about the Apache processes.

If it is normal to have these processes running, should they be running forever? Remember, these processes stay on and don't shut off. (See attached image)

When we see it happening again, we will check out the Apache logs to see what is happening.

Is this the command to view the logs? tail -f /usr/local/apache/logs/error_log

What do we look for in the logs? I guess what I am asking, is there something we focus on in the logs, like the PID ID or something?

Thanks again Harry!

Yes, it's normal that there are a number of them running. Apache forks a number of these processes to be able to handle multiple requests quicker.

This is my output of the same command
In my setup apache writes in /var/log/apache, so just check where your apache writes. There are however more files than only the error_log. access_log and ssl_engine_log come to mind. It however depends on the configuration of the server.

excerpt of access_log
'tail -f' only shows the last lines and new incoming lines; for analysis 'less' might be more usefull.

I appreciate your advice. I will let you know how it works after we get the new drive installed and if it happens again, we will check out the logs you mentioned to see what happens.

Thanks!