LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices



Reply
 
Search this Thread
Old 06-18-2009, 12:13 PM   #1
ajames3436
LQ Newbie
 
Registered: Jun 2009
Posts: 3

Rep: Reputation: 0
High server load... might be this process: /usr/local/apache/bin/httpd -DSSL


Before I mention my issue, let me disclose that I know very little about Linux. I have just been learning as our server load has been spiking lately.

We run a pretty active site that uses Joomla as its CMS system. It is an active site, but nowhere near enough to result in server loads above 10-20. We have a dual processor with the following specs:

Command Used: more /proc/cpuinfo

vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Pentium(R) 4 CPU 3.06GHz
stepping : 9
cpu MHz : 3067.078
cache size : 1024 KB
physical id : 0

Command Used: dmesg | grep ^Memory:

Memory: 3110640k/3143552k available (2043k kernel code, 31712k reserved, 846k data, 232k init, 2226048k highmem)

Most of the time, it does fine during peak hours, but a few times out of the week, we see huge load issues.

There are two things that really stand out to me that might be causing the problem.

1) We have received some alerts about unreadable sectors (Device: /dev/sda, 1 Currently unreadable (pending) sectors). We have done some “SMART DATA SCANS” on the drive and it does say that we should backup the drive as it will fail in less than 24 hours. It has been saying this for a long time now, but it runs every day. We have ordered a new HD which will be replaced this weekend. So this might be what is causing the random high load issues.

2) When we run the command ps -aux we see the following processes over and over again:

nobody 29504 0.6 0.5 28588 17256 ? S 11:12 0:17 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29522 0.6 0.5 29748 18052 ? S 11:12 0:19 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29536 0.7 0.5 29032 17464 ? S 11:13 0:20 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29542 0.4 0.6 30736 19492 ? S 11:13 0:12 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29552 0.4 0.5 29824 18064 ? S 11:13 0:13 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29554 11.3 0.6 31176 19524 ? S 11:13 5:18 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29558 0.5 0.5 29468 17832 ? S 11:13 0:14 \_ /usr/local/apache/bin/httpd –DSSL
nobody 29559 7.7 0.6 33772 19188 ? S 11:13 3:38 \_ /usr/local/apache/bin/httpd –DSSL
nobody 30050 0.4 0.5 28720 17484 ? S 11:21 0:10 \_ /usr/local/apache/bin/httpd –DSSL
nobody 31168 0.3 0.5 29908 18400 ? S 11:41 0:03 \_ /usr/local/apache/bin/httpd –DSSL

Right now you don’t see that they are only opened for a short time. The longest being 5:18. That is because we manually killed all of these processes 5:18 ago. They restart as soon as we kill them. Before we killed them, they have been open since the last time we killed them a few days ago.

Essentially these will keep running/open until they are manually killed. When the load is high and we are actually able to SSH in to the machine and run the TOP command, one of these HTTPD processes seems to be requiring enormous amounts of %memory. Regardless, these processes should be running forever.

So those are the two things we have been able to find out that might be causing the high load issues.

Does anyone out there recognize the /usr/local/apache/bin/httpd –DSSL processes?

We did locate an article online that referenced these processes as hacker programs. You can view that article here: http://www.webmasterworld.com/websit...gy/3415033.htm

Your help will be greatly appreciated!
 
Old 06-18-2009, 05:18 PM   #2
harry edwards
Member
 
Registered: Nov 2007
Location: Lincolnshire, UK
Distribution: CentOS, Fedora, and Suse
Posts: 365

Rep: Reputation: 48
The httpd process is the main Apache process (the process that servers your website). The -DSSL switch is used to start Apache with SSL support. It is quite normal to have multiple copies of this process. Have you looked at the Apache logs to see what the processes are doing at the time of the problem. The same theory applies to /var/log/messages.

Finally, I would tend to think your underling problem is the dodgy disk and would get this fixed before investigating too much.
 
Old 06-18-2009, 10:34 PM   #3
ajames3436
LQ Newbie
 
Registered: Jun 2009
Posts: 3

Original Poster
Rep: Reputation: 0
High server load... might be this process: /usr/local/apache/bin/httpd -DSSL

Harry,

Thanks for the quick reply. Yes, we are changing out the drive this weekend. So hopefully that will take care of the issue.

Quick question about the Apache processes.

If it is normal to have these processes running, should they be running forever? Remember, these processes stay on and don't shut off. (See attached image)

When we see it happening again, we will check out the Apache logs to see what is happening.

Is this the command to view the logs? tail -f /usr/local/apache/logs/error_log

What do we look for in the logs? I guess what I am asking, is there something we focus on in the logs, like the PID ID or something?

Thanks again Harry!
Attached Images
File Type: gif Untitled-1.gif (16.7 KB, 3 views)
 
Old 06-19-2009, 12:50 AM   #4
Wim Sturkenboom
Senior Member
 
Registered: Jan 2005
Location: Roodepoort, South Africa
Distribution: Slackware 10.1/10.2/12, Ubuntu 12.04, Crunchbang Statler
Posts: 3,786

Rep: Reputation: 282Reputation: 282Reputation: 282
Yes, it's normal that there are a number of them running. Apache forks a number of these processes to be able to handle multiple requests quicker.

This is my output of the same command
Code:
wim@btd-techweb01:~$ ps aux |grep httpd
root     19969  0.0  2.1 75784 5376 ?        Ss   Jun10   0:00 /usr/sbin/httpd -DSSL
nobody   19970  0.0  2.9 77020 7484 ?        S    Jun10   0:37 /usr/sbin/httpd -DSSL
nobody   19971  0.0  4.0 79792 10248 ?       S    Jun10   1:40 /usr/sbin/httpd -DSSL
nobody   19973  0.0  4.0 79804 10264 ?       S    Jun10   1:15 /usr/sbin/httpd -DSSL
nobody   19974  0.0  3.0 77288 7752 ?        S    Jun10   0:49 /usr/sbin/httpd -DSSL
nobody   19977  0.0  3.9 79684 10140 ?       S    Jun10   1:11 /usr/sbin/httpd -DSSL
nobody   19979  0.0  3.9 79820 10216 ?       S    Jun10   1:12 /usr/sbin/httpd -DSSL
nobody   19980  0.0 10.3 98136 26524 ?       S    Jun10   1:42 /usr/sbin/httpd -DSSL
nobody   21796  0.0  4.1 80168 10624 ?       S    Jun11   1:07 /usr/sbin/httpd -DSSL
nobody   26892  0.0  3.6 78836 9292 ?        S    Jun14   0:40 /usr/sbin/httpd -DSSL
nobody   26976  0.0  3.1 77564 8024 ?        S    Jun14   0:20 /usr/sbin/httpd -DSSL
In my setup apache writes in /var/log/apache, so just check where your apache writes. There are however more files than only the error_log. access_log and ssl_engine_log come to mind. It however depends on the configuration of the server.

excerpt of access_log
Code:
wim@btd-techweb01:~$ tail -f /var/log/apache/access_log
10.17.130.44 - - [19/Jun/2009:06:43:15 +0200] "GET /sortorder.php HTTP/1.0" 200 4558
10.17.130.44 - - [19/Jun/2009:06:43:22 +0200] "GET /select_platform.php HTTP/1.0" 200 4796
10.17.130.44 - - [19/Jun/2009:06:43:24 +0200] "GET /select_platform.php?id=is7&submit=Select HTTP/1.0" 200 4219
10.17.130.44 - - [19/Jun/2009:06:43:25 +0200] "GET /main.php HTTP/1.0" 200 23428
10.17.130.44 - - [19/Jun/2009:06:43:27 +0200] "GET /incident_open.php HTTP/1.0" 200 31510
10.17.130.44 - - [19/Jun/2009:06:43:48 +0200] "POST /incident.php HTTP/1.0" 200 2046
10.17.130.44 - - [19/Jun/2009:06:43:49 +0200] "GET /main.php HTTP/1.0" 200 11865
10.17.130.44 - - [19/Jun/2009:06:44:17 +0200] "POST /main.php HTTP/1.0" 200 12576
10.17.130.44 - - [19/Jun/2009:06:44:20 +0200] "POST /main.php HTTP/1.0" 200 12576
10.17.130.44 - - [19/Jun/2009:06:44:24 +0200] "GET /incident.php?id=469844 HTTP/1.0" 200 6561
10.17.130.44 - - [19/Jun/2009:06:45:34 +0200] "POST /incident_open.php HTTP/1.0" 200 31599
10.17.130.44 - - [19/Jun/2009:06:46:33 +0200] "POST /incident.php HTTP/1.0" 200 2046
10.17.130.44 - - [19/Jun/2009:06:46:34 +0200] "GET /main.php HTTP/1.0" 200 11865
10.17.130.44 - - [19/Jun/2009:06:46:57 +0200] "POST /main.php HTTP/1.0" 200 12576
10.17.130.44 - - [19/Jun/2009:06:46:58 +0200] "GET /incident.php?id=469866 HTTP/1.0" 200 6561
10.17.130.44 - - [19/Jun/2009:06:46:58 +0200] "POST /incident_open.php HTTP/1.0" 200 4376
10.17.130.44 - - [19/Jun/2009:06:47:00 +0200] "GET /incident_view.php?id=840 HTTP/1.0" 200 7575
'tail -f' only shows the last lines and new incoming lines; for analysis 'less' might be more usefull.
 
Old 06-19-2009, 08:45 AM   #5
ajames3436
LQ Newbie
 
Registered: Jun 2009
Posts: 3

Original Poster
Rep: Reputation: 0
I appreciate your advice. I will let you know how it works after we get the new drive installed and if it happens again, we will check out the logs you mentioned to see what happens.

Thanks!
 
  


Reply

Tags
high, load average


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
/usr/local/apache2/bin/apachectl: line 65: 4407 Segmentation fault $HTTPD -k $A onlin85 Linux - Software 2 06-18-2009 07:54 AM
Server high load, testing hardware reliability and limiting process stelar Linux - Enterprise 1 10-20-2006 01:58 AM
Starting httpd: /usr/sbin/httpd: symbol lookup error: /usr/local/lib/libaprutil-0.so. bijuhpd Linux - Newbie 1 10-30-2005 06:07 PM
path in services wrong for clamav updated frm 0.75 to 0.80 usr/bin vs usr/local/bin Emmanuel_uk Linux - Newbie 3 04-22-2005 02:02 AM
Webmin: The Apache server executable /usr/local/apache2/bin does not exist hektor Linux - Software 0 01-14-2005 07:48 AM


All times are GMT -5. The time now is 11:49 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration