LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 04-28-2012, 07:39 AM   #1
pawwa
LQ Newbie
 
Registered: Dec 2008
Posts: 15

Rep: Reputation: 0
httpd increasing memory usage over time


When I restart the httpd service, httpd child processses consume resident set memory of size ~6224, but at the end of the day they consume ~28544 kilobytes which is a lot I would say.

The main difference is in single 'anon' block of allocated memory, here's a snippet of interest:

Code:
$ diff -y /tmp/pmap_before_restart /tmp/pmap_after_restart
00007f737e313000   19636   19316   19316 rw---    [ anon ]    |	00007fff264bf000      84      32      32 rw---    [ stack ]
----------------  ------  ------  ------			----------------  ------  ------  ------
total kB          215048   28544   25480		      |	total kB          195232    6224    5584
Can someone with more experience in memory management and httpd explain what is happening here? Is this caching taking place?

*Edit*: typo (bytes vs kilobytes).

Last edited by pawwa; 04-30-2012 at 06:49 AM.
 
Old 04-29-2012, 04:23 PM   #2
kyrunner
LQ Newbie
 
Registered: Apr 2012
Location: New York
Distribution: Centos,Debian
Posts: 29

Rep: Reputation: 1
Try these settings in your apache config file.

StartServers 1
MinSpareServers 3
MaxSpareServers 6
ServerLimit 24
MaxClients 24
MaxRequestsPerChild 3000
 
Old 04-29-2012, 04:55 PM   #3
sag47
Senior Member
 
Registered: Sep 2009
Location: Philly, PA
Distribution: Kubuntu x64, RHEL, Fedora Core, FreeBSD, Windows x64
Posts: 1,459
Blog Entries: 35

Rep: Reputation: 363Reputation: 363Reputation: 363Reputation: 363
Just so you're clear, according to the values you have provided your server starts out taking up 6MB of memory and end the day with around ~28MB of memory. That is not very much and not abnormal. So there really isn't anything you should worry about. Some of the high availability web servers I run take up several gigabytes in memory and still run just fine.

So worry not.

--- *Edit*
Another thing I'd like to note, you didn't provide very much information. We don't know the specs of your server nor the normal load you expect it to handle (i.e. requests per second or connections daily). And that generic output of what appears to be a stack trace is non-descriptive either. What commands did you use to obtain that information besides catting a temporary file? The only reason why I gave you the values I did is because of what you originally stated ("~28544 bytes") when your catted file states the measurements are in "Total kB". Details like that matter when analyzing your own performance.

So lack of information, plus lack of explanation, make it so that nobody on this forum could give you a concise answer to your perceived problem or even begin to point you in the right direction.

SAM

Last edited by sag47; 04-29-2012 at 05:04 PM.
 
1 members found this post helpful.
Old 04-30-2012, 07:32 AM   #4
pawwa
LQ Newbie
 
Registered: Dec 2008
Posts: 15

Original Poster
Rep: Reputation: 0
Thanks and sorry for the lack of information, I thought that the presented would be sufficient. I'm always afraid that vast details detract paying attention to the problem. I have edited the bytes/kilobytes typo too.

The load on the server is relatively low. The VPS instance is a micro EC2: 2 ECUs (bursting), 613 MB RAM, low I/O performance. The LAMP environment is Apache httpd 2.2 (prefork, mod_ruid2), php 5.3 (mod_php DSO), mysql 5.1, latest stable Wordpress. No opcode caching of course.

I have tuned the LAMP environment to utilize only the necessary memory. The temporary files have 'pmap', 'before_restart' and 'after_restart' in their names suggesting that I have used the pmap command to get the output (pmap -x httpd_child_PID) before and after 'service httpd restart' :-)

I am interested in why httpd child processes increases memory utilization over time, but I supose that it's not a straight forward answer?
 
Old 04-30-2012, 09:30 AM   #5
sag47
Senior Member
 
Registered: Sep 2009
Location: Philly, PA
Distribution: Kubuntu x64, RHEL, Fedora Core, FreeBSD, Windows x64
Posts: 1,459
Blog Entries: 35

Rep: Reputation: 363Reputation: 363Reputation: 363Reputation: 363
What version of httpd are you using and what modules do you have currently enabled? Are there modules which you don't need that can be disabled? The reason why I ask is sometimes there can be a memory leak in a given module which httpd can't do anything about. I found this bug report with a work around as an example of a module which leaks memory.

A few quick google searches lead me to some forum articles which help diagnose the problem.

Why are you investigating the memory utilization of httpd? Are you currently experiencing performance problems due to it or is this more of a personal exercise? Either way it's fine. If you need to have the server restart every once in a while then perhaps writing a script and running it in a subshell to check memory and when memory is low to restart apache to free up memory would be a possible (hacked) solution.

You could also try tweaking httpd.conf to limit its ability to grab resources.

While there's not a clear answer to what is currently causing your problem (whether it's the http application you're using or httpd itself), there are steps you can take which will help the management of your memory.

Are there any errors in your logs indicating what could be the problem? For instance too many open file descriptors, etc.?

Last edited by sag47; 04-30-2012 at 09:02 PM.
 
Old 04-30-2012, 10:15 AM   #6
sag47
Senior Member
 
Registered: Sep 2009
Location: Philly, PA
Distribution: Kubuntu x64, RHEL, Fedora Core, FreeBSD, Windows x64
Posts: 1,459
Blog Entries: 35

Rep: Reputation: 363Reputation: 363Reputation: 363Reputation: 363
Another thing you can do is to write your own "monitoring script" which monitors the memory usage of httpd throughout the day. By doing that you'll be able to see when it starts eating memory and check the logs during that time to see what's happening. You may have to set the log level to debug for more verbose log output. An example script to monitor the memory for httpd would be something like this. I made it easy for you to change the user and the process name to whatever you want.

Code:
#!/bin/bash
# GNU bash, version 4.2.20(1)-release (x86_64-redhat-linux-gnu)
# GNU Awk 4.0.0
# Linux 3.2.7-1.fc16.x86_64 x86_64 GNU/Linux

# CONFIGURE VARIABLES
user="apache"
proc="httpd"
#END CONFIGURE VARIABLES

sysmem="$(awk '$1 == "MemTotal:" {print $2/1024}' /proc/meminfo)"
while true;do
#  perc="$(ps axo user,pid,ppid,%mem,comm | awk '($5 == "'$proc'") && ($1 == "'$user'") && (last != $4) {last=$4;print $4}')"
  perc="$(ps axo user,pid,ppid,%mem,comm | awk 'BEGIN{memusage=0.0}; ($5 == "'$proc'") && ($1 == "'$user'"){memusage=memusage+$4}; END{print memusage}')"
  if [ -z "$perc" ];then
    mem="0.0"
  else
    mem="$(echo "$perc" | awk '{print $1*'$sysmem'/100}')"
  fi
  mem_perc="$(echo "$mem" | awk '{print $1/'$sysmem'*100}')"
  current="$proc Memory = $mem / $sysmem MB ($mem_perc %)"
  date="$(date '+%Y-%m-%d %I:%M:%S %p')"
  dates="$(date '+%s')"
  if [ "$current" != "$last" ];then
    echo "$current; $date; $dates";
  fi
  last="$current"
  sleep 1;
done
There are many silly ways like that to get what you want

You can start it by
Code:
su - -c "/path/to/httpd_monitor.sh > /var/log/httpd_monitor.log &"
And read it with
Code:
tail -f /var/log/httpd_monitor.log
SAM

Last edited by sag47; 04-30-2012 at 09:20 PM.
 
Old 04-30-2012, 05:49 PM   #7
pawwa
LQ Newbie
 
Registered: Dec 2008
Posts: 15

Original Poster
Rep: Reputation: 0
OK, I got the culprit - it was PHP DSO and wordpress that used the extra memory.

I have tried a few experiments with PHP's memory_get_usage() function and concluded that when I hit the wordpress PHP page it would instantly increase httpd's memory for about 18MB. It didn't increase over time actually. So, no memory leaks, just normal behaviour that I didn't notice as I was hitting some static HTML tests before that.

sag47, thanks for all the comments and recommendations. This was all just for my curiosity, and I hope this thread will also help someone else :-)
 
Old 04-30-2012, 09:06 PM   #8
sag47
Senior Member
 
Registered: Sep 2009
Location: Philly, PA
Distribution: Kubuntu x64, RHEL, Fedora Core, FreeBSD, Windows x64
Posts: 1,459
Blog Entries: 35

Rep: Reputation: 363Reputation: 363Reputation: 363Reputation: 363
No problem, exercises are fun and encouraged which is basically why I was throwing scenarios and solutions left and right. It's good to want to learn

I fixed the httpd monitoring script to account for forking httpd processes. I also added an interpreter comment and some other misc environment information for the conditions in which I successfully tested the script.

FYI if you use vim I inserted environment info like this in normal mode...

:r!awk --version | head -n1

Last edited by sag47; 04-30-2012 at 09:23 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Bind memory usage keeps increasing CTRLBREAK Linux - Server 6 06-17-2011 05:34 PM
Memory usage increasing Debian Lenny Apache2 server johnsan Linux - Newbie 4 05-11-2010 06:50 AM
Memory usage constantly increasing reverse Debian 1 11-11-2007 03:15 AM
mysql increasing memory usage over time jman623 Linux - Server 0 08-21-2007 07:44 PM
Strange httpd memory usage TTiZ Linux - Server 1 05-25-2007 08:19 AM


All times are GMT -5. The time now is 12:01 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration