Does anyone know how to limit the total amount of memory usage by apache? Ive got a problem with my servers crasing and i believe its down to apache consuming all the memory. I have tried using rlimitmem but that doesnt seem to have any affect.

Limit the number of child process requests, spare servers and max servers

Also look at the keepalive time settings, maybe make them more agressive.

Keep an eye on the log files for access to dynamic content, cgi, php etc
make efforts to speed the scripts up...

Regards,
Peter

What Dist do you use ?

In debian you can edit these limits in the /etc/security/limits.conf
for user nobody, if you are running apache as nobody.

If you dont have the limits.conf file you can modify the .bashrc file with ulimit

Look in the man ulimit for more info

Ive got a problem with my servers crasing and i believe its down to apache consuming all the memory.

Please first try to make pretty certain you're looking at the right area by gathering data about memory usage. That's better that vague guesstimating (and not giving any stats and system info as well). Try running sar/atsar-like commands to give you data about memory/cpu/network/http counters. I prefer using "atop" (freshmeat) for periodic memory reports.

When you're sure data supports Apache *is* the culprit look at your hardware and see if it needs expanding in the cpu/mem field, Linux VM settings, like FredrikN says, the PAM per-user settings. Then tweak your /proc and Apache settings like peter_robb says.

// Moderator note: I'm grafting this to /General, this is about measuring system/application performance and system configuration.

Apache isnt using too much memory 99.9% of the time so its not that my system isnt capable of handling the load. I believe there is a malicous script on the system. I am using Red Hat Linux 7.2 and have allrady tried using /etc/security/limits.conf to limit the user nobody but with no sucess

Not to be too harsh, but your reply is contradicting your original post. Again, if you're guesstimating (as in I believe, I think, I assume), try *pinpoint* what the problem is.
If you made sure there is a "malicious" script then trying to work ulimits is trying to mend the problem by the *symptoms*, not the *cause*!

You should inspect your scripts instead, ofcourse....

I have 9 servers with about 500 sites on each server so its not possible to just look at the scripts and see whats causing it. I can tell from /var/log/messages thtas its httpd processes that are taking up the memory when the server crashes.

When the server crashes nothing will work so its not possible to see what sites are causing it.

I have actually seen in top once that it was httpd processes consuming the memory, but i wasnt able to determine what site it was from. I need to be able to stop them crashing the server so i can determine what script is causing it.