Huge memory problem on Debian w/ qmail & spamassassin

raccoon · 01-02-2006, 08:03 AM

I've recently discovered that our qmail based server is reporting extremely high usage of memory, apparently from the spamd daemon. We have 1gb of RAM in the machine, of which ~800MB is free after a fresh boot.

Now, after almost 24 hours since the last reboot, the amount of free memory is down to ~450MB and it is decreasing all the time as long as the server is being used, slowly but yet irritating. Before our last reboot the server was down to only 30MB available memory. Worth mentioning is that our swap partition is not used at all, for some reason!

I've done alot of research on this matter but can't seem to find any logical answers. I'm running spamd in the foreground with daemontools supervising it, so that I can use multilog correctly. However, running spamd as a daemon does not affect the memory usage at all, at least not noticeably.

Here is some output from top, about 24 hours after the last reboot:

Code:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
30480 spamd     16   0 31812  28m 2220 S  0.0  3.2   0:00.12 spamd
  731 spamd     16   0 31024  27m 2172 S  0.0  3.1   0:00.05 spamd
 9087 root      16   0 29968  27m 2500 S  0.0  3.1   0:00.42 spamd

"ps aufx" (trimmed for relevancy):

Code:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      3590  0.0  0.1   2636  1268 ?        Ss   Dec27   0:00 /bin/sh /command/svscanboot
root      3617  0.0  0.0   1608   348 ?        S    Dec27   0:00  \_ svscan /service
root      3628  0.0  0.0   1448   300 ?        S    Dec27   0:00  |   \_ supervise log
spamd     9100  0.0  0.0   1456   348 ?        S    10:00   0:00  |   |   \_ /usr/local/bin/multilog t s1000000 n20 /var/log/spamd
root      9086  0.0  0.0   1444   296 ?        S    10:00   0:00  |   \_ supervise spamd
root      9087  0.0  3.0  29968 27712 ?        S    10:00   0:00  |       \_ /usr/bin/perl -T -w /usr/bin/spamd --max-conn-per-child=10 -m 5 -x -H /home/spamassassin -u spamd -s st
derr
spamd    30480  0.0  3.2  31812 29276 ?        S    14:47   0:00  |           \_ spamd child
spamd      731  0.0  3.1  31024 28504 ?        S    15:30   0:00  |           \_ spamd child
root      3618  0.0  0.0   1440   252 ?        S    Dec27   0:00  \_ readproctitle service errors: ......

vmstat:

Code:

procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
 1  0      0 441868  87116 181008    0    0     2    27  322    43  0  0 99  1

The machine is running Debian (Sarge) on a 2.6.14.4 kernel with SMP support. Has anyone encountered any problems like this one?

During nights, when noone is using the server for mail purposes, the memory usage does not increase - but as soon as the clock hits 7 or 8 in the morning and all our customers starts using it again, it sure does. Check the following image for an idea of the problem:

Ram usage

(We rebooted the machine around 16:00 yesterday, therefore the huge increase in available memory), then it slowly decreases again.

stress_junkie · 01-02-2006, 08:31 AM

Is this a problem? I don't think so. I think that the computer is operating as expected. You mentioned that the swap space isn't even being used. Overnight the RAM usage decreases. It sounds to me like the server is working fine. Even if it did start to swap a bit it would not necessarily be a problem. What happens if you let it run for 30 days, 60 days, 90 days, etc.? If you get the same behavior and if the machine is performing its tasks in a reasonable time then I would assess it to be working okay.

If it works then don't "fix" it. (Except for security problems.)

raccoon · 01-03-2006, 03:15 AM

Alright, I'll let this be for a week or so - it just seemed strange that it kept "allocating" memory but never released it.

Now to another (yet similar) problem; I recently put a an additional pair of 1GB sticks in the server, making it a total of 3GB RAM. However, Linux only reports 1GB of RAM even though BIOS/POST sees it all. Is this normal aswell?

Hangdog42 · 01-03-2006, 08:59 AM

Linux uses memory very differently from Windows, so if your expectations about "free" memory were learned in the Windows world, it might be worth a read through this.

As for the increase in memory, you might need to recompile your kernel to see all the memory. Or perhaps one of the Debian repositories has a kernel designed to handle greater than 1G of RAM.

amosf · 01-03-2006, 09:19 AM

Recompile your kernel for greater ram support - like say the 4gig option.

And as said, learn a little about how linux uses ram. It does USE it. It doesn't just let it sit there. What is the point of having a gig of ram just sitting there when it could be doing something useful like a buffer of cache...