Serious Memory leak on a BP6 running Linux, please help!

MayDay · 09-09-2004, 09:55 PM

I have had my Abit BP6 Dual Celeron 466Mhz for a long time, once it was even a Dual 333@500 until i got two 466Mhz for free. It ran heavenly under Win2k and WinXP up until i changed it for a P4 1.6Ghz.

For the last year i started running Linux on it, Debian stable, unstable, testing. kernel version 2.2, 2.4 and 2.6. All kinds of different combinations.
At first it was just to test and have fun. But since 5 months ago i had to use it for more serious stuff and thats about the time i started to notice the serious memory leak.

I always thought it was something i compiled wrong or forgot. I tried alot of combinations but the memory leak remains. So two weeks ago the harddrive crashed because of the heat. Now i JUST reinstalled and are running Debian Sarge with kernel version 2.4.26-1-386 which is not a custom compiled kernel, but then again i know I cant compile a kernel which removes the memory leak so this single cpu kernel is fine until i find and remove the leak.

I have been running bios version RU for a long time, just patched to RV yesterday, the APIC Errors i used to get are gone i think, i actually thought that was the problem until i read about what it was on the forum, i still tried to disable it in the bios to see if it made a difference, it diden't.

Following are 2 TOP dumps, first is RIGHT after boot, the second is a little later, only SSHD and apt-get was used in the meantime:

Code:

top - 01:08:43 up 18 min,  1 user,  load average: 0.00, 0.01, 0.00
Tasks:  25 total,   1 running,  24 sleeping,   0 stopped,   0 zombie
Cpu(s):   0.4% user,   0.0% system,   0.0% nice,  99.6% idle
Mem:    516364k total,    50080k used,   466284k free,     4732k buffers
Swap:   497972k total,        0k used,   497972k free,    35180k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
  740 root      13   0  1908 1904 1664 S  0.0  0.4   0:00.06 sshd
  708 Debian-e   9   0  1468 1464 1248 S  0.0  0.3   0:00.00 exim4
  721 root       9   0  1440 1440 1312 S  0.0  0.3   0:00.01 sshd
  743 root      18   0  1404 1404 1120 S  0.0  0.3   0:00.02 bash
  677 root       9   0  1344 1344  476 S  0.0  0.3   0:00.25 klogd
  745 root      19   0  1036 1036  844 R  0.4  0.2   0:00.02 top
  728 root       9   0   712  712  608 S  0.0  0.1   0:00.01 cron
  585 root       9   0   664  664  548 S  0.0  0.1   0:00.00 dhclient
  725 daemon     9   0   624  624  548 S  0.0  0.1   0:00.00 atd
  674 root       9   0   612  612  520 S  0.0  0.1   0:00.04 syslogd
  714 root       9   0   520  520  464 S  0.0  0.1   0:00.01 inetd
    1 root      10   0   508  504  456 S  0.0  0.1   0:00.15 init
  734 root       9   0   472  472  420 S  0.0  0.1   0:00.01 getty
  735 root       9   0   472  472  420 S  0.0  0.1   0:00.00 getty
  736 root       9   0   472  472  420 S  0.0  0.1   0:00.00 getty
  737 root       9   0   472  472  420 S  0.0  0.1   0:00.00 getty
  738 root       9   0   472  472  420 S  0.0  0.1   0:00.00 getty
  739 root      10   0   472  472  420 S  0.0  0.1   0:00.01 getty
    2 root       8   0     0    0    0 S  0.0  0.0   0:00.00 keventd
    3 root      19  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd_CPU0
    4 root       9   0     0    0    0 S  0.0  0.0   0:00.00 kswapd
    5 root       9   0     0    0    0 S  0.0  0.0   0:00.00 bdflush
    6 root       9   0     0    0    0 S  0.0  0.0   0:00.00 kupdated
  105 root       9   0     0    0    0 S  0.0  0.0   0:00.03 kjournald
  370 root       9   0     0    0    0 S  0.0  0.0   0:00.00 khubd

Code:

top - 09:00:34 up  9:10,  1 user,  load average: 0.00, 0.00, 0.00
Tasks:  25 total,   1 running,  24 sleeping,   0 stopped,   0 zombie
Cpu(s):   0.0% user,   0.0% system,   0.0% nice, 100.0% idle
Mem:    516364k total,   269728k used,   246636k free,    15204k buffers
Swap:   497972k total,        0k used,   497972k free,   233652k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
  1458 root      12   0  1908 1904 1664 S  0.0  0.4   0:00.04 sshd
  705 Debian-e   9   0  1468 1464 1248 S  0.0  0.3   0:00.01 exim4
  1005 root       9   0  1440 1440 1312 S  0.0  0.3   0:00.02 sshd
  1461 root      14   0  1404 1404 1120 S  0.0  0.3   0:00.02 bash
  674 root       9   0  1344 1344  476 S  0.0  0.3   0:00.28 klogd
  1463 root      19   0  1036 1036  844 R  0.0  0.2   0:00.06 top
  725 root       9   0   712  712  608 S  0.0  0.1   0:00.00 cron
  585 root       9   0   664  664  548 S  0.0  0.1   0:00.02 dhclient
  722 daemon     9   0   624  624  548 S  0.0  0.1   0:00.00 atd
  671 root       9   0   612  612  520 S  0.0  0.1   0:00.04 syslogd
  711 root       9   0   520  520  464 S  0.0  0.1   0:00.00 inetd
    1 root       7   0   508  504  456 S  0.0  0.1   0:00.20 init
  731 root       9   0   472  472  420 S  0.0  0.1   0:00.00 getty
  732 root       9   0   472  472  420 S  0.0  0.1   0:00.00 getty
  733 root       9   0   472  472  420 S  0.0  0.1   0:00.00 getty
  734 root       9   0   472  472  420 S  0.0  0.1   0:00.01 getty
  735 root       9   0   472  472  420 S  0.0  0.1   0:00.00 getty
  736 root       9   0   472  472  420 S  0.0  0.1   0:00.00 getty
    2 root       8   0     0    0    0 S  0.0  0.0   0:00.00 keventd
    3 root      19  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd_CPU0
    4 root       9   0     0    0    0 S  0.0  0.0   0:00.00 kswapd
    5 root       9   0     0    0    0 S  0.0  0.0   0:00.00 bdflush
    6 root       9   0     0    0    0 S  0.0  0.0   0:00.00 kupdated
  105 root       9   0     0    0    0 S  0.0  0.0   0:00.49 kjournald
  368 root       9   0     0    0    0 S  0.0  0.0   0:00.00 khubd

50Mb after boot, 270Mb 9 hours later by doing nothing...

Im all out of ideas now, i dont understand why i am the ONLY one with this problem, and why its only under linux, not windows.

BTW. I know from the last 5 months of looking at memory graphs that it goes all the way up to 500'ish Memory use and then i have seen the graph suddenly DUMP to 90-120Mb usage in the middle of the night.

And thats with programs running on it, where 80Mb is what it used after boot with that setup.

I actually thought about switching to FreeBSD

, but i know nothing about it and i LOVE debian, apt truely has super cow powers

If anyone can help me got damnit i will name something after you!

Looking forward to replies with much anticipation...

MayDay

Electro · 09-09-2004, 11:18 PM

Windows users always say that Linux eats memory up like crazy but if you look at the two different top dumps. You will see the difference. Look at the cache and buffers. Do you see the difference. Linux caches and buffers data so that it does not resort to the hard drive to get data. If you compare it to Windows 2000/XP, you will be using your swap at the end of the day. Linux will still be using physical memory at the end of the day.

Your system is working perfectly. You do not have any memory leaks. Memory leaks happens when a program is running is taking more memory over time than it was designed for.

BTW, its good that you did the top dumps because you or someone else will not understand how Linux handles memory.

dthacker · 09-09-2004, 11:50 PM

I agree with the poster above, you don't show any processes grabbing big chunks of memory.
Are you getting console errors? Is the kernel attempting to shut down processes because it's out of memory? If not you don't have a problem.

Do a man vmstat, and run that for awhile after you get the syntax. Look for paging. If you don't have any, you have no problems, for now....

Dave

J.W. · 09-10-2004, 12:21 AM

Electro is right -- you have no memory leak. It only looks that way because Linux manages memory differently than Windows. A good, clearly written description can be found here -- J.W.

MayDay · 09-10-2004, 12:33 AM

Hmm okay, i actually thought of this but i thought only the buffer was what of the used memory that was not _really_ used, but if you account in the cached it all makes sense!

It actually felt like it was running very slow when it was up to 500Mb "used" memory, but i guess it was all in my head.

Thanks much guys

J.W. · 09-10-2004, 01:10 AM

Are you using memory-intensive WM such as Gnome or KDE? If so, you may want to switch to a "lighter" window manager.

Either way, Welcome to LQ! --J.W.

MayDay · 09-10-2004, 08:17 AM

No, not at all. its just "light" use of terminal programs without X running at all.

TeamSpeak2 Server, ProFTPD, Eggdrops, Apache, mrtg, Game server stats.

Before i found out about the memory thing yesterday i had run apache with 1 initial server and min 1, max 1 in reserve. Is that a good idea? Its only for clan members so the traffic is extremely light. When i set it like this it still starts 2, one by root and the rest by www-data. Im never gonna have more then 10 people on my webserver at a time, how should i set it up?

MayDay · 09-11-2004, 09:15 PM

Okay while i did learn something very importent im now posetive that there was a problem "before".

After i found out about the free command i notice that was what i was using for my mrtg.

Code:

#!/bin/sh

USED=`free -b|grep cache:|cut -d ":" -f2|cut -c1-11`
FREE=`free -b|grep cache:|cut -d ":" -f2|cut -c12-22`

echo $USED
echo $FREE

This is live: http://pp.lir.dk:8080/mrtg/ppx.memory.html

Im using the server extensively while im setting it up and im 1 program away "eggdrop" from having the same load as before and it dosent show at all.

Im thinking its the "acpi?" fixes in the last bios update or i got lucky with the kernel i compiled.

Anyways, thanks much...