Help diagnose memory leak problem

itibet · 11-06-2010, 07:24 AM

Hi,
I'm running several SHOUTcast server instances and a WowzaMediaServer instance on a CentOS machine.
I'm experiencing a memory leak problem, but I can't figure out which processes are eating memory.

TOP command reports as follows:

Cpu(s): 0.9%us, 1.7%sy, 0.0%ni, 96.8%id, 0.0%wa, 0.1%hi, 0.6%si, 0.0%st
Mem: 8153084k total, 7533528k used, 619556k free, 240984k buffers
Swap: 1052464k total, 0k used, 1052464k free, 6929692k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
15011 root 40 0 1440m 89m 7788 S 4.7 1.1 1:03.01 java
5873 centovac 40 0 36060 2236 732 S 1.0 0.0 12:01.07 sc_serv
5439 centovac 40 0 36060 2336 732 S 0.7 0.0 11:56.51 sc_serv
5465 centovac 40 0 36060 2304 732 S 0.7 0.0 12:16.37 sc_serv
5597 centovac 40 0 36060 2368 732 S 0.7 0.0 13:19.11 sc_serv
5850 centovac 40 0 36060 2112 732 S 0.7 0.0 11:26.37 sc_serv
5894 centovac 40 0 34264 2432 728 S 0.7 0.0 12:16.28 sc_serv
5381 centovac 40 0 36060 2472 728 S 0.3 0.0 12:59.27 sc_serv
5642 centovac 40 0 36060 3000 732 S 0.3 0.0 13:05.48 sc_serv
5855 centovac 40 0 36060 2164 732 S 0.3 0.0 12:03.41 sc_serv
5864 centovac 40 0 36060 2160 728 S 0.3 0.0 11:58.29 sc_serv
5885 centovac 40 0 36060 2164 732 S 0.3 0.0 12:05.75 sc_serv
1 root 40 0 2068 620 536 S 0.0 0.0 0:01.36 init
2 root 40 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:00.06 migration/0
4 root 20 0 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
...

Something misterious to me (I'm still a Linux newbie) is that TOP reports a total of 7.5GB used ram but very small percentage for single process (0-1%).

Memory consumption starts at 1GB/8GB after reboot and in three days running gradually increases up to 8GB.

I'm practising with Linux, but I still miss a lot to understand what's happening on my system.
Can anybody suggest me a way to help diagnose the problem?
For instance, are there linux kernel logs saved somewhere that I can look at?

Thanks!

AlucardZero · 11-06-2010, 07:57 AM

http://www.linuxatemyram.com/

itibet · 11-06-2010, 08:29 AM

Quote:

Originally Posted by AlucardZero

http://www.linuxatemyram.com/

Thanks a lot, linux strange/wonderful world

Flyte · 11-06-2010, 09:09 AM

Nice website!

You may want to have a look at 'atop' which is an advanced version of top.
This will tell you how much memory is being used, and how much of it is used by the cache, as highlighted below:

Code:

ATOP - dade          2010/11/06  14:06:30          ------            10s elapsed
PRC | sys    0.02s | user   0.01s | #proc    117 | #zombie    0 | #exit      0 |
CPU | sys       0% | user      0% | irq       0% | idle    200% | wait      0% |
cpu | sys       0% | user      0% | irq       0% | idle    100% | cpu000 w  0% |
cpu | sys       0% | user      0% | irq       0% | idle    100% | cpu001 w  0% |
CPL | avg1    0.00 | avg5    0.00 | avg15   0.00 | csw      916 | intr   10025 |
MEM | tot     2.0G | free   76.0M | cache   1.3G | buff  340.1M | slab  144.2M |
SWP | tot     3.9G | free    3.9G |              | vmcom 540.4M | vmlim   4.9G |
LVM | p00-LogVol00 | busy      0% | read       0 | write     26 | avio 0.00 ms |
DSK |          sdb | busy      0% | read       0 | write     16 | avio 0.00 ms |
NET | transport    | tcpi       5 | tcpo       6 | udpi       0 | udpo       0 |
NET | network      | ipi        5 | ipo        6 | ipfrw      0 | deliv      5 |
NET | eth0      0% | pcki       1 | pcko       2 | si    0 Kbps | so    1 Kbps |
NET | lo      ---- | pcki       4 | pcko       4 | si    0 Kbps | so    0 Kbps |

  PID  SYSCPU  USRCPU  VGROW  RGROW  RDDSK   WRDSK ST EXC S  CPU  CMD        1/1
 5170   0.02s   0.01s     0K     0K     0K      0K --   - R   0%  atop
 2854   0.00s   0.00s     0K     0K     0K      4K --   - S   0%  smbd
 4026   0.00s   0.00s     0K     0K     0K      0K --   - S   0%  sshd
 2586   0.00s   0.00s     0K     0K     0K      0K --   - S   0%  nmbd
 2112   0.00s   0.00s     0K     0K     0K     16K --   - S   0%  syslogd
 2185   0.00s   0.00s     0K     0K     0K      0K --   - S   0%  irqbalance
  423   0.00s   0.00s     0K     0K     0K      4K --   - S   0%  kjournald

My poor underused server

itibet · 11-06-2010, 11:56 AM

Thanks, very useful!