LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 10-17-2005, 06:48 AM   #1
guarriman
Member
 
Registered: Nov 2004
Posts: 101

Rep: Reputation: 15
Out Of Memory problems. My server is down. Memory logs attached.


Hi.

Using Fedora Core 2 '2.6.9-1.667', I'm suffering memory problems. This night
I had to restart this server at 22:45h because most of daemons were killed.
The problem started at 21:03h.

From my '/var/log/message':
Code:
Oct 16 21:01:16 www crond(pam_unix)[995]: session closed for user root
Oct 16 21:03:10 www kernel: oom-killer: gfp_mask=0x1d2
[---NO LOG HERE! (??)---]
Oct 16 21:32:03 www kernel: DMA per-cpu:
...
Oct 16 21:36:43 www kernel: cpu 0 hot: low 2, high 6, batch 1
Oct 16 21:41:36 www kernel: cpu 0 cold: low 0, high 2, batch 1
...
Oct 16 21:44:52 www kernel: Normal per-cpu:
...
Oct 16 21:50:02 www kernel: cpu 0 hot: low 32, high 96, batch 16
...
Oct 16 21:50:22 www kernel: cpu 0 cold: low 0, high 32, batch 16
...
Oct 16 22:04:03 www kernel: HighMem per-cpu: empty
...
Oct 14 22:16:43 www crond(pam_unix)[1066]: session closed for user john
Oct 14 22:16:43 www kernel: 
Oct 14 22:16:45 www crond(pam_unix)[1096]: session closed for user mailman
Oct 14 22:16:45 www crond(pam_unix)[1105]: session closed for user root
Oct 14 22:45:56 www syslogd 1.4.1: restart.
Oct 14 22:45:56 www syslog: syslogd startup succeeded

I'm monitorizing memory uses each 10 minutes, and these are the outputs
at 21:00h.

Code:
[]# uptime
21:00:32 up 5 days, 10:49,  0 users,  load average: 22.53, 6.15, 2.23

[]# free -m
             total       used       free     shared    buffers     cached
Mem:           496        494          1          0          1         12
-/+ buffers/cache:        480         15
Swap:         1058        547        511

[]# vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
 0 32 560456   1592   1084  12572    0    2    13    75   30    21 10  1 88  1
 0 32 560200   2104   1048  11968  748    0  1108     0 1092   213  2  3  0 95
 0 32 559996   1976   1008  12148  768    0  1676     8 1138   258  2  5  0 93
 0 30 559692   1720   1024  12024  940    0  1800     0 1863   361  2  2  0 96
 0 31 559548   1912   1004  11764  484    0   948     4 1077   196  3  3  0 94

[]# ps auxf
USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  2868  480 ?        S    Oct09   0:01 init [3]                  
root         2  0.0  0.0     0    0 ?        SN   Oct09   0:00 [ksoftirqd/0]
root         3  0.0  0.0     0    0 ?        S<   Oct09   0:00 [events/0]
root         4  0.0  0.0     0    0 ?        S<   Oct09   0:00  \_ [khelper]
root         5  0.0  0.0     0    0 ?        S<   Oct09   0:00  \_ [kacpid]
root        20  0.0  0.0     0    0 ?        S<   Oct09   0:00  \_ [kblockd/0]
root        30  0.0  0.0     0    0 ?        D    Oct09   0:00  \_ [pdflush]
root        33  0.0  0.0     0    0 ?        S<   Oct09   0:00  \_ [aio/0]
root      4261  0.0  0.0     0    0 ?        D    Oct12   0:01  \_ [pdflush]
root        21  0.0  0.0     0    0 ?        S    Oct09   0:00 [khubd]
root        32  0.0  0.0     0    0 ?        S    Oct09   0:09 [kswapd0]
root       106  0.0  0.0     0    0 ?        S    Oct09   0:00 [kseriod]
root       175  0.0  0.0     0    0 ?        D    Oct09   0:25 [kjournald]
root      1055  0.0  0.0  1648  356 ?        S<s  Oct09   0:00 udevd
root      1809  0.0  0.0  2292  492 ?        Ds   Oct09   0:04 syslogd -m 0
root      1813  0.0  0.0  2072  380 ?        Ss   Oct09   0:00 klogd -x
root      1847  0.0  0.0  1808  504 ?        Ss   Oct09   0:00 rpc.idmapd
root      1866  0.0  0.0  2200  388 ?        Ss   Oct09   0:00 /usr/sbin/acpid
root      1897  0.0  0.2  5456 1044 ?        Ss   Oct09   0:03 /usr/sbin/sshd
root      1948  0.0  0.1  5404  524 ?        S    Oct09   0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/run/mysqld/mysqld.pid
mysql     1980  0.0  0.8 79880 4440 ?        S    Oct09   0:19  \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --socket=/var
mysql     1981  0.0  0.8 79880 4440 ?        S    Oct09   0:23      \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --socket=
mysql     1982  0.0  0.8 79880 4440 ?        S    Oct09   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1983  0.0  0.8 79880 4440 ?        S    Oct09   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1984  0.0  0.8 79880 4440 ?        S    Oct09   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1985  0.0  0.8 79880 4440 ?        S    Oct09   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1986  0.0  0.8 79880 4440 ?        S    Oct09   0:01          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1987  0.0  0.8 79880 4440 ?        S    Oct09   0:01          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1988  0.0  0.8 79880 4440 ?        S    Oct09   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1989  0.0  0.8 79880 4440 ?        S    Oct09   0:02          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql    30766  0.0  0.8 79880 4448 ?        S    19:25   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql    32508  0.0  0.8 79880 4448 ?        S    20:29   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql    32581  0.0  0.8 79880 4448 ?        S    20:31   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql    32675  0.0  0.8 79880 4448 ?        S    20:36   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      612  0.0  0.8 79880 4448 ?        S    20:43   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      694  0.0  0.8 79880 4448 ?        S    20:45   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      727  0.0  0.8 79880 4448 ?        S    20:48   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      728  0.0  0.8 79880 4448 ?        S    20:48   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      929  0.2  0.8 79880 4448 ?        S    20:59   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      938  0.1  0.8 79880 4448 ?        S    20:59   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      952  0.2  0.8 79880 4448 ?        S    20:59   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      956  0.0  0.8 79880 4448 ?        S    20:59   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      971  0.1  0.8 79880 4448 ?        D    20:59   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      979  0.2  0.8 79880 4448 ?        S    20:59   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      980  0.2  0.8 79880 4448 ?        D    20:59   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      983  0.0  0.8 79880 4448 ?        S    20:59   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql      984  0.0  0.8 79880 4448 ?        D    20:59   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1011  0.1  0.8 79880 4448 ?        S    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1012  0.0  0.8 79880 4448 ?        D    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1013  0.0  0.8 79880 4448 ?        D    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1014  0.0  0.8 79880 4448 ?        S    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1015  0.0  0.8 79880 4448 ?        D    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1016  0.0  0.8 79880 4448 ?        S    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1017  0.0  0.8 79880 4448 ?        D    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1018  0.0  0.8 79880 4448 ?        S    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1019  0.0  0.8 79880 4448 ?        D    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1020  0.0  0.8 79880 4448 ?        D    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1021  0.0  0.8 79880 4448 ?        D    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1022  0.0  0.8 79880 4448 ?        S    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1023  0.0  0.8 79880 4448 ?        S    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1024  0.0  0.8 79880 4448 ?        S    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1025  0.0  0.8 79880 4448 ?        S    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1026  0.0  0.8 79880 4448 ?        S    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1027  0.0  0.8 79880 4448 ?        S    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
mysql     1048  0.0  0.8 79880 4448 ?        S    21:00   0:00          \_ /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-locking --soc
postgres  2014  0.0  0.1 19668  916 ?        S    Oct09   0:00 /usr/bin/postmaster -p 5432 -D /var/lib/pgsql/data
postgres  2016  0.0  0.1 10468  920 ?        S    Oct09   0:00  \_ postgres: stats buffer process                    
postgres  2017  0.0  0.1  9476  916 ?        S    Oct09   0:00      \_ postgres: stats collector process                 
root      2038  0.0  0.2  8440 1524 ?        Ss   Oct09   0:00 sendmail: rejecting connections on daemon MTA: load average: 21
smmsp     2047  0.0  0.2  6816 1340 ?        Ss   Oct09   0:00 sendmail: Queue runner@01:00:00 for /var/spool/clientmqueue
daemon    2084  0.0  0.0  3224  480 ?        Ss   Oct09   0:00 /usr/sbin/atd
dbus      2093  0.0  0.1  2408  520 ?        Ss   Oct09   0:00 dbus-daemon-1 --system
root      2102  0.0  0.2  6776 1336 ?        Ss   Oct09   1:43 hald
mailman   2191  0.0  0.2 10560 1172 ?        Ss   Oct09   0:00 /usr/bin/python /usr/lib/mailman/bin/mailmanctl -s -q start
mailman   2192  0.0  0.3 11024 1820 ?        S    Oct09   0:02  \_ /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=ArchRunner:0:1 -s
mailman   2193  0.0  0.3 10760 1852 ?        S    Oct09   0:03  \_ /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=BounceRunner:0:1 -s
mailman   2194  0.0  0.3 10144 1776 ?        S    Oct09   0:02  \_ /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=CommandRunner:0:1 -s
mailman   2195  0.0  0.3 10216 1828 ?        S    Oct09   0:02  \_ /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=IncomingRunner:0:1 -s
mailman   2196  0.0  0.3 10112 1812 ?        S    Oct09   0:02  \_ /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=NewsRunner:0:1 -s
mailman   2197  0.0  0.3 13624 1956 ?        D    Oct09   0:03  \_ /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=OutgoingRunner:0:1 -s
mailman   2209  0.0  0.3 12320 1928 ?        D    Oct09   0:03  \_ /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=VirginRunner:0:1 -s
mailman   2210  0.0  0.2 10084 1180 ?        S    Oct09   0:00  \_ /usr/bin/python /usr/lib/mailman/bin/qrunner --runner=RetryRunner:0:1 -s
root      2204  0.0  0.1  3552  548 ?        S    Oct09   0:00 su - ogo -c /usr/local/sbin/ogo-webui-1.0a -WOPort 20000 -WOHttpAllowHost 127.0.0.1,www.mydomain.com,localhost -OGoMinimumActiveSessionCou
ogo       2221  0.0  0.1 14812  908 ?        S    Oct09   0:00  \_ /usr/local/sbin/ogo-webui-1.0a -WOPort 20000 -WOHttpAllowHost 127.0.0.1,www.mydomain.com,localhost -OGoMinimumActiveSessionCount 0
ogo       2316  0.0  0.2 40644 1160 ?        S    Oct09   0:12      \_ /usr/local/sbin/ogo-webui-1.0a -WOPort 20000 -WOHttpAllowHost 127.0.0.1,www.mydomain.com,localhost -OGoMinimumActiveSessionCount 0
root      2212  0.0  0.1  3308  544 ?        S    Oct09   0:00 su - ogo -c /usr/local/sbin/ogo-xmlrpcd-1.0a -WOPort 20002
ogo       2230  0.0  0.1 13576  908 ?        S    Oct09   0:00  \_ /usr/local/sbin/ogo-xmlrpcd-1.0a -WOPort 20002
ogo       2317  0.0  0.1 14008  972 ?        S    Oct09   0:00      \_ /usr/local/sbin/ogo-xmlrpcd-1.0a -WOPort 20002
root      2215  0.0  0.1  2208  544 ?        S    Oct09   0:00 su - ogo -c /usr/local/sbin/ogo-zidestore-1.3 -WOPort 23000 -WOHttpAllowHost 127.0.0.1,www.mydomain.com,localhost -SxExplain YES -WOCachin
ogo       2237  0.0  0.1 12996  908 ?        S    Oct09   0:00  \_ /usr/local/sbin/ogo-zidestore-1.3 -WOPort 23000 -WOHttpAllowHost 127.0.0.1,www.mydomain.com,localhost -SxExplain YES -WOCachingEnabled
ogo       2315  0.0  0.1 17840  980 ?        S    Oct09   0:00      \_ /usr/local/sbin/ogo-zidestore-1.3 -WOPort 23000 -WOHttpAllowHost 127.0.0.1,www.mydomain.com,localhost -SxExplain YES -WOCachingEna
root      2299  0.0  0.3 10144 1992 ?        Ss   Oct09   0:00 /usr/bin/perl -wT /usr/sbin/pop-before-smtp --daemon=/var/run/pop-before-smtp.pid
root      2308  0.0  0.1  5488  676 ?        S    Oct09   0:00 /usr/sbin/vsftpd /etc/vsftpd/vsftpd.conf
root      2318  0.0  0.3  6476 2024 ?        Ss   Oct09   0:00 /usr/bin/perl /usr/libexec/webmin/miniserv.pl /etc/webmin/miniserv.conf
root      2328  0.0  0.1  2768  568 ?        Ss   Oct09   0:01 xinetd -stayalive -pidfile /var/run/xinetd.pid
root      2332  0.0  0.0  2416  320 tty1     Ss+  Oct09   0:00 /sbin/mingetty tty1
root      2333  0.0  0.0  2484  320 tty2     Ss+  Oct09   0:00 /sbin/mingetty tty2
root      2334  0.0  0.0  2708  320 tty3     Ss+  Oct09   0:00 /sbin/mingetty tty3
root      2335  0.0  0.0  2540  320 tty4     Ss+  Oct09   0:00 /sbin/mingetty tty4
root      2336  0.0  0.0  2128  320 tty5     Ss+  Oct09   0:00 /sbin/mingetty tty5
root      2337  0.0  0.0  2152  320 tty6     Ss+  Oct09   0:00 /sbin/mingetty tty6
root      6705  0.0  1.9 22760 10084 ?       Ss   Oct11   0:00 /usr/sbin/httpd
apache   29970  0.8  2.2 34120 11572 ?       D    18:52   1:06  \_ /usr/sbin/httpd
apache   30263  0.7  2.2 33808 11184 ?       D    19:02   0:54  \_ /usr/sbin/httpd
apache   31201  0.7  2.1 33972 11160 ?       S    19:35   0:39  \_ /usr/sbin/httpd
apache   32465  0.9  2.8 34768 14332 ?       S    20:28   0:18  \_ /usr/sbin/httpd
apache   32479  0.9  2.2 33684 11528 ?       S    20:28   0:18  \_ /usr/sbin/httpd
apache   32670  0.6  2.8 34380 14360 ?       S    20:36   0:08  \_ /usr/sbin/httpd
apache     455  0.7  2.0 33756 10532 ?       S    20:39   0:09  \_ /usr/sbin/httpd
apache     499  0.5  2.9 34052 14992 ?       D    20:40   0:07  \_ /usr/sbin/httpd
apache     643  0.5  2.7 34036 13820 ?       D    20:44   0:05  \_ /usr/sbin/httpd
apache     695  0.4  2.3 33668 11936 ?       D    20:45   0:04  \_ /usr/sbin/httpd
apache     921  1.3  2.0 33604 10548 ?       S    20:59   0:00  \_ /usr/sbin/httpd
apache     927  0.9  2.5 33900 13004 ?       S    20:59   0:00  \_ /usr/sbin/httpd
apache     928  0.5  2.5 33740 13100 ?       D    20:59   0:00  \_ /usr/sbin/httpd
apache     934  0.6  2.3 33888 12104 ?       S    20:59   0:00  \_ /usr/sbin/httpd
apache     935  0.4  2.4 32516 12412 ?       S    20:59   0:00  \_ /usr/sbin/httpd
apache     936  0.9  2.7 32576 14008 ?       S    20:59   0:00  \_ /usr/sbin/httpd
apache     937  0.7  2.5 32444 13004 ?       S    20:59   0:00  \_ /usr/sbin/httpd
apache     943  0.7  2.4 32080 12340 ?       D    20:59   0:00  \_ /usr/sbin/httpd
apache     944  1.2  2.7 33944 13776 ?       D    20:59   0:00  \_ /usr/sbin/httpd
apache     945  0.2  2.1 30500 10712 ?       S    20:59   0:00  \_ /usr/sbin/httpd
apache     947  0.4  2.1 31800 11140 ?       D    20:59   0:00  \_ /usr/sbin/httpd
apache     948  0.4  2.5 31800 12864 ?       D    20:59   0:00  \_ /usr/sbin/httpd
apache     949  0.5  2.5 33852 13148 ?       S    20:59   0:00  \_ /usr/sbin/httpd
apache     950  1.3  2.7 33948 13856 ?       D    20:59   0:00  \_ /usr/sbin/httpd
apache     957  1.3  2.2 33604 11460 ?       D    20:59   0:00  \_ /usr/sbin/httpd
apache     958  0.6  2.6 32080 13524 ?       D    20:59   0:00  \_ /usr/sbin/httpd
apache     959  0.8  2.5 32092 12820 ?       S    20:59   0:00  \_ /usr/sbin/httpd
apache     960  1.5  2.3 34240 11788 ?       S    20:59   0:00  \_ /usr/sbin/httpd
apache     961  0.8  2.2 33464 11456 ?       D    20:59   0:00  \_ /usr/sbin/httpd
apache     962  0.8  2.1 33464 10836 ?       S    20:59   0:00  \_ /usr/sbin/httpd
apache     963  0.8  2.6 33888 13368 ?       D    20:59   0:00  \_ /usr/sbin/httpd
apache     964  0.2  2.1 28992 10680 ?       D    20:59   0:00  \_ /usr/sbin/httpd
apache    1002  0.3  2.0 28992 10548 ?       S    21:00   0:00  \_ /usr/sbin/httpd
apache    1003  0.6  2.5 32316 12916 ?       D    21:00   0:00  \_ /usr/sbin/httpd
apache    1004  0.3  2.0 28992 10596 ?       S    21:00   0:00  \_ /usr/sbin/httpd
apache    1005  0.4  2.0 30356 10656 ?       S    21:00   0:00  \_ /usr/sbin/httpd
apache    1006  0.3  2.1 28992 10880 ?       D    21:00   0:00  \_ /usr/sbin/httpd
apache    1007  0.0  2.1 22888 10808 ?       D    21:00   0:00  \_ /usr/sbin/httpd
apache    1008  0.4  2.0 30356 10640 ?       S    21:00   0:00  \_ /usr/sbin/httpd
apache    1010  0.3  2.1 28992 10952 ?       D    21:00   0:00  \_ /usr/sbin/httpd
apache    1042  0.1  2.4 24524 12700 ?       D    21:00   0:00  \_ /usr/sbin/httpd
apache    1043  0.0  2.0 22760 10632 ?       D    21:00   0:00  \_ /usr/sbin/httpd
apache    1045  0.0  2.0 22760 10200 ?       S    21:00   0:00  \_ /usr/sbin/httpd
apache    1046  0.0  2.0 22760 10200 ?       S    21:00   0:00  \_ /usr/sbin/httpd
apache    1050  0.0  2.0 22760 10200 ?       S    21:00   0:00  \_ /usr/sbin/httpd
named     8748  0.0  0.2 38256 1092 ?        Ssl  Oct11   0:00 /usr/sbin/named -u named
root     25605  0.0  0.1  4484  544 ?        Ss   Oct11   0:00 crond
root       995  0.0  0.1  5008  584 ?        S    21:00   0:00  \_ crond
root      1029  0.0  0.1  3756  696 ?        Ss   21:00   0:00  |   \_ /bin/sh -c /root/memmon.sh >> /root/memmon.txt
root      1034  0.0  0.1  2148  756 ?        S    21:00   0:00  |       \_ /bin/bash /root/memmon.sh
root      1051  0.2  0.1  2504  832 ?        R    21:00   0:00  |           \_ ps auxf --width=200
root       996  0.0  0.1  5008  628 ?        S    21:00   0:00  \_ crond
peter    1030  0.2  0.1  4064  656 ?        Ss   21:00   0:00  |   \_ /bin/bash /home/peter/tools/update
peter    1040  0.0  0.0  4380  424 ?        D    21:00   0:00  |       \_ svn --username peter --password berlin co http://www.mydomain.com/svn/myproject
root       997  0.0  0.1  5008  640 ?        S    21:00   0:00  \_ crond
peter    1031  0.1  0.1  2212  764 ?        Ss   21:00   0:00  |   \_ /bin/sh /home/peter/tools/peter-script.sh
peter    1039  0.0  0.0  6356  368 ?        D    21:00   0:00  |       \_ /usr/local/bin/php peter-script.php
root       998  0.0  0.1  5008  584 ?        S    21:00   0:00  \_ crond
root      1033  0.0  0.1  5004  580 ?        S    21:00   0:00  |   \_ crond
root       999  0.0  0.1  5008  584 ?        S    21:00   0:00  \_ crond
512       1028  0.1  0.1  3932  768 ?        Ss   21:00   0:00  |   \_ /bin/sh /home/john/tools/john-script.sh
512       1038  0.0  0.0  6696  392 ?        D    21:00   0:00  |       \_ /usr/local/bin/php john-script.php
root      1000  0.0  0.1  5004  584 ?        S    21:00   0:00  \_ crond
root      1001  0.0  0.1  5004  596 ?        S    21:00   0:00  \_ crond
root      1009  0.0  0.1  5004  688 ?        S    21:00   0:00  \_ crond
Does anybody have any clue of the reason of my Memory problem? Thank you very much.
 
Old 10-17-2005, 06:57 AM   #2
jtshaw
Senior Member
 
Registered: Nov 2000
Location: Seattle, WA USA
Distribution: Ubuntu @ Home, RHEL @ Work
Posts: 3,892
Blog Entries: 1

Rep: Reputation: 66
You seam to have plenty of swap space free... The thing that concerns me is the load of 22.53. Can you run top while this is happening and tell us what process(es) are hogging all that CPU time?
 
Old 10-17-2005, 09:31 AM   #3
guarriman
Member
 
Registered: Nov 2004
Posts: 101

Original Poster
Rep: Reputation: 15
Hi.

Thank you very much, jtshaw, for your answer. I didn't notice the figures you point.

It's a very high load, but the unique log I've got it's the 'ps aux' output I reported
Isn't it the same information as 'top'?

Regards.
 
Old 10-17-2005, 11:52 AM   #4
TigerOC
Senior Member
 
Registered: Jan 2003
Location: Devon, UK
Distribution: Debian Etc/kernel 2.6.18-4K7
Posts: 2,380

Rep: Reputation: 49
just looking at that log and specifically looking at the mysql and httpd makes me suspicious that perhaps you have a php script querying the database that is not closing. I have experienced this myself and what it does it leaves the query open. You have 38% use by mysql. Have you added any new query scripts recently?
 
Old 10-18-2005, 04:08 AM   #5
guarriman
Member
 
Registered: Nov 2004
Posts: 101

Original Poster
Rep: Reputation: 15
Hi TigerOC. Thank you very much for your answer.

I'm not very sure about how to interpret the memory use.

If I add up every 'mysql' processes at the time of the incident, I get 36-38%, and if I add up every proccess, I
get 149%. So I consider it's the reason of my problem, right?

But If I add up every proccess just now, I get 228%, but my server runs ok right now. :?

Could anybody explain it to me?
 
Old 10-18-2005, 05:09 AM   #6
TigerOC
Senior Member
 
Registered: Jan 2003
Location: Devon, UK
Distribution: Debian Etc/kernel 2.6.18-4K7
Posts: 2,380

Rep: Reputation: 49
This article may help you pinpoint the problem. I see from the log that you are running the memmon script so have you checked the memmon.txt file?

Last edited by TigerOC; 10-18-2005 at 05:12 AM.
 
Old 10-18-2005, 11:35 AM   #7
guarriman
Member
 
Registered: Nov 2004
Posts: 101

Original Poster
Rep: Reputation: 15
Hi TigerOC.

I'm actually checking that document to find out any explanation.

Thank you very much!!!
 
Old 06-10-2008, 07:46 AM   #8
sudhi_naidu
LQ Newbie
 
Registered: Jun 2008
Posts: 7

Rep: Reputation: 0
Smile

If I add up every 'mysql' processes at the time of the incident, I get 36-38%, and if I add up every proccess, I
get 149%. So I consider it's the reason of my problem, right?

But If I add up every proccess just now, I get 228%, but my server runs ok right now. :?


This could be because, individual process memory stats show the shared libraries also that are loaded. Where as in the total memory usage memory used by shared libraries is counted only once.... You can probably check how much memory exactly a process is using with pmap <pid> command. (my guessing)

Thanks,
Sudheendra
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
tool to detect number of memory slots, memory module type, speed, and size stefanwolf Linux - Hardware 8 04-22-2013 09:48 AM
Missing memory in USB stick, flash memory, removable hd etc bamboo_spider Linux - Newbie 3 06-14-2006 05:39 PM
Memory Leak when using memory debugging C program on SuSE SLES8 babalina Linux - Distributions 0 10-06-2003 09:39 AM
Help!?! RH 8 Memory Mapping -High Memory-Virtural Memory issues.. Merlin53 Linux - Hardware 2 06-18-2003 04:48 PM
RH 8.0 Mapping Virtual Memory to get access to VMIC Reflective Memory PCI card. Merlin53 Linux - Hardware 0 05-05-2003 12:50 PM


All times are GMT -5. The time now is 06:03 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration