Linux isn't using all of its RAM

syg00 · 06-01-2007, 08:12 PM

Quote:

Originally Posted by Tinkster

Humour me and post the output of free -m

Next week Tink - it's heading well towards midday Saturday here.
I can't even dial into it as I turn it off over the weekend - test box only after all.

rsmccain · 06-01-2007, 08:19 PM

Quote:

Originally Posted by DirkDiggler

Hey,

I made it home and used the 'free -m' command. My total is 756, used is 317 (thats with klotski running, a game), cached is 133. I started up GIMP, total is 756, used 355, cached is 154. Left klotski gimp open, opened up Open Office word processor and: total 756, used 472, cached 253. So, in my case, it doesn't cache more unless more stuff is opened. I'm running a fresh install of Fedora 7, in case you were wondering.

Regards,

Brandon

Well, it looks like "I" stand corrected. Any idea when this changed? Maybe between the 2.4x and 2.6x kernels? Or maybe I never had it right in the first place.

Quakeboy02 · 06-01-2007, 08:32 PM

I haven't seen swappiness mentioned, so here's my obligatory link to the swappiness article: http://gentoo-wiki.com/FAQ_Linux_Mem...2.6_kernels.29

For an exhaustive argument about the subject of swappiness among noted kernel developers, look here: http://kerneltrap.org/node/3000

DirkDiggler · 06-01-2007, 08:59 PM

The plot thickens. I've had an up time of 1 hour 42 minutes and my used is at 748 with 518 cached out of 756. This is idle, nothing running but terminal. Maybe, the memory has to be used first in order for it to be cached? Unfortunately, I'm not the expert. Maybe somewhere down the line I will be.

Regards,

Brandon

rsmccain · 06-01-2007, 09:01 PM

Quote:

Originally Posted by DirkDiggler

The plot thickens. I've had an up time of 1 hour 42 minutes and my used is at 748 with 518 cached out of 756. This is idle, nothing running but terminal. Maybe, the memory has to be used first in order for it to be cached? Unfortunately, I'm not the expert. Maybe somewhere down the line I will be.

Regards,

Brandon

hahaha.

Yes, this is strange. I'm 90% sure it used to be the way I described earlier but maybe it changed in the 2.6 kernel. I'm trying to find an official statement on the topic somewhere but haven't had any luck as of yet. Maybe someone here can clarify all of it.

Quakeboy02 · 06-01-2007, 09:27 PM

Quote:

I'm 90% sure it used to be the way I described earlier but maybe it changed in the 2.6 kernel.

From the swappiness link I posted: "Since 2.6, there has been a way to tune how much Linux favors swapping out to disk compared to shrinking the caches when memory gets full."

rsmccain · 06-01-2007, 10:18 PM

Quote:

Originally Posted by Quakeboy02

From the swappiness link I posted: "Since 2.6, there has been a way to tune how much Linux favors swapping out to disk compared to shrinking the caches when memory gets full."

Make sense now. Thanks for clarifying.

syg00 · 06-03-2007, 11:37 PM

As requested: first list is immediately after boot, second is an hour later after starting X (Gnome 2.18.2) and Firefox (2.0.0.4), and running updatedb from a terminal. It will sit at these allocations for days.

Code:

             total       used       free     shared    buffers     cached
Mem:          3729         42       3687          0          3         16
-/+ buffers/cache:         23       3706
Swap:         1000          0       1000

             total       used       free     shared    buffers     cached
Mem:          3729        247       3482          0         70         90
-/+ buffers/cache:         86       3642
Swap:         1000          0       1000

rsmccain · 06-04-2007, 11:05 AM

Quote:

Originally Posted by syg00

As requested: first list is immediately after boot, second is an hour later after starting X (Gnome 2.18.2) and Firefox (2.0.0.4), and running updatedb from a terminal. It will sit at these allocations for days.

Code:

             total       used       free     shared    buffers     cached
Mem:          3729         42       3687          0          3         16
-/+ buffers/cache:         23       3706
Swap:         1000          0       1000

             total       used       free     shared    buffers     cached
Mem:          3729        247       3482          0         70         90
-/+ buffers/cache:         86       3642
Swap:         1000          0       1000

Thanks...

Here's my dilema.

top - 10:58:57 up 2 days, 12:43, 2 users, load average: 7.20, 7.13, 7.10
Tasks: 208 total, 2 running, 205 sleeping, 0 stopped, 1 zombie
Cpu(s): 50.6% us, 0.2% sy, 0.0% ni, 48.9% id, 0.3% wa, 0.0% hi, 0.0% si
Mem: 1034228k total, 282076k used, 752152k free, 26764k buffers
Swap: 524280k total, 13080k used, 511200k free, 137416k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
30830 root 25 0 1492 496 412 R 99.5 0.0 55:03.92 udevinfo
30974 root 16 0 385m 44m 12m S 1.0 4.4 0:19.15 ndsd
820 root 16 0 2812 1464 1088 R 0.7 0.1 0:01.16 top
7348 root 16 0 91772 1764 1400 S 0.3 0.2 18:42.52 httpstkd
1 root 16 0 588 68 44 S 0.0 0.0 0:26.64 init
2 root RT 0 0 0 0 S 0.0 0.0 0:00.07 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 0:00.05 ksoftirqd/0
4 root RT 0 0 0 0 S 0.0 0.0 0:00.10 migration/1
5 root 34 19 0 0 0 S 0.0 0.0 0:00.03 ksoftirqd/1
6 root 5 -10 0 0 0 S 0.0 0.0 0:00.02 events/0
7 root 5 -10 0 0 0 S 0.0 0.0 0:00.01 events/1
8 root 5 -10 0 0 0 S 0.0 0.0 0:00.03 khelper
9 root 15 -10 0 0 0 S 0.0 0.0 0:00.00 kacpid
46 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 kblockd/0
47 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 kblockd/1
66 root 15 0 0 0 0 S 0.0 0.0 0:00.00 kirqd
71 root 16 0 0 0 0 S 0.0 0.0 1:18.29 kswapd0
ServerA:/etc/init.d # free -m
total used free shared buffers cached
Mem: 1009 275 734 0 26 134
-/+ buffers/cache: 115 894
Swap: 511 12 499

As you can see "udevinfo" is maxing out the CPU at 99% however you can see no extra memory isn't being allocated.

This happens when migrate a cluster service over to this servers, which is part of a cluster. It also maxes out for about 10 minutes when the system boots up.

Any ideas?

Tinkster · 06-04-2007, 07:06 PM

Quote:

Originally Posted by syg00

As requested: first list is immediately after boot, second is an hour later after starting X (Gnome 2.18.2) and Firefox (2.0.0.4), and running updatedb from a terminal. It will sit at these allocations for days.

Code:

             total       used       free     shared    buffers     cached
Mem:          3729         42       3687          0          3         16
-/+ buffers/cache:         23       3706
Swap:         1000          0       1000

             total       used       free     shared    buffers     cached
Mem:          3729        247       3482          0         70         90
-/+ buffers/cache:         86       3642
Swap:         1000          0       1000

Interesting ... what is this server serving? :} Next time I have to
re-boot one of the servers here I'll look at this, too.

Anyway ... I still think we have to agree that on rsmccain's
machine things should be different; considering his swap-usage
and buffers the used memory *should* have climbed up.

Cheers,
Tink

Tinkster · 06-04-2007, 07:09 PM

Quote:

Originally Posted by rsmccain

Thanks...

Here's my dilema.

As you can see "udevinfo" is maxing out the CPU at 99% however you can see no extra memory isn't being allocated.

This happens when migrate a cluster service over to this servers, which is part of a cluster. It also maxes out for about 10 minutes when the system boots up.

Any ideas?

Do you get anything in dmesg or the system logs regarding this?
And did you find anything in grub-/lilo configuration that would
explain the lack of memory use?

Cheers,
Tink

syg00 · 06-04-2007, 07:49 PM

Quote:

Originally Posted by Tinkster

Interesting ... what is this server serving? :}

Nothing ...
Merely offered as evidence that Linux doesn't (unilaterally) allocate all the memory.
I use it for things like cpusets testing, and kernel patch testing - especially early (pre-release) patch sets.

As for the OPs problem, let's see that /proc/meminfo again now. Also maybe slabtop

Code:

slabtop | tee slabtop.txt

As soon as you see some info enter<q>, and post the txt file.

rsmccain · 06-05-2007, 11:20 AM

Quote:

Originally Posted by Tinkster

Do you get anything in dmesg or the system logs regarding this?
And did you find anything in grub-/lilo configuration that would
explain the lack of memory use?

Cheers,
Tink

Everything was normal. No strange kernel parameters. I found out that when the system boots it maxes out the CPU at 100% on udevinfo and hwscan for about 20 minutes then things settle down.

I'm running Novell Cluser Services. When I migrate a cluser service such as mail or proxy over to the server udevinfo will max out the cpu at 100% until I manually kill it.

Here is output from ps:
ServerA:~ # ps aux | grep udevinfo
root 30829 0.0 0.0 3464 636 ? S Jun04 0:00 sh -c /usr/bin/udevinfo -d 2>/dev/null
root 30830 99.7 0.0 1492 496 ? R Jun04 1514:05 /usr/bin/udevi

..

I have no idea why a cluster service would kick off udevinfo

rsmccain · 06-05-2007, 11:24 AM

Quote:

Originally Posted by syg00

Nothing ...
Merely offered as evidence that Linux doesn't (unilaterally) allocate all the memory.
I use it for things like cpusets testing, and kernel patch testing - especially early (pre-release) patch sets.

As for the OPs problem, let's see that /proc/meminfo again now. Also maybe slabtop

Code:

slabtop | tee slabtop.txt

As soon as you see some info enter<q>, and post the txt file.

the slaptop.txt file came up empty, however, here is the output of the "slabtop" command:

Active / Total Objects (% used) : 511967 / 520450 (98.4%)
Active / Total Slabs (% used) : 37075 / 37080 (100.0%)
Active / Total Caches (% used) : 103 / 145 (71.0%)
Active / Total Size (% used) : 144641.91K / 145567.62K (99.4%)
Minimum / Average / Maximum Object : 0.01K / 0.28K / 128.00K

OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
210496 210472 99% 0.15K 8096 26 32384K dentry_cache
143940 143937 99% 0.62K 23990 6 95960K ext3_inode_cache
69935 69872 99% 0.05K 985 71 3940K buffer_head
19000 18729 98% 0.02K 95 200 380K dm_io
19000 18729 98% 0.02K 95 200 380K dm_tio
13514 13462 99% 0.06K 233 58 932K size-64
8932 8930 99% 0.27K 638 14 2552K radix_tree_node
3240 2408 74% 0.08K 72 45 288K vm_area_struct
3210 3038 94% 0.12K 107 30 428K size-128
2960 2935 99% 0.38K 296 10 1184K proc_inode_cache
2820 2818 99% 0.38K 282 10 1128K inode_cache
2500 982 39% 0.01K 10 250 40K anon_vma
2128 2051 96% 0.03K 19 112 76K size-32
1845 1472 79% 0.25K 123 15 492K filp
800 73 9% 0.02K 4 200 16K tcp_bind_bucket
696 529 76% 0.06K 12 58 48K inet_peer_cache

..Heres /proc/meminfo
MemTotal: 1034228 kB
MemFree: 341920 kB
Buffers: 261244 kB
Cached: 180996 kB
SwapCached: 6780 kB
Active: 239420 kB
Inactive: 283832 kB
HighTotal: 131008 kB
HighFree: 240 kB
LowTotal: 903220 kB
LowFree: 341680 kB
SwapTotal: 524280 kB
SwapFree: 511480 kB
Dirty: 28 kB
Writeback: 0 kB
Mapped: 113584 kB
Slab: 154284 kB
Committed_AS: 676152 kB
PageTables: 1080 kB
VmallocTotal: 114680 kB
VmallocUsed: 7896 kB
VmallocChunk: 106636 kB
HugePages_Total: 0
HugePages_Free: 0
Hugepagesize: 4096 kB

..

I posted those as udevinfo is maxing out the CPU at 100%

Thanks.

syg00 · 06-05-2007, 06:12 PM

Nothing seems too out of order there.

Quote:

I posted those as udevinfo is maxing out the CPU at 100%

You appear to have a problem with udevinfo, not with memory. I would also find it hard to believe that the memory is causing the udevinfo to go ballistic. Possible but unlikely.

Code:

Committed_AS: 676152 kB

This is a "best guess" of how much (total) memory you would currently need to satisfy all requests - including mallocs not actually used yet.

Kill the udevinfo, and chase that up. Relax about the memory (non-)usage.
As for the minimal swap usage, I don't know but I'd just ignore it. Happens over time anyway.

BTW - when posting output, use "code" tags so it formats properly.