LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices



Reply
 
Search this Thread
Old 01-17-2010, 10:32 AM   #1
a6speedo
LQ Newbie
 
Registered: Jan 2010
Posts: 5

Rep: Reputation: 0
Unhappy top used memory vs ps processes memory


I found from command 'top' that 8GB memory are used. However, using command 'ps' with some options to grep the running processes and then summing up the memory used by the running processes are less than 2 GB. Where has the used memory gone ? what command to check this? Thks.
 
Old 01-17-2010, 10:36 AM   #2
johnsfine
Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,144

Rep: Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127
The "used" memory reported by top includes "buffers" and "cached", which are categories of memory use that are not included in any individual process in ps.

The whole topic is more complex than you might expect. But if you want a nearly correct greatly simplified view, you should consider "buffers" and "cached" to be free memory.

So the actual free memory is the sum of what top reports as "free", "buffers" and "cached". The actual used memory is what top reports as "used" minus both "buffers" and "cached"

The free command gives you used and free values both before and after that adjustment. top only gives you the unadjusted values.

Quote:
Originally Posted by a6speedo View Post
using command 'ps' with some options to grep the running processes and then summing up the memory used by the running processes are less than 2 GB.
I expect even that value is more than the actual memory used by your running processes.

If you enable the RSS column in ps and sum that, you will get a value that is at least the total memory use of the processes, but may be significantly more than the total memory use. I don't know anything else you might have summed that wouldn't be even further from correct.

Last edited by johnsfine; 01-17-2010 at 10:47 AM.
 
Old 01-18-2010, 10:59 AM   #3
a6speedo
LQ Newbie
 
Registered: Jan 2010
Posts: 5

Original Poster
Rep: Reputation: 0
Hi Johnsfine. Thank you very much for your info.
 
Old 06-11-2010, 09:36 PM   #4
dsb45
LQ Newbie
 
Registered: Dec 2009
Posts: 2

Rep: Reputation: 0
Hi Johnsfine,

You have the best understanding of Linux memory than anyone I've worked with!
I wonder if you know about the Active memory metric reported from /proc/meminfo
and also vmstat -a. The metrics from top and free line up perfectly - you can see how used - cache - buffers = [actual] used and that number subtracted from Total = the [real] free amount.

I can see that the Inactive number from /proc/meminfo and vmstat -a map to Buffer amount, but I have tried every combination I can think of and still do not see what the Active number is derived from. Does it include cache ? How is it used ? I've been told that one can use free, top, /proc/meminfo, or vmstat and get the same results, but that does not appear to be quite true.
What am I missing ?????????

Really appreciate your time and expertise!
Thanks.
 
Old 06-11-2010, 09:52 PM   #5
johnsfine
Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,144

Rep: Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127Reputation: 1127
I don't know that aspect of it.

Quote:
Originally Posted by dsb45 View Post
I can see that the Inactive number from /proc/meminfo and vmstat -a map to Buffer amount
Certainly not so on my system.

Free plus Inactive plus Active appears to be total ram. So Inactive plus Active is the reported Used memory, including Buffers and Cache.

But I don't know what distinguishes Active from Inactive.
 
Old 06-12-2010, 01:43 AM   #6
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 12,504

Rep: Reputation: 1079Reputation: 1079Reputation: 1079Reputation: 1079Reputation: 1079Reputation: 1079Reputation: 1079Reputation: 1079
Active and inactive are merely classifications the memory manager code uses to determine suitability of pages for reclaim. Recently referenced pages stay on the "active" queue. After a (essentially indeterminate) period of non-reference, they migrate to the inactive q, where they are amongst the first considered for reclaim.
Trying to be exact with memory allocation is a waste of time - too many unknown variables. I would expect active pus inactive (including anon a+i) plus slab should approximate the total non page-cache in use on a "quiet" system.
It is possible for pages no longer allocated to remain on any of those queues even after the owning process has terminated. There is no way to account for those that I know of (in userspace).
 
Old 06-12-2010, 05:46 PM   #7
dsb45
LQ Newbie
 
Registered: Dec 2009
Posts: 2

Rep: Reputation: 0
Hi johnsfine, and syg00,

Thank you both for responding so quickly, especially considering it's a weekend!
I've been on other Q&A sites where questions never get a response - so thanks again.

I did a top, free, cat /proc/meminfo, and vmstat -a on a relatively quiet(right now) Linux machine right before my post. Here's pertinent output:

--------------------------------------------------------------------------------
bl460-5:$ top

Cpu(s): 0.0%us, 0.0%sy, 0.0%ni, 99.7%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 16633024k total, 5745336k used, 10887688k free, 626460k buffers
Swap: 2097144k total, 0k used, 2097144k free, 3603556k cached

--------------------------------------------------------------------------------
bl460-5:$ free
total used free shared buffers cached
Mem: 16633024 5745832 10887192 0 626468 3603564
-/+ buffers/cache: 1515800 15117224
Swap: 2097144 0 2097144

--------------------------------------------------------------------------------
bl460-5:$ cat /proc/meminfo

MemTotal: 16633024 kB
MemFree: 10887688 kB
Buffers: 626468 kB
Cached: 3603564 kB
SwapCached: 0 kB
Active: 4941408 kB
Inactive: 646928 kB
HighTotal: 15858008 kB
HighFree: 10877288 kB
LowTotal: 775016 kB
LowFree: 10400 kB
SwapTotal: 2097144 kB
SwapFree: 2097144 kB
Dirty: 32 kB

-----------------------------------------------------------------------------------
bl460-5:$ vmstat 1 2 -a

r b swpd free inact active si so bi bo in cs us sy id wa st
0 0 0 10887316 646908 4941476 0 0 0 1 0 0 0 0 100 0 0
0 0 0 10887316 646904 4941480 0 0 0 20 1012 43 0 0 100 0 0

-----------------------------------------------------------------------------------

johnsfine, the top and /proc/meminfo Buffers number looked so close to the vmstat
inact number, that I was thinking it reflected the same thing. Not so ?

sysg00, thanks for admonition about a B-A-C-K-U-P. I ALWAYS take a backup before
ANY changes, and test changes on a machine that's not being used. It only takes one time of being burned to teach you that lesson (I was indeed once burned).

What we're really trying to determine is which number to report for the amount of
memory that was used during the benchmark of an application ? As I mentioned, several people have said that ALL the measurement tools (top, free, etc)
source the information they report from the same "place", and that /proc/meminfo
is the most reliable. Since the Active number from /proc/meminfo and vmstat
are a little different, it's not clear what the most accurate, "best" number
is to report.

Sorry this is so long, guys. Appreciate your help and interest.
Hope I can return the favor down the line.
 
Old 06-13-2010, 01:23 AM   #8
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 12,504

Rep: Reputation: 1079Reputation: 1079Reputation: 1079Reputation: 1079Reputation: 1079Reputation: 1079Reputation: 1079Reputation: 1079
Quote:
Originally Posted by dsb45 View Post
...the top and /proc/meminfo Buffers number looked so close to the vmstat inact number, that I was thinking it reflected the same thing. Not so ?
coincidence.
Quote:
What we're really trying to determine is which number to report for the amount of memory that was used during the benchmark of an application ?
Using system-wide metrics to infer the usage by a single process (and children ?) would seem especially unpredictable - and unrepeatable.
Quote:
As I mentioned, several people have said that ALL the measurement tools (top, free, etc) source the information they report from the same "place", and that /proc/meminfo is the most reliable. Since the Active number from /proc/meminfo and vmstat are a little different, it's not clear what the most accurate, "best" number is to report.
All these numbers are sampled from /proc/xxx
That means they are read at different times - each are as "correct" as any other. For that instant in time.
I would be inclined to use /proc/meminfo for the system-wide numbers, and /proc/<pid>/{stat,statm,status,smaps} for the process depending on what data format you need. You'll have to sample these yourself, or use something like sysstat or collectl.
Again these are only samples, so are not necessarily reproducable, or applicable to different environments.

Edit: if getrusage was fully implemented in Linux, the GNU time command would fit this requirement beautifully.

Last edited by syg00; 06-13-2010 at 01:26 AM.
 
  


Reply

Tags
memory, top, used


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
top shows high memory used but processes don't add up mohitanchlia Linux - General 2 04-08-2009 11:08 PM
Difference between resident memory,shared memory and virtual memory in system monitor mathimca05 Linux - Newbie 1 11-11-2007 05:05 AM
Processes and memory? Itzuke Slackware 1 04-11-2006 02:57 PM
how 2 c the memory used by processes imppayel Linux - General 3 12-08-2004 08:40 AM
Help!?! RH 8 Memory Mapping -High Memory-Virtural Memory issues.. Merlin53 Linux - Hardware 2 06-18-2003 05:48 PM


All times are GMT -5. The time now is 09:11 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration