Memory reporting corrupt?
 

First, apologies - I posted about this before in the SuSE board, but perhaps it's not a SuSE issue. And it really troubles me.

My system (SuSE 10 amd64, newly installed) seems to report inconsistent data for memory usage. This is what I see in top:

while free gives me

So, java is using 1.5 GIGAbytes of virtual memory? 200 mb in physical, and 1.3 gigabytes swapped out? This can't be, since I am only using 380 + 212 (physical + swap) mb of ram. Java seems to take up more virtual memory than I have physical+virtual memory in my system...

Also, I see similar things with knotify (virtual memory of over a gigabyte). It isn't just java, but is most pronounced there.

I have read the man pages several times, and googled for help on this. My only conclusion after all of this is that something is corrupt in my system's memory reporting?

it's a little daunting but its normal.

linux links demanding programs to lots of disk cache in memory which is good.

That cache (over 1gig) is phyisical RAM
its used to reduce Hadr disk reads/writes/fragmentation.

since it is cache though it will quickly go away for other programs to use, hense it is actually really free to use.

that is why you have one free 20MB with
free Cache -+ 1GB

you really have 1GB+20MB free RAM if you need it.




think of it like
Low priority for software to use the CPU.

low means a program can only use the CPU when nothing else is.

DiskCache can use the RAM only if nothing else needs it.

i love the Disk Cache Quirk
i paid a lot of money for 4GB ram and Windows uses 512MB and insists on using 200MB pagefile.

Thanks carl0ski, but I still don't understand.

I realize Linux is using all my physical RAM as cache, hence just a few megs free physical memory. I am not worried about that. What worries me is that java is reported to use 1.5 Gigabytes of RAM! A single application, that takes more RAM than physical + swap that I have... :-(

ok the thing is to understand what is virtual memory.
it doesn't have anything to do with physical anything accept proccessor bandwidth.
all 32 bit processors have 4 gigs of virtual memory
that is 2 to the 32nd power distinct values. If you allow each value to address a different byte of memory, you get 2 to the 32nd power bytes, which equals four gigabytes.
so on a 32 bit processor all things get 4 gigs virtual address space.
this is total how many unique values can be in address space that then can point or not sometimes and not to real places in memory or not on a disk or wherever some puter info might be living.

I really don't understand, I guess.

Please tell me one thing: based on the data I quoted above, is java using 1.5 gigabytes of memory (physical+swap), or not?

If not, then please tell me this: how can I tell how much physical and swap memory java (or any other process) is using? I simply want to know how much physical memory is used by it, and how much of my swap file is used by it.

Thanks in advance.

my java does it too,

in my case i have azureus leaching Suse

but that is very Hard drive intensive and data going out is frequently tthe same so it is remembered.

it is cleared if i need it.

well it's extreemly difficult to tell what physical swap is being used. all swap above is total virtual minus RSS. not a representation of actual swap use.
But the number you should be looking at is RES or "resident set size". That's basically stuff in actual physical memory or RESIDENT in physical memory. That would be 203m for your java process.

So, what you are saying, foo_bar_foo, is that there is NO WAY to know how much of my swap file a process is using?

If that is true, then what is the meaning of the SWAP field appearing in top? It's just garbage?

if you use KDE
open Kcontrol

Configure you desktop.

there will be and information tab>
memory

will give you a graphical display to give you a clear veiw of whats going on.

orange is Disk Cache.



P.S i just went back to Mandriva 32bit, it wouldnt detect more than 800MB ram :S

By the way, java is using the amount of memory it is allowed to.

Look at the script launching your java application for the -Xms and -Xmx options, and you'll find the initial and maximal amount of virtual memory set.

Like all other userland programs, java only knows about virtual memory, only the kernel deals with physical RAM.

please try following commands:

# pmap [pid of that java process]
# cat /proc/[pid]/maps

maybe we can see in detail where are all those 1.5G used. maybe just some big shared libraries that really do nothing but sitting there.

Hmm, thanks, pczou, I didn't know of those commands (well cat I of course knew, but not to do that...).

Ok, I did it, and it's very very long. So I won't post it all. But the interesting part is this:
As for cat /proc, it's also too long to post here. Also, I couldn't find a line(s) that correspond directly to the above (but this seems the closest:
2aaaae2c3000-2aaaedc93000 rwxp 2aaaae2c3000 00:00 0 - seems useless to me, though, but I don't know).

The [ anon ] means that there isn't a name corresponding to the memory chunk, I see in the documentation. So, is there anything else I can do to find out what is taking a gigabyte of RAM?

Also, I ask again: if this process is indeed of size 1.5 gigabytes, where does it reside? My physical+swap isn't big enough for it. I must not understand something fundamental here.

memory allocation pool of java can be limited by option -Xmx, e.g. 'java -Xms16m -Xmx128m'.

the 1.5G memory is just the virtual memory space taken by the process, physical memory is allocated only on demand. for example, if a large chunk of memory ls malloc'd or large file is mmap'd, the virtual memory grows but actually no physical memory is alloced (yet). so the 'RES' field of top output shows the resident memory really consumed by the java process, no need to worry too much about the 'VIRT' field.

The funny thing is, java takes 1.6 gigabytes (it rose since my original post) even though I ran it with -Xmx64m. And Knotify now uses 619 mb. Something just seems 'wrong' with all of this. Ah well...

So Linux has no problem allocating more ram than I have physical+swap? It does this and hopes that not all of it will actually be used?