LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 01-26-2006, 04:37 AM   #1
kripkenstein
LQ Newbie
 
Registered: Jan 2006
Posts: 10

Rep: Reputation: 0
Question Memory reporting corrupt?


First, apologies - I posted about this before in the SuSE board, but perhaps it's not a SuSE issue. And it really troubles me.

My system (SuSE 10 amd64, newly installed) seems to report inconsistent data for memory usage. This is what I see in top:

Code:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  SWAP COMMAND
11917 food      15   0 1576m 203m  26m S 14.3 41.0  30:48.23 1.3g java
while free gives me

Code:
             total       used       free     shared    buffers     cached
Mem:        508784     503576       5208          0      23588      98040
-/+ buffers/cache:     381948     126836
Swap:       939760     212892     726868
So, java is using 1.5 GIGAbytes of virtual memory? 200 mb in physical, and 1.3 gigabytes swapped out? This can't be, since I am only using 380 + 212 (physical + swap) mb of ram. Java seems to take up more virtual memory than I have physical+virtual memory in my system...

Also, I see similar things with knotify (virtual memory of over a gigabyte). It isn't just java, but is most pronounced there.

I have read the man pages several times, and googled for help on this. My only conclusion after all of this is that something is corrupt in my system's memory reporting?
 
Old 01-26-2006, 10:08 AM   #2
carl0ski
Member
 
Registered: Sep 2004
Location: Melbourne, Victoria Australia
Distribution: Support those that support you :)
Posts: 872
Blog Entries: 12

Rep: Reputation: 30
Quote:
Originally Posted by kripkenstein
First, apologies - I posted about this before in the SuSE board, but perhaps it's not a SuSE issue. And it really troubles me.

My system (SuSE 10 amd64, newly installed) seems to report inconsistent data for memory usage. This is what I see in top:

Code:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  SWAP COMMAND
11917 food      15   0 1576m 203m  26m S 14.3 41.0  30:48.23 1.3g java
while free gives me

Code:
             total       used       free     shared    buffers     cached
Mem:        508784     503576       5208          0      23588      98040
-/+ buffers/cache:     381948     126836
Swap:       939760     212892     726868
So, java is using 1.5 GIGAbytes of virtual memory? 200 mb in physical, and 1.3 gigabytes swapped out? This can't be, since I am only using 380 + 212 (physical + swap) mb of ram. Java seems to take up more virtual memory than I have physical+virtual memory in my system...

Also, I see similar things with knotify (virtual memory of over a gigabyte). It isn't just java, but is most pronounced there.

I have read the man pages several times, and googled for help on this. My only conclusion after all of this is that something is corrupt in my system's memory reporting?

Code:
             total       used       free     shared    buffers     cached
Mem:        508784     503576       5208          0      23588      98040
-/+ buffers/cache:     381948     126836
Swap:       939760     212892     726868
it's a little daunting but its normal.

linux links demanding programs to lots of disk cache in memory which is good.

That cache (over 1gig) is phyisical RAM
its used to reduce Hadr disk reads/writes/fragmentation.

since it is cache though it will quickly go away for other programs to use, hense it is actually really free to use.

that is why you have one free 20MB with
free Cache -+ 1GB

you really have 1GB+20MB free RAM if you need it.




think of it like
Low priority for software to use the CPU.

low means a program can only use the CPU when nothing else is.

DiskCache can use the RAM only if nothing else needs it.
 
Old 01-26-2006, 10:11 AM   #3
carl0ski
Member
 
Registered: Sep 2004
Location: Melbourne, Victoria Australia
Distribution: Support those that support you :)
Posts: 872
Blog Entries: 12

Rep: Reputation: 30
i love the Disk Cache Quirk
i paid a lot of money for 4GB ram and Windows uses 512MB and insists on using 200MB pagefile.
 
Old 01-26-2006, 12:53 PM   #4
kripkenstein
LQ Newbie
 
Registered: Jan 2006
Posts: 10

Original Poster
Rep: Reputation: 0
Thanks carl0ski, but I still don't understand.

I realize Linux is using all my physical RAM as cache, hence just a few megs free physical memory. I am not worried about that. What worries me is that java is reported to use 1.5 Gigabytes of RAM! A single application, that takes more RAM than physical + swap that I have... :-(
 
Old 01-26-2006, 03:50 PM   #5
foo_bar_foo
Senior Member
 
Registered: Jun 2004
Posts: 2,553

Rep: Reputation: 51
ok the thing is to understand what is virtual memory.
it doesn't have anything to do with physical anything accept proccessor bandwidth.
all 32 bit processors have 4 gigs of virtual memory
that is 2 to the 32nd power distinct values. If you allow each value to address a different byte of memory, you get 2 to the 32nd power bytes, which equals four gigabytes.
so on a 32 bit processor all things get 4 gigs virtual address space.
this is total how many unique values can be in address space that then can point or not sometimes and not to real places in memory or not on a disk or wherever some puter info might be living.
 
Old 01-26-2006, 04:08 PM   #6
kripkenstein
LQ Newbie
 
Registered: Jan 2006
Posts: 10

Original Poster
Rep: Reputation: 0
I really don't understand, I guess.

Please tell me one thing: based on the data I quoted above, is java using 1.5 gigabytes of memory (physical+swap), or not?

If not, then please tell me this: how can I tell how much physical and swap memory java (or any other process) is using? I simply want to know how much physical memory is used by it, and how much of my swap file is used by it.

Thanks in advance.
 
Old 01-26-2006, 07:01 PM   #7
carl0ski
Member
 
Registered: Sep 2004
Location: Melbourne, Victoria Australia
Distribution: Support those that support you :)
Posts: 872
Blog Entries: 12

Rep: Reputation: 30
Quote:
Originally Posted by kripkenstein
Thanks carl0ski, but I still don't understand.

I realize Linux is using all my physical RAM as cache, hence just a few megs free physical memory. I am not worried about that. What worries me is that java is reported to use 1.5 Gigabytes of RAM! A single application, that takes more RAM than physical + swap that I have... :-(
my java does it too,

in my case i have azureus leaching Suse

but that is very Hard drive intensive and data going out is frequently tthe same so it is remembered.

it is cleared if i need it.
 
Old 01-26-2006, 10:45 PM   #8
foo_bar_foo
Senior Member
 
Registered: Jun 2004
Posts: 2,553

Rep: Reputation: 51
Quote:
Originally Posted by kripkenstein
I really don't understand, I guess.

Please tell me one thing: based on the data I quoted above, is java using 1.5 gigabytes of memory (physical+swap), or not?

If not, then please tell me this: how can I tell how much physical and swap memory java (or any other process) is using? I simply want to know how much physical memory is used by it, and how much of my swap file is used by it.

Thanks in advance.
well it's extreemly difficult to tell what physical swap is being used. all swap above is total virtual minus RSS. not a representation of actual swap use.
But the number you should be looking at is RES or "resident set size". That's basically stuff in actual physical memory or RESIDENT in physical memory. That would be 203m for your java process.
 
Old 01-27-2006, 01:51 AM   #9
kripkenstein
LQ Newbie
 
Registered: Jan 2006
Posts: 10

Original Poster
Rep: Reputation: 0
So, what you are saying, foo_bar_foo, is that there is NO WAY to know how much of my swap file a process is using?

If that is true, then what is the meaning of the SWAP field appearing in top? It's just garbage?
 
Old 01-27-2006, 03:08 AM   #10
carl0ski
Member
 
Registered: Sep 2004
Location: Melbourne, Victoria Australia
Distribution: Support those that support you :)
Posts: 872
Blog Entries: 12

Rep: Reputation: 30
Quote:
Originally Posted by kripkenstein
So, what you are saying, foo_bar_foo, is that there is NO WAY to know how much of my swap file a process is using?

If that is true, then what is the meaning of the SWAP field appearing in top? It's just garbage?
if you use KDE
open Kcontrol

Configure you desktop.

there will be and information tab>
memory

will give you a graphical display to give you a clear veiw of whats going on.

orange is Disk Cache.



P.S i just went back to Mandriva 32bit, it wouldnt detect more than 800MB ram :S
 
Old 01-27-2006, 04:13 AM   #11
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,522

Rep: Reputation: 365Reputation: 365Reputation: 365Reputation: 365
By the way, java is using the amount of memory it is allowed to.

Look at the script launching your java application for the -Xms and -Xmx options, and you'll find the initial and maximal amount of virtual memory set.

Like all other userland programs, java only knows about virtual memory, only the kernel deals with physical RAM.
 
Old 01-27-2006, 04:34 AM   #12
pczou
LQ Newbie
 
Registered: Aug 2005
Posts: 14

Rep: Reputation: 0
Quote:
Originally Posted by kripkenstein
I really don't understand, I guess.

Please tell me one thing: based on the data I quoted above, is java using 1.5 gigabytes of memory (physical+swap), or not?

If not, then please tell me this: how can I tell how much physical and swap memory java (or any other process) is using? I simply want to know how much physical memory is used by it, and how much of my swap file is used by it.

Thanks in advance.
please try following commands:

# pmap [pid of that java process]
# cat /proc/[pid]/maps

maybe we can see in detail where are all those 1.5G used. maybe just some big shared libraries that really do nothing but sitting there.
 
Old 01-27-2006, 05:04 AM   #13
kripkenstein
LQ Newbie
 
Registered: Jan 2006
Posts: 10

Original Poster
Rep: Reputation: 0
Hmm, thanks, pczou, I didn't know of those commands (well cat I of course knew, but not to do that...).

Ok, I did it, and it's very very long. So I won't post it all. But the interesting part is this:
Code:
Address           Kbytes     RSS    Anon  Locked Mode   Mapping
00002aaaae2b3000 1042304       -       -       - rwx--    [ anon ]
As for cat /proc, it's also too long to post here. Also, I couldn't find a line(s) that correspond directly to the above (but this seems the closest:
2aaaae2c3000-2aaaedc93000 rwxp 2aaaae2c3000 00:00 0 - seems useless to me, though, but I don't know).

The [ anon ] means that there isn't a name corresponding to the memory chunk, I see in the documentation. So, is there anything else I can do to find out what is taking a gigabyte of RAM?

Also, I ask again: if this process is indeed of size 1.5 gigabytes, where does it reside? My physical+swap isn't big enough for it. I must not understand something fundamental here.
 
Old 01-27-2006, 06:24 AM   #14
pczou
LQ Newbie
 
Registered: Aug 2005
Posts: 14

Rep: Reputation: 0
memory allocation pool of java can be limited by option -Xmx, e.g. 'java -Xms16m -Xmx128m'.

the 1.5G memory is just the virtual memory space taken by the process, physical memory is allocated only on demand. for example, if a large chunk of memory ls malloc'd or large file is mmap'd, the virtual memory grows but actually no physical memory is alloced (yet). so the 'RES' field of top output shows the resident memory really consumed by the java process, no need to worry too much about the 'VIRT' field.
 
Old 01-27-2006, 10:04 AM   #15
kripkenstein
LQ Newbie
 
Registered: Jan 2006
Posts: 10

Original Poster
Rep: Reputation: 0
The funny thing is, java takes 1.6 gigabytes (it rose since my original post) even though I ran it with -Xmx64m. And Knotify now uses 619 mb. Something just seems 'wrong' with all of this. Ah well...

So Linux has no problem allocating more ram than I have physical+swap? It does this and hopes that not all of it will actually be used?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
df reporting incorrectly bigtl Linux - General 2 06-23-2005 06:47 AM
Reporting bugs in 10.1? RedDwarf Mandriva 1 02-01-2005 11:34 AM
Error reporting in c george_mercury Programming 2 11-20-2004 07:22 PM
Gkrellm and CPU/memory usage reporting dalek Linux - Software 6 06-25-2004 02:42 AM
Help!?! RH 8 Memory Mapping -High Memory-Virtural Memory issues.. Merlin53 Linux - Hardware 2 06-18-2003 05:48 PM


All times are GMT -5. The time now is 01:31 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration