LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 11-17-2006, 11:56 PM   #1
Rayven
LQ Newbie
 
Registered: Oct 2006
Distribution: Red Hat 4, Ubuntu
Posts: 21

Rep: Reputation: 15
Using Valgrind and top for leak checking


I decided to make this a new post since the previous post (Does free( ) immediately free the memory?) has been inactive for a while now. After many hours of debugging a fork'd c program, Valgrind reports that I have 0 memory leaks and 0 memory issues. I have ran the application for over 3days processing over 100 thousand files, and the reports still never showed any leaks. However, when I use the top command to check the free and available memory, it still lists
8097678K Total, 7968xxxK used, 1000-2000K free. Top reports the memory is at the 7GB used mark after the first day, but will never go over the 8GB used mark. The swap space is never used up beyond 160KB, and the application is not taking a speed hit (I have built in speed checking). Is there some issue with running top in reporting the memory, does it report the memory that is in the de-allocation pool as being used? Is there a better utility, other that top, for checking the memory usage?

I am really not sure if I have a memory problem now, I let the program run over this weekend to see if it terminates or crashes the system, but I don't believe it will. Any further suggestions would be greatly appreciated.

Thanks in advance!
 
Old 11-19-2006, 01:31 PM   #2
Mara
Moderator
 
Registered: Feb 2002
Location: Grenoble
Distribution: Debian
Posts: 9,536

Rep: Reputation: 148Reputation: 148
Look into /proc/meminfo and /proc/vmstat. They provide much more detailed info. In fact, short form of that info you can get after running 'free' command. My guess is that you have huge amount in the 'buffers' and 'cached' column, what means that the systems keeps that memory.
 
Old 11-20-2006, 09:26 AM   #3
Rayven
LQ Newbie
 
Registered: Oct 2006
Distribution: Red Hat 4, Ubuntu
Posts: 21

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by Mara
Look into /proc/meminfo and /proc/vmstat. They provide much more detailed info. In fact, short form of that info you can get after running 'free' command. My guess is that you have huge amount in the 'buffers' and 'cached' column, what means that the systems keeps that memory.
You are correct, the buffered and cached memory keeps increasing. Does this effect the performance of the system? As I understood, Red Hat keeps the memory buffers for quicker access on sucessive mallocs. Is there a way to periodically release this memory from the buffers/cache? If not, will using realloc instead of malloc help?

Thanks
 
Old 11-20-2006, 12:44 PM   #4
Quigi
Member
 
Registered: Mar 2003
Location: Cambridge, MA, USA
Distribution: Ubuntu (Dapper and Heron)
Posts: 377

Rep: Reputation: 31
Ravyen, are you observing any problem? What you report sounds like everything is working right. Maybe you make an implicit assumption about memory management and your observations don't confirm it?

Unix processes traditionally never shrink. When you allocate memory (e.g., malloc), eventually memory management will need to call brk(2) and increase the end of the data segment. This increases the "used memory" visible from outside, e.g., top. When you later free() memory, your process keeps it for subsequent allocation. The size of your process does not decrease. At least this has been the Unix model for decades -- if I'm outdated, please someone correct me! There are some ways around this (if you need to make some of the space available for other processes), but I haven't used any.

Don't worry about buffer/cache. Linux puts unused memory to use in buffers and caches, to give you better performance. The kernel will reclaim it as needed. The credo is "Free memory is wasted memory". I don't think you'd want to release this memory from the buffers/cache.
 
Old 11-20-2006, 04:43 PM   #5
Rayven
LQ Newbie
 
Registered: Oct 2006
Distribution: Red Hat 4, Ubuntu
Posts: 21

Original Poster
Rep: Reputation: 15
Thumbs up

Quote:
Originally Posted by Quigi
Ravyen, are you observing any problem? What you report sounds like everything is working right. Maybe you make an implicit assumption about memory management and your observations don't confirm it?

Unix processes traditionally never shrink. When you allocate memory (e.g., malloc), eventually memory management will need to call brk(2) and increase the end of the data segment. This increases the "used memory" visible from outside, e.g., top. When you later free() memory, your process keeps it for subsequent allocation. The size of your process does not decrease. At least this has been the Unix model for decades -- if I'm outdated, please someone correct me! There are some ways around this (if you need to make some of the space available for other processes), but I haven't used any.

Don't worry about buffer/cache. Linux puts unused memory to use in buffers and caches, to give you better performance. The kernel will reclaim it as needed. The credo is "Free memory is wasted memory". I don't think you'd want to release this memory from the buffers/cache.
No, no problem is observed, I was just assuming that when you free'd the memory, that the OS would then reduce the number that was reported as being "used". As long as Linux will know that other programs will need to utilize this memory in the future, I am ok with things! Thanks for the great response!!
 
Old 11-20-2006, 09:26 PM   #6
Quigi
Member
 
Registered: Mar 2003
Location: Cambridge, MA, USA
Distribution: Ubuntu (Dapper and Heron)
Posts: 377

Rep: Reputation: 31
Quote:
Originally Posted by Rayven
I was just assuming that when you free'd the memory, that the OS would then reduce the number that was reported as being "used". As long as Linux will know that other programs will need to utilize this memory in the future, I am ok with things!
Normally, free'd memory is only available within the process. Other programs won't be able to utilize it until the process exits. But this doesn't have directly to do with the small amount of free memory -- it's typical to see a small figure here. When a big process exits, you can see that free memory temporarily goes up correspondingly, but slowly it gets put to "better use" in caches and buffers.

That said, there appear to be mechanisms by which a process can give memory back to the OS -- I've seen emacs's total size go down after killing a big buffer. Not sure how that's accomplished.
 
Old 11-20-2006, 11:02 PM   #7
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,499

Rep: Reputation: 355Reputation: 355Reputation: 355Reputation: 355
Quote:
Originally Posted by Quigi
Normally, free'd memory is only available within the process. Other programs won't be able to utilize it until the process exits. But this doesn't have directly to do with the small amount of free memory -- it's typical to see a small figure here. When a big process exits, you can see that free memory temporarily goes up correspondingly, but slowly it gets put to "better use" in caches and buffers.

That said, there appear to be mechanisms by which a process can give memory back to the OS -- I've seen emacs's total size go down after killing a big buffer. Not sure how that's accomplished.
This has already been discussed and explained in the afore mentioned previous thread.
 
  


Reply

Tags
memory, top, valgrind


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Valgrind - an answer. cyent Programming 6 02-11-2010 01:49 PM
Valgrind reports memory leak at opendir Rayven Programming 11 11-02-2006 02:03 PM
Any memory leak checking tool? George2 Programming 2 06-21-2006 10:26 AM
valgrind with C++ yashwantpinge Programming 2 02-16-2006 03:56 PM
Valgrind 2.2.0 works sridurai Linux - General 0 02-18-2005 05:51 AM


All times are GMT -5. The time now is 04:41 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration