LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
LinkBack Search this Thread
Old 10-30-2006, 08:19 PM   #1
Rayven
LQ Newbie
 
Registered: Oct 2006
Distribution: Red Hat 4, Ubuntu
Posts: 21

Rep: Reputation: 15
Question Does free( ) immediatly free the memory?


I have been using Valgrind on some C code and have gotten all memory leaks patched (except one here that has not been answered yet that still plaques me,HINT HINT please respond here if you have an idea!). I am running this on both Red Hat AS and Solaris 8. When running on Red Hat, the memory usage will slowly climb until the system crashes. On Solaris however, the memory will slowly climb, and then quickly return back to the original amount. I know that there used to be a malloc library on Solaris that when linked to, immediately free'd the memory when free( ) was called, else the "standard" version free'd the memory, but kept it in a reserve until a later time. Is this the case with Linux/Red Hat? Are there any commands to force free( ) to allow access to all the memory? The application in question is a background daemon process that processes files and does not shut down.

Thanks oh so much!
Rayven
 
Old 10-31-2006, 01:03 AM   #2
dalek
Senior Member
 
Registered: Jul 2003
Location: Mississippi USA
Distribution: Gentoo
Posts: 2,058
Blog Entries: 2

Rep: Reputation: 65
First, the command free does not free anything. It just tells what memory is being used and what is not being used. From man free:

Code:
free  displays  the  total amount of free and used physical and swap memory in 
the system, as well as the buffers used by the kernel.  The shared memory column
 should be ignored; it is obsolete.
Maybe this link will help you a little:

http://gentoo-wiki.com/FAQ_Linux_Memory_Management

That one has some Gentoo specifics but a lot of it is the same for Linux in general. This is a distro independant I think:

http://www.linuxhq.com/guides/TLK/mm/memory.html

Linux handles memory a lot different than windoze.

Hope that helps.

 
Old 10-31-2006, 07:12 AM   #3
Rayven
LQ Newbie
 
Registered: Oct 2006
Distribution: Red Hat 4, Ubuntu
Posts: 21

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by dalek
First, the command free does not free anything. It just tells what memory is being used and what is not being used. From man free:

Code:
free  displays  the  total amount of free and used physical and swap memory in 
the system, as well as the buffers used by the kernel.  The shared memory column
 should be ignored; it is obsolete.
Maybe this link will help you a little:

http://gentoo-wiki.com/FAQ_Linux_Memory_Management

That one has some Gentoo specifics but a lot of it is the same for Linux in general. This is a distro independant I think:

http://www.linuxhq.com/guides/TLK/mm/memory.html

Linux handles memory a lot different than windoze.

Hope that helps.

This was not a question of the command line free call, but to the 'C' routine free( ) (as in malloc( ) and free( ) for memory usage. I apologize if I was unclear about that. This also does not have to work on windoze, but just Red Hat and Solaris.
 
Old 10-31-2006, 08:07 AM   #4
dalek
Senior Member
 
Registered: Jul 2003
Location: Mississippi USA
Distribution: Gentoo
Posts: 2,058
Blog Entries: 2

Rep: Reputation: 65
No apologies needed. I was going by your post count and thought you were new to Linux in general but at the same time your post made it sound like you were familiar with Linux. I wasn't really sure so I tried to include information that would then get additional info if you still had questions. Most people coming from windoze systems freak out when it says most all of the ram is in use. They don't realize that Linux caches as much as possible. That was the reason for the windoze comment.

Maybe someone with knowledge of 'C' routines will shed more light.

 
Old 11-01-2006, 02:30 AM   #5
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,459

Rep: Reputation: 352Reputation: 352Reputation: 352Reputation: 352
Quote:
Originally Posted by Rayven
I am running this on both Red Hat AS and Solaris 8. When running on Red Hat, the memory usage will slowly climb until the system crashes. On Solaris however, the memory will slowly climb, and then quickly return back to the original amount.
How are you measuring the memory usage on both systems ?
 
Old 11-01-2006, 01:13 PM   #6
Mara
Moderator
 
Registered: Feb 2002
Location: Grenoble
Distribution: Debian
Posts: 9,527

Rep: Reputation: 147Reputation: 147
free() call usually frees the memory just when it's called. System crash shouldn't happen in such case, if free is really called. Caches may be in use in your case (there are many different factors here), but should not lead to a crash, definitelly.

I wouldn't blame Linux at this point. You may have a subtle bug in your code that causes such behaviour (valgrind is good, but it can't find all the possible problems).

Make a small test. Run the program with memory limit per process (using ulimit). Check if it reaches the limit and/or if it crashes after that.
 
Old 11-01-2006, 02:49 PM   #7
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,459

Rep: Reputation: 352Reputation: 352Reputation: 352Reputation: 352
Actually freed memory is returned to the system only under certain conditions.

Traditional malloc libraries only alloc on the heap and never use brk to lower it. With them, allocated memory is never returned to the O/S. This is documented in the Solaris free manual page.

On linux, glibc malloc use either the heap or mmap, depending on the size of the requested size. Memory is guaranteed to be free only with mmap. With heap allocation, this can happen only if there is no allocated areas live between the just freed one and the heap limit.

Finally, don't forget all this is about virtual memory. Physical usage is mostly unrelated.
 
Old 11-01-2006, 07:37 PM   #8
orgcandman
Member
 
Registered: May 2002
Location: dracut MA
Distribution: Ubuntu; PNE-LE; LFS (no book)
Posts: 594

Rep: Reputation: 102Reputation: 102
Sorry to go slightly offtopic, but

Quote:
On linux, glibc malloc use either the heap or mmap, depending on the size of the requested size. Memory is guaranteed to be free only with mmap. With heap allocation, this can happen only if there is no allocated areas live between the just freed one and the heap limit.
I'm not sure what you mean by this. Lets use an example. I have processes A, B, C, and D. Each allocates some chunks of heap memory. After some time A, and C finish execution, call their free()s and exit. Say then, that A allocated(and released) 1gb of memory, B currently holds 1.2mb of memory, C allocated 13gb of memory, and D holds 10 bytes of heap. My interpretation of your statement is that some 14gb of memory will be unavailable until B and D finish executing. Is that right?

Is that all within the context of a single process? Multiple processes? It seems to me that your statement needs some clarification.
 
Old 11-02-2006, 12:40 AM   #9
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,459

Rep: Reputation: 352Reputation: 352Reputation: 352Reputation: 352
You got this wrong, sorry for having been not clear enough.

Malloc and free span is limited to a process, so allocations make by A have no relationship with allocations make by B, C or D. They are using unrelated virtual memory addresses (what my last sentence was telling)

My statement is valid if a single process is doing these 4 allocations though.
 
Old 11-02-2006, 10:40 AM   #10
Rayven
LQ Newbie
 
Registered: Oct 2006
Distribution: Red Hat 4, Ubuntu
Posts: 21

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by jlliagre
How are you measuring the memory usage on both systems ?
On the RHEL systems, I am using "top". On Solaris I am using vmstat.

Quote:
Originally Posted by jlliagre
Actually freed memory is returned to the system only under certain conditions.

Traditional malloc libraries only alloc on the heap and never use brk to lower it. With them, allocated memory is never returned to the O/S. This is documented in the Solaris free manual page.

On linux, glibc malloc use either the heap or mmap, depending on the size of the requested size. Memory is guaranteed to be free only with mmap. With heap allocation, this can happen only if there is no allocated areas live between the just freed one and the heap limit.

Finally, don't forget all this is about virtual memory. Physical usage is mostly unrelated.
So should I be using mmap instead of malloc? Will munmap then completely free the memory instead of just returning it into the malloc pool?
 
Old 11-02-2006, 12:49 PM   #11
Hko
Senior Member
 
Registered: Aug 2002
Location: Groningen, The Netherlands
Distribution: ubuntu
Posts: 2,530

Rep: Reputation: 108Reputation: 108
Quote:
Originally Posted by Rayven
So should I be using mmap instead of malloc? Will munmap then completely free the memory instead of just returning it into the malloc pool?
Just use malloc(). The mmap() stuff jlliagre was talking about is about the internal implementation of malloc() inside glibc. Whether mmap() is used behind the scenes is of no concern to you as an application programmer, except when wondering where free()d memory will go...

Last edited by Hko; 11-02-2006 at 12:52 PM.
 
Old 11-02-2006, 01:14 PM   #12
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,459

Rep: Reputation: 352Reputation: 352Reputation: 352Reputation: 352
Quote:
Originally Posted by Rayven
On the RHEL systems, I am using "top". On Solaris I am using vmstat.
And what column(s) are you watching on both of them ?
Quote:
So should I be using mmap instead of malloc? Will munmap then completely free the memory instead of just returning it into the malloc pool?
You may want to use mmap with Solaris if you wish to release memory to the system, as neither glibc nor mmap based allocation libraries are available (as far as I know, I didn't investigate).
As Hko stated, there is no interest building your own allocation library under Gnu/Linux, as malloc does already it for you.
 
Old 11-02-2006, 01:57 PM   #13
Rayven
LQ Newbie
 
Registered: Oct 2006
Distribution: Red Hat 4, Ubuntu
Posts: 21

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by jlliagre
And what column(s) are you watching on both of them ?
Yes, I am watching both. The swap space is 8GB free, 2GB swap which I usually restart the computer before swap is touched.

Quick valgrind question, pertaining to this topic: if I am receiving multiple messages about "Invalid Read Size of 8" and "Use of Unitialized Value of size 8", would they make the memory leaks? I thought that these two warnings did not have much to do with memory leaks but just warnings of possible "bad" things. http://valgrind.org/docs/manual/mc-m...nual.errormsgs, maybe I read the documentation incorrectly.
Thanks again!
 
Old 11-02-2006, 02:56 PM   #14
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,459

Rep: Reputation: 352Reputation: 352Reputation: 352Reputation: 352
Quote:
Originally Posted by Rayven
Yes, I am watching both. The swap space is 8GB free, 2GB swap which I usually restart the computer before swap is touched.
You misread my question.
I'm asking what values are you looking at in the vmstat and the top output.
Quote:
Quick valgrind question, pertaining to this topic: if I am receiving multiple messages about "Invalid Read Size of 8" and "Use of Unitialized Value of size 8", would they make the memory leaks?
No.
Quote:
I thought that these two warnings did not have much to do with memory leaks but just warnings of possible "bad" things.
Real bad things actually, these are bugs that should be fixed.
 
Old 11-02-2006, 03:22 PM   #15
Mara
Moderator
 
Registered: Feb 2002
Location: Grenoble
Distribution: Debian
Posts: 9,527

Rep: Reputation: 147Reputation: 147
Quote:
Originally Posted by Rayven
Quick valgrind question, pertaining to this topic: if I am receiving multiple messages about "Invalid Read Size of 8" and "Use of Unitialized Value of size 8", would they make the memory leaks?
Not leaks, but it means you access memeory outside of your allocated space. In short: you're probably reading garbage. When used in certain ways, it may lead to crashes. Look into it.
 
  


Reply

Tags
free, malloc


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Clear Free Memory lowlifeish Linux - General 3 12-12-2005 12:12 PM
How much memory is really 'free'? robbiemorgan Linux - Newbie 4 09-19-2004 07:05 PM
Free, free,FREE=? no money, = Freedom? murshed Linux - Newbie 8 01-20-2003 07:01 AM
Memory free problem edreddy Programming 1 11-09-2002 10:59 PM


All times are GMT -5. The time now is 11:34 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration