LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 02-08-2006, 02:18 PM   #1
xinminhua
LQ Newbie
 
Registered: Feb 2006
Posts: 5

Rep: Reputation: 0
free() does not clear memory space in LINUX


When porting C codes from SGI platform to LINUX, we often had core dumps due to memory problems. In SGI machine, when a pointer is allocated certain memory, one only needs to free it when it is no longer useful.

However, the same codes sometimes will core dump in LINUX. We found that in order to fix this problem, one needs to set pointer to NULL after calling free. For example:

int *p;
p = (int *) malloc(n);
....
free(p);
p = NULL;

My question is: Why in LINUX, free() is not enough to clear the memory?
Is our fix by setting pointers to NULL a correct way of fixing this kind problem?
Shall LINUX be updated to clear memory when calling free()?

Thanks.
 
Old 02-08-2006, 03:00 PM   #2
rstewart
Member
 
Registered: Feb 2005
Location: Sunnyvale, CA
Distribution: Ubuntu
Posts: 205

Rep: Reputation: 38
Specifically what behavior leads you to believe that free() isn't deallocating previously malloced memory? You mentioned that sometimes you got core dumps. There are multiple reasons for a program to core dump. What reason was given? Is/was your program trying to use the pointer for some other reason after you called free()?
 
Old 02-08-2006, 08:06 PM   #3
dogpatch
Member
 
Registered: Nov 2005
Location: Central America
Distribution: Mepis
Posts: 162
Blog Entries: 1

Rep: Reputation: 18
I have written programs in Linux that make extensive use of memory allocation and deallocation calls. I don't set the pointer to NULL after calling free(), and have never had a problem.

I, too, suspect that something else is causing the core dumps, unless you can show more specifically how you know that the memory blocks are not in fact being deallocated by the free() call. Can you supply more specifics?
 
Old 02-09-2006, 01:32 AM   #4
aluser
Member
 
Registered: Mar 2004
Location: Massachusetts
Distribution: Debian
Posts: 557

Rep: Reputation: 42
what does "clear the memory" mean? rstewart and dogpatch seem to assume it means "deallocate", i.e. add it to the free list. But even forgetting to free() a pointer doesn't directly cause a core dump, just a memory leak. Is your program using lots of memory when it shouldn't?

Assigning a value (NULL or anything else) to a pointer has no effect on the memory to which it points. (er, unless it points to itself..)
 
Old 02-09-2006, 08:38 AM   #5
xinminhua
LQ Newbie
 
Registered: Feb 2006
Posts: 5

Original Poster
Rep: Reputation: 0
free() does not clear memory space in LINUX

Thanks to all who replied.

Q: Specifically what behavior leads you to believe that free() isn't deallocating previously malloced memory?
Q: I, too, suspect that something else is causing the core dumps, unless you can show more specifically how you know that the memory blocks are not in fact being deallocated by the free() call. Can you supply more specifics?

A: When core dump happened, we used gdb to debug the codes and found that pointer p is not empty after calling free(p). For example, before calling free(p), *p = "ABCD", after free(p), *p does not equal "", but contains some garbage. Then, if we added p =NULL, program no longer core dumps.

Q: Was your program trying to use the pointer for some other reason after you called free()?
A: No. The same program runs on SGI machine quite well.
 
Old 02-09-2006, 09:28 AM   #6
jim mcnamara
Member
 
Registered: May 2002
Posts: 964

Rep: Reputation: 34
Without going thru how malloc works, Linux uses a version of Doug Lea's malloc program. When it allocates memory it uses two longs, and they muist live right next to each other for free to work. The first long is the length of the memory block allocated, the second is a pointer to the start of the block. The second one is the pointer that malloc returns.

If you inadvertantly swap the second one (the pointer ) with another pointer - ie., one that does not live nextdoor to the correct length value, free corrupts the blocks of heap memory that malloc handles. This causes all sorts of problems - free segfaulting, heap memory being incorrectly overwriteen, other segfaults.

This is what you have. Some other unix systems base memory allocation on where the pointer is "aimed" not where the pointer "lives in memory".
 
Old 02-09-2006, 10:45 AM   #7
rstewart
Member
 
Registered: Feb 2005
Location: Sunnyvale, CA
Distribution: Ubuntu
Posts: 205

Rep: Reputation: 38
Quote:
Thanks to all who replied.

Q: Specifically what behavior leads you to believe that free() isn't deallocating previously malloced memory?
Q: I, too, suspect that something else is causing the core dumps, unless you can show more specifically how you know that the memory blocks are not in fact being deallocated by the free() call. Can you supply more specifics?

A: When core dump happened, we used gdb to debug the codes and found that pointer p is not empty after calling free(p). For example, before calling free(p), *p = "ABCD", after free(p), *p does not equal "", but contains some garbage. Then, if we added p =NULL, program no longer core dumps.

Q: Was your program trying to use the pointer for some other reason after you called free()?
A: No. The same program runs on SGI machine quite well.
"*p = "ABCD", after free(p), *p does not equal "", but contains some garbage". That is absolutely correct, it is the way it is supposed to work. Calling free() on all of the 'nix machines I have ever used - Linux, UNIX series III, UNIX system V, Xenix, BSD, Sunos, and Solaris do not guarantee what the contents of the pointer will be after the free occurs. It may contain NULL, or it may contain an invalid pointer - it is implementation dependent. If your application code does not correctly handle this, then it may core dump after the free. As a very simple test, on your SGI machine, perform a malloc, do something with the malloced memory, and then call free() to free it up. After the free, print out the value of the pointer.

Something along the lines of:

Code:
void MyFunction( void )
{
    unsigned char *p;

    p = (unsigned char *)malloc( 256 );
    if ( !p )
    {
        // Error
        printf( "Insufficient memory needed to run application!\n" );
        return;
    }
    printf( "p: %08X\n", p );
    memcpy( (void *)p, "This is a test", 14 );
    free( p );
    printf( "p: %08X\n", p );
}
I suspect that the SGI implementation may zero out the pointer automatically for you. Also in your application, you said that it core dumps. Where in the code did it core dump? Right after the call to free()? Inside the call to free()? Or some time later in the program? If some time later, how do you know that it was failing due to the previously freed memory?

Please can you provide additional information - we are trying to help you understand what is going on (and ourselves at the same time).
 
Old 02-09-2006, 11:35 AM   #8
aluser
Member
 
Registered: Mar 2004
Location: Massachusetts
Distribution: Debian
Posts: 557

Rep: Reputation: 42
Quote:
Calling free() on all of the 'nix machines I have ever used - Linux, UNIX series III, UNIX system V, Xenix, BSD, Sunos, and Solaris do not guarantee what the contents of the pointer will be after the free occurs. It may contain NULL, or it may contain an invalid pointer - it is implementation dependent.
It sounds like you're saying free() can change the value of the pointer passed to it, rather than the contents of the memory to which the pointer points. This isn't true; look at the prototype for free(). It takes a pointer argument, and arguments are passed by value in C. The pointer will *always* point to the same memory location after free() as before it. What happens to exist at that location might or might not change with free(); the program isn't supposed to depend on it. My bad if I completely misunderstood the quote.

A thought about the OP's problem: some (all?) free() implementations ignore a NULL argument. Perhaps you're double free()ing your pointer, which is solved (somewhat) by setting it to NULL before attempting to free it the second time. If this is the case, it implies that you may be reading or writing memory after it's free()d, which could very well happen to work on one architecture and not another. Different malloc implementations will handle a double free differently; some actually generate warnings about it, others could crash like you're seeing.
 
Old 02-09-2006, 12:15 PM   #9
rstewart
Member
 
Registered: Feb 2005
Location: Sunnyvale, CA
Distribution: Ubuntu
Posts: 205

Rep: Reputation: 38
Quote:
Quote:
Calling free() on all of the 'nix machines I have ever used - Linux, UNIX series III, UNIX system V, Xenix, BSD, Sunos, and Solaris do not guarantee what the contents of the pointer will be after the free occurs. It may contain NULL, or it may contain an invalid pointer - it is implementation dependent.

It sounds like you're saying free() can change the value of the pointer passed to it, rather than the contents of the memory to which the pointer points.
Yes, that is what I am trying to say. It is implementation dependent what happens to the actual memory location that contains the pointer's address. Eg: "p" is a variable that references a memory address. "*p" is also a variable it references the memory address specified in "p". Some implementations may set "p" to NULL after the call to free(). All of the 'nix's I have used leave "p" alone. However referencing "*p" after free() will generate a fault as you are then trying to access memory that no longer is mapped to your process address space. You are still free to use "p" all you want, you can assign it NULL, or you can leave it alone and do nothing with it. BTW: I have used some non-'nix systems (VMS, and a few realtime OSes) that do set "p" to NULL as part of the call to free() as a way for the application to test if "p" is a pointer pointing to accessable memory or not. Maybe the SGI system is one of those systems that change "p" to NULL as part of the free() call?

Quote:
This isn't true; look at the prototype for free(). It takes a pointer argument, and arguments are passed by value in C. The pointer will *always* point to the same memory location after free() as before it. What happens to exist at that location might or might not change with free(); the program isn't supposed to depend on it. My bad if I completely misunderstood the quote.
Yes but, it is implementation dependent what the pointer will contain after the call to free(). The Linux implementation you are referencing does not change the pointer, I agree.

Quote:
A thought about the OP's problem: some (all?) free() implementations ignore a NULL argument. Perhaps you're double free()ing your pointer, which is solved (somewhat) by setting it to NULL before attempting to free it the second time. If this is the case, it implies that you may be reading or writing memory after it's free()d, which could very well happen to work on one architecture and not another. Different malloc implementations will handle a double free differently; some actually generate warnings about it, others could crash like you're seeing.
Exactly! That is the line that I was trying to explore. Either double freeing, or trying to use the pointer as if it was pointing to something valid.
 
Old 02-09-2006, 12:26 PM   #10
aluser
Member
 
Registered: Mar 2004
Location: Massachusetts
Distribution: Debian
Posts: 557

Rep: Reputation: 42
You're saying free() is sometimes a macro which modifies its argument?
 
Old 02-09-2006, 12:28 PM   #11
aluser
Member
 
Registered: Mar 2004
Location: Massachusetts
Distribution: Debian
Posts: 557

Rep: Reputation: 42
Does that mean that this is not always valid code?

Code:
char *p;
p = malloc(4);
free(p + 1 - 1);
 
Old 02-09-2006, 12:52 PM   #12
rstewart
Member
 
Registered: Feb 2005
Location: Sunnyvale, CA
Distribution: Ubuntu
Posts: 205

Rep: Reputation: 38
Quote:
You're saying free() is sometimes a macro which modifies its argument?
No. To an application it is a runtime library function. Its specific behavior is implementation dependent. Some runtime libraries have it modifying its argument, current Linux implementations do not.

Example:

Code:
char *p;                   // "p" points to "something" - implementation
                           // dependent it may be a nonsense value or it
                           // may be NULL

printf( "P;  %08X\n" p );  // This will tell implementation initialization

p = (char *)malloc( 256 ); // After assignment "p" will point to a 256 byte
                           // memory block located in accessable virtual
                           // addressing space

printf( "P:  %08X\n", p ); // This will print out address of virtual memory
                           // block just allocated

free( p );                 //   <<<<<  OR >>>>>  (use only one)
//free( p + 1 - 1 );       // Both return 256 byte memory block back
                           // to free pool

printf( "P: %08X\n", p );  // This will tell if implementation does anything
                           // to value of "p" or not.  Some implementations
                           // print out same value as what was returned by
                           // call to malloc, some print out 00000000
 
Old 02-09-2006, 01:07 PM   #13
aluser
Member
 
Registered: Mar 2004
Location: Massachusetts
Distribution: Debian
Posts: 557

Rep: Reputation: 42
how do you write a C function which modifies its argument?
 
Old 02-09-2006, 01:21 PM   #14
rstewart
Member
 
Registered: Feb 2005
Location: Sunnyvale, CA
Distribution: Ubuntu
Posts: 205

Rep: Reputation: 38
Quote:
how do you write a C function which modifies its argument?
Obviously you don't. The compiler would need to treat the free function as a special case and generate the second level of indirection automatically, or alternatively as an inline code (or macro) wrapper function.

Again, the point that I am trying to make is that it is all implementation dependent.
 
Old 02-09-2006, 03:49 PM   #15
xinminhua
LQ Newbie
 
Registered: Feb 2006
Posts: 5

Original Poster
Rep: Reputation: 0
free() does not clear memory space in LINUX

This is to reply to rstewart:

Quote:
"*p = "ABCD", after free(p), *p does not equal "", but contains some garbage". That is absolutely correct, it is the way it is supposed to work. Calling free() on all of the 'nix machines I have ever used - Linux, UNIX series III, UNIX system V, Xenix, BSD, Sunos, and Solaris do not guarantee what the contents of the pointer will be after the free occurs. It may contain NULL, or it may contain an invalid pointer - it is implementation dependent.
If your application code does not correctly handle this, then it may core dump after the free. As a very simple test, on your SGI machine, perform a malloc, do something with the malloced memory, and then call free() to free it up. After the free, print out the value of the pointer.

My reply:
You are right. The core dump did not happen when you call free(), but at some point later totally innocente (like exit from a function). We were not able to determine the exact cause of core dump. But we found setting freed pointers to NULL will remove the core dumps. This happened several times, so we have the experience in proting to LINUX: if program core dumps with no obvious reason, set some freed pointers to NULL. So far it works. But we want to understand it.

Thanks again.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Clear Free Memory lowlifeish Linux - General 3 12-12-2005 12:12 PM
How do I clear the memory cache on my linux server? robgo777 Linux - Software 2 01-09-2005 01:30 AM
Not enough free space on hard drive with 50g of free space??? auoq Suse/Novell 5 10-13-2004 08:21 PM
Formating free space: WinXP pro and RH9 dualboot with free space on 3rd drive Vermicious Linux - General 2 03-22-2004 05:10 AM
Clear out Linux Mandrake disk space? aaronlc Linux - Software 5 02-18-2003 07:09 PM


All times are GMT -5. The time now is 06:11 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration