LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 08-25-2006, 03:21 PM   #1
hedpe
Member
 
Registered: Jan 2005
Location: Boston, MA
Distribution: Debian
Posts: 380

Rep: Reputation: 30
c/c++ buffer overrun detection tool?


Hey guys,

I am having problems with a program I have written, where its seg faulting in random places like malloc's or fprintf's. This to me is a signal that I have overrun somewhere earlier in the execution and overwritten a valid pointer or space somewhere else in memory. However, I've spent many hours debugging and trying to find it but I can't.

I was wondering if anyone knows any tools that will monitor the stack and detect when you try to access memory out of bounds? I have used ElectricFence in the past, linking my programs with -lefence, but I keep getting an electric fence error trying to do so with this program:
Code:
lanthanum-ini blinc_code # ./blinc 0 300

ElectricFence: Registering with atexit().
ElectricFence: If this hangs, change the library load order with LD_PRELOAD.
ElectricFence: Registration was successful.
ElectricFence Exiting: 
Electric Fence: mprotect() failed: Cannot allocate memory
It's not an immediate exit, after about 1 minute it exits. Maybe this is a sign of something though?

I'd greatly appreciate any help and tool suggestions.

Thanks!
George
 
Old 08-25-2006, 03:51 PM   #2
mhcox
Member
 
Registered: Aug 2005
Location: Albuquerque, NM
Distribution: Fedora
Posts: 30

Rep: Reputation: 15
Quote:
Originally Posted by hedpe
Hey guys,

I am having problems with a program I have written, where its seg faulting in random places like malloc's or fprintf's. This to me is a signal that I have overrun somewhere earlier in the execution and overwritten a valid pointer or space somewhere else in memory. However, I've spent many hours debugging and trying to find it but I can't.

I was wondering if anyone knows any tools that will monitor the stack and detect when you try to access memory out of bounds? I have used ElectricFence in the past, linking my programs with -lefence, but I keep getting an electric fence error trying to do so with this program:
Code:
lanthanum-ini blinc_code # ./blinc 0 300
 
ElectricFence: Registering with atexit().
ElectricFence: If this hangs, change the library load order with LD_PRELOAD.
ElectricFence: Registration was successful.
ElectricFence Exiting: 
Electric Fence: mprotect() failed: Cannot allocate memory
It's not an immediate exit, after about 1 minute it exits. Maybe this is a sign of something though?

I'd greatly appreciate any help and tool suggestions.

Thanks!
George
Valgrind is freely available, although I've never used it (my personal favorite is PurifyPlus, but that costs $$$).
You can get it here http://www.valgrind.org/

Mike
 
Old 08-26-2006, 06:40 PM   #3
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 9,078
Blog Entries: 4

Rep: Reputation: 3187Reputation: 3187Reputation: 3187Reputation: 3187Reputation: 3187Reputation: 3187Reputation: 3187Reputation: 3187Reputation: 3187Reputation: 3187Reputation: 3187
Throughout every bit of your code, go and do this:
  • Make sure that every single pointer is initialized to NULL.
  • When you dispose of allocated memory, in the very next statement set the pointer back to NULL.
  • When you allocate memory, ask for four bytes more than you need. Don't tell anyone about those extra four bytes.
  • Explicitly set all dynamically allocated memory to binary zeroes immediately upon allocation.
The problems will go away for good.
 
Old 08-27-2006, 11:09 AM   #4
mhcox
Member
 
Registered: Aug 2005
Location: Albuquerque, NM
Distribution: Fedora
Posts: 30

Rep: Reputation: 15
Quote:
Originally Posted by sundialsvcs
Throughout every bit of your code, go and do this:
  • Make sure that every single pointer is initialized to NULL.
  • When you dispose of allocated memory, in the very next statement set the pointer back to NULL.
  • When you allocate memory, ask for four bytes more than you need. Don't tell anyone about those extra four bytes.
  • Explicitly set all dynamically allocated memory to binary zeroes immediately upon allocation.
The problems will go away for good.
This strategy may hide your problem, but you will still have the problem (not freeing or double freeing an application, overrunning a block of allocated memory, etc.). Instead of expending the effort to do the above, I'd review the code (maybe with someone else; the more eyes the better) one more time, counting your malloc/frees and making sure they're paired appropriately, look for casts and make sure they're appropriate (e.g. casting a malloced pointer to a pointer of a type that is not larger than the malloced size), and if that still fails, use valgrind or some other tool to find the bug, not hide it.

BTW, if you try the above as a last resort, use calloc() which does the last step above.

Mike
 
Old 08-27-2006, 01:14 PM   #5
xhi
Senior Member
 
Registered: Mar 2005
Location: USA::Pennsylvania
Distribution: Slackware
Posts: 1,065

Rep: Reputation: 45
Quote:
Originally Posted by mhcox
This strategy may hide your problem, but you will still have the problem (not freeing or double freeing an application, overrunning a block of allocated memory, etc.). Instead of expending the effort to do the above, I'd review the code (maybe with someone else; the more eyes the better) one more time, counting your malloc/frees and making sure they're paired appropriately, look for casts and make sure they're appropriate (e.g. casting a malloced pointer to a pointer of a type that is not larger than the malloced size), and if that still fails, use valgrind or some other tool to find the bug, not hide it.

BTW, if you try the above as a last resort, use calloc() which does the last step above.

Mike
how do you see the above as a way to hide the problem? i would see it as a way to solve it by getting to the root of the problem.

>Instead of expending the effort to do the above
you should be expending that effort anyhow. those are good programming practices, any good c programmer will tell you that.

although sundialsvcs i am still undecided about this one..

> When you allocate memory, ask for four bytes more than you need. Don't tell anyone about those extra four bytes.

maybe i just have not had the right experience of something running past the end of a block when i have done everything correctly. i would think that if you manage the memory you allocate correctly, you can just allocate what you need and be safe. i remember seeing you say this somewhere before though. is there an example you could give of why you prefer to do this.
 
Old 08-27-2006, 01:36 PM   #6
cupubboy
Member
 
Registered: May 2003
Location: Bucharest,Romania
Distribution: Fedora Core 7
Posts: 109

Rep: Reputation: 15
I'm not sure I'm correct here ....

But theese four extra bytes are in case you accidentally overun the known size ?? they're a fail-safe ...

I think that it would just be better to keep stright to the previous rule .. because as possible as it is to run out one byte .. it's just as possible to overrun 5 bytes .. or 10 bytes .. not to mention that in case you overrun the buffer 3 bytes (but everything is cool because you have 4 hidden bytes) it might be hard to track the error in the program afterwards

Just my 2 cents .. I maybe wrong though
 
Old 08-27-2006, 01:43 PM   #7
xhi
Senior Member
 
Registered: Mar 2005
Location: USA::Pennsylvania
Distribution: Slackware
Posts: 1,065

Rep: Reputation: 45
> But theese four extra bytes are in case you accidentally overun the known size ?? they're a fail-safe ...

thats how i see it also, if you are writing good code you should not need a fail safe, and if you need a fail safe then you are probably not writing good code.

thats why i was wondering about an example, cuz i couldnt think of a good reason. but thats not to say there isnt one.
 
Old 08-27-2006, 01:50 PM   #8
paulsm4
LQ Guru
 
Registered: Mar 2004
Distribution: SusE 8.2
Posts: 5,863
Blog Entries: 1

Rep: Reputation: Disabled
Xhi -

You're correct. Sundialsvc's advice is good (with the exception of the "four byte" thing), and, more importantly, it is indeed a way to get to the ROOT of the problem. It is emphatically not "hiding" the problem - it's simply Good Practice.

Hedpe - please take sundialsvcs's suggestions to heart - maybe even formalize them in coding guidelines for your team, and enforce them in team code reviews:
Quote:
I advise the following self-disciplines with respect to dynamically allocated memory.

1. "A pointer's value shall be NULL or it shall point to a valid block of storage." Therefore, the statement immediately following should be p = NULL;.
2. "All dynamically-allocated blocks of storage shall be immediately initialized to a known value, such as zeroes." Either use a storage allocation function that does this, or do it yourself.
3. "Allocate all buffers four to eight bytes larger than they need to be, make sure all those bytes are known-zero, and don't tell anyone they exist when you quote the size of the buffer."
(Personally, I disagree with this one - and find it at odds with 5) below...)
4. "When manipulating strings, always use functions that require you to give the size of the buffer and that will not write beyond that." Remember the trailing-null that some routines stomp past the end of the buffer.
5. "Always check. Don't assume. When something is wrong, fail noisily."
(This is VERY true. "Recovering gracefully" is sometimes the WORST strategy: but anything that helps you find the "root cause" is always good...)
6. "Insert debugging code into your program until you are certain that it is completely debugged. Then, and only then ... leave it in. "
This is from post http://www.linuxquestions.org/questi...49#post2380649

Mhcox added the following:
Quote:
7. Regularly compile your code with memory error detection tools like Purify, valgrind, etc. and run the code through a suite of unit-tests.
Again: this is all (very!) good advice. Heed it.
 
Old 08-27-2006, 02:59 PM   #9
xhi
Senior Member
 
Registered: Mar 2005
Location: USA::Pennsylvania
Distribution: Slackware
Posts: 1,065

Rep: Reputation: 45
> This is from post http://www.linuxquestions.org/questi...49#post2380649

yes sir, that is exactly where i remember these guidelines from
 
Old 08-27-2006, 02:59 PM   #10
mhcox
Member
 
Registered: Aug 2005
Location: Albuquerque, NM
Distribution: Fedora
Posts: 30

Rep: Reputation: 15
Quote:
Originally Posted by xhi
how do you see the above as a way to hide the problem? i would see it as a way to solve it by getting to the root of the problem.
Using tools like PurifyPlus (which I've used) or valgrind (assuming it has the same functionality as PurifyPlus; I've never used it, but have heard good things about it) will narrow it down to the line of code quickly, without having to make manual changes to the code (PurifyPlus "instruments" the object code and replaces malloc free with their own versions; I assume valgrind does the same).

Mike
 
Old 08-27-2006, 07:03 PM   #11
xhi
Senior Member
 
Registered: Mar 2005
Location: USA::Pennsylvania
Distribution: Slackware
Posts: 1,065

Rep: Reputation: 45
Quote:
Originally Posted by mhcox
Using tools like PurifyPlus (which I've used) or valgrind (assuming it has the same functionality as PurifyPlus; I've never used it, but have heard good things about it) will narrow it down to the line of code quickly, without having to make manual changes to the code (PurifyPlus "instruments" the object code and replaces malloc free with their own versions; I assume valgrind does the same).

Mike
i still dont understand why you think the above mentioned practices are hiding the problem. if you are not following those things that sundialsvcs laid out, that is your problem. if you would follow those from the beginning you would not have this sort of problem to fix.

Valgrind and the like are not meant to be a replacement for good programming.
 
Old 08-28-2006, 03:56 AM   #12
mhcox
Member
 
Registered: Aug 2005
Location: Albuquerque, NM
Distribution: Fedora
Posts: 30

Rep: Reputation: 15
Quote:
Originally Posted by xhi
i still dont understand why you think the above mentioned practices are hiding the problem. if you are not following those things that sundialsvcs laid out, that is your problem. if you would follow those from the beginning you would not have this sort of problem to fix.

Valgrind and the like are not meant to be a replacement for good programming.
Of course not . But sometimes us mere mortals forget a free() or three, free memory through one pointer, but use the freed memory through another, etc. Also, tools like PurifyPlus and valgrind catch other memory corruption errors that have nothing to do with dynamic memory allocation. And they're so easy to use with a unit-test framework.

I think the mentioned practices hide problems because:
  • It is a problem/bug if there exists a path of execution in your program that causes a pointer to be freed twice without an intervening malloc(). By setting a pointer to NULL after you have freed it, you hide a double-free() bug, since free ignores NULL pointers passed to it. And setting a pointer to NULL after free()-ing it doesn't help at all in the case of multiple pointers pointed to the same block of memory.
  • It is a problem/bug if your logic overruns a buffers length (dynamically or statically allocated). By adding extra bytes at the end of a malloced memory buffer, you hide this buffer overrun (see message #6). Although, I have seen this general idea used in debug malloc/free libraries, where the extra space is set to a known byte pattern and then on a free() the extra space is checked for corruption, but you shouldn't be explicitly doing this in your code.
Mike
 
Old 08-28-2006, 01:52 PM   #13
paulsm4
LQ Guru
 
Registered: Mar 2004
Distribution: SusE 8.2
Posts: 5,863
Blog Entries: 1

Rep: Reputation: Disabled
Hi, mhcox -

I totally agree with 2) ...and totally disagree with 1).

BOTTOM LINE:
Setting pointers to NULL when you declare a pointer, and explicitly setting pointers again to NULL when you free the pointer, is simply Good Practice.

1. It's good discipline: it makes you look at precisely those things that might later "bite you".

2. It emphatically does not "hide" the problem. Under the right circumstances, it might actually cause you to discover and track down a latent bug QUICKER!

3. It's a habit that carries over well to garbage-collected environments (like Java or C#)

Let's please agree that setting pointers to "NULL" is a Good Thing.

PS:
Quid pro quo: I'll agree with you that adding those extra bytes is at best a band-aid, and in fact a totally bogus, Mickey Mouse kind of band-aid. OK ;-)?
 
Old 08-28-2006, 04:43 PM   #14
mhcox
Member
 
Registered: Aug 2005
Location: Albuquerque, NM
Distribution: Fedora
Posts: 30

Rep: Reputation: 15
Quote:
Originally Posted by paulsm4
Let's please agree that setting pointers to "NULL" is a Good Thing.
I'm sorry, but I guess we'll have to agree to disagree.

Quote:
Originally Posted by paulsm4
2. It emphatically does not "hide" the problem. Under the right circumstances, it might actually cause you to discover and track down a latent bug QUICKER!
I think I've shown that "under the right circumstances" (double-free()-ing) you do hide the problem. Also, "under the right circumstances", setting a pointer to NULL that points to malloc()-ed memory, you create a memory leak. And, as you point out, dereferencing a NULL pointer (the right circumstances) will cause you to find the bug quicker, but if you could be omniscient and know what your next bug was going to be, you would fix it.

Quote:
Originally Posted by paulsm4
3. It's a habit that carries over well to garbage-collected environments (like Java or C#)
Java and C#'s garbage-collectors CYA, but in C/C++ you have no such safety-net. Setting Java/C# references to null is a good practice in those languages (that we can agree on ).

We can also agree on the practice that variables should be initialized when declared. Since C99 adopted the variable declaration conventions of C++, i.e. variables can be declared anywhere in a scope, one can adopt some very good practices presented in C++ Coding Standards: 101 Rules, Guidelines, and Best Practices, in particular #18 (Declare variables as locally as possible.) and #19 (Always initialize variables.) see http://safari.awprofessional.com/0321113586/part03


Mike

P.S. I think this horse has been beaten to within an inch of its life.
 
Old 08-28-2006, 05:18 PM   #15
cupubboy
Member
 
Registered: May 2003
Location: Bucharest,Romania
Distribution: Fedora Core 7
Posts: 109

Rep: Reputation: 15
Quote:
Quote:
Originally Posted by paulsm4
Let's please agree that setting pointers to "NULL" is a Good Thing.
I'm sorry, but I guess we'll have to agree to disagree.
I think he meant .. initializing pointers to null .. and setting them to null after a the memory they are pointing to has been freed
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Serial Port Buffer Overrun damiendusha Linux - Hardware 1 06-22-2006 02:56 AM
What is the difference between the free buffer and buffer in the buffer hash queue? Swagata Linux - Enterprise 0 05-25-2006 11:57 PM
Tool for Hardware Detection gsrikanth Linux - Hardware 1 10-21-2004 05:09 AM
Buffer Overrun? gnashley Slackware 0 07-28-2004 04:12 PM
Squid Delay Pools being overrun Redleg Linux - Networking 8 07-03-2004 07:04 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 06:24 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration