I have a memory leak in C. Need help interpreting valgrind's output.

adz · 09-02-2008, 07:32 AM

I've written a small GTK program and it definitely has a memory leak that I can see with top or ps. However, I can't make heads or tails of the valgrind output. I've looked at some simple tutorials and I don't get any of the messages that they get (e.g. "Conditional jump", "Invalid write", "Mismatched free ()", etc). I do, however, get the following leak summary:

Code:

==7987== LEAK SUMMARY:
==7987==    definitely lost: 80,548 bytes in 260 blocks.
==7987==    indirectly lost: 159,932 bytes in 7,901 blocks.
==7987==      possibly lost: 1,159,784 bytes in 3,699 blocks.
==7987==    still reachable: 35,094,535 bytes in 180,314 blocks.
==7987==         suppressed: 800 bytes in 20 blocks.

Which seems to suggest multiple leaks. For reference, the command I used to run valgrind was:

Code:

valgrind --tool=memcheck --leak-check=yes --show-reachable=yes --num-callers=20 --track-fds=yes --log-file=valgrind.log ./GTKalendar

I've included the entire log file here for people to have a look at. The only lines that it complains about that are actually in my code are:

line 22

Code:

gtk_init (&argc, &argv);

and line 20

Code:

PangoFontDescription *prev_next = pango_font_description_from_string ("6");

both of which are super simple and rather essential. I've also posted the code here. To compile (you'll need GTK development libraries), just untar and cd into the directory and type make GTKalendar. When you run it, you'll need to click on the systray icon (a fancy upper-case "C") to make the window visible. To then trigger the memory leak, you need to move through the months either by the scrollwheel or clicking the arrow buttons. The direction (forwards or backwards) doesn't seem to matter as both cause the amount of memory used to go up. There's no easy way to quit yet, you'll just have to kill it or make the window decorations visible (assuming your window manager supports that) and click the X.

Be aware that it will create a settings file named ~/.GTKalendar that you may want to delete afterwards.

SciYro · 09-02-2008, 09:57 AM

Look at the loss records, those show the backtrace of where in your code the allocation occurred that was not unallocated.

By the looks of things, a lot of stuff seems to have been allocated by various libraries, yet has not be unallocated. You should probably start by checking your program properly closes down. If you have 7 open file descriptors at shutdown, then you probably forgot to tell your libraries to un-init, and your widgets to destroy themselves, etc.

shane_kerr · 09-02-2008, 02:26 PM

First off, you shouldn't be using your header file, GTKalender.h, to store your functions. Header files are not meant to be libraries, they are meant to contain declarations (function declarations, constants, and so on) that one or more of the non-header C files will need to compile properly.

Valgrind is hard to interpret because of the way GTK works. Rather than driving the program yourself, you register a bunch of event handlers, and then GTK calls you as necessary. This is why Valgrind thinks that all of your memory is coming from gtk_init().

I think the other poster is correct, and the reports are merely because your program is not cleaning up when it shuts down. So, when you invoke gtk_event_box_new(), then any memory needed for this is "leaked", because when the program shuts down it is not freed.

What I recommend is building a debug version of the program which cleans up memory on shutdown. This way you can use Valgrind to see if there are any "true" leaks. But in normal operation it is better simply to exit, since the OS will reclaim all memory used by the process. A good way to do this is by something like:

Code:

#ifdef MEMORY_DEBUG
    memory_cleanup();
#endif

Use this right before the program ends. And then you can have a function that frees things:

Code:

#define NUM_BOXES (sizeof(day_box) / sizeof(day_box[0]))

void
memory_cleanup (void)
{
    int i;

    for (i=0; i < NUM_BOXES; i++) {
        gtk_event_box_free(day_box[i]);   // or whatever...
        gtk_label_free(day_label[i]);     // or whatever...
    }

    // and so on... 
}

You'll also want to save labels in globals for cleanup, and so on.

Good luck!

ta0kira · 09-02-2008, 09:22 PM

I'm not particularly familiar with valgrind, but I do know that some developers consider a one-time allocation that's never deallocated a "non-leak." The reasoning is that the memory is implicitly released upon process exit, that there is still a pointer to it until that point (immediately disqualifying it as a leak,) and that it doesn't occur more than once. Some memory pool implementations use such rationale, so maybe GTK+ uses a similar pattern with gtk_init. I don't know, but it's something to think about.
ta0kira

adz · 09-02-2008, 11:40 PM

Hi all. Thanks for replying. I wanted you all to know I haven't abandoned this thread and I'm investigating some of the suggestions put forward.

Quote:

Originally Posted by SciYro

Look at the loss records, those show the backtrace of where in your code the allocation occurred that was not unallocated.

Yeah I looked at those and that's what was confusing me. I had no idea how to track it down to anything meaningful.

Quote:

Originally Posted by SciYro

By the looks of things, a lot of stuff seems to have been allocated by various libraries, yet has not be unallocated. You should probably start by checking your program properly closes down. If you have 7 open file descriptors at shutdown, then you probably forgot to tell your libraries to un-init, and your widgets to destroy themselves, etc.

I'm pretty much with shane_kerr and ta0kira on this one. They seem to me to be one-time allocations and that's why I didn't really care. Not to mention, in the GTK tutorial, they didn't destroy all of their widgets on quitting so I didn't think I had to either. I may still try shane_kerr's clean up suggestion.

Quote:

Originally Posted by shane_kerr

First off, you shouldn't be using your header file, GTKalender.h, to store your functions. Header files are not meant to be libraries, they are meant to contain declarations (function declarations, constants, and so on) that one or more of the non-header C files will need to compile properly.

I've fixed that up now. Thanks for the pointer.

Quote:

Originally Posted by shane_kerr

Valgrind is hard to interpret because of the way GTK works. Rather than driving the program yourself, you register a bunch of event handlers, and then GTK calls you as necessary. This is why Valgrind thinks that all of your memory is coming from gtk_init().

I think the other poster is correct, and the reports are merely because your program is not cleaning up when it shuts down. So, when you invoke gtk_event_box_new(), then any memory needed for this is "leaked", because when the program shuts down it is not freed.

I had come to a similar conclusion myself about most of those "leaks", however, there actually is an honest-to-goodness leak in the code, over and above these one-time allocations which top and ps show.

Quote:

Originally Posted by shane_kerr

What I recommend is building a debug version of the program which cleans up memory on shutdown. This way you can use Valgrind to see if there are any "true" leaks. But in normal operation it is better simply to exit, since the OS will reclaim all memory used by the process. A good way to do this is by something like:
<snip>

I haven't done this yet but probably will later. I have isolated, more-or-less, where the leak is coming from but I have no idea why it happens or how to fix it. If I comment out the gtk_widget_modify_bg () and gtk_widget_modify_fg () function calls in update_month () then the leak goes away. I have no idea why this would cause a leak nor how to plug it.

Quote:

Originally Posted by ta0kira

I'm not particularly familiar with valgrind, but I do know that some developers consider a one-time allocation that's never deallocated a "non-leak." The reasoning is that the memory is implicitly released upon process exit, that there is still a pointer to it until that point (immediately disqualifying it as a leak,) and that it doesn't occur more than once. Some memory pool implementations use such rationale, so maybe GTK+ uses a similar pattern with gtk_init. I don't know, but it's something to think about.
ta0kira

Yes, I agree with you. In fact, I saw a thread on this somewhere in my travels and I think that's exactly what GTK does. It doesn't explain why there's an actual runtime leak, though.

adz

ta0kira · 09-03-2008, 06:55 AM

Quote:

Originally Posted by adz

Not to mention, in the GTK tutorial, they didn't destroy all of their widgets on quitting so I didn't think I had to either. I may still try shane_kerr's clean up suggestion.

This doesn't always mean that they aren't cleaned up. I'm sure the examples always insert the new widgets into containers of other widgets, making a tree originating at a common base window. The exit call could free the whole tree by accessing the base object because each widget will be indirectly accessible through it. But you probably are right about that in this case.
ta0kira

adz · 09-03-2008, 08:08 AM

Quote:

Originally Posted by ta0kira

This doesn't always mean that they aren't cleaned up. I'm sure the examples always insert the new widgets into containers of other widgets, making a tree originating at a common base window. The exit call could free the whole tree by accessing the base object because each widget will be indirectly accessible through it. But you probably are right about that in this case.
ta0kira

Yes, you're right. They do do that and so do I. Everything in my app is packed immediately into something that is in turn in some way packed into the main top level window that I call destroy on eventually. Even if I didn't, the widgets are only instantiated once and would be killed (and free'd) by force when the app exited (like you said in your previous post).

adz · 09-03-2008, 11:46 PM

Hi all.

I've managed to substantially reduce the memory leak, essentially by calling gtk_widget_modify_* less often, and as a side effect made my app considerably more efficient. I still don't know why those function calls inflate memory usage. At least the problem is considerably smaller now.

ta0kira · 09-04-2008, 08:16 AM

Remember that GTK+ creates pseudo-polymorphism through extensive use of macros. C doesn't have the built-in inheritance support that C++ does, so it all has to be done at a higher level of code. Some of those operations can be very complex to implement by hand and (as far as I know) it's done at the source level rather than at the binary level as with C++. I say "as far as I know" because the constructs have to be able to interface with macros in C.
ta0kira

adz · 09-04-2008, 09:21 PM

Hi again.

I've just recently written a super simple GTK app that just inverts the foreground and background colours every second as a test case. Now, as I watch the output of top, I see both VIRT and RES slowly creeping up. Have I got it completely wrong or is this a memory leak? Here is the code I used:

Code:

#include <gtk/gtk.h>

GtkWidget *window;
GtkWidget *label;
GdkColor *fg;
GdkColor *bg;

int toggle = 0;

static void destroy (GtkWidget *widget, gpointer data) {
	gtk_main_quit ();
}


gint toggle_colours () {
	if (toggle == 0) {
		gtk_widget_modify_bg (window, GTK_STATE_NORMAL, fg);
		gtk_widget_modify_fg (label, GTK_STATE_NORMAL, bg);
		toggle = 1;
	}
	else {
		gtk_widget_modify_bg (window, GTK_STATE_NORMAL, bg);
		gtk_widget_modify_fg (label, GTK_STATE_NORMAL, fg);
		toggle = 0;
	}
	
	return TRUE;
}


int main (int argc, char *argv[]) {

	gtk_init (&argc, &argv);
	
	window = gtk_window_new (GTK_WINDOW_TOPLEVEL);
	g_signal_connect (G_OBJECT (window), "destroy", G_CALLBACK (destroy), NULL);
	
	label = gtk_label_new ("Look at the output of top or ps for memory leaks.");
	gtk_container_add (GTK_CONTAINER (window), label);
	
	g_timeout_add (1000, toggle_colours, NULL);
	
	gtk_widget_show (label);
	
	gtk_widget_show (window);
	
	fg = window->style->bg;
	bg = window->style->fg;
	
	gtk_main ();
	
	return 0;
}

To compile, save it as something then run the command: gcc -Wall -g <insert filename>.c -o <insert filename> `pkg-config --cflags --libs gtk+-2.0`. Have I missed something in my code?

Thanks in advance.

jlliagre · 09-05-2008, 02:49 AM

No leak observed here with your test code under Solaris Express.

adz · 09-05-2008, 03:05 AM

Really? No difference in the number spat out by top or ps? Hmmm... Might be a problem specific to Linux and/or Debian. May I ask, how long did you run the program for and what version of the GTK libraries you have?

One more question. When free() is called under Solaris, is the memory free'd straightaway? I read somewhere that that was the case, or at least it was a possibility if one compiles with a special header file. As I understand it, under Linux, free'd memory just goes back into the malloc() pool and will only get truly free'd when the program exits.

Edit: I've been running my app for almost 2 hours now and the RES column of top is up to 51m!

jlliagre · 09-05-2008, 06:19 AM

Quote:

Originally Posted by adz

May I ask, how long did you run the program for and what version of the GTK libraries you have?

I first ran it for a couple of minutes and saw no evolution in the SIZE and RSS columns. 69 MB of VM, 13 MB of RAM.
Then I modified your code to toggle the background every 10 ms instead of every second and the values are still exactly the same after 40000 swaps (roughly 10 hours equivalent time if I'm not mistaken).
The GTK version I use is 2.12.9.1209.9.

Quote:

One more question. When free() is called under Solaris, is the memory free'd straightaway?

No. The standard libc free function do alloc from the heap (brk). Only the free libraries based on mapped memory can release memory and only when the allocated/free'd size is large enough.
Note that this is only about virtual memory. The RAM might be recovered after a while regardless of how memory is allocated/free'd.

Quote:

I read somewhere that that was the case, or at least it was a possibility if one compiles with a special header file. As I understand it, under Linux, free'd memory just goes back into the malloc() pool and will only get truly free'd when the program exits.

Same happens with Linux. Only the working set of memory is using RAM, the rest can be swapped out should real memory is required or really free'd if mapped memory is used.

Quote:

Edit: I've been running my app for almost 2 hours now and the RES column of top is up to 51m!

You do not tell at what rate do the memory values rise on your system.
How was the RES column at startup ?

adz · 09-05-2008, 06:44 AM

Quote:

Originally Posted by jlliagre

No. The standard libc free function do alloc from the heap (brk). Only the free libraries based on mapped memory can release memory and only when the allocated/free'd size is large enough.
Note that this is only about virtual memory. The RAM might be recovered after a while regardless of how memory is allocated/free'd.

I see

Quote:

Originally Posted by jlliagre

You do not tell at what rate do the memory values rise on your system.
How was the RES column at startup ?

RES starts out at about 6.8m and VIRT is consistently ~RES + 9. Now, after about 5 hours and 20 min, RES and VIRT are at 139m and 147m respectively. So I'm looking at about 0.4m per minute. I may have to file a bug report with Debian.