Creating a memory leak in c++

HarryBoy · 12-16-2010, 07:19 AM

I need to write a small program that eats away at availabe memory. I need to creaty a memory leak to test how other programs cope.

I need to run this program on linux and see if the available memory is decreasing.
So I have done:

Code:

int main()
{
	int *buffer;
	while(1)
	{
		buffer = (int*)malloc(100000);
		sleep(1);
	}
	return 0;
}

I run this program in the background and then I use:

Code:

free -m

To see the output of available memory like so:

Code:

total     used       free     shared    buffers     cached
3293      3168        124          0        237       1205

The problem is that I don't see the 'free' value decrease over time.

Is my approach wrong here??

I have checked that my process is running using ps-ef and it is.
Thanks

dwhitney67 · 12-16-2010, 07:34 AM

If you are running your app under Linux, the OOM (Out of Memory) Killer will terminate one or more apps when memory runs low. Most likely the app that is killed will be the one sucking up all the memory, but there's no guarantee.

You can read more about the OOM Killer here.

HarryBoy · 12-16-2010, 07:39 AM

Thanks, thats ok if my app is killed.

I still need to find a way to detect the amount of available memory. Is this possible?

dwhitney67 · 12-16-2010, 07:42 AM

Parse the contents of /proc/meminfo.

ntubski · 12-16-2010, 09:15 AM

Linux won't allocate any memory until you actually use it, try

Code:

int main()
{
	int *buffer;
	while(1)
	{
		buffer = (int*)malloc(100000);
                memset(buffer, 0, 100000);
		sleep(1);
	}
	return 0;
}

johnsfine · 12-16-2010, 09:17 AM

Quote:

Originally Posted by HarryBoy

buffer = (int*)malloc(100000);

That just "commits" the memory, very little is actually allocated.

Unless you have unusual settings for "over commit" there is no effect on other processes from this process committing a lot of memory that it doesn't actually allocate.

If you want to actually allocate the memory, you have to write to all of it (write to at least one byte out of every 4096 bytes).

Also, you should be aware that below a certain request size malloc will group allocations together, so several will be committed from memory malloc started with, then a lot will be committed at once when the initial pool runs out and used for the next several requests.

So to see commit and allocation and other aspects all change in sync each second (as I expect you want) you need a large enough request in addition to writing to the memory that malloc gives you. I don't know whether 100000 is "large enough" because that varies by version of malloc. But I suspect it is not large enough.

orgcandman · 12-16-2010, 12:41 PM

Quote:

Originally Posted by johnsfine

Also, you should be aware that below a certain request size malloc will group allocations together, so several will be committed from memory malloc started with, then a lot will be committed at once when the initial pool runs out and used for the next several requests.

Do you have a pointer to some info on this effect? The only "size-matters" issue I'm aware of with malloc is a malloc request of <= 64 bytes using the skb() system call, and > 64 bytes using anonymous backed mmap call. I'm interested in the effect you describe (coalescence of memory-block requests). Are these related to the internal "arena" system?

Additionally, I was under the impression that a call to mlockall(MCL_CURRENT | MCL_FUTURE); would skip requiring to tickle pages to force commit (as long as you have CAP_IPC_LOCK permissions a.k.a. root). I could be wrong on this, however.

johnsfine · 12-16-2010, 05:50 PM

Quote:

Originally Posted by orgcandman

Do you have a pointer to some info on this effect? The only "size-matters" issue I'm aware of with malloc is a malloc request of <= 64 bytes using the skb() system call, and > 64 bytes using anonymous backed mmap call. I'm interested in the effect you describe (coalescence of memory-block requests). Are these related to the internal "arena" system?

I don't know the amount on which that decision is based, but it is far higher than 64 bytes. It isn't even practical below 4096 bytes and I'm sure it isn't that low. Maybe you meant 64K bytes.

Small requests to malloc typically come from memory previously gotten from the OS by (I forget the name, but you said skb and that is likely correct). If the memory isn't available from previous skb then a new skb will be used to commit far more than the current request.

Large requests to malloc look first in the same pool of previously committed memory, but almost never are satisfied that way. Then they use mmap.

Sergei Steshenko · 12-16-2010, 06:25 PM

Quote:

Originally Posted by HarryBoy

I need to write a small program that eats away at availabe memory. I need to creaty a memory leak to test how other programs cope.

I need to run this program on linux and see if the available memory is decreasing.
So I have done:

Code:

int main()
{
	int *buffer;
	while(1)
	{
		buffer = (int*)malloc(100000);
		sleep(1);
	}
	return 0;
}

I run this program in the background and then I use:

Code:

free -m

To see the output of available memory like so:

Code:

total     used       free     shared    buffers     cached
3293      3168        124          0        237       1205

The problem is that I don't see the 'free' value decrease over time.

Is my approach wrong here??

I have checked that my process is running using ps-ef and it is.
Thanks

Your 'buffer' variable is not used anywhere, it is not in the RHS (Righ Hand Side). It is also of class 'auto', i.e. not global, so it can't be used by, say a separate function. So, the compiler has the right to optimize out the whole

Code:

buffer = (int*)malloc(100000);

line.

...

ntubski made a correct suggestion how to make 'buffer' actually used.

johnsfine · 12-16-2010, 08:39 PM

Quote:

Originally Posted by Sergei Steshenko

Your 'buffer' variable is not used anywhere, it is not in the RHS (Righ Hand Side). It is also of class 'auto', i.e. not global, so it can't be used by, say a separate function. So, the compiler has the right to optimize out the whole

Code:

buffer = (int*)malloc(100000);

line.

Not correct.

The function call cannot be optimized out even though the lhs variable can.

Quote:

ntubski made a correct suggestion

Yes.

Quote:

how to make 'buffer' actually used.

But not for that reason.

orgcandman · 12-17-2010, 12:22 PM

Quote:

Originally Posted by johnsfine

I don't know the amount on which that decision is based, but it is far higher than 64 bytes. It isn't even practical below 4096 bytes and I'm sure it isn't that low. Maybe you meant 64K bytes.

Small requests to malloc typically come from memory previously gotten from the OS by (I forget the name, but you said skb and that is likely correct). If the memory isn't available from previous skb then a new skb will be used to commit far more than the current request.

Large requests to malloc look first in the same pool of previously committed memory, but almost never are satisfied that way. Then they use mmap.

Don't want to hijack any more.

Firstly, the call is sbrk() / brk(). My bad on that. Too much time in network driver land.

Second, it seems the current thresholds are 128K as min mmap size (this is from my reading of the current malloc in the glibc git repository). Anything below that will fall into an sbrk/brk call. 128*1024 as your malloc size would get you there faster, as johnsfine points out. My understanding of how the internal glibc malloc works is quite outdated.