C, Malloc, and Memory: Just Curious

CoderMan · 01-20-2010, 07:35 PM

I don't do as much C programming any more (Mostly C++, Java, a little Perl and PHP) but it seems like I remember being told repeatedly to always check the return value of malloc, so that you know whether or not the system actually was able to allocate the memory you requested.

But I was sitting in a lecture the other day and the speaker stated that because Linux does "lazy allocation," malloc might actually return a non-null pointer even if the memory doesn't actually exist, so you might not know the memory is not available until later in the program.

My first thought was, "How would you know later on if the memory was not available?" Would you just have to wait for the program to crash, or do you check some flag or signal? I talked to the lecturer and he wasn't sure.

So I'm wondering: Is this guy right even right about the lazy allocation issue? And if not, then what is the answer to my question? (There is virtual memory as well, I know, but eventually you have to run out of swap space...)

Sergei Steshenko · 01-20-2010, 07:41 PM

Quote:

Originally Posted by CoderMan

I don't do as much C programming any more (Mostly C++, Java, a little Perl and PHP) but it seems like I remember being told repeatedly to always check the return value of malloc, so that you know whether or not the system actually was able to allocate the memory you requested.

But I was sitting in a lecture the other day and the speaker stated that because Linux does "lazy allocation," malloc might actually return a non-null pointer even if the memory doesn't actually exist, so you might not know the memory is not available until later in the program.

My first thought was, "How would you know later on if the memory was not available?" Would you just have to wait for the program to crash, or do you check some flag or signal? I talked to the lecturer and he wasn't sure.

So I'm wondering: Is this guy right even right about the lazy allocation issue? And if not, then what is the answer to my question? (There is virtual memory as well, I know, but eventually you have to run out of swap space...)

The answer is in

man 3 malloc
.

wje_lq · 01-20-2010, 07:57 PM

Quote:

Originally Posted by CoderMan

I talked to the lecturer and he wasn't sure.

The lecturer was blowing smoke. malloc() returns NULL if and only if the memory allocation failed.

CoderMan · 01-20-2010, 08:32 PM

Quote:

Originally Posted by Sergei Steshenko

The answer is in

man 3 malloc
.

Well, I guess it proves what he is saying. Here is a quote from malloc(3):

Code:

       By  default, Linux follows an optimistic memory allocation strategy.  This means that when malloc() returns non-NULL there is no
       guarantee that the memory really is available.  This is a really bad bug.  In case it turns out that the system is out  of  mem‐
       ory,  one  or  more processes will be killed by the infamous OOM killer.  In case Linux is employed under circumstances where it
       would be less desirable to suddenly lose some randomly picked processes, and moreover the kernel version is sufficiently recent,
       one can switch off this overcommitting behavior using a command like:

           # echo 2 > /proc/sys/vm/overcommit_memory

I'm not sure that entirely answers my question, since most people probably don't have overcommit_memory set to 2. (It seems to be set to 0 on my system.) So you can't really be sure that your malloc will not over-commit in most places that it is deployed.

CoderMan · 01-20-2010, 09:17 PM

So, I looked up OOM Killer and got this page:

http://linux-mm.org/OOM_Killer

If I read this correctly, then in "optimistic mode" OOM is supposed to keep processes from running out of memory by killing off the "undeserving" processes until there is enough memory left over for everybody else. But it doesn't describe what happens if some process tries to take even more memory than is needed just to run the kernel (or basic system utilities, for the matter).