How to simulate out of memory errors?

lowpro2k3 · 10-20-2005, 03:46 PM

Are there any tools designed to do this? I want to test some calls to realloc(), specifically what would happen when realloc fails.

jailbait · 10-20-2005, 04:26 PM

"Are there any tools designed to do this? I want to test some calls to realloc(), specifically what would happen when realloc fails."

You might take a look at electric fence.

http://www.die.net/doc/linux/man/man3/efence.3.html

-----------------------
Steve Stites

Hko · 10-20-2005, 04:43 PM

Or just try to realloc() a really huge amount of memory.

lowpro2k3 · 10-20-2005, 06:05 PM

I wrote this little program to try and make the system run out of memory - it doesn't seem to work, because my system is still 100% usable and 'ps aux' only shows memory usage as 0.1% for the process.

Code:

/* keep calling realloc() until it runs out of memory */
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main()
{
 char *ptr;
 size_t len;
 len = 1;

 ptr = (char*) malloc(len);

 while(len *= 2)
 {
  if((ptr = (char*) realloc(ptr, len)) == NULL)
   fprintf(stderr, "realloc failed trying to allocate %d bytes\n", len);

  if(len > 67108865) /* 64MB */
  {
   fprintf(stdout, "going to sleep\n");
   sleep(45);
  }
 }

 return 0;
}

Before I added the 64MB check, realloc seemed to fail when integer overflow happened, not when the system ran out of memory.

Basically I'm spending time worrying about this for my little shell program. I'm starting to wonder whether its worth it to check whether realloc failed when I try to allocate size for a line of any length. I'm starting to really doubt it...

I initially malloc LINE_MAX number of bytes, and if thats not enough I call realloc when necessary.

jailbait · 10-20-2005, 06:11 PM

"I wrote this little program to try and make the system run out of memory - it doesn't seem to work, because my system is still 100% usable and 'ps aux' only shows memory usage as 0.1% for the process."

You can make the problem a little easier by getting rid of swap during the experiment. Stop swap with:
swapoff

Start swap again with:
swapon -a

---------------------
Steve Stites

rsheridan6 · 10-20-2005, 10:43 PM

I haven't actually tried this, but I think using setrlimit to reduce RLIMIT_DATA would help. That would tell the kernel not to give your process so much heap memory.

jlliagre · 10-21-2005, 03:25 PM

Allocating memory is not enough, it reserves pages of virtual memory but does nothing else.

If you want to really simulate a memory shortage, you need to make each of these pages mapped to physical memory, the simpler would be to read or write one byte per page.

primo · 10-21-2005, 04:32 PM

"Unfortunately", it depends on the malloc implementation. From the malloc manpage:

Quote:

By default, Linux follows an optimistic memory allocation strategy.
This means that when malloc() returns non-NULL there is no guarantee
that the memory really is available. This is a really bad bug. In case
it turns out that the system is out of memory, one or more processes
will be killed by the infamous OOM killer. In case Linux is employed
under circumstances where it would be less desirable to suddenly lose
some randomly picked processes, and moreover the kernel version is suf-
ficiently recent, one can switch off this overcommitting behavior using
a command like
# echo 2 > /proc/sys/vm/overcommit_memory
See also the kernel Documentation directory, files vm/overcommit-
accounting and sysctl/vm.txt.

There are malloc implementations out there that use mmap(2) to get file-backed memory in /tmp. It's hard to make robust implementations fail, but this OOM killer is something I turn off. Try with RLIMIT_DATA as rsheridan6 pointed out. Try setting malloc() options too. If you're trying to make your program robust, then check NULLs and -1's. Installing a signal handler for SIGSEGV doesn't make sense. Note that not every malloc implementation sets errno. I usually create a wrapper like this:

Code:

void *emalloc(size_t size)
{
     void *addr;

     errno = 0;
     addr = malloc(size);
     if (addr == NULL && !errno)
         errno = ENOMEM;

     return addr;
}

The same with realloc() and friends

Dark_Helmet · 10-21-2005, 04:47 PM

Ok, I may be way off, but what's trying to be tested here? The system as a whole or a program running in a constrained memory environment?

If it's the latter, I'd just write a wrapper for realloc() and free(). Some pseudo code:

Code:

function realloc_wrapper( request_size )
{
  if ( request_size + currently_allocated_memory > memory_allowed )
    return NULL;
  else
  {
    realloc_ptr = realloc( request_size );
    if( realloc_ptr != NULL )
      currently_allocated_memory += request_size;
    return realloc_ptr;
  }
}

function free_wrapper( memory_size, memory_block )
{
  currently_allocated_memory -= memory_size;
  free( memory_block );
}

Then either set memory_allowed to some static value with a #define or come up with some fancy-schmancy way of real-time modification to simulate requests made by other processes.

EDIT:
Didn't mean to rehash what primo said... I think we were both headed in the same direction.