[SOLVED] Explorations in memory allocation in ANSI C

ejspeiro · 02-08-2011, 09:49 AM

Hi there!

I could start by writing a little bit about me but I'm not that interesting XD I may do it later!

The question itself is this one: I'm performing some explorations in terms of memory allocations in ANSI C, using the following compiler:

gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)

and I started to try to make the compiler complain for a huge data structure, so I coded this:

Code:

/* 10,000 MILLIONS: */
#define TWODCRASHER 10000000000

char pool3[TWODCRASHER][TWODCRASHER];

and it effectively tells me the following:

Code:

$ gcc -o alloc alloc.c
alloc.c:34: error: size of array ‘pool3’ is too large

now... can anyone tell me, why does gcc does this? I mean, are there minimum / maximum sizes for an array to be statically allocated? Could anyone point me towards some documentation?

Another scenario I tried is to allocate a 1D array defined as:

Code:

/* 100,000,000 MILLIONS OF MILLIONS: */
#define WARNING     100000000000000000000

char pool2[WARNING];

I tried then to do this:

Code:

int main(void) {

  pool2[0] = 'a';
  pool2[WARNING - 1] = 'z';
  printf("Apparently, if I print this, the allocation was successful...\n");
  printf("Some values: %c and %c.\n", pool2[0], pool2[WARNING - 1]);

  ...
}

I get the following complains in compilation time:

Code:

$ gcc -o alloc alloc.c
alloc.c:32:12: warning: integer constant is too large for its type
alloc.c:58:9: warning: integer constant is too large for its type
alloc.c:60:55: warning: integer constant is too large for its type

And I get this, when I execute it:

Code:

$ ./alloc
Killed
$

Any explanations / useful links?

I mean, I can always read the man entry for gcc but I rather ask because I can then have interaction... I mean, isn't that the whole purpose of forums?

Thanks!

paulsm4 · 02-08-2011, 10:20 AM

Hi -

The problem isn't the *size* of the array. The array *index* (at least for a 32-bit system) needs to fit in 32 bits.

ejspeiro · 02-08-2011, 10:23 AM

That is what I though!

But what happens if it is larger? Some sort of overflow in compilation time perhaps?

Thanks for reply!

And btw, if you have a reference for that it would be cool, NOT BECAUSE I DON'T TRUST YOU, but because I'm writing all of this and a reference may be need later so I'm trying to collect them

Sergei Steshenko · 02-08-2011, 10:29 AM

Quote:

Originally Posted by ejspeiro

...
Any explanations / useful links?
...

http://en.wikipedia.org/wiki/Central_processing_unit -> http://en.wikipedia.org/wiki/Addressing_mode -> http://www.osdata.com/topic/language/asm/address.htm -> http://www.osdata.com/system/physical/memory.htm ;

http://en.wikipedia.org/wiki/Binary_arithmetic ;
http://en.wikipedia.org/wiki/Random-access_memory .

ejspeiro · 02-08-2011, 10:42 AM

This is cool, because I have a lot of links now to reference this! Thank you guys!

I also tried to allocate in the stack memory by doing this:

Code:

void AllocateFromStack(long int bytes) {

  char pool[bytes];

  printf("It seems that allocating from the stack worked!\n");
  pool[0] = 'a';
  printf("It worked! pool[0] = %c\n", pool[0]);
}

and then calling it from the main module. At first I thought that I shouldn't work but I was surprised by finding out that it did! Because I could successfully allocate in execution time

Now, should I be surprised? Is that true that pool, in this context is actually being allocated on the stack memory. I think so but I would like further insights!

Thanks!

Sergei Steshenko · 02-08-2011, 12:02 PM

Quote:

Originally Posted by ejspeiro

This is cool, because I have a lot of links now to reference this! Thank you guys!

I also tried to allocate in the stack memory by doing this:

Code:

void AllocateFromStack(long int bytes) {

  char pool[bytes];

  printf("It seems that allocating from the stack worked!\n");
  pool[0] = 'a';
  printf("It worked! pool[0] = %c\n", pool[0]);
}

and then calling it from the main module. At first I thought that I shouldn't work but I was surprised by finding out that it did! Because I could successfully allocate in execution time

Now, should I be surprised? Is that true that pool, in this context is actually being allocated on the stack memory. I think so but I would like further insights!

Thanks!

It works because it's mandated by C99 standard: http://www.open-std.org/JTC1/SC22/wg...docs/n1124.pdf .

You better use 'unsigned' (and probably 'unsigned long') for 'bytes'. At all, unless you expect negatives values, use 'unsigned' - (signed) 'int' is actually rarely needed.

ejspeiro · 02-08-2011, 02:28 PM

Yeah but it works until an specific value of "bytes", besides, I'm still not sure if just by allocating pool inside a function like this:

Code:

int main(void) {

  pool2[0] = 'a';
  pool2[WARNING - 1] = 'z';
  printf("Apparently, if I print this, the allocation was successful...\n");
  printf("Some values: %c and %c.\n", pool2[0], pool2[WARNING - 1]);

  ...
}

actually causes for pool to be allocated in the stack memory.

What do you think? I know that this might be solved with me reading more about heap and stack memory, and I will, but still, I want some feedback.

johnsfine · 02-08-2011, 03:42 PM

Quote:

Originally Posted by ejspeiro

Yeah but it works until an specific value of "bytes"

In Linux, the maximum stack size is typically controlled by a setting in ulimit. It is often quite small (compared to what works for the size of a static or heap object).

Quote:

I'm still not sure if just by allocating pool inside a function like this:

actually causes for pool to be allocated in the stack memory.

I think you copied the wrong earlier example into that post.

I'm sure (at least for x86 and x86_64) the correct example (in post #5) does do stack allocation because I have stepped through equivalent code with a debugger and watched the stack allocation occur.

I'm pretty sure the C standard won't tell you that this type of allocation is on the stack, because even having a stack at all is an implementation detail. The C standard does not specify such implementation details.

I don't know if GCC documentation mentions that such variables are on the stack. It certainly doesn't mention its specific choice for most implementation details of the C standard.

ejspeiro · 02-08-2011, 05:36 PM

You're right! I copied the wrong code as example!

I meant the one that was quoted in post #5.

0k so, what about dynamic allocation within a function? It is of my understanding that every dynamic memory allocation takes resources FROM THE HEAP, but is this true even for functions? If I code this:

Code:

void AllocateFromStack(long int bytes) {

  char *pool;

  printf("  Called with %ld.\n", bytes);
  pool = (char *) malloc (bytes);
  if (pool != NULL) {
    printf("It seems that allocating from the stack worked!\n");
    pool[0] = 'a';
    printf("It worked! pool[0] = %c\n", pool[0]);
  } else {
    printf("It seems that allocating from the stack DIDN't work!\n");
  }
  free(pool);
}

is pool being allocated on the stack or in the heap?

johnsfine · 02-08-2011, 07:35 PM

Quote:

Originally Posted by ejspeiro

It is of my understanding that every dynamic memory allocation takes resources FROM THE HEAP, but is this true even for functions?

Yes.

Quote:

is pool being allocated on the stack or in the heap?

Heap.

More exactly, pool is a pointer, which is 4 or 8 bytes long (x86 or x86_64). That 4 or 8 bytes is from the stack. That pointer points to a chunk of bytes characters allocated from the heap.

paulsm4 · 02-08-2011, 07:51 PM

Hi, again -

As long as you're using GCC, you might find this of interest:

http://www.gnu.org/s/libc/manual/htm...llocation.html

As far as C/C++ in general, the "Storage duration specifiers" are:

Quote:

auto
http://en.wikipedia.org/wiki/Automatic_variable

automatic variable is a lexically-scoped variable which is allocated and de-allocated automatically when program flow enters and leaves the variable's scope. The term local variable is usually synonymous with automatic variable, since these are the same thing in many programming languages.

Automatic variables may be allocated in the stack frame of the procedure in which they are declared; this has the useful effect of allowing recursion and re-entrancy. (For efficiency, the optimizer will try to allocate some of these variables in processor registers.)

Quote:

static
http://en.wikipedia.org/wiki/Static_variable

A variable that has been allocated statically — whose lifetime extends across the entire run of the program. This is in contrast to the more ephemeral automatic variables (local variables), whose storage is allocated and deallocated on the call stack; and in contrast to objects whose storage is dynamically allocated.

Quote:

register
http://en.wikipedia.org/wiki/C_synta...ion_specifiers

The auto and register specifiers may only be used within functions and function argument declarations; as such, the auto specifier is always redundant.
...
Objects with automatic storage are local to the block in which they were declared and are discarded when the block is exited. Additionally, objects declared with the register storage class may be given higher priority by the compiler for access to registers; although they may not actually be stored in registers, objects with this storage class may not be used with the address-of (&) unary operator. Objects with static storage persist upon exit from the block in which they were declared. In this way, the same object can be accessed by a function across multiple calls. Objects with allocated storage duration are created and destroyed explicitly with malloc, free, and related functions.

Quote:

extern
http://en.wikipedia.org/wiki/C_synta...ion_specifiers

The extern storage class specifier indicates that the storage for an object has been defined elsewhere. When used inside a block, it indicates that the storage has been defined by a declaration outside of that block. When used outside of all blocks, it indicates that the storage has been defined outside of the compilation unit. The extern storage class specifier is redundant when used on a function declaration. It indicates that the declared function has been defined outside of the compilation unit.

ejspeiro · 02-14-2011, 06:39 PM

Thank you guys!

I learned a lot by reading and asking in this thread!

Regards!