Declare large arrays in C

Thodoris21 · 05-27-2012, 07:13 AM

Do you know how to declare large int arrays?
For example i want to declare a 2d array.I declare it as int A[4096][4096] but when the program starts it shows me segmentation fault.I think with malloc i can do it but i'm not sure.

dwhitney67 · 05-27-2012, 07:32 AM

When you declared the 2d array on the stack, you most likely exceeded the limits imposed by your system. You can check the stack size limit by issuing the command 'ulimit -s'. On my system, it is set to 8192 Kbytes. Your array far exceeds this size (btw, I believe your array has a size of 65536 Kbytes).

If you want to allocate from the heap this large array, then use something like:

Code:

#include <stdlib.h>
...
const int ROWS = 4096;
const int COLS = 4096;

int** array = malloc(ROWS * sizeof(int*));    // allocate the rows

for (int r = 0; r < ROWS; ++r)
{
    array[r] = malloc(COLS * sizeof(int));    // allocate the columns
}
...
array[10][50] = 10;    // insert data at row 10, column 50
...
for (int r = 0; r < ROWS; ++r)
{
    free(array[r]);    // this frees the columns
}
free(array);    // this frees the rows

An alternative is to declare a 1d array, and then programmatically manage access to the 1d array as if it were a 2d array; for example:

Code:


	Code:
	#include <stdlib.h>

...
const int ROWS = 4096
const int COLS = 4096;

int* array = malloc(ROWS * COLS * sizeof(int));
...
array[10 * ROWS + 50] = 10;    // insert data at row 10, column 50
...
free(array);    // frees allocated memory

Thodoris21 · 05-27-2012, 07:52 AM

I take segmentation fault again!

dwhitney67 · 05-27-2012, 07:54 AM

Quote:

Originally Posted by Thodoris21

I take segmentation fault again!

You will need to post your code (within CODE tags!) to get additional help. Your comment above is of no help.

Thodoris21 · 05-27-2012, 08:00 AM

Ok this is my code

Code:

#define dim1 4096
#define dim2 4096

int main (int argc, char *argv[]) {
       int prevkMap[dim1][dim2],num,i,j;
       for (i=0; i<dim1; i++) {
   	  for (j=0; j<dim2; j++) {
		num=rand() % 6;
		prevkMap[i][j]=num;			
          }
       }
return 0;
}

When dim1 and dim2 is 512 it runs without problem.With values above 512 there is problem...

dwhitney67 · 05-27-2012, 08:02 AM

Quote:

Originally Posted by Thodoris21

When dim1 and dim2 is 512 it runs without problem.With values above 512 there is problem...

Again, from my previous post...

Quote:

When you declared the 2d array on the stack, you most likely exceeded the limits imposed by your system. You can check the stack size limit by issuing the command 'ulimit -s'. On my system, it is set to 8192 Kbytes. Your array far exceeds this size (btw, I believe your array has a size of 65536 Kbytes).

Thodoris21 · 05-27-2012, 08:04 AM

And on my system, stack size limit is set to 8192 Kbytes.I did it using malloc but i take the same message.

dwhitney67 · 05-27-2012, 08:13 AM

This works fine for me:

Code:

#include <stdlib.h>
#include <time.h>

int main()
{
    const int ROWS = 4096;
    const int COLS = 4096;

    int** prevkMap = malloc(ROWS * sizeof(int*));

    for (int r = 0; r < ROWS; ++r)
    {
        prevkMap[r] = malloc(COLS * sizeof(int));
    }

    srand(time(NULL));

    for (int r = 0; r < ROWS; ++r)
    {
        for (int c = 0; c < COLS; ++c)
        {
            prevkMap[r][c] = rand() % 6;
        }
    }

    // TODO: Free allocated memory.

    return 0;
}

Thodoris21 · 05-27-2012, 08:23 AM

Nothing again!What is going on?I run Ubuntu 12.04 on Vmware through windows 7.

Thodoris21 · 05-27-2012, 08:38 AM

Ok i fix it.It was my mistake!Thanks!

johnsfine · 05-27-2012, 08:38 AM

Quote:

Originally Posted by Thodoris21

Do you know how to declare large int arrays?
For example i want to declare a 2d array.I declare it as int A[4096][4096] but when the program starts it shows me segmentation fault.I think with malloc i can do it but i'm not sure.

You could do it with malloc, but if you want to use the stack you can do so.

You just need a little extra code to allow use of an excessive size stack. The following program adjusts its own stack limit to 1MB greater than what is needed for a 4Kx4K int array, then it calls a function that uses that 4Kx4K array.

Code:

#include <sys/resource.h>

void stack_consumer()
{
    int A[4096][4096];
    A[0][0] = 1;
    A[4095][4095] = 1;
    printf ("A was allocated at address %x\n", A );
}

int main (int argc, char **argv)
{
    const rlim_t StackSize = sizeof(int) * 4096 * 4096 + 1048576;
    struct rlimit rl;
    int result;

    result = getrlimit(RLIMIT_STACK, &rl);
    if (result == 0)
    {
        printf("Changing stack limit from %d to %d\n", rl.rlim_cur, StackSize );
        rl.rlim_cur = StackSize;
        result = setrlimit(RLIMIT_STACK, &rl);
    }
    stack_consumer();
    return 0;
}

Thodoris21 · 05-27-2012, 08:48 AM

Thank you very much!