Segmentation Fault

felipebil · 11-18-2008, 04:53 PM

Im with a problem to run a C program, maybe you help me to find the solution.
Its a simple code, but uses a lot of memory. One matrix for example have 10 million elements. The program runs normally when i reduce the size of some of that matrices, but in the wanted size doesnt work. As it run sometimes, i think the problem is memory.
I have used:

% ulimit -s unlimited

and it appears to help a little, but didnt solve the problem. Had also tried to use malloc and free to free some memory, but had no effect. I know that much bigger programs run in this computer, so i really need help because i dont know what to do.

below the variables declaration:

Code:

#define T_MIN 5000
#define T_MAX 15000
#define E_MIN 0.0
#define E_MAX 0.1
#define E_NUM 100
#define N_MAPS 1000

int main(void){

int tempo, map, e, i, start, end;
int y_maxtab[N_MAPS][T_MAX/100], k[N_MAPS], n_ymax[N_MAPS];
double eps, coup, step = (double)(E_MAX-E_MIN)/(double)(E_NUM-1);
double mean, new_x;
double alfa[N_MAPS], x0[N_MAPS], y0[N_MAPS];
double x_dat[N_MAPS], y_dat[N_MAPS][T_MAX];
double rnmed_tab[2][E_NUM], rn_tab[T_MAX];
double mcos, msin, fi,deno;

the y_dat matrix is the biggest here.

p.s.: normally i use icc to compile my programs, but in gcc i get the same errors. i had also tried to use gdb, but didnt discover nothing.

Quakeboy02 · 11-18-2008, 05:03 PM

Is your RAM + swap space large enough for the total memory footprint of the program?

chrism01 · 11-18-2008, 05:29 PM

IME segment violation is usually caused by writing off the end of an array. Are you absolutely sure that isn't the issue here, especially as you claim you have run bigger programs than this (I assume you mean progs with bigger array declarations)

paulsm4 · 11-18-2008, 06:05 PM

I believe you're probably encountering stack overflow (in other words, I believe the "segmentation violation" is actually "stack too small", and I believe that "ulimit -s unlimited" is in fact *very* limited...)

Here are a couple of links that might help:
http://cs.nyu.edu/exact/core/doc/stackOverflow.txt
http://ubuntuforums.org/archive/index.php/t-114071.html
http://linux.about.com/library/cmd/b..._setrlimit.htm

graemef · 11-18-2008, 06:11 PM

Check these links http://www.linuxquestions.org/questi...-fault-672145/ or http://www.linuxquestions.org/questi...ze-in-c-673742, they may assist.

The_Nerd · 11-18-2008, 10:39 PM

I believe paulsm4 is right. When you statically allocate memory for an array (as you are doing), it gets "allocated" (per se) onto the stack. What you are doing isn't really a very good thing todo. Try dynamically allocating your memory via new, delete or malloc, free. Either way, make sure the memory allocated actually gets allocated, for example, DON'T do this:

Code:

int *ptr = malloc(huge_amt);
/* Bad! */
p[0] = 1;

This is what you should do.

Code:

int *ptr = malloc(huge_amt);
assert(ptr == NULL); //check for error
p[0] = 1;

I would like to add as a final note, that if you allocate outside your RAM, you will start using swap space, which is very slow. If you allocate outside of both, all bets are off (though malloc should fail).

ta0kira · 11-19-2008, 12:16 AM

Quote:

Originally Posted by paulsm4

I believe you're probably encountering stack overflow (in other words, I believe the "segmentation violation" is actually "stack too small", and I believe that "ulimit -s unlimited" is in fact *very* limited...)

Yes, I was about to say that.

This looks like a major abuse of the stack. You should either make all of the arrays static or malloc them. How long is the program? Have you tried gdbing it?
ta0kira

ta0kira · 11-19-2008, 12:32 AM

Quote:

Originally Posted by The_Nerd

I believe paulsm4 is right. When you statically allocate memory for an array (as you are doing), it gets "allocated" (per se) onto the stack. What you are doing isn't really a very good thing todo.

Careful with how you use the word static! You're going to confuse people. Though semantically accurate, "statically allocate" sounds suspiciously like "static data". I would have said "the allocation is static" because "statically allocated" implies allocation in the static data area.
ta0kira

felipebil · 11-19-2008, 08:53 PM

Well, at least i'm understanding what is the problem.
But I'm a little bit confuse with the many kinds of memory.
ram, stack, heap, swap, buffer...

in my case, i think that occurred a stack overflow, despite a have no idea of why it happened. I make a simple calculation, and estimate the footprint memory of the program in no more than 200MB.
And as you can see below, the result of the free -lm (my job wasnt running at this time):

Code:

   
               total       used       free     shared    buffers     cached
Mem:          2028MB      600MB     1427MB        0MB        0MB       75MB
Low:           876MB        7MB      869MB
High:         1151MB      592MB      558MB
-/+ buffers/cache:        525MB     1503MB
Swap:            0MB        0MB        0MB

so how can i know the limit of the stack memory when i define "ulimit -s unlimited"? and what is the relation of this with the ram? if i use malloc to allocate memory for my matrices anything in the arrange of the memory changes?

thanks a lot

The_Nerd · 11-19-2008, 09:05 PM

Quote:

Originally Posted by felipebil

Well, at least i'm understanding what is the problem.
But I'm a little bit confuse with the many kinds of memory.
ram, stack, heap, swap, buffer...

in my case, i think that occurred a stack overflow, despite a have no idea of why it happened. I make a simple calculation, and estimate the footprint memory of the program in no more than 200MB.
And as you can see below, the result of the free -lm (my job wasnt running at this time):

Code:

   
               total       used       free     shared    buffers     cached
Mem:          2028MB      600MB     1427MB        0MB        0MB       75MB
Low:           876MB        7MB      869MB
High:         1151MB      592MB      558MB
-/+ buffers/cache:        525MB     1503MB
Swap:            0MB        0MB        0MB

so how can i know the limit of the stack memory when i define "ulimit -s unlimited"? and what is the relation of this with the ram? if i use malloc to allocate memory for my matrices anything in the arrange of the memory changes?

thanks a lot

It is simple, when you write a program and compile it, the compiler allocates a heap of memory, a memory block if you will, this is where all the program variables, statically allocated arrays, etc... are put. This is also used as a memory space for variable creation, function calling, etc...

When you dynamically allocate memory via a function called at runtime, the memory is "given to you" from where-ever the OS can find it. Usually it will look fist for space in the RAM, if ALL of RAM is full, the OS will start using a space on the hard-drive called "swap space". This slows down memory operations considerably as the hard-drive is WAY slower than RAM.

So to sum it up:

Code:

/* Statically allocated (basically, this means the compiler allocates space for it on the stack) */
int MyStaticArray[100];
/* Dynamic memory allocation */
int *p = malloc(100 * sizeof(int));
/* Now RAM is full (hypothetically), so now the OS will give you space on the hard-drive. This is low level, so to you it works the same as memory in RAM... it is just way slower. */
int *hard_drive_alloc = malloc(100 * sizeof(int));
/*... error testing...*/
free(p);
free(hard_drive_alloc);

Hope that clears things up for you.

ErV · 11-19-2008, 10:01 PM

Quote:

Originally Posted by felipebil

in my case, i think that occurred a stack overflow, despite a have no idea of why it happened. I make a simple calculation, and estimate the footprint memory of the program in no more than 200MB.

There was another thread about same problem. The data allocated on stack will cause stack overflow, if data size is above 8megabytes. Not sure how ulimit affects that.
Solutions:
1) Make large variables (defined within functions) static.
or
2) Make large variables global. I.e. don't put declaration of anything huge within function call, unless it is static.
or
3) Allocate arrays using malloc.