LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 11-10-2003, 12:21 AM   #1
ymei
LQ Newbie
 
Registered: Sep 2003
Distribution: suse8.2
Posts: 7

Rep: Reputation: 0
segmentation fault when array size exceed 1GB


My OS is RH9.0
When I order an array whose size exceed 1GB, then It fails with segmentation fault.
How can I solve this problem?
 
Old 11-10-2003, 12:45 AM   #2
Palin
Member
 
Registered: Feb 2003
Location: A Meatlocker, well feels like one
Distribution: Gentoo
Posts: 292

Rep: Reputation: 30
What in the world could require one gig of memory? Which programming language and is there any way to optimize your code so you don't need to use so much memory?
 
Old 11-10-2003, 01:09 AM   #3
Stack
Member
 
Registered: Oct 2003
Distribution: FreeBSD
Posts: 325

Rep: Reputation: 30
Quote:
<antiroach> i wrote a program that recurses x number of times. when x is 25000 it core dumps. i ran a debugger and it said a stack overflow happened. is there any way i can increase stack space? or how else can i avoid running out of stack space?
<Sparr> antiroach: stop using the stack
<StoneCyph> antiroach: yes, you can increase stack space. how to prevent that problem? don't recurse to a depth of 5 digits.
<antiroach> how can i increase the stack space
<Kniht> antiroach: use an iterative solution instead
<antiroach> it has to be recursive
<antiroach> this algorithm finds a best move given a position.
<Kniht> no, it doesn't have to be
<antiroach> its like a game thing. yes it does. its part of the specs.
<Kniht> any recursive algorithm can be made iterative
<StoneCyph> antiroach: you're not listening. this is like saying "i built a car that goes fast. when i drive it into the wall at 500 miles per hour, it crumples. can i make it stronger?" "yes, but in general, stop driving into walls."
<[eloi]> StoneCyph: no, more like, "can i move the wall back a bit?"
Pretty similar to what you are asking... All in all you cant solve that problem and you need to work around it, unless you intend to expand your computers memory...
 
Old 11-10-2003, 02:29 AM   #4
worldmagic
Member
 
Registered: Oct 2003
Location: Europe/Sweden
Distribution: RedHat
Posts: 78

Rep: Reputation: 15
Actualy I think this is a valid questions, maybe he do have a program that will use that amount of data? I myself been playing with terraforming rutins, and physics engiens for these worlds.. The worlds themself are much containted on harddrive and swapped up and such, but the "current state" data is huge to, and If I would ever continue my work, its not unreasonble to exspect to use 1Gig of memory for these calculations. Recursive calls kills the stack, yes it does, yes thats a wall you should avoid. But if you have 2Gig in your machine, or 512Mb and 4Gig swap... Why shouldnt you be able to do this: struct stateData[1000000000] currentState.. ?
 
Old 11-10-2003, 03:21 AM   #5
ymei
LQ Newbie
 
Registered: Sep 2003
Distribution: suse8.2
Posts: 7

Original Poster
Rep: Reputation: 0
I use Fortran for scientific computing. My PC has 2 giga bytes memory.

Quote:
Originally posted by Palin
What in the world could require one gig of memory? Which programming language and is there any way to optimize your code so you don't need to use so much memory?
 
Old 11-10-2003, 03:28 AM   #6
ymei
LQ Newbie
 
Registered: Sep 2003
Distribution: suse8.2
Posts: 7

Original Poster
Rep: Reputation: 0
Sorry, It is my fault. I should have told you I have 2 giga bytes memory fixed in my PC. And if I reduce the size of that array to 800MB, I can run the program with 2 threads at the same time, which cost 1.5GB memory.
And also I can change the stacksize to unlimited, but it still refuse to work.

Quote:
Originally posted by Stack
Pretty similar to what you are asking... All in all you cant solve that problem and you need to work around it, unless you intend to expand your computers memory...
 
Old 11-10-2003, 05:55 AM   #7
MartinN
Member
 
Registered: Nov 2003
Location: Ronneby, Sweden
Posts: 555

Rep: Reputation: 30
Could you try doing the same in a short C program to see if that fails too?

Martin
 
Old 11-10-2003, 05:59 AM   #8
ymei
LQ Newbie
 
Registered: Sep 2003
Distribution: suse8.2
Posts: 7

Original Poster
Rep: Reputation: 0
I don't think it will be easy to translate the code.

Quote:
Originally posted by MartinN
Could you try doing the same in a short C program to see if that fails too?

Martin
 
Old 11-10-2003, 06:18 AM   #9
MartinN
Member
 
Registered: Nov 2003
Location: Ronneby, Sweden
Posts: 555

Rep: Reputation: 30
I take it your allocation is more complicated than:

char *i = new char[1000000000];

I'm not familiar with fortran, is it possible to allocate memory both on the stack and on the heap? The above C example is stack allocation, this is heap allocation:

char i[100000000];

Couldn't you try to do just the allocation part without the rest of the program logic? I'm sorry I can't do it myself. My bad excuse for a computer has only 192 MB

Regards
Martin
 
Old 11-10-2003, 09:35 AM   #10
jim mcnamara
Member
 
Registered: May 2002
Posts: 964

Rep: Reputation: 34
We don't know if your FORTRAN compiler allocates arrays on the stack on in heap memory. There is a BIG difference. Allocating to heap is the way to go for large arrays.

Are there compiler switches that change how memory allocates for variables? ie., make the array static (heap), not volatile.
 
Old 11-10-2003, 01:35 PM   #11
jim mcnamara
Member
 
Registered: May 2002
Posts: 964

Rep: Reputation: 34
PS: some FORTRAN compilers used to have support for virtual arrays - ie., arrays kept mostly on disk and paged into/out of memory as needed. I don't know if your FORTRAN compiler does that, but it's worth investigating.
 
Old 11-11-2003, 06:18 AM   #12
nowonmai
Member
 
Registered: Jun 2003
Posts: 481

Rep: Reputation: 48
Quote:
Originally posted by MartinN
char *i = new char[1000000000];

The above C example is stack allocation
the above C example is C++
 
Old 11-11-2003, 08:44 AM   #13
Misel
Member
 
Registered: Mar 2003
Location: Berlin
Distribution: Slackware current
Posts: 310

Rep: Reputation: 30
Quote:
Originally posted by nowonmai
the above C example is C++
care to explain to a C newbie?
 
Old 11-11-2003, 09:02 AM   #14
jim mcnamara
Member
 
Registered: May 2002
Posts: 964

Rep: Reputation: 34
The new keyword is not a C keyword, it is C++

in C it would be:
Code:
char *i;
i=(char *) malloc(1000000000);
............. play with data here

free(i);  /* release memory */
 
Old 11-11-2003, 10:27 AM   #15
MartinN
Member
 
Registered: Nov 2003
Location: Ronneby, Sweden
Posts: 555

Rep: Reputation: 30
Quote:
Originally posted by nowonmai
the above C example is C++
Whoops Sorry. I think it's pretty obvious what my language of choice is now...

I took the liberty to test this a little. This code snippet (which is C++) shows what happens on my machine:
Code:
try
  {
  char *i = new char[1000000000]; // 1GB
  }
catch(bad_alloc)
  {

    cout << "Caught a bad alloc exception" << endl;

  }
Without the try block, the program exits with a core dump. Otherwise the exception line is printed.

One interesting thing to note is that I can easily allocate 300 MB 3 times, like this:
Code:
char *i = new char[300000000]; // 300 MB
char *j = new char[300000000]; 
char *k = new char[300000000];
The memory is never really allocated as physical memory until I start using it. Every memory request is checked to see if it can be satisfied, but the allocator routine doesn't know about other requests that haven't been used.

Martin
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
yast segmentation fault, system freezing - nvidia driver at fault? BaltikaTroika Suse/Novell 2 12-02-2005 09:34 AM
Segmentation fault after declaring a large array. oulevon Programming 6 11-08-2005 02:41 AM
segmentation fault vibhory2j Programming 7 10-16-2005 07:06 AM
PERL: Size of an array of an Array inspleak Programming 2 03-10-2004 02:24 PM
su = segmentation fault kud0ze General 5 02-21-2002 08:19 PM


All times are GMT -5. The time now is 04:58 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration