LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 05-01-2011, 05:28 AM   #1
hydraMax
Member
 
Registered: Jul 2010
Location: Skynet
Distribution: Debian + Emacs
Posts: 467
Blog Entries: 60

Rep: Reputation: 51
Why the stack size limit?


Why is it in Linux that there is a stack size set by default? And why is it so small? (My system is set to 8192 kbytes.) And why is there a default limit on the stack size when the max memory and virtual memory size are, by default, unlimited? (Aren't they both fed from the same place ultimately?)

Reason I ask: I want to use recursive functions in my programming a lot more. Problem is, if the language (or implementation) doesn't happen to support tail-call recursion, then I can be pretty well certain that the first huge problem that gets thrown at my function is going to kill my program because the stack size limit is going to be quickly reached. Obviously, I can change the stack size limit for my own computers, but it doesn't feel so great knowing that most of the people who copy and execute my code will have probably have overlooked this.

Anyway, does anyone know: is this small default stack size limit just one of those historical artifacts, or is there some technical reason for it?
 
Click here to see the post LQ members have rated as the most helpful post in this thread.
Old 05-01-2011, 05:29 PM   #2
Hidden Windshield
Member
 
Registered: Jul 2010
Distribution: Fedora
Posts: 65

Rep: Reputation: 26
There has to be a set stack size, because the stack is stored contiguously in memory, and if it grows too big, it will start overwriting the heap. Increasing the stack size would involve moving the heap, which would invalidate every single heap pointer in the program (which, typically, would be the vast majority of pointers).

As for why it's so small, it's because the individual stack frames are small. For instance, on my system, gcc only uses 32 bytes per stack frame (plus whatever variables I have). I wrote a quick C program to test, and I got more than 300,000 recursions before it finally segfaulted.

My advice would be to go ahead and write your program and ignore the stack size. If you do run in to the stack size limit during testing, you can use the pthreads library to create a thread with a larger stack size, and do all your recursion in the thread.
 
2 members found this post helpful.
Old 05-01-2011, 08:55 PM   #3
hydraMax
Member
 
Registered: Jul 2010
Location: Skynet
Distribution: Debian + Emacs
Posts: 467
Blog Entries: 60

Original Poster
Rep: Reputation: 51
Quote:
Originally Posted by Hidden Windshield View Post
There has to be a set stack size, because the stack is stored contiguously in memory, and if it grows too big, it will start overwriting the heap. Increasing the stack size would involve moving the heap, which would invalidate every single heap pointer in the program (which, typically, would be the vast majority of pointers).

As for why it's so small, it's because the individual stack frames are small. For instance, on my system, gcc only uses 32 bytes per stack frame (plus whatever variables I have). I wrote a quick C program to test, and I got more than 300,000 recursions before it finally segfaulted.

My advice would be to go ahead and write your program and ignore the stack size. If you do run in to the stack size limit during testing, you can use the pthreads library to create a thread with a larger stack size, and do all your recursion in the thread.
Thank you for helpful response. However, some things are not quite clear to me: So, if I set stack limit x, does that meant x amount of memory is reserved for the stack of each process? If this is the case, what happens when I set the stack limit to unlimited? (My ulimit command allows me to do this.)

Code:
enigma ~ # ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 16002
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 16002
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
enigma ~ # ulimit -s unlimited
enigma ~ # ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 16002
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 16002
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
enigma ~ #
I tried this with a small C program implementing a slow recursive function: so far it has consumed 300M of memory and presumably will continue until I run out.

So are you saying that the stack size for a process is immutably fixed to an actual number (which doesn't seem to fit the above evidence) or that there is actually an internal stack limit but that it is adjusted when necessary (during runtime)?
 
Old 05-02-2011, 04:08 AM   #4
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1190Reputation: 1190Reputation: 1190Reputation: 1190Reputation: 1190Reputation: 1190Reputation: 1190Reputation: 1190Reputation: 1190
Quote:
Originally Posted by hydraMax View Post
why is there a default limit on the stack size when the max memory and virtual memory size are, by default, unlimited?
In a 32 bit system, user space virtual memory is more fundamentally limited to 3GB. Setting ulimit above 3GB or unlimited can't get around that constraint.

Quote:
Originally Posted by Hidden Windshield View Post
it's because the individual stack frames are small. For instance, on my system, gcc only uses 32 bytes per stack frame (plus whatever variables I have). I wrote a quick C program to test, and I got more than 300,000 recursions before it finally segfaulted.
One might have quite a lot of local variables and/or recurse deeper than 300,000. I frequently need a larger stack limit.

Quote:
Originally Posted by Hidden Windshield View Post
My advice would be to go ahead and write your program and ignore the stack size. If you do run in to the stack size limit during testing, you can use the pthreads library to create a thread with a larger stack size, and do all your recursion in the thread.
Yuck!
If your program is expected to use a lot of stack, provide a safe margin up front. Don't trust testing to hit the cases that real use wll hit.
I also dislike that pthreads idea.

Quote:
Originally Posted by hydraMax View Post
So, if I set stack limit x, does that meant x amount of memory is reserved for the stack of each process?
Each process has its own virtual memory space. A larger stack limit reserves virtual memory, not physical.
Quote:
If this is the case, what happens when I set the stack limit to unlimited?
I don't know.
 
1 members found this post helpful.
Old 05-02-2011, 09:07 PM   #5
Hidden Windshield
Member
 
Registered: Jul 2010
Distribution: Fedora
Posts: 65

Rep: Reputation: 26
Quote:
Originally Posted by johnsfine View Post
One might have quite a lot of local variables and/or recurse deeper than 300,000. I frequently need a larger stack limit.
I've never hit the stack size limit unless I had an infinite recursion bug (or I was trying to write a na´ve version of the Ackermann function). Of course, most of my projects are fairly small, for myself, which might have something to do with it.



Quote:
Originally Posted by johnsfine View Post
Yuck!
If your program is expected to use a lot of stack, provide a safe margin up front. Don't trust testing to hit the cases that real use wll hit.
I also dislike that pthreads idea.
The question is, how do you increase the margin? Have your program run ulimit, then re-exec itself? AFAIK, there's no way to change the stack size of a program once it's started, and no way for it to request more stack before it starts. (I suppose you could put "you must have x bytes of stack for this program" in the manual, but then your end users wouldn't read the manual. )
And I didn't like the pthreads idea either. I just couldn't think of another way of getting a bigger stack on an end-user's computer.



Quote:
Originally Posted by johnsfine View Post
Quote:
If this is the case, what happens when I set the stack limit to unlimited?
I don't know.
Ok, I guess it's been a while since I took computer architecture in college. I had no idea that it was possible to set an unlimited stack size. As I said earlier, the stack limit basically keeps the stack from overwriting the heap. It also keeps an infinite recursion from eating all your memory and crashing your system. I presume that setting an unlimited stack size just removes that protection.

Of course, how you're going to convince your end users to do that is another problem.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to set Stack/Heap Limit ? gloridel Linux - Kernel 1 06-11-2007 03:56 AM
Max Stack Size limit on Linux 2.4 rhegde Debian 0 08-08-2006 03:18 PM
Stack size decision atul_mehrotra Programming 2 10-08-2004 05:32 AM
Stack Size Decision atul_mehrotra Linux - General 0 10-07-2004 02:01 AM
Removing 4k stack size limit on 2.6.7-1.456 Corallis Fedora 2 07-23-2004 10:46 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 04:16 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration