Is there some way to reduce memory cost of threads

Fedora15 32bit.

I write a test program, it creates new thread continually, the thread does nothing but sleep.

I find virtual memory increases up almost 10Mb when a new thread is created. and when there's more than 200 threads, the virtual memory used by the program is 3Gb, and now cann't create new thread.

but on windows, it costs little memory.

What can I do to config the operation system to take less memory on threads?

thanks.

Hi,

The trick is to keep track of your threads. Dont expect Linux to wave a magic wand and poof them away... because, well, when is a thread "not needed"???

In the "other one", it is possible to keep stuffing the thing until you overwrite something. Did that once, and got a "boot error" - and a dead os...but, those were bad times (I had to develop for the "thing)...

You need to find a way to let the main/calling system know what thread can be killed and get it to kill that thread too...

Linux works for you, it does not persé decide what has to be done, it is your system, you're not its appendix...

Thor

Originally Posted by cplus I write a test program...

It would be nice to see this test program, namely to see which programming language you are using. Also, what tool are you using to measure the amount of (virtual) memory being consumed?

If you are developing in C, then the successful completion of a thread should be followed by a call to pthread_join(). That should release any resources used by the thread.

Originally Posted by dwhitney67 It would be nice to see this test program, namely to see which programming language you are using. Also, what tool are you using to measure the amount of (virtual) memory being consumed? If you are developing in C, then the successful completion of a thread should be followed by a call to pthread_join(). That should release any resources used by the thread.

I use c++, Qt framework. use QThread to create thread.

I found that after ulimit -s 1024, it will take less memory.

that's it is, ulimit -s 1024 set the stack size, each thread has it own stack.

Originally Posted by cplus that's it is, ulimit -s 1024 set the stack size, each thread has it own stack.

QThread::setStackSize(). You may also want to look at QtConcurrent and QFuture. Qt has very good documentation, so use it.

Originally Posted by cplus I use c++, Qt framework. use QThread to create thread. I found that after ulimit -s 1024, it will take less memory.

Why not skip the Qt threads and just use pthreads? It seems like the Qt thread class is meant for writing code that isn't limited to *nix operating systems. You can also look at man 2 clone for a more "customized" version if you really want to get your hands dirty.
Kevin Barry

Sure... "don't create 200 threads."

Stop and consider how many hamburgers a group of about eight people can serve during a typical lunch hour, and how many hundreds of people can be placing those orders. Yet, "everything works," because the workflow (i.e. burgers) is not related to the number of workers who are doing the job.

If fast-food restaurants worked like too-many web sites do, the scenario would be like this:

1. You walk into the store.
2. The owner of the store pulls out a red box and pours a drop of water on it.
3. "Just Add Water!" A new employee magically appears!!
4. The employee takes your order, then walks into a crowded kitchen full of similar employees and starts slugging it out, trying to get your order ready.
5. The employee hands you your burger, then drops dead on the spot.

Originally Posted by SigTerm QThread::setStackSize(). You may also want to look at QtConcurrent and QFuture. Qt has very good documentation, so use it.

Thanks, you let me know stacksize can be set by program it self.

Originally Posted by ta0kira Why not skip the Qt threads and just use pthreads? It seems like the Qt thread class is meant for writing code that isn't limited to *nix operating systems. You can also look at man 2 clone for a more "customized" version if you really want to get your hands dirty. Kevin Barry

I use QThread for the test program, in fact, our project use pthread_create.

Originally Posted by sundialsvcs Sure... "don't create 200 threads." Stop and consider how many hamburgers a group of about eight people can serve during a typical lunch hour, and how many hundreds of people can be placing those orders. Yet, "everything works," because the workflow (i.e. burgers) is not related to the number of workers who are doing the job. If fast-food restaurants worked like too-many web sites do, the scenario would be like this: You walk into the store. The owner of the store pulls out a red box and pours a drop of water on it. "Just Add Water!" A new employee magically appears!! The employee takes your order, then walks into a crowded kitchen full of similar employees and starts slugging it out, trying to get your order ready. The employee hands you your burger, then drops dead on the spot.

thanks, sundialsvcs. your example is very lively.

our program is a realtime network program, it works well on Windows for many years, these days we migrated it to Linux, and get the problem.

We solved it by setting stack size of Linux at last. and we needn't to change our code.

Windows uses an entirely different thread-management protocol, and manages its affairs very differently especially with regards to memory. (Read some docs in the Wine project for a discussion of some of those.)

IMHO, your "solution" is at best a temporary workaround and management needs to know that redesign of this aspect of the application needs to become a high-priority budget item. The project does not have to be unmercifully costly, and it will reap significant performance and therefore business benefits. (The same reasoning would greatly improve the Windows application, too, if it is also still in service.)

On either system, this design is over-committing the system and doing so without adequate controls, opening the way e.g. to a denial of service. Linux/Unix is simply making you pay the price more visibly.