Linux Multi-heap Multi-thread application

We are porting a multi-process application to multi-threaded architecture. We have the same application running on Windows and it is very performant.
For Linux we are using the pthread libraries. In order to avoid memory contention we have custom heaps, each thread having its own heap. We are using mspace for this custom heap implemention. However the issue is that this approach is causing a lot of performance overhead. The mspace memory allocations are very slow as compared to the native malloc. This is the bottleneck. We tried the Hoard allocator but that is much worse.

Is there any alternative to acheive multi-heap which is also performant?

I think you can reach the best performannce without any custom heap handler. You can try valgrind to catch memory handling problem.

Looking through http://gee.cs.oswego.edu/pub/misc/malloc.c, my intuition is that your library is compiled with USE_LOCKS:

USE_LOCKS default: 0 (false) Causes each call to each public routine to be surrounded with pthread or WIN32 mutex lock/unlock. (If set true, this can be overridden on a per-mspace basis for mspace versions.) If set to a non-zero value other than 1, locks are used, but their implementation is left out, so lock functions must be supplied manually, as described below.

Originally Posted by pan64 I think you can reach the best performannce without any custom heap handler. You can try valgrind to catch memory handling problem.

Thanks for the reply. However it is not possible to use a common heap since we have lots & lots - in terms of millions per hour - allocations and a single heap is getting fragmented and very less performant.
Also I want to make sure each thread has a separate chunk of memory to avoid leak/overrun since the threads can run custom code and plugins.
Any suggestions?

Originally Posted by sundialsvcs Looking through http://gee.cs.oswego.edu/pub/misc/malloc.c, my intuition is that your library is compiled with USE_LOCKS:

Thanks for the reply. This is a really good option. However I did explore this earlier and don't see any improvements

How is it that Linux doesn't have native multi heap implementation? When pthread supports all if not more features of Windows multi-thread, it should also have supported multiple heap concept...

Hi all,

Are there any other linux custom heap allocators? I have tried mspace & hoard.

Do you think the issue with mspace is that it runs in user-mode and not in kernel-mode?

This is a generic problem for any huge application. Are there any huge applications that is multi-threaded in linux? Or is it that all are multi-process?

Does anybody else have same issue? If so, I think we should put this feature into linux kernel itself...

Please do provide inputs.

probably you need clone? http://www.linuxforu.com/2011/08/lig...linux-threads/

Originally Posted by pan64 probably you need clone? http://www.linuxforu.com/2011/08/lig...linux-threads/

Thanks for suggestion. Do you mean use "clone" APIs instead of "pthread"? I do not think this is the bottle neck. I think the issue is Linux lacking multi-heap APIs and mspace not being performant in user mode under heavy load...

Hello everyone,
I am sure multi heap is a requirement for using multi threaded applications in Linux. Any huge application will end up getting its memory fragmented and insecure if it lets same heap for all threads and plugins.

Please share your valuable inputs.

Is this under requirements for Linux kernel?
Is there any good algorithm for multi heap in linux?