What is the maximum allocatable heap size for a process?
 

I'm wondering what's the maximum heap space a process can use (not necessarily by a single malloc()) in Ubuntu x86_64. Which parameter determines the size?

Thanks.

Though I do not have a direct answer to your question but still you can look at the pmap command.

and IMO you should have posted this question in the programming forum instead of this Ubuntu forum as

I read your question as follows:
"What is the maximum allocatable heap size for a process in a 64 bit Linux OS ?"

Here Ubuntu or redhat does not matter I suppose .

pmap gives you information about the *current* memory usage of a process, but not about the *maximum possible* message usage.

I'll post this kind of questions in the programming forum later on.

Thanks for your reply and suggestion.

You might get a better answer if you explain why you want to know.

For most purposes, the answer is "bigger than the practical limits".

If you plan to actually use the heap space, you need your swap space plus a moderate fraction of your ram space to exceed the total heap allocation.

I'm not sure of the actual limits nor the parameters to tweak them, but I'm pretty sure that for any reasonable cost PC you won't have enough swap space to approach the limit before you exceed the practical limit.

If you want to allocate a lot of heap but not actually use it, so you want to be immune from the check that there is enough swap space, you can change the parameters controlling over commit checking.

Since no one is expected to hit the implemented limit before hitting a lower practical limit from swap size, I expect most Linux experts could not tell you the implemented limit.

The theoretical limit from the architecture would be nearly 128TB. Possibly, there is no Linux implementation detail cutting you short of that theoretical limit. So if you had 128TB of disk space and made it all into swap partitions, you could use 128TB of heap. Or maybe there are some limits on number of swap partitions and size of swap partitions that would leave you well short of 128TB even if you could attach that much disk space. I don't know those details.

I had tested the pmap command before posting the reply . I know it shows only the current memory usage !

Actually I had reported your thread to the moderators for being moved to the programming forum, but may be the moderators had different views so they didn't move it !

I waited for one day but no one replied to your question.

Then I thought mentioning the pmap command as a reply will give a bump to your thread and then may be some senior programmer will see it !

That all the story :)

Thanks, anishakaul. Your action was very considerate. :-)

johnsfine, thanks for your reply. What I'm trying to do is statically partition the entire heap space into multiple sub-heaps. So, I needed to know what's the maximum theoretical limit of heap space. How did you get the number "128TB"?

In X86_64 architecture, the top 17 bits of any pointer must be all copies of the same bit value (all ones if the pointer is in kernel space and all zeroes if the pointer is in user space).

So a user space pointer has its top 17 bits zero and its bottom 47 bits available to address 128TB of memory.

I expect that means you will be allocating virtual address space far in excess of the amount the swap file size would let you actually use.

To do that, you probably need to change vm.overcommit_memory to 1.
As root, execute the command
/sbin/sysctl vm.overcommit_memory=1

Most welcome !