LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 09-15-2012, 02:37 PM   #1
mfkraft
LQ Newbie
 
Registered: Sep 2012
Location: USA
Distribution: SuSE, RedHat
Posts: 2

Rep: Reputation: Disabled
Lightbulb Virtual Memory limit exhausted at 1TB? malloc/mmap failures even with free memory


Curious,

even with ulimit for virtual mem set to unlimited or 0, I get malloc failures after about 1 TB virtual is exhausted, with malloc size > 128kB that should do a private mmap. Disk-supported mmap still seem to work beyond one terabyte.

Anybody encountered similar? Could it be my testing distro is a "desktop" instead of an enterprise version? Many thanks.

SuSE Linux Kernel 2.6.37.6-0.20-desktop #1 SMP PREEMPT 2011-12-19 23:39:38 +0100 x86_64 x86_64 x86_64 GNU/Linux

24 GB RAM + 36 GB Swap, 1020 GB can be allocated (of course not used). mmap against some 1TB+ disks in raid, malloc fails with ENOMEM after approx 1TB with 0% swap and 30% ram real usage.

Just wondering if anybody has some ideas.

I thought the virtual limit to be 128TB, sometimes 64TB, on Intel distros, not just 1TB.....

Thanks and Cheers, Martin
 
Old 09-15-2012, 05:23 PM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,103

Rep: Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117
I would be suspecting something like control block shortage. Keep an eye on /proc/meminfo and /proc/slabinfo ("cat" them to disk for later perusal).
There is also slabtop which can be handy in "normal" storage shortages, but don't use it when the system (i.e. kernel controlled) storage is in crisis. Your system might just appear to "go away" - maybe for several hours.
 
1 members found this post helpful.
Old 09-16-2012, 08:27 AM   #3
mfkraft
LQ Newbie
 
Registered: Sep 2012
Location: USA
Distribution: SuSE, RedHat
Posts: 2

Original Poster
Rep: Reputation: Disabled
Cool

Thanks, in this case the mem chunks are quite big since I increased them beyond reasonable sizes, so to eliminate control structure shortages.

And found the solution: turn vm overcommit checks completely off, apparently the default heuristics stop at the reasonable size of 1 TB:

Code:
echo 1 > /proc/sys/vm/overcommit_memory
Now I can overcommit well beyond 1 TB, such that TOP displays a queston mark, is that a bug maybe, VIRT=???

TOP output
Code:
top - 09:13:06 up  1:00,  3 users,  load average: 10.34, 7.08, 5.32
Tasks: 191 total,   1 running, 190 sleeping,   0 stopped,   0 zombie
Cpu(s): 99.0%us,  0.4%sy,  0.0%ni,  0.6%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:     24152M total,    15088M used,     9064M free,       34M buffers
Swap:    36863M total,        0M used,    36863M free,      513M cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 6453 martin    20   0     ?  11g 2392 S 1190 50.4  61:41.11 mchess
Nevertheless, immediate problem solved.

Cheers, Martin
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
ffmpeg: make: virtual memory exhausted simopal6 Linux - Software 15 05-30-2011 03:31 PM
Application Virtual address space memory allocation - memory does not get free chamara82 Linux - General 4 01-01-2011 08:19 PM
In Linux, apart from malloc() what can increase the virtual memory of a process? gmwebway Programming 3 10-27-2010 09:37 AM
Can't malloc all free physical memory, CommitLimit too low FEWP Programming 2 07-23-2009 12:56 PM
not 'free'ing 'malloc'ed memory Hammo Programming 9 03-03-2004 07:33 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 06:26 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration