Hello,
I've some memory allocation problems on my embedded Linux system
I think they are linked with the overcommit_memory setting
sysctl vm.overcommit_memory=2
Which was set in order to get a deterministic behavior of the system.
My problem is that all my malloc failed after some system operation even if there is still free memory.
If I test the max size of RAM that can be "malloced" by an external custom tools launched in commandline, I got 12MB (1k blocks).
I did some tests with a piece of code integrated in my application:
- memory allocation of a single 100KB block: only 0.1 MB more used
- thread creation just after malloc (just a while(1) {usleep(); } thread) : +15MB !!!
So I suspect a huge quantity of memory allocation done by glibc when a thread is created since stack is only of 1MB.
I read that Glibc could allocate a lot of memory as memory pools for thread ( something like that in
https://udrepper.livejournal.com/20948.html).
What do you think about this ?
How can I reduce the amount of "shadow" allocated memory ?
Thanks in advance !
-----------------------------------------
Application langage: C++ using STL and BOOST
Glibc version: 2.13
Kernel: Linux 2.6.39 #4 SMP PREEMPT Fri Apr 10 17:10:02 CEST 2015 i686 GNU/Linux
Target: Atom 1 core with hyperthreading enabled
top command (in bold lines: the process composing my application)
Code:
Mem: 508336k total, 139472k used, 368864k free, 1436k buffers
Swap: 0k total, 0k used, 0k free, 36808k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
19247 root 20 0 2484 1224 1020 S 25 0.2 0:08.48 net_disconnect_
3062 root 20 0 2340 956 780 R 2 0.2 0:00.01 top
19988 root -56 0 51728 36m 1264 S 2 7.4 0:00.62 analyse_and_cal
20074 root -56 0 63292 14m 11m S 2 2.9 0:02.16 system_loader
1 root 20 0 1824 88 20 S 0 0.0 0:00.97 init
2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd
3 root 20 0 0 0 0 S 0 0.0 0:00.01 ksoftirqd/0
4 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/0:0
5 root 20 0 0 0 0 S 0 0.0 0:00.01 kworker/u:0
6 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/0
7 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/1
8 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/1:0
9 root 20 0 0 0 0 S 0 0.0 0:00.00 ksoftirqd/1
10 root 20 0 0 0 0 S 0 0.0 0:04.63 kworker/0:1
11 root 0 -20 0 0 0 S 0 0.0 0:00.00 khelper
166 root 20 0 0 0 0 S 0 0.0 0:00.03 sync_supers
168 root 20 0 0 0 0 S 0 0.0 0:00.00 bdi-default
170 root 0 -20 0 0 0 S 0 0.0 0:00.00 kblockd
249 root 0 -20 0 0 0 S 0 0.0 0:00.00 ata_sff
257 root 20 0 0 0 0 S 0 0.0 0:00.01 khubd
362 root 0 -20 0 0 0 S 0 0.0 0:00.00 rpciod
363 root 20 0 0 0 0 S 0 0.0 0:03.05 kworker/1:1
380 root 20 0 0 0 0 S 0 0.0 0:01.75 kswapd0
381 root 20 0 0 0 0 S 0 0.0 0:00.00 fsnotify_mark
382 root 0 -20 0 0 0 S 0 0.0 0:00.00 nfsiod
404 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/u:2
430 root 20 0 0 0 0 S 0 0.0 0:00.00 scsi_eh_0
433 root 20 0 0 0 0 S 0 0.0 0:00.01 scsi_eh_1
511 root 0 -20 0 0 0 S 0 0.0 0:00.00 ttm_swap
548 root 20 0 0 0 0 S 0 0.0 0:00.06 kjournald
555 root 20 0 0 0 0 S 0 0.0 0:00.00 kjournald
571 root 16 -4 2036 304 0 S 0 0.1 0:00.08 udevd
637 root 18 -2 2048 304 0 S 0 0.1 0:00.00 udevd
638 root 18 -2 2048 304 0 S 0 0.1 0:00.00 udevd
654 root 0 -20 0 0 0 S 0 0.0 0:00.00 intr_work
660 root 20 0 2072 284 180 S 0 0.1 0:01.09 syslogd
695 root 20 0 3776 1448 0 S 0 0.3 0:00.00 dhcpd
700 root 20 0 2116 76 0 S 0 0.0 0:00.00 vsftpd
706 root 20 0 4040 268 0 S 0 0.1 0:00.00 sshd
739 root 20 0 2484 216 0 S 0 0.0 0:00.04 S29fyinstalld.b
761 root 20 0 1888 144 40 S 0 0.0 0:00.01 cron
786 root 20 0 2616 812 500 S 0 0.2 0:04.40 fyinstaller.bas
1106 root 20 0 1816 52 0 S 0 0.0 0:00.00 getty
1107 root 20 0 1816 56 0 S 0 0.0 0:00.00 getty
1108 root 20 0 1816 56 0 S 0 0.0 0:00.00 getty
1110 root 20 0 1816 52 0 S 0 0.0 0:00.00 getty
1111 root 20 0 1816 56 0 S 0 0.0 0:00.00 getty
1112 root 20 0 1816 48 0 S 0 0.0 0:00.00 getty
1113 root 20 0 1816 52 0 S 0 0.0 0:00.00 getty
2401 root 20 0 1680 368 324 S 0 0.1 0:00.00 sleep
3045 root 20 0 1680 376 324 S 0 0.1 0:00.00 sleep
3773 root 20 0 6736 944 428 S 0 0.2 0:14.38 sshd
3869 root 20 0 6600 488 0 S 0 0.1 0:00.11 sshd
3883 root 20 0 2536 812 600 S 0 0.2 0:00.34 bash
3972 root 20 0 3724 240 0 S 0 0.0 0:00.02 sftp-server
11849 root 20 0 0 0 0 S 0 0.0 0:00.00 flush-8:0
19391 root 20 0 2612 1280 1036 S 0 0.3 0:00.13 S18_cached
19436 root 20 0 2480 1224 1024 S 0 0.2 0:00.03 runInit.bash
19530 root -56 0 11424 1224 996 S 0 0.2 0:00.40 init
19987 root -56 0 51728 36m 1264 S 0 7.4 0:00.59 analyse_and_cal
Code:
cat /proc/meminfo
MemTotal: 508336 kB
MemFree: 369012 kB
Buffers: 1448 kB
Cached: 36832 kB
SwapCached: 0 kB
Active: 90412 kB
Inactive: 31112 kB
Active(anon): 83316 kB
Inactive(anon): 316 kB
Active(file): 7096 kB
Inactive(file): 30796 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 83308 kB
Mapped: 14504 kB
Shmem: 388 kB
Slab: 6936 kB
SReclaimable: 2936 kB
SUnreclaim: 4000 kB
KernelStack: 800 kB
PageTables: 800 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 254168 kB
Committed_AS: 162312 kB
VmallocTotal: 520980 kB
VmallocUsed: 9172 kB
VmallocChunk: 510796 kB
DirectMap4k: 14656 kB
DirectMap4M: 503808 kB
ulimits when application is started:
Code:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 3968
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 1024
cpu time (seconds, -t) unlimited
max user processes (-u) 3968
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
