ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I have been using Valgrind on some C code and have gotten all memory leaks patched (except one here that has not been answered yet that still plaques me,HINT HINT please respond here if you have an idea!). I am running this on both Red Hat AS and Solaris 8. When running on Red Hat, the memory usage will slowly climb until the system crashes. On Solaris however, the memory will slowly climb, and then quickly return back to the original amount. I know that there used to be a malloc library on Solaris that when linked to, immediately free'd the memory when free( ) was called, else the "standard" version free'd the memory, but kept it in a reserve until a later time. Is this the case with Linux/Red Hat? Are there any commands to force free( ) to allow access to all the memory? The application in question is a background daemon process that processes files and does not shut down.
First, the command free does not free anything. It just tells what memory is being used and what is not being used. From man free:
Code:
free displays the total amount of free and used physical and swap memory in
the system, as well as the buffers used by the kernel. The shared memory column
should be ignored; it is obsolete.
First, the command free does not free anything. It just tells what memory is being used and what is not being used. From man free:
Code:
free displays the total amount of free and used physical and swap memory in
the system, as well as the buffers used by the kernel. The shared memory column
should be ignored; it is obsolete.
Linux handles memory a lot different than windoze.
Hope that helps.
This was not a question of the command line free call, but to the 'C' routine free( ) (as in malloc( ) and free( ) for memory usage. I apologize if I was unclear about that. This also does not have to work on windoze, but just Red Hat and Solaris.
No apologies needed. I was going by your post count and thought you were new to Linux in general but at the same time your post made it sound like you were familiar with Linux. I wasn't really sure so I tried to include information that would then get additional info if you still had questions. Most people coming from windoze systems freak out when it says most all of the ram is in use. They don't realize that Linux caches as much as possible. That was the reason for the windoze comment.
Maybe someone with knowledge of 'C' routines will shed more light.
Distribution: Solaris 11.4, Oracle Linux, Mint, Ubuntu/WSL
Posts: 9,783
Rep:
Quote:
Originally Posted by Rayven
I am running this on both Red Hat AS and Solaris 8. When running on Red Hat, the memory usage will slowly climb until the system crashes. On Solaris however, the memory will slowly climb, and then quickly return back to the original amount.
How are you measuring the memory usage on both systems ?
free() call usually frees the memory just when it's called. System crash shouldn't happen in such case, if free is really called. Caches may be in use in your case (there are many different factors here), but should not lead to a crash, definitelly.
I wouldn't blame Linux at this point. You may have a subtle bug in your code that causes such behaviour (valgrind is good, but it can't find all the possible problems).
Make a small test. Run the program with memory limit per process (using ulimit). Check if it reaches the limit and/or if it crashes after that.
Distribution: Solaris 11.4, Oracle Linux, Mint, Ubuntu/WSL
Posts: 9,783
Rep:
Actually freed memory is returned to the system only under certain conditions.
Traditional malloc libraries only alloc on the heap and never use brk to lower it. With them, allocated memory is never returned to the O/S. This is documented in the Solaris free manual page.
On linux, glibc malloc use either the heap or mmap, depending on the size of the requested size. Memory is guaranteed to be free only with mmap. With heap allocation, this can happen only if there is no allocated areas live between the just freed one and the heap limit.
Finally, don't forget all this is about virtual memory. Physical usage is mostly unrelated.
On linux, glibc malloc use either the heap or mmap, depending on the size of the requested size. Memory is guaranteed to be free only with mmap. With heap allocation, this can happen only if there is no allocated areas live between the just freed one and the heap limit.
I'm not sure what you mean by this. Lets use an example. I have processes A, B, C, and D. Each allocates some chunks of heap memory. After some time A, and C finish execution, call their free()s and exit. Say then, that A allocated(and released) 1gb of memory, B currently holds 1.2mb of memory, C allocated 13gb of memory, and D holds 10 bytes of heap. My interpretation of your statement is that some 14gb of memory will be unavailable until B and D finish executing. Is that right?
Is that all within the context of a single process? Multiple processes? It seems to me that your statement needs some clarification.
Distribution: Solaris 11.4, Oracle Linux, Mint, Ubuntu/WSL
Posts: 9,783
Rep:
You got this wrong, sorry for having been not clear enough.
Malloc and free span is limited to a process, so allocations make by A have no relationship with allocations make by B, C or D. They are using unrelated virtual memory addresses (what my last sentence was telling)
My statement is valid if a single process is doing these 4 allocations though.
How are you measuring the memory usage on both systems ?
On the RHEL systems, I am using "top". On Solaris I am using vmstat.
Quote:
Originally Posted by jlliagre
Actually freed memory is returned to the system only under certain conditions.
Traditional malloc libraries only alloc on the heap and never use brk to lower it. With them, allocated memory is never returned to the O/S. This is documented in the Solaris free manual page.
On linux, glibc malloc use either the heap or mmap, depending on the size of the requested size. Memory is guaranteed to be free only with mmap. With heap allocation, this can happen only if there is no allocated areas live between the just freed one and the heap limit.
Finally, don't forget all this is about virtual memory. Physical usage is mostly unrelated.
So should I be using mmap instead of malloc? Will munmap then completely free the memory instead of just returning it into the malloc pool?
So should I be using mmap instead of malloc? Will munmap then completely free the memory instead of just returning it into the malloc pool?
Just use malloc(). The mmap() stuff jlliagre was talking about is about the internal implementation of malloc() inside glibc. Whether mmap() is used behind the scenes is of no concern to you as an application programmer, except when wondering where free()d memory will go...
Distribution: Solaris 11.4, Oracle Linux, Mint, Ubuntu/WSL
Posts: 9,783
Rep:
Quote:
Originally Posted by Rayven
On the RHEL systems, I am using "top". On Solaris I am using vmstat.
And what column(s) are you watching on both of them ?
Quote:
So should I be using mmap instead of malloc? Will munmap then completely free the memory instead of just returning it into the malloc pool?
You may want to use mmap with Solaris if you wish to release memory to the system, as neither glibc nor mmap based allocation libraries are available (as far as I know, I didn't investigate).
As Hko stated, there is no interest building your own allocation library under Gnu/Linux, as malloc does already it for you.
And what column(s) are you watching on both of them ?
Yes, I am watching both. The swap space is 8GB free, 2GB swap which I usually restart the computer before swap is touched.
Quick valgrind question, pertaining to this topic: if I am receiving multiple messages about "Invalid Read Size of 8" and "Use of Unitialized Value of size 8", would they make the memory leaks? I thought that these two warnings did not have much to do with memory leaks but just warnings of possible "bad" things. http://valgrind.org/docs/manual/mc-m...nual.errormsgs, maybe I read the documentation incorrectly.
Thanks again!
Distribution: Solaris 11.4, Oracle Linux, Mint, Ubuntu/WSL
Posts: 9,783
Rep:
Quote:
Originally Posted by Rayven
Yes, I am watching both. The swap space is 8GB free, 2GB swap which I usually restart the computer before swap is touched.
You misread my question.
I'm asking what values are you looking at in the vmstat and the top output.
Quote:
Quick valgrind question, pertaining to this topic: if I am receiving multiple messages about "Invalid Read Size of 8" and "Use of Unitialized Value of size 8", would they make the memory leaks?
No.
Quote:
I thought that these two warnings did not have much to do with memory leaks but just warnings of possible "bad" things.
Real bad things actually, these are bugs that should be fixed.
Quick valgrind question, pertaining to this topic: if I am receiving multiple messages about "Invalid Read Size of 8" and "Use of Unitialized Value of size 8", would they make the memory leaks?
Not leaks, but it means you access memeory outside of your allocated space. In short: you're probably reading garbage. When used in certain ways, it may lead to crashes. Look into it.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.