Physical Memory leaked by user application process Is not reclaimed back after exit
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I don't wish to belabor the point, but ... your interpretation of the numbers is not correct.
When a user application process terminates, all of the memory allocated by it is released.
There are, however, many "reasons" why "memory" can become associated with a process, and these commands reflect all of those "reasons," not merely "discretionary malloc() calls."
The root cause of "your pain" is: your application has a bug in it. Maybe a subtle one, but it's enough to build up. You're going to have to sleuth it, and find it.
The root cause of "your pain" is: your application has a bug in it. Maybe a subtle one, but it's enough to build up. You're going to have to sleuth it, and find it.
If my application has bug or any leak then the memory increased should show in the 'top' command output for that process.
Even the memory details in /proc/<PID>/status file are not changed from start to end.
There is some summary info about kernel memory use available in the first five lines of the display from the command
Code:
slabtop
If you can repeat your experiment, that should tell you how much the kernel is using and how much it is reserving.
One theory that fits your symptoms, but probably isn't a problem, would be that your program makes the kernel allocated objects stored in slabs in such a way that the objects are released when your program exits, but the slabs aren't. So the second time you run the program, the kernel would reuse those slabs and not appear to leak physical memory again.
A more serious problem might have the symptom that the kernel objects allocated because of your program don't go away when the program exits. That should be practical to estimate by seeing the way the slab information changes as your program runs and then as it exits.
In between (but probably serious in an "appliance") your program might be gradually causing the kernel to accumulate objects that go away only when your program exits. Assuming your program is meant to run for a very long time, that would be a serious memory leak. The fact that the kernel slabs don't go away when the objects go away, might just be a secondary symptom.
More detailed (too detailed) info about slab use can be put into a file using
Code:
cp /proc/slabinfo file_name
If you capture that before, during and after your program runs, you might be able to find the biggest differences among those three to understand how your program affects kernel memory use.
I ran free and looked at its output. I think there is no real problem. We have just been misinterpreting the information.
The first output line from free is heading titles. The first heading, indented from the left side of the screen, is "total". The headings are
total, used, free, shared, buffers, cached.
The numbers printed after Mem: at the left side of the screen are
in the last screen. This shows your system has 1007 total MB with 472 used in earlier screen, where 534 units are free. Then, in the last screen, 454 units are used and 552 are free. So 18 more units were in use while your application was running, but they were freed when your application exited. I now believe that you do not have a memory leak problem.
In some cases, including the two cited above your used + free add up to only 1006, which is one less than the 1007 displayed in the total column. A MB appears to be missing. But in other cases the used + free number add up to 1007. This disturbed me until I ran free without the -m option which displays values in MB. Without the -m option, free displays the values in KB. When I ran free without the -m option, the totals of used + free always added up to the total value. I infer that the MB values may be truncated instead of rounded, or that, if they are rounded, that the rounding algorithm may have problems. Since Linux deals with memory in 4KB pages (at least, that is the default), the memory sizes in KB are never rounded nor truncated. The numbers in used and free are also always integral multiples of 4.
Another option you can use with free is -t <seconds>, such as
free -t 5
This will run the command at 5 second intervals until you stop it by typing CTL-C. This will give you a sequence of memory usage snapshots. You will probably notice that the values of used and free bounce around quite a bit on successive times that free runs, even if you do nothing else on the machine. Memory usage depends on what processes may be running when free runs each time. You may also notice your hard drive access indicator flashing even if you have not been using the computer for a while. The network interface activity, if your computer has such indicator, may similarly blink. These are only a couple of the many processes which may use more or less memory at different times behind the scenes.
You can pull up a console terminal and review the man page for free:
man free
to review the online manual.
I'm sorry I didn't look at it closer berfore I muddied the water with my earlier postings in this thread.
'cuz if you felt embarrassed for misinterpreting something that a computer seemed to be telling you, then I daresay everyone around here would be walking around with more-or-less permanently red faces.
Last edited by sundialsvcs; 05-08-2010 at 08:52 AM.
[QUOTE]
Thank you ArthurSittler and johnsfine for spending your valuable time on my problem.
[/QUOTE}
My appliance logs some network traffic and attacks data.
What my application will do is fetch the logged data and parse for reporting.
Attack log files will be copied from some location in appliance to my application's installation path.I used just 'cp' command to copy.
I observed memory usage increasing by 1M upon copying certain number of files.
To fetch traffic I need to run certain appliance specific commands.I ran these commands using bash script. If traffic data present memory usage is increasing.
This proved that my application really do not have any potential memory leaks.
Also I did one more experiment.
I written small program which 1K memory leak.I ran this program in a infinite loop.
At after certain period of time my system got slowed and os start using swap also.After test program aborted as no memory to allocate to my program.
Now i run the free -m command.
Now used memory come down to two digits number.(77M).
Even the memory which was increased while my application is running but released also got released now.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.