LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 05-06-2010, 11:15 PM   #16
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,659
Blog Entries: 4

Rep: Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939

I don't wish to belabor the point, but ... your interpretation of the numbers is not correct.

When a user application process terminates, all of the memory allocated by it is released.

There are, however, many "reasons" why "memory" can become associated with a process, and these commands reflect all of those "reasons," not merely "discretionary malloc() calls."

The root cause of "your pain" is: your application has a bug in it. Maybe a subtle one, but it's enough to build up. You're going to have to sleuth it, and find it.
 
Old 05-06-2010, 11:36 PM   #17
ravikumarv
LQ Newbie
 
Registered: Jun 2007
Posts: 27

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by sundialsvcs View Post
The root cause of "your pain" is: your application has a bug in it. Maybe a subtle one, but it's enough to build up. You're going to have to sleuth it, and find it.
If my application has bug or any leak then the memory increased should show in the 'top' command output for that process.
Even the memory details in /proc/<PID>/status file are not changed from start to end.
 
Old 05-07-2010, 07:14 AM   #18
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
Why are you so sure the memory wasn't taken by some other process?
 
Old 05-07-2010, 07:34 AM   #19
ravikumarv
LQ Newbie
 
Registered: Jun 2007
Posts: 27

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by johnsfine View Post
Why are you so sure the memory wasn't taken by some other process?
Because my box is a appliance and so I knew the process running and their memory usage
 
Old 05-07-2010, 07:53 AM   #20
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197Reputation: 1197
There is some summary info about kernel memory use available in the first five lines of the display from the command
Code:
slabtop
If you can repeat your experiment, that should tell you how much the kernel is using and how much it is reserving.

One theory that fits your symptoms, but probably isn't a problem, would be that your program makes the kernel allocated objects stored in slabs in such a way that the objects are released when your program exits, but the slabs aren't. So the second time you run the program, the kernel would reuse those slabs and not appear to leak physical memory again.

A more serious problem might have the symptom that the kernel objects allocated because of your program don't go away when the program exits. That should be practical to estimate by seeing the way the slab information changes as your program runs and then as it exits.

In between (but probably serious in an "appliance") your program might be gradually causing the kernel to accumulate objects that go away only when your program exits. Assuming your program is meant to run for a very long time, that would be a serious memory leak. The fact that the kernel slabs don't go away when the objects go away, might just be a secondary symptom.

More detailed (too detailed) info about slab use can be put into a file using
Code:
cp /proc/slabinfo file_name
If you capture that before, during and after your program runs, you might be able to find the biggest differences among those three to understand how your program affects kernel memory use.

Last edited by johnsfine; 05-07-2010 at 07:56 AM.
 
Old 05-07-2010, 09:46 PM   #21
ArthurSittler
Member
 
Registered: Jul 2008
Distribution: Slackware
Posts: 124

Rep: Reputation: 31
now I feel silly

We may have all been chasing our own tails.

I ran free and looked at its output. I think there is no real problem. We have just been misinterpreting the information.
The first output line from free is heading titles. The first heading, indented from the left side of the screen, is "total". The headings are

total, used, free, shared, buffers, cached.

The numbers printed after Mem: at the left side of the screen are

1007, 472, 534, 0, 125, 109 <-notice that 472 + 534 = 1006

in the second-to-last screen, and

1007, 454, 552, 0, 125, 109 <- again, 452 + 554 = 1006

in the last screen. This shows your system has 1007 total MB with 472 used in earlier screen, where 534 units are free. Then, in the last screen, 454 units are used and 552 are free. So 18 more units were in use while your application was running, but they were freed when your application exited. I now believe that you do not have a memory leak problem.

In some cases, including the two cited above your used + free add up to only 1006, which is one less than the 1007 displayed in the total column. A MB appears to be missing. But in other cases the used + free number add up to 1007. This disturbed me until I ran free without the -m option which displays values in MB. Without the -m option, free displays the values in KB. When I ran free without the -m option, the totals of used + free always added up to the total value. I infer that the MB values may be truncated instead of rounded, or that, if they are rounded, that the rounding algorithm may have problems. Since Linux deals with memory in 4KB pages (at least, that is the default), the memory sizes in KB are never rounded nor truncated. The numbers in used and free are also always integral multiples of 4.

Another option you can use with free is -t <seconds>, such as

free -t 5

This will run the command at 5 second intervals until you stop it by typing CTL-C. This will give you a sequence of memory usage snapshots. You will probably notice that the values of used and free bounce around quite a bit on successive times that free runs, even if you do nothing else on the machine. Memory usage depends on what processes may be running when free runs each time. You may also notice your hard drive access indicator flashing even if you have not been using the computer for a while. The network interface activity, if your computer has such indicator, may similarly blink. These are only a couple of the many processes which may use more or less memory at different times behind the scenes.

You can pull up a console terminal and review the man page for free:

man free

to review the online manual.

I'm sorry I didn't look at it closer berfore I muddied the water with my earlier postings in this thread.
 
Old 05-08-2010, 08:51 AM   #22
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,659
Blog Entries: 4

Rep: Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939Reputation: 3939
Don't feel silly ...

'cuz if you felt embarrassed for misinterpreting something that a computer seemed to be telling you, then I daresay everyone around here would be walking around with more-or-less permanently red faces.

Last edited by sundialsvcs; 05-08-2010 at 08:52 AM.
 
Old 05-09-2010, 11:55 PM   #23
ravikumarv
LQ Newbie
 
Registered: Jun 2007
Posts: 27

Original Poster
Rep: Reputation: 15
[QUOTE]
Thank you ArthurSittler and johnsfine for spending your valuable time on my problem.
[/QUOTE}
My appliance logs some network traffic and attacks data.
What my application will do is fetch the logged data and parse for reporting.

Attack log files will be copied from some location in appliance to my application's installation path.I used just 'cp' command to copy.
I observed memory usage increasing by 1M upon copying certain number of files.

To fetch traffic I need to run certain appliance specific commands.I ran these commands using bash script. If traffic data present memory usage is increasing.

This proved that my application really do not have any potential memory leaks.

Also I did one more experiment.

I written small program which 1K memory leak.I ran this program in a infinite loop.
At after certain period of time my system got slowed and os start using swap also.After test program aborted as no memory to allocate to my program.

Now i run the free -m command.

Now used memory come down to two digits number.(77M).
Even the memory which was increased while my application is running but released also got released now.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Reserving physical memory for a named process??? bortbortresson Linux - Server 12 12-09-2009 03:45 AM
how kernel process is loaded in the physical memory.. Kesavaraj Linux - Software 1 07-23-2008 08:53 AM
Free Memory Not Reclaimed By System Sheado Programming 3 09-28-2007 03:38 PM
mechanics of mapping process memory addresses to physical addresses on amd64 Tischbein Linux - Kernel 2 02-01-2007 08:09 PM
Command to show each application/process memory usage and its total gn00kie Linux - Enterprise 2 09-21-2006 03:08 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 02:23 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration