LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 11-02-2006, 05:02 PM   #16
osvaldomarques
Member
 
Registered: Jul 2004
Location: Rio de Janeiro - Brazil
Distribution: Conectiva 10 - Conectiva 8 - Slackware 9 - starting with LFS
Posts: 519

Rep: Reputation: 34

Could you show us the beginning of the function with its declarations and possible use of the dp pointer before the line where valgrind reports possible memory leak?
 
Old 11-03-2006, 08:44 AM   #17
Rayven
LQ Newbie
 
Registered: Oct 2006
Distribution: Red Hat 4, Ubuntu
Posts: 21

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by jlliagre
You misread my question.
I'm asking what values are you looking at in the vmstat and the top output.
I am looking at the used and available memory for both commands. I am not using any special format commands.
 
Old 11-03-2006, 11:00 AM   #18
Rayven
LQ Newbie
 
Registered: Oct 2006
Distribution: Red Hat 4, Ubuntu
Posts: 21

Original Poster
Rep: Reputation: 15
Post

Quote:
Originally Posted by osvaldomarques
Could you show us the beginning of the function with its declarations and possible use of the dp pointer before the line where valgrind reports possible memory leak?
Code:
char *inputDir; // This value is set at program startup from an
                // environment variable (getenv). It is free'd when
                // the used pressed Ctrl-C to terminate the
                // application (if ever). 

// pollDirectory( ) is called within a for( ;; ) statement, if the
// return value is < 0, the application terminates
int pollDirectory( ){
   pid_t pid;
   struct direct *dirp;
   struct stat statbuf;
   struct stat filestat;
   char inPath[256];
   char tempPath[256];
   char msg[1024];
   int inSz = 0;
   int currNumScreened = 0;
   DIR *dp;

   // Ensure directory exists
   if( lstat( inputDir, &statbuf ) < 0 ) {
      // Report error
      return -1;
   }

   // path is a directory
   if( S_ISDIR( statbuf.st_mode ) == 0 ) {
      // Report error
      return -1;
   }

   // Code to open directory and parse through filenames
      // fork child processes to handle the files, child process renames
      // and moves the listed files. 
   // After all the directory listing has been processed, closedir( ) 
   // and return the number of files processed
}
This just occured to me while typing this post: if I am removing the files from the directoy, would that cause a memory leak within the dirent/DIR structures? It doesent seem like that should cause any leaks, but I am not totally sure.
 
Old 11-03-2006, 03:06 PM   #19
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,523

Rep: Reputation: 365Reputation: 365Reputation: 365Reputation: 365
Quote:
Originally Posted by Rayven
I am looking at the used and available memory for both commands. I am not using any special format commands.
Are you looking at the memory numbers or the swap ones ?
By the way, vmstat does not report used memory or swap, just free memory and swap.
 
Old 11-03-2006, 03:32 PM   #20
Rayven
LQ Newbie
 
Registered: Oct 2006
Distribution: Red Hat 4, Ubuntu
Posts: 21

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by jlliagre
Are you looking at the memory numbers or the swap ones ?
By the way, vmstat does not report used memory or swap, just free memory and swap.
In top I am looking at the available/free memory, total memory, and swap available/free and total. vmstat I am looking at the available/free memory. I am typically closing the application prior to the swap space being used.

Also, I ran the program through Sun Studio 10 and found a few more memory leaks, but not on the opendir call...this is VERY difficult!
 
Old 11-03-2006, 05:04 PM   #21
osvaldomarques
Member
 
Registered: Jul 2004
Location: Rio de Janeiro - Brazil
Distribution: Conectiva 10 - Conectiva 8 - Slackware 9 - starting with LFS
Posts: 519

Rep: Reputation: 34
Hi Rayven,

Do you use the "dp" pointer in the forked process for any other purpose? Normally, a memory leak happens when you use a pointer to get a new chunk of memory without freeing a previous one. On your case, it would happen if you use "opendir" without a "closedir" for a previous opened directory. Is your function recursive? Do you fork for opening another subdirectory?
 
Old 11-04-2006, 01:54 AM   #22
Rayven
LQ Newbie
 
Registered: Oct 2006
Distribution: Red Hat 4, Ubuntu
Posts: 21

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by osvaldomarques
Hi Rayven,

Do you use the "dp" pointer in the forked process for any other purpose? Normally, a memory leak happens when you use a pointer to get a new chunk of memory without freeing a previous one. On your case, it would happen if you use "opendir" without a "closedir" for a previous opened directory. Is your function recursive? Do you fork for opening another subdirectory?
No, dp is not used inside of the child process. The process flow is read the directory contents (no subdirectories), create a temporary filename by appending a prefix to the filename, rename the file to the temporary name (so I do re-evaluate the file), fork the process, child process evaluates file contents, parent moves to next filename. After the entire directory is evaluated, dp is closed, the pollDirectory function returns the number of files evaluated, and sleeps for x number of seconds and then repeats the entire process.
 
Old 11-04-2006, 02:20 AM   #23
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,523

Rep: Reputation: 365Reputation: 365Reputation: 365Reputation: 365
Quote:
Originally Posted by Rayven
In top I am looking at the available/free memory, total memory, and swap available/free and total. vmstat I am looking at the available/free memory.
vmstat doesn't reports available memory.
Quote:
I am typically closing the application prior to the swap space being used.
Gnu/Linux and Solaris manage swap space quite differently.

With Solaris, the total swap space reported (for example by swap -s) is the sum of the RAM size plus the sum of swap areas on disk. A malloc will fail if no page is available in this whole swap space.

With Gnu/Linux, the total swap space reported is the size of the swap areas on disk. By default, a malloc won't fail if no more virtual memory is available. This is the controversial lazy memory allocation method. Controversial as it breaks malloc semantics and because of the OOM killer infamous behaviour.

See http://lwn.net/Articles/104179/ for a reference.

Quote:
Also, I ran the program through Sun Studio 10 and found a few more memory leaks, but not on the opendir call...
Where are these other memory leaks, in libraries or in your code ?
Quote:
this is VERY difficult!
What is very difficult ?
 
Old 11-05-2006, 10:09 PM   #24
Rayven
LQ Newbie
 
Registered: Oct 2006
Distribution: Red Hat 4, Ubuntu
Posts: 21

Original Poster
Rep: Reputation: 15
Quote:
Where are these other memory leaks, in libraries or in your code ?
I fixed a few in my source code. I still have a couple more ti figure out if they are in libraries or in my source.

Quote:
What is very difficult ?
Debugging memory leaks is VERY difficult. In school I was never taught about good memory management for large scale programs. It is something I have had to learn on my own, with the help of forums such as these!
 
Old 11-06-2006, 03:24 AM   #25
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,523

Rep: Reputation: 365Reputation: 365Reputation: 365Reputation: 365
Quote:
Debugging memory leaks is VERY difficult. In school I was never taught about good memory management for large scale programs. It is something I have had to learn on my own, with the help of forums such as these!
Yes, memory bugs nightmares with C and C++ code are one of the reasons Java is so successful these days.

Last edited by jlliagre; 11-06-2006 at 03:55 AM.
 
  


Reply

Tags
free, malloc


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Clear Free Memory lowlifeish Linux - General 3 12-12-2005 01:12 PM
How much memory is really 'free'? robbiemorgan Linux - Newbie 4 09-19-2004 08:05 PM
Free, free,FREE=? no money, = Freedom? murshed Linux - Newbie 8 01-20-2003 08:01 AM
Memory free problem edreddy Programming 1 11-09-2002 11:59 PM


All times are GMT -5. The time now is 08:36 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration