LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 02-18-2010, 09:01 AM   #1
Martin_J
LQ Newbie
 
Registered: Feb 2010
Posts: 6

Rep: Reputation: 0
java - Too many files open


Hi, i am creating program that gets 'key' string and directory path and finds if key string is present in any file under given path,
everything seemed OK but when i put path like "/" after 5 mins i get error : Too many opened files. I've minimalized opened File object at the same time as much as posible, tried to call System.gc(),tried to close every stream every opened file, google but i still cannot search in bigger directories.

Algorithm (mine one is much bigger so i tried to make it as small as possible:

read 'path';
read 'key'(set as public);
#&
call method get('path') {
#&
String[] files = all files in 'path' directory;
for(String s : files) {
if File('path' + s).isDirectory() call again method get: get(path + "/" + s); //Searching in subdirectories
if File('path' + s).isFile() open it and search if 'key' is there -if yes
write file;
#&
}
}

#& - where i tried System.gc()

Anyone has some idea why I always get that runtime error ?
(I need it to search hundreds of GB so when my 40GB partition do this what will bigger one do )
 
Old 02-18-2010, 09:14 AM   #2
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 454Reputation: 454Reputation: 454Reputation: 454Reputation: 454
Quote:
Originally Posted by Martin_J View Post
Hi, i am creating program that gets 'key' string and directory path and finds if key string is present in any file under given path,
everything seemed OK but when i put path like "/" after 5 mins i get error : Too many opened files. I've minimalized opened File object at the same time as much as posible, tried to call System.gc(),tried to close every stream every opened file, google but i still cannot search in bigger directories.

Algorithm (mine one is much bigger so i tried to make it as small as possible:

read 'path';
read 'key'(set as public);
#&
call method get('path') {
#&
String[] files = all files in 'path' directory;
for(String s : files) {
if File('path' + s).isDirectory() call again method get: get(path + "/" + s); //Searching in subdirectories
if File('path' + s).isFile() open it and search if 'key' is there -if yes
write file;
#&
}
}

#& - where i tried System.gc()

Anyone has some idea why I always get that runtime error ?
(I need it to search hundreds of GB so when my 40GB partition do this what will bigger one do )
Do you ever close unneeded open files ?
 
Old 02-18-2010, 09:43 AM   #3
Martin_J
LQ Newbie
 
Registered: Feb 2010
Posts: 6

Original Poster
Rep: Reputation: 0
You mean I/O streams ? Yea, I always close them all.

OOh - does that error mean that i have unclosed I/O Streams or too many File objects ?

Last edited by Martin_J; 02-18-2010 at 09:50 AM.
 
Old 02-18-2010, 09:50 AM   #4
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 454Reputation: 454Reputation: 454Reputation: 454Reputation: 454
Quote:
Originally Posted by Martin_J View Post
You mean I/O streams ? Yea, I always close them all.
No, I mean files. OS deals with files.
 
Old 02-18-2010, 10:05 AM   #5
Martin_J
LQ Newbie
 
Registered: Feb 2010
Posts: 6

Original Poster
Rep: Reputation: 0
Yea, I always invoke 'stream'.close();
 
Old 02-18-2010, 11:52 AM   #6
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 454Reputation: 454Reputation: 454Reputation: 454Reputation: 454
Quote:
Originally Posted by Martin_J View Post
Yea, I always invoke 'stream'.close();
OSes have utilities for reporting open files - in Linux it's 'lsof'. So check your assumptions with such a utility.
 
Old 02-19-2010, 01:26 AM   #7
Martin_J
LQ Newbie
 
Registered: Feb 2010
Posts: 6

Original Poster
Rep: Reputation: 0
And when i have opened instance of java.io.File class does it counts as opened file or it has nothing to do with that?
 
Old 02-19-2010, 02:26 AM   #8
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 454Reputation: 454Reputation: 454Reputation: 454Reputation: 454
Quote:
Originally Posted by Martin_J View Post
And when i have opened instance of java.io.File class does it counts as opened file or it has nothing to do with that?
Your question does not contain enough details to give you a definite answer. Every file opened in terms of

man 3 fopen

(or http://linux.die.net/man/3/fopen ) counts.

You need to understand how Java stuff translates into OS stuff - I am not a Java guy.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
cant compile java files in fedora 8.. java/ maranganesan Linux - Software 4 12-31-2007 12:21 PM
PHP/Java/Anything that can split files, while files stay on Server? Lamborghini Kid Linux - Software 1 04-27-2007 04:53 PM
LXer: Java Look-Alike Project Pushed Sun To Make Java Open Source LXer Syndicated Linux News 0 11-15-2006 03:54 AM
Firefox refuses to load Java jnlp files - plugin and java ok Melsync Linux - Software 1 06-25-2006 04:09 PM
Java IOException: Too many open files naijaguy Programming 0 02-01-2005 05:20 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 02:18 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration