LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Programming (http://www.linuxquestions.org/questions/programming-9/)
-   -   java - Too many files open (http://www.linuxquestions.org/questions/programming-9/java-too-many-files-open-789940/)

Martin_J 02-18-2010 09:01 AM

java - Too many files open
 
Hi, i am creating program that gets 'key' string and directory path and finds if key string is present in any file under given path,
everything seemed OK but when i put path like "/" after 5 mins i get error : Too many opened files. I've minimalized opened File object at the same time as much as posible, tried to call System.gc(),tried to close every stream every opened file, google but i still cannot search in bigger directories.

Algorithm (mine one is much bigger so i tried to make it as small as possible:

read 'path';
read 'key'(set as public);
#&
call method get('path') {
#&
String[] files = all files in 'path' directory;
for(String s : files) {
if File('path' + s).isDirectory() call again method get: get(path + "/" + s); //Searching in subdirectories
if File('path' + s).isFile() open it and search if 'key' is there -if yes
write file;
#&
}
}

#& - where i tried System.gc()

Anyone has some idea why I always get that runtime error ?
(I need it to search hundreds of GB so when my 40GB partition do this what will bigger one do :( )

Sergei Steshenko 02-18-2010 09:14 AM

Quote:

Originally Posted by Martin_J (Post 3868156)
Hi, i am creating program that gets 'key' string and directory path and finds if key string is present in any file under given path,
everything seemed OK but when i put path like "/" after 5 mins i get error : Too many opened files. I've minimalized opened File object at the same time as much as posible, tried to call System.gc(),tried to close every stream every opened file, google but i still cannot search in bigger directories.

Algorithm (mine one is much bigger so i tried to make it as small as possible:

read 'path';
read 'key'(set as public);
#&
call method get('path') {
#&
String[] files = all files in 'path' directory;
for(String s : files) {
if File('path' + s).isDirectory() call again method get: get(path + "/" + s); //Searching in subdirectories
if File('path' + s).isFile() open it and search if 'key' is there -if yes
write file;
#&
}
}

#& - where i tried System.gc()

Anyone has some idea why I always get that runtime error ?
(I need it to search hundreds of GB so when my 40GB partition do this what will bigger one do :( )

Do you ever close unneeded open files ?

Martin_J 02-18-2010 09:43 AM

You mean I/O streams ? Yea, I always close them all.

OOh - does that error mean that i have unclosed I/O Streams or too many File objects ?

Sergei Steshenko 02-18-2010 09:50 AM

Quote:

Originally Posted by Martin_J (Post 3868205)
You mean I/O streams ? Yea, I always close them all.

No, I mean files. OS deals with files.

Martin_J 02-18-2010 10:05 AM

Yea, I always invoke 'stream'.close();

Sergei Steshenko 02-18-2010 11:52 AM

Quote:

Originally Posted by Martin_J (Post 3868235)
Yea, I always invoke 'stream'.close();

OSes have utilities for reporting open files - in Linux it's 'lsof'. So check your assumptions with such a utility.

Martin_J 02-19-2010 01:26 AM

And when i have opened instance of java.io.File class does it counts as opened file or it has nothing to do with that?

Sergei Steshenko 02-19-2010 02:26 AM

Quote:

Originally Posted by Martin_J (Post 3869041)
And when i have opened instance of java.io.File class does it counts as opened file or it has nothing to do with that?

Your question does not contain enough details to give you a definite answer. Every file opened in terms of

man 3 fopen

(or http://linux.die.net/man/3/fopen ) counts.

You need to understand how Java stuff translates into OS stuff - I am not a Java guy.


All times are GMT -5. The time now is 08:00 PM.