Java: Reading from a text file slow on Tomcat

robbbert · 08-20-2006, 12:40 PM

I'm using this function to read from a text file:

Code:

	public static String getTextFromFile(String fileName, String encoding)
	throws IOException
	{
		BufferedReader input = new BufferedReader(
		new InputStreamReader(new FileInputStream(
		fileName), encoding));
		String line = input.readLine();
		StringBuilder stringBuilder = new StringBuilder();
		while (line != null)
		{
			stringBuilder.append(line);
			stringBuilder.append("\n");
			line = input.readLine();
		}
		input.close();

		return stringBuilder.toString();
	}

For a file with 3000 lines, this will take 0.25 seconds on my PC (when run as a Java application).
Being run as a Tomcat servlet, this takes minutes...

By debugging the application, I found the part that takes so long on Tomcat is the loop.
I've also tried to use the StringBuffer class.

This is on Java 1.5, Tomcat 5.5.

Any ideas?
Thanks

robbbert · 08-22-2006, 01:34 AM

Hmmm, it became fast again after a reboot... I'm using Eclipse to invoke and start Tomcat (both, in run and debug mode), and apparently, it's getting unreliable (and/or slower) after several starts (even though any Java processes have been ended, properly).

That's what my current findings are...

ntubski · 08-22-2006, 04:17 PM

I think reading one line at a time would be rather slow. Furthermore you create and then discard a String object for every loop iteration (might even be 2 String's per loop, I seem to remember using a String literal, the "\n", creates a new String). You might try to read all the bytes at once:

Code:

...
//read in the file in raw byte form
byte[] buf = new byte[fileSize];
fin = new FileInputStream(fileName);
fin.read(buf);
...
return new String(buf, encoding); //translate into encoding

This requires knowing the file size though. There is a more sophisticated method that takes an estimate for file size here.

robbbert · 08-23-2006, 01:10 PM

Thanks ntubski. My goal would always be to make the standard methods work first.

ntubski · 08-23-2006, 08:48 PM

Umm, what do you mean by that exactly?

robbbert · 08-24-2006, 11:44 AM

I mean, the code in my first post is what you can read in any basic Java book, or by searching Google on "java read from text file". And while your code might work - or might not -, that's a rather uncommon, experimental approach. I.e., it contains alot of FIXME comments.

ntubski · 08-24-2006, 06:21 PM

Well, I found that experimental piece of code through Googling actually. Although today I see I was totally on the wrong track; I had read somewhere that read a text file into memory was called "slurping" so I was searching for that, but today when I searched java read file string I see things are much simpler. I somehow missed the available() method in FileInputStream.
So this should work fine:

Code:

...
//read in the file in raw byte form
fin = new FileInputStream(fileName);
byte[] buf = new byte[fin.available()];
fin.read(buf);
...
return new String(buf, encoding); //translate into encoding

Although I really feel that you ought to be able to just attach a FileInputStream to a StringWriter and have all the low-level stuff be taken care automagically, oh well...

robbbert · 08-25-2006, 12:07 PM

Thanks. - However,

Quote:

new FileInputStream(fileName).available()

does not guarantee to return the size of the file it has opened.

I agree, though, that a valid approach was to loop until .available() becomes 0.

- The readLine() approach, in contrast, appears to have severe limitations when no line feed characters exist, and the file is really large.