Hi,

I have a software which outputs lines of numbers one by one. In another java program, I use BufferedReader as follows:

BufferedReader input = new BufferedReader(new InputStreamReader(pr.getInputStream(),"US-ASCII"));

where pr is the process which runs the software. For each line in the output stream, I try to check whether the first number in the line is greater than a certain value and if it is, I write to another file.

The problem is, when I run the software repeatedly (in a linux cluster) like 10 simulations simultaneously, one of them have got @^@^.... characters followed by some huge numbers. So, I tried eliminating that line alone by using String.matches command but now, it seems even after that I get the error - so, I think it might mean that it is actually during the writing process that the problem occurs. I use BufferedWriter like the following to write it:
BufferedWriter out=new BufferedWriter(new FileWriter(tmp_file));

I would greatly appreciate if someone can help. This thing really seems to be a huge bottleneck!

Thanks,
Senthil

Without knowing how the data are created, I can only guess. The problem might be that you are specifying a character encoding that doesn't agree with the input, or the source is emitting raw binary data, or something else.

It's all guesswork without seeing the data and the code that created it.

Hi! Thanks.

Its just lines of numbers like the following:
0 7200 0 0 0 0 0 .....
2 7124 9 8 27 10 ....

When the simulator is run, it usually outputs these lines of numbers to the command line, which I am now reading using BufferedStream.

Does this help at all?

Thanks and looking forward to your post!

If the source data was a simple list of numerical text, you wouldn't be having a problem. But you are having a problem, so you need to look more closely at the input data and its source. Is it really just text, or is it numerical data that happens to be printed as text in your example?

One way to proceed is to capture the data in a text file and then examine it using hexdump:

Anything outside the numerical range 0 - 127 (hex 00 - 7f) isn't plain ASCII and needs to be investigated further.

Thanks! I captured a part of the output and then, tried the command you have suggested. I am unable to discern anything out of it - I guess, they are all in the range of 00-7f - is there anyway you could help? Thanks, once again!

00000000 30 2e 30 20 37 32 30 30 20 30 20 30 20 30 20 30 |0.0 7200 0 0 0 0|
00000010 20 30 20 30 20 30 20 30 20 30 20 30 20 30 20 30 | 0 0 0 0 0 0 0 0|
*
00000190 20 30 20 30 20 30 20 0a 32 2e 31 35 37 31 37 30 | 0 0 0 .2.157170|
000001a0 39 34 35 33 32 39 31 37 30 38 20 37 31 36 32 20 |9453291708 7162 |
000001b0 39 20 34 20 32 20 30 20 30 20 30 20 30 20 30 20 |9 4 2 0 0 0 0 0 |
000001c0 30 20 30 20 30 20 30 20 30 20 30 20 30 20 30 20 |0 0 0 0 0 0 0 0 |
*
00000330 30 20 30 20 30 20 30 20 30 20 30 20 30 20 0a 33 |0 0 0 0 0 0 0 .3|
00000340 2e 38 33 31 37 34 38 37 32 39 36 31 37 38 34 34 |.831748729617844|
00000350 35 20 37 31 30 32 20 31 32 20 37 20 39 20 32 20 |5 7102 12 7 9 2 |
00000360 30 20 31 20 30 20 30 20 30 20 30 20 30 20 30 20 |0 1 0 0 0 0 0 0 |
00000370 30 20 30 20 30 20 30 20 30 20 30 20 30 20 30 20 |0 0 0 0 0 0 0 0 |
*
000004e0 30 20 30 20 30 20 0a 35 2e 33 37 38 37 36 31 31 |0 0 0 .5.3787611|
000004f0 36 31 39 34 38 39 32 33 20 37 30 35 32 20 31 31 |61948923 7052 11|
00000500 20 39 20 33 20 38 20 34 20 32 20 30 20 31 20 30 | 9 3 8 4 2 0 1 0|
00000510 20 30 20 30 20 30 20 30 20 30 20 30 20 30 20 30 | 0 0 0 0 0 0 0 0|
*
00000680 20 30 20 30 20 30 20 30 20 30 20 30 20 0a 36 2e | 0 0 0 0 0 0 .6.|
00000690 38 38 33 33 32 31 34 37 32 38 34 32 32 31 36 20 |883321472842216 |
000006a0 37 30 31 35 20 39 20 39 20 35 20 35 20 35 20 33 |7015 9 9 5 5 5 3|
000006b0 20 32 20 32 20 31 20 30 20 30 20 30 20 30 20 30 | 2 2 1 0 0 0 0 0|
000006c0 20 30 20 30 20 30 20 30 20 30 20 30 20 30 20 30 | 0 0 0 0 0 0 0 0|
*
00000830 20 30 20 0a 38 2e 33 39 33 32 31 30 38 35 39 32 | 0 .8.3932108592|
00000840 38 32 38 32 34 20 36 39 36 36 20 39 20 36 20 36 |82824 6966 9 6 6|
00000850 20 37 20 33 20 33 20 30 20 32 20 33 20 30 20 32 | 7 3 3 0 2 3 0 2|
00000860 20 31 20 30 20 31 20 30 20 30 20 30 20 30 20 30 | 1 0 1 0 0 0 0 0|
00000870 20 30 20 30 20 30 20 30 20 30 20 30 20 30 20 30 | 0 0 0 0 0 0 0 0|
*
000009d0 20 30 20 30 20 30 20 30 20 0a 39 2e 39 31 33 37 | 0 0 0 0 .9.9137|
000009e0 34 38 39 32 39 30 39 37 36 31 20 36 38 39 35 20 |4892909761 6895 |
000009f0 31 30 20 39 20 31 30 20 33 20 36 20 33 20 30 20 |10 9 10 3 6 3 0 |
00000a00 31 20 31 20 33 20 32 20 30 20 30 20 30 20 31 20 |1 1 3 2 0 0 0 1 |
00000a10 31 20 31 20 31 20 30 20 30 20 30 20 30 20 30 20 |1 1 1 0 0 0 0 0 |
00000a20 30 20 30 20 30 20 30 20 30 20 30 20 30 20 30 20 |0 0 0 0 0 0 0 0 |
*
00000b80 0a 31 31 2e 34 34 34 32 33 36 36 32 37 34 31 33 |.11.444236627413|
00000b90 32 32 34 20 36 38 32 30 20 31 34 20 39 20 38 20 |224 6820 14 9 8 |
00000ba0 37 20 33 20 35 20 32 20 32 20 31 20 33 20 30 20 |7 3 5 2 2 1 3 0 |
00000bb0 30 20 31 20 31 20 30 20 31 20 30 20 31 20 31 20 |0 1 1 0 1 0 1 1 |
00000bc0 31 20 31 20 30 20 30 20 30 20 30 20 30 20 30 20 |1 1 0 0 0 0 0 0 |
00000bd0 30 20 30 20 30 20 30 20 30 20 30 20 30 20 30 20 |0 0 0 0 0 0 0 0 |
*
00000d20 30 20 30 20 30 20 30 20 0a 31 33 2e 30 30 31 32 |0 0 0 0 .13.0012|
00000d30 35 31 30 32 39 31 35 32 38 31 39 20 36 37 36 31 |51029152819 6761|
00000d40 20 35 20 36 20 39 20 37 20 34 20 35 20 31 20 33 | 5 6 9 7 4 5 1 3|
00000d50 20 32 20 33 20 30 20 31 20 31 20 31 20 30 20 31 | 2 3 0 1 1 1 0 1|
00000d60 20 31 20 30 20 30 20 30 20 32 20 30 20 33 20 30 | 1 0 0 0 2 0 3 0|
00000d70 20 30 20 30 20 30 20 30 20 30 20 30 20 30 20 30 | 0 0 0 0 0 0 0 0|
*
00000ec0 20 30 20 30 20 30 20 30 20 30 20 30 20 30 20 0a | 0 0 0 0 0 0 0 .|
00000ed0 31 34 2e 36 30 36 34 35 36 30 30 38 34 31 33 30 |14.6064560084130|
00000ee0 35 33 20 36 36 35 37 20 31 32 20 34 20 39 20 38 |53 6657 12 4 9 8|
00000ef0 20 34 20 32 20 33 20 33 20 36 20 31 20 32 20 32 | 4 2 3 3 6 1 2 2|
00000f00 20 31 20 30 20 30 20 30 20 30 20 33 20 30 20 30 | 1 0 0 0 0 3 0 0|
00000f10 20 30 20 33 20 31 20 30 20 30 20 30 20 31 20 31 | 0 3 1 0 0 0 1 1|
00000f20 20 30 20 30 20 30 20 30 20 30 20 30 20 30 20 30 | 0 0 0 0 0 0 0 0|
*
00001070 20 30 20 30 20 30 20 0a | 0 0 0 .|
00001078