LinuxQuestions.org - [SOLVED] Ada : read > 8 M from file = Memory fault

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - Ada : read > 8 M from file = Memory fault (https://www.linuxquestions.org/questions/programming-9/ada-read-8-m-from-file-%3D-memory-fault-929968/)

Ada : read > 8 M from file = Memory fault

Hi,

I am trying to find a way to read large size blocks from a text file in Ada. Whatever I try, however, I get memory faults when trying to read more than 8.5 M. Anyone an idea how I should really do this ?

This programme: memory fault

Code:

with ada.streams.stream_io ;use ada.streams.stream_io;

with ada.text_io;

procedure memfault is

  bsize:constant integer :=8_376_929;

file:file_type;

ostream:stream_access;

str:string(1..bsize);

begin



open(file, in_file,name=>"file.txt");

ostream:=stream(file);

string'read(ostream,str);

close(file);

end memfault;

This programme : memory fault

Code:

with ada.text_io; use ada.text_io;

with ada.direct_io;



procedure memfault is

  bsize:constant integer:=8_386_529;

  type buf is new string (1..bsize);

  package directio is new ada.direct_io(buf);

begin



null;



end memfault;

I guess the string is being allocated on the stack so the segfault is caused by stack overflow. You'll need to allocate the memory from the heap; I gather from this page that ada calls this "pool access".

Thank you

I made the streams version work by using a pointer. Unfortunately streams turn out to be extremely slow, I had not thought of that.
A small check with a 4 M buffer with direct_io shows that direct_io does have an acceptable speed. I have not found a solution for that package however, because it needs a type as an argument.

But for the streams variant, this works:

Code:

with ada.streams.stream_io ;use ada.streams.stream_io;

with ada.text_io;

procedure memfault is

  bsize:constant integer :=28_000_000;

  type bufarr is  new string(1..bsize);

  type buf is access all bufarr;



file:file_type;

ostream:stream_access;

str:buf:=new bufarr;

begin



open(file, in_file,name=>"test.txt");

ostream:=stream(file);

bufarr'read(ostream,str.all);

ada.text_io.put(str.all(69800));

ada.text_io.new_line;

close(file);

ada.text_io.put_line("done");

end memfault;

I compiled it with -gnatp -O2. But it is way too slow, on my antique system 3.9 seconds against 0.08 seconds for c++. direct_io seems to be the only option.

well I found it
the code should be

Code:

with ada.text_io, Interfaces.C_Streams;

use interfaces.c_streams;

procedure memfault is

  bsize:constant integer:=28_000_000;

  type bufarr is  new string(1..bsize);

  type buf is access all bufarr;

  strbuf:buf:=new bufarr;

  str:string:="test.txt"&ASCII.NUL;

  ms:constant string:="rw";

  fio: FILEs;

  res:interfaces.c_streams.size_t;

  resi:integer;

begin

  fio:=fopen(str'address,ms'address);

  res:=fread(strbuf.all'address,size_t(bsize),1,fio);

  ada.text_io.put(strbuf.all(69800));

  ada.text_io.new_line;

  resi:=fclose(fio);

  ada.text_io.put_line("done");

end memfault;

I abandoned Ada last year after learning it, exactly for the reason that you cannot get around the C-libraries, and need to work with the translations between type names in interfaces.C. Well, no choice.
The above is just as fast as the c++code below. My input files are utf-8, but the seperator tokens are all ascii, so this does not pose a problem.
My guess is that gcc compiles it to about the same thing as

Code:

#include <iostream>

#include <fstream>

#define BSIZE 28000000



using namespace std;



int main(){



char * strbuf= new char [BSIZE+1];

ifstream fio;



fio.open("test.txt");

fio.read(strbuf,BSIZE);



 cout << strbuf[69799]<<endl;

 cout << "done"<< endl;

 fio.close();

 return 0;

}