LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 02-17-2012, 05:28 PM   #1
muggabug
LQ Newbie
 
Registered: Aug 2011
Posts: 18

Rep: Reputation: Disabled
Ada : read > 8 M from file = Memory fault


Hi,

I am trying to find a way to read large size blocks from a text file in Ada. Whatever I try, however, I get memory faults when trying to read more than 8.5 M. Anyone an idea how I should really do this ?

This programme: memory fault
Code:
with ada.streams.stream_io ;use ada.streams.stream_io;
with ada.text_io;
procedure memfault is
   bsize:constant integer :=8_376_929;
file:file_type;
ostream:stream_access;
str:string(1..bsize);
begin

open(file, in_file,name=>"file.txt");
ostream:=stream(file);
string'read(ostream,str);
close(file);
end memfault;
This programme : memory fault
Code:
with ada.text_io; use ada.text_io;
with ada.direct_io;

procedure memfault is
   bsize:constant integer:=8_386_529;
   type buf is new string (1..bsize);
   package directio is new ada.direct_io(buf);
begin

null;

end memfault;
 
Old 02-18-2012, 06:15 PM   #2
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian
Posts: 2,454

Rep: Reputation: 841Reputation: 841Reputation: 841Reputation: 841Reputation: 841Reputation: 841Reputation: 841
I guess the string is being allocated on the stack so the segfault is caused by stack overflow. You'll need to allocate the memory from the heap; I gather from this page that ada calls this "pool access".
 
Old 02-19-2012, 11:26 AM   #3
muggabug
LQ Newbie
 
Registered: Aug 2011
Posts: 18

Original Poster
Rep: Reputation: Disabled
Thank you

I made the streams version work by using a pointer. Unfortunately streams turn out to be extremely slow, I had not thought of that.
A small check with a 4 M buffer with direct_io shows that direct_io does have an acceptable speed. I have not found a solution for that package however, because it needs a type as an argument.

But for the streams variant, this works:
Code:
with ada.streams.stream_io ;use ada.streams.stream_io;
with ada.text_io;
procedure memfault is
   bsize:constant integer :=28_000_000;
   type bufarr is  new string(1..bsize);
   type buf is access all bufarr;

file:file_type;
ostream:stream_access;
str:buf:=new bufarr;
begin

open(file, in_file,name=>"test.txt");
ostream:=stream(file);
bufarr'read(ostream,str.all);
ada.text_io.put(str.all(69800));
ada.text_io.new_line;
close(file);
ada.text_io.put_line("done");
end memfault;
I compiled it with -gnatp -O2. But it is way too slow, on my antique system 3.9 seconds against 0.08 seconds for c++. direct_io seems to be the only option.
 
Old 02-19-2012, 04:58 PM   #4
muggabug
LQ Newbie
 
Registered: Aug 2011
Posts: 18

Original Poster
Rep: Reputation: Disabled
solution

well I found it
the code should be
Code:
with ada.text_io, Interfaces.C_Streams;
use interfaces.c_streams;
procedure memfault is
   bsize:constant integer:=28_000_000;
   type bufarr is  new string(1..bsize);
   type buf is access all bufarr;
   strbuf:buf:=new bufarr;
   str:string:="test.txt"&ASCII.NUL;
   ms:constant string:="rw";
   fio: FILEs;
   res:interfaces.c_streams.size_t;
   resi:integer;
begin
   fio:=fopen(str'address,ms'address);
   res:=fread(strbuf.all'address,size_t(bsize),1,fio);
   ada.text_io.put(strbuf.all(69800));
   ada.text_io.new_line;
   resi:=fclose(fio);
   ada.text_io.put_line("done");
end memfault;
I abandoned Ada last year after learning it, exactly for the reason that you cannot get around the C-libraries, and need to work with the translations between type names in interfaces.C. Well, no choice.
The above is just as fast as the c++code below. My input files are utf-8, but the seperator tokens are all ascii, so this does not pose a problem.
My guess is that gcc compiles it to about the same thing as
Code:
#include <iostream>
#include <fstream>
#define BSIZE 28000000

using namespace std;

int main(){

char * strbuf= new char [BSIZE+1];
ifstream fio;

fio.open("test.txt");
fio.read(strbuf,BSIZE);

 cout << strbuf[69799]<<endl;
 cout << "done"<< endl;
 fio.close();
 return 0;
}

Last edited by muggabug; 02-19-2012 at 05:16 PM. Reason: correction
 
  


Reply

Tags
segfault


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Ada: How do I read large blocks of characters and get strings ? muggabug Programming 0 02-10-2012 11:46 AM
How do I buffer textinput from file in Ada (Ada newbie) muggabug Programming 3 11-03-2011 01:28 AM
ADA Development tools in Linux for ADA 83 and ADA95 ehudnatan Linux - Software 2 05-11-2011 09:58 AM
c++ in linux to read a PGM file (portable graymap) onto the memory and print hight singh.vikas85 Linux - Newbie 4 05-06-2009 07:14 PM
gdb error "Failed to read a valid object file image from memory." Fejimush Programming 0 03-11-2007 01:40 PM


All times are GMT -5. The time now is 11:38 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration