C++ text stream random access

Orkie · 10-26-2005, 11:57 AM

First, some explanation as to what I'm doing and why (just in case anybody spots any problems with the logic of my program).
I am writing an interpreter for a Logo-based programming language (yes the one with the turtle, but even the original Logo can do more than just draw pictures). I have got it to the stage where you can set a background colour, pen colour and draw lines. Now I want to add in the REPEAT command.

Basically, how I am approaching this is that in the function which calls the red command, validate command and execute command functions, I am making it detect if the REPEAT command has been used (after reading and validation) and if it has, instead of sending it down a different route than if it weren't the REPEAT command (if REPEAT is not picked up, it goes straight onto executing the command). The command reading function reads it as if it were a normal text file (so with << and >>).
Down this other path, the first thing that it does is push_back() the current location in the file (obtained with the tellg() ifstream member function) onto a vector<ios:: off_type> (the space is in there to prevent smilification) named stack. It then enters a while loop which will keep going on until it has looped for the same number of times as was passed to REPEAT as a parameter. This while loop calls another function named repeatCommand(). In this function is a call to readCommand(), validateCommand and executeCommand() plus the same special REPEAT branch as before (to enable embedded loops).
After it has called repeatCommand() and that has returned, it check to see if the ']' character has been found (which indicates the end of a code block) and if it has, it reads the file location that I stored on the stack earlier on and seekg()s back to it (so the next thing to be read should be the first command inside the code block). Then increments a loop variable so that it knows when to stop REPEATing the commands. Now it goes back and executes repeatCommand() and this is where it fails. It works fine, executing the commands inside the code block before the file location is moved back but as soon as it is moved back, it fails -- the program closes straight away since one of the built-in error checks in readCommand() has been triggered. it is a fatal error and 'cout <<'s "EOF reached before command ending" (which is a fairly self explanatory error).

Now, I have stepped through the program with GDB many, many times and have decided that it can't possibly be anything other than the point at which the file pointer is moved. All the other parts are working perfectly up until that is moved. So, does anybody have any idea what is going wrong here?

Here is the code to the two functions which contain the file pointer move:

processFile() (calls repeatCommand())

Code:

void processFile(char *filename)
{
     SDL_Surface *screen = SDL_GetVideoSurface();

     // Prepare screen for drawing
     boxRGBA(screen, 0, 0, screen->w, screen->h, backgroundColour.r, backgroundColour.g, backgroundColour.b, 255);
     
     // Open code file for reading only (no point in a full fstream)
     ifstream file;
     file.open(filename);
     
     // Create the vector which will store the commands temporarily
     vector<string> command;
     if(file.is_open())
     {
                       while(!file.eof())
                       {
                             command = readCommand(file);
                             if(validateCommand(command))
                             {
                                                         // If the repeat command is not found, execute command as normal. However, if it IS found, run it in that special REPEATish way
                                                         if(command[0] != "REPEAT")
                                                         {
                                                             if(!executeCommand(command))
                                                             {
                                                                                         error("A problem occured whilst executing the command");
                                                             }
                                                         }
                                                         else
                                                         {
                                                             stack.push_back(file.tellg());
                                                             unsigned int loop = 0;
                                                             unsigned int repeat = atoi(command[1].c_str());
                                                             
                                                             while(loop != repeat)
                                                             {
                                                                        repeatCommand(file, command);
                                                                        
                                                                        if(command[0] == "]")
                                                                        {
                                                                                      // Set file pointer to location that was stored on the stack - the last entry for LIFO
                                                                                      file.seekg(stack[stack.size() - 1], ios::beg);
                                                                                      
                                                                                      // Increment 'loop';
                                                                                      loop++;
                                                                        }
                                                             }
                                                             
                                                             // Pop that file location off the end of the stack
                                                             stack.erase(stack.end(), stack.end());
                                                         }
                             }
                             else
                             {
                                 error("Invalid syntax for command");
                             }
                       }
                       
                       file.close();
     }
     else
     {
         // Just in case something has happened in those few milliseconds which ruined the file...
         error("No file was specified or specified file couldn't be opened", false, false);
     }
     return;
}

repeatCommand()

Code:

void repeatCommand(ifstream &file, vector<string> &command)
{
     command = readCommand(file);
     if(validateCommand(command))
     {
                                 // If the repeat command is not found, execute command as normal. However, if it IS found, run it in that special REPEATish way
                                 if(command[0] != "REPEAT")
                                 {
                                               if(!executeCommand(command))
                                               {
                                                                           error("A problem occured whilst executing the command");
                                               }
                                 }
                                 else
                                 {
                                               stack.push_back(file.tellg());
                                               unsigned int loop = 0;
                                               unsigned int repeat = atoi(command[1].c_str());
                                                             
                                               while(loop != repeat)
                                               {
                                                          repeatCommand(file, command);
                                                                        
                                                          if(command[0] == "]")
                                                          {
                                                                        // Set file pointer to location that was stored on the stack - the last entry for LIFO
                                                                        file.seekg(stack[stack.size() - 1], ios::beg);
                                                                                      
                                                                        // Increment 'loop';
                                                                        loop++;
                                                          }
                                               }
                                                             
                                               // Pop that file location off the end of the stack
                                               stack.erase(stack.end(), stack.end());
                                 }
     }
     else
     {
      error("Invalid syntax for command");
     }
}

This is the repeat code that is stored in the program file:

Code:

REPEAT 10 
[ 
	BACKWARD 200 ; 
]

I'm not interested in making the code efficient at the moment, just getting it working is my priority.
Thanks

naf · 10-26-2005, 01:02 PM

First, avoid using stack. It may cause confusion with the STL version of stack.

I am having difficulty getting the line input correctly. Perhaps if you provide readCommand.

Have you considered using a syntax parser like lex/yacc combo?

Code:

#include <iostream>
#include <fstream>
#include <vector>
#include <string>

using namespace std;

void processFile( char *filename );
void repeatCommand( ifstream &file, vector< string > &command );

vector<ifstream::pos_type> pstack;
vector<string> command_buffer;


int main( int argc, char **argv )
{
    
    if( argc > 1 )
        processFile( argv[1] );
	else
		processFile( "test.txt" );
    
    
    return 0;
}

void error( const char *message, ... )
{
    cerr << message << endl;
}

bool validateCommand( ... )
{
    // stub in for now.
    return true;
}

bool executeCommand( ... )
{
    // stub in for now.
    return true;
}

vector<string> &readCommand( ifstream &file )
{
    string entry;
    char buffer[1024];
    while( file >> entry )
        command_buffer.push_back( entry );
    
    return command_buffer;
}

void processFile(char *filename)
{
/*
    SDL_Surface *screen = SDL_GetVideoSurface();
    
    // Prepare screen for drawing
    boxRGBA(screen, 0, 0, screen->w, screen->h, backgroundColour.r, backgroundColour.g, backgroundColour.b, 255);
*/
    
    // Open code file for reading only (no point in a full fstream)
    ifstream file;
    file.open(filename);
    
    // Create the vector which will store the commands temporarily
    vector<string> command;
	if( file.is_open() )
    {
        while(!file.eof())
        {
            command = readCommand( file );

            if( validateCommand( command ) )
            {
                // If the repeat command is not found, execute command as normal. However, if it IS found, run it in that special REPEATish way
                if( command[0] != "REPEAT" )
                {
                    if( ! executeCommand( command ) )
                    {
                        error( "A problem occured whilst executing the command" );
                    }
                }
                else
                {
                    pstack.push_back( file.tellg() );
                    unsigned int loop = 0;
                    unsigned int repeat = atoi( command[1].c_str() );
                    
                    while(loop != repeat)
                    {
                        repeatCommand(file, command);
                        
                        if(command[0] == "]")
                        {
                            // Set file pointer to location that was stored on the stack - the last entry for LIFO
                            file.seekg( pstack[pstack.size() - 1], ios::beg );
                            
                            // Increment 'loop';
                            loop++;
                        }
                    }
                    
                    // Pop that file location off the end of the stack
                    pstack.erase( pstack.end(), pstack.end() );
                }
            }
            else
            {
                error("Invalid syntax for command");
            }
        }
        
        file.close();
    }
    else
    {
        // Just in case something has happened in those few milliseconds which ruined the file...
        error( "No file was specified or specified file couldn't be opened", false, false );
    }
    return;
}

void repeatCommand(ifstream &file, vector<string> &command)
{
    command = readCommand(file);
    if(validateCommand(command))
    {
        // If the repeat command is not found, execute command as normal. However, if it IS found, run it in that special REPEATish way
        if(command[0] != "REPEAT")
        {
            if(!executeCommand(command))
            {
                error("A problem occured whilst executing the command");
            }
        }
        else
        {
            pstack.push_back( file.tellg() );
            unsigned int loop = 0;
            unsigned int repeat = atoi( command[1].c_str() );
            
            while( loop != repeat )
            {
                repeatCommand( file, command );
                
                if( command[0] == "]" )
                {
                    // Set file pointer to location that was stored on the stack - the last entry for LIFO
                    file.seekg( pstack[pstack.size() - 1], ios::beg );
                    
                    // Increment 'loop';
                    loop++;
                                                          }
            }
            
            // Pop that file location off the end of the stack
			
            pstack.erase( pstack.end(), pstack.end() );
        }
    }
    else
    {
        error("Invalid syntax for command");
    }
}

Quigi · 10-26-2005, 01:08 PM

Regarding the design, I might have decided to read the commands in a REPEAT block into a vector (or some more sophisticated, symbolic representation), and then repeat that, so I never have to back up on the file. That way, it could work interactively by reading standard input (which doesn't let you back up -- use /dev/fd/0 as the filename).

I first suspected a buffering issue, but my trivial test program (at the bottom) seems to work just fine. I don't think it matters, but tellg() returns a ios::pos_type, and the one-argument seekg is sufficient.

I do see a problem, but it should happen after the REPEAT loop completes:

Code:

// Pop that file location off the end of the stack
stack.erase(stack.end(), stack.end());

The two argument erase erases the elements from the initial iterator up to but excluding the final iterator. In your case they are the same, so nothing gets popped. I'd use the single argument erase, i.e.,

Code:

stack.erase(stack.end()-1);

Anyway, here's my sample program:

Code:

#include <stdlib.h>
#include <string>
#include <iostream>
#include <fstream>

int main(int argc, char**argv) {
  std::ifstream file;
  file.open(argv[1]);
  std::string word;
  file >> word; std::cout << "a0 read " << word << "." << std::endl;
  file >> word; std::cout << "a1 read " << word << "." << std::endl;
  file >> word; std::cout << "a2 read " << word << "." << std::endl;
  std::ios::pos_type pos = file.tellg();
  std::cout << "Now at " << pos << "." << std::endl;
  file >> word; std::cout << "b0 read " << word << "." << std::endl;
  file >> word; std::cout << "b1 read " << word << "." << std::endl;
  file.seekg(pos);
  file >> word; std::cout << "c0 read " << word << "." << std::endl;
  file >> word; std::cout << "c1 read " << word << "." << std::endl;
  file >> word; std::cout << "c2 read " << word << "." << std::endl;
  file.close();
}

Orkie · 10-26-2005, 01:16 PM

I have considered them but they are yet another library which I would have to learn to use. I would like to get it working with this approach which I understand before I start fiddling with anything else.

readCommand()

Code:

// Contains the readCommand() function which reads an entire command (up to the
// semicolon) and shives it all into a vector

vector<string> readCommand(ifstream &file)
{
       vector<string> command;
       
       // Read up until a semicolon is found and just shove everything straight into a vector (in raw state)
       string temp;
       while(true)
       {
                  // Extract one word/character from the program file
                  file >> temp;
                  
                  // If semicolon is found, the command must be over
                  if(temp == ";")
                  {
                          break;
                  }
                  
                  // If the 'begin code block' character (i.e. '[') is found, command is over if the command is REPEAT
                  if(temp == "[" && command[0] == "REPEAT")
                  {
                          break;
                  }
                  
                  // If the semicolon or '[' hasn't been found, store it in a vector
                  command.push_back(temp);
                  
                  // If the ']' command was found (end of code block), break but only after adding it to the command vector
                  if(temp == "]")
                  {
                          break;
                  }
                  
                  // If EOF is reached before the end of a command, something is wrong
                  if(file.eof())
                  {
                                error("EOF reached before command ending", false, false);
                  }
       }
       
       return command;
}

I'll have a look at the 'read it all into a vector' approach, that could well work.

Quigi · 10-26-2005, 01:57 PM

One caveat with readCommand: if the first temp read is "[", then command[0] isn't defined yet where you compare it with "REPEAT". Of course that wouldn't happen in a correct Logo program. But if it does, it would be preferrable to blame the broken input file, rather than have an error (or undefined behavior?) in the interpreter.

Orkie · 10-26-2005, 02:11 PM

Thanks for pointing that out, I'll have a look at it when I get chance.
I've got the REPEAT command to work with the 'read it all into a vector' approach to some degree now. Now I'm just ironing out the bugs (currently it will only repeat one command, then stop).

EDIT: Get repeat working perfectly but then realised that my formulae for FORWARD and BACKWARD are wrong... I'm sure I can fix those though. I also managed to fix the readCommand() function whilst creating the REPEAT command.

naf · 10-26-2005, 02:48 PM

I am not familiar with the complete Logo syntax. It has been twenty years. Try this test:

Code:

REPEAT 10 
[ 
	BACKWARD 200 ; 
]
LEFT 90 ;
REPEAT 5
[
	FORWARD 10 ;
	RIGHT 20 ;
]

on this modification:

Code:

#include <iostream>
#include <fstream>
#include <vector>
#include <string>

using namespace std;

void processFile( char *filename );
bool executeCommandLine( ifstream &file );

vector< ifstream:: pos_type > pstack;
vector< string > command_buffer;


int main( int argc, char **argv )
{
    
    if( argc > 1 )
        processFile( argv[1] );
	else
		processFile( "test.txt" );
    
    
    return 0;
}

void error( const char *message, ... )
{
    cerr << message << endl;
}

bool validateCommand( ... )
{
    // stub in for now.
    return true;
}

bool executeCommand( vector< string > &command )
{
    // stub in for now.
	vector< string >::size_type index;

	cout << "Executing: ";

	for( index = 0; index < command.size(); index++ )
		cout << command[index] << " ";

	cout << endl;

    return true;
}

vector<string> &readCommand2( ifstream &file )
{
    string entry;
    while( file >> entry )
        command_buffer.push_back( entry );
    
    return command_buffer;
}

vector< string > &readCommand( ifstream &file, vector< string > &command )
{
    // Clear the current comamnd.
    command.clear();
    
    // Read up until a semicolon is found and just shove everything straight into a vector (in raw state)
    while( true )
    {
        string temp;

        // Extract one word/character from the program file
        file >> temp;
        
        // If semicolon is found, the command must be over
        if( temp == ";" )
        {
            break;
        }

        // If the 'begin code block' character (i.e. '[') is found, command is over if the command is REPEAT
        if( temp == "[" && command[0] == "REPEAT" )
        {
            break;
        }
                  
        // If the semicolon or '[' hasn't been found, store it in a vector
        command.push_back( temp );
        
        // If the ']' command was found (end of code block), break but only after adding it to the command vector
        if( temp == "]" )
        {
            break;
        }
        
        // If EOF is reached before the end of a command, something is wrong
        if( file.eof() )
        {
            error( "EOF reached before command ending", false, false );
			break;
        }
    }
    
    return command;  // <-- Costly!!! Remember that this is a copy operation.
}

void processFile(char *filename)
{
/*
    SDL_Surface *screen = SDL_GetVideoSurface();
    
    // Prepare screen for drawing
    boxRGBA(screen, 0, 0, screen->w, screen->h, backgroundColour.r, backgroundColour.g, backgroundColour.b, 255);
*/
    
    // Open code file for reading only (no point in a full fstream)
    ifstream file;
    file.open(filename);

    // Create the vector which will store the commands temporarily
    vector<string> command;
	if( ! file.is_open() )
        // Just in case something has happened in those few milliseconds which ruined the file...
        error( "No file was specified or specified file couldn't be opened", false, false );
    else
    {
        while( ! file.eof() ) 
			if( ! executeCommandLine( file ) )  // <-- Alternatively, consider using a try {} catch {} scheme.
				break;
        
        file.close();
    }

    return;
}

bool executeCommandLine( ifstream &file )
{
	vector< string > command;
    bool retval;

    for( retval = true; ( ! file.eof() ) && ( readCommand( file, command ) ).size() > 0; )
    {    
        // Verify valid command.  (Is this necessary?  I ask because you are parsing right now.)
        if( ! ( retval = validateCommand( command ) ) )
        {
            error( "Invalid command encountered." );  // <-- Alternatively, consider using throw
            break;
        }
        else
        {
			string subcommand = command[0];
            if( command[0] == "]" )
                break;
            else if( command[0] == "REPEAT" )
            {
                // Process a REPEAT block with the syntax of:
                // REPEAT <count> [ <stmt>* ]
                fstream:: pos_type pos;
                unsigned int loop;
                unsigned int repeat; 
                
                for( pos = file.tellg(), retval = true, repeat = atoi( command[1].c_str() ), loop = 0; retval && ( loop < repeat ); loop++ )
                    file.seekg( pos ), retval = executeCommandLine( file );
            }
            // If the repeat command is not found, execute command as normal. However, if it IS found, run it in that special REPEATish way
            else 
            {
                if( ! executeCommand( command ) )
                {
                    retval = false;
                    error( "A problem occured whilst executing the command" );
                }
            }
        }
    }
 
    // Return true if successfully executed command, false otherwise.
	return retval;
}

I get the following output:

Code:

Executing: BACKWARD 200 
Executing: BACKWARD 200 
Executing: BACKWARD 200 
Executing: BACKWARD 200 
Executing: BACKWARD 200 
Executing: BACKWARD 200 
Executing: BACKWARD 200 
Executing: BACKWARD 200 
Executing: BACKWARD 200 
Executing: BACKWARD 200 
Executing: LEFT 90 
Executing: FORWARD 10 
Executing: RIGHT 20 
Executing: FORWARD 10 
Executing: RIGHT 20 
Executing: FORWARD 10 
Executing: RIGHT 20 
Executing: FORWARD 10 
Executing: RIGHT 20 
Executing: FORWARD 10 
Executing: RIGHT 20 
EOF reached before command ending
Executing: