Hi,
In using flex++ (with c++), my problem is that the lexer skips a character (be it a space or some other character) after every token form the input file. It does not skip and works fine when I do not give a "return" statement in the c++ code segment for tokens. I dont know whats going on, and how can I make the lexer not skip any characters and also use "return" in the code. I am pasting my lex file here:
%{
#include <iostream>
#include <fstream>
using namespace std;
const int OVER = 256;
const int NAME = 257;
const int SYMBOL = 258;
const int ERROR = 259;
const int SPACE = 260;
%}
space [ \t]
alpha [A-Za-z_]
digit [0-9]
alphanum {alpha}|{digit}
nonalphanum [^A-Za-z0-9_]
%%
{space} { cout<<"SPACE"<<endl; return SPACE; }
over/{nonalphanum} { cout<<"OVER: '"<<yytext<<"'"<<endl; return OVER; }
{alpha}{alphanum}* { cout<<"Id: '"<<yytext<<"'"<<endl; return NAME; }
[,;:\.+=\-()] { cout<<"Symbol: '"<<yytext<<"'"<<endl; return SYMBOL; }
. { return ERROR; }
%%
int main(int argc, char** argv)
{
int token = 0;
FlexLexer* lexer = new yyFlexLexer;
ifstream inp;
inp.open(argv[1]);
while(token = lexer->yylex(&inp))
{
cout<<"TOKEN: "<<token<<" ("<<lexer->YYText()<<")"<<endl;
}
inp.close();
}
----------------------------------------------------------------
a segment of input file:
int count;
int y;
;
default cell=cell;
----------------------------------------------------------------------
a segment of the output:
Id: 'int'
TOKEN: 257 (int)
Id: 'count'
TOKEN: 257 (count)
Id: 'int'
TOKEN: 257 (int)
Id: 'y'
TOKEN: 257 (y)
Symbol: ';'
TOKEN: 258 (
Id: 'default'
TOKEN: 257 (default)
Id: 'cell'
TOKEN: 257 (cell)
Id: 'cell'
TOKEN: 257 (cell)
-------------------------------------------------------------
Here you can see that the semicolon (
is missing in the output (so are spaces) from the first line. Consider the last line, the equal to sign is missing (=), so is the semicolon again.
When I remove the return statements from my code (eg. eturn OVER
, then it works fine for the character after that particular token, but not when i give the return statement.
can someone please tell me what am I doing wrong here, or what is going on?
FLEX++ skipping characters
