FLEX/LEX input question

ankit4u1 · 05-31-2006, 12:53 PM

Hi,

I want to give my input for my compiler like this:

{1,2,3,4} + {4,5,2,7,8}

i.e. set union set ,where + stands for set.

So, for that purpose, I have to take the set as a token.

I thought like this:

set [{(a-zA-Z0-9),)+}]

so that, when this token is scanned,

[{(a-zA-Z0-9),)+}] {return set}

i can return the token in yytext in this manner {1,2,3,4}

I am a bit skeptical about the opening parenthesis and comma. Is this the correct way ( the one shown above)?

Hope to get a reply from you soon. Thank you.

ankit4u1 · 06-01-2006, 12:09 PM

Hey freinds. I am desperately waiting for yr replies. I am a newbie to this tool lex.

Hope to get a reply soon.

I would be grateful to you, if u can help me.

taylor_venable · 06-01-2006, 08:45 PM

I can't give you an explicit solution, since your problem is still a little fuzzy in my mind. But if you want to create a sort of language that can take the union of two sets expressed like that, I would do it a different way. Lex was written to be (obviously) a lexer, which handles syntactic forms at the lowest level possible, translating lexemes (found in the source code) into tokens (which have actual programatic meaning). Yacc, often referred to as Lex's companion, was created to take these tokens and use the context in which they occur to grant them meaning, that is, to transform syntax into semantics. So, I would do something like this (sorry, no source code now; but I may be able to come up with something later if you want; it's been a while since I wrote Lex or Yacc):

Use Lex to detect the important tokens in your set - '{' to start, '[A-Za-z0-9]+' for an element, ',' to separate them, and '}' to close. Plus the union operator '+'.

Then use Yacc to put together a complete set, something like: SET := SET_START [SET_ELEMENT SET_ELEMENT_SEPARATOR]+ SET_END. Also, create the rule for taking a union: UNION := SET + SET.

Last, write the C code for what should happen when the compiler reduces to the UNION rule. At this point, I believe you'd have a language capable of dealing with set unions, like you wanted.

ankit4u1 · 06-02-2006, 02:20 AM

Yes. that's a great idea. But I am am not able to understand how to give opening parenthesis i.e. { as input. Coz, as per my thinking, the opening brace is for action and not for rules. I mean to say sumthing like this:

[a-z]+ { return xyz}

So, here the scanner (LEX/FLEX) takes opening parenthesis as action. So i want to know how to give this opening brace as input. Also, how to give comma as input? bcoz of these two things i am trapped.

And yes, as per your suggestion of giving the input in 3 token form: like opening parenthesis, then elements, then separator and finally again closing parenthesis, I cannot do that, because I am using STL (Standard Template Library).

In STL, there are funtions which take input parameters, which in this case of set operations will be whole set e.g.{1,2,3,4} in the form of array. So, for that purpose, I need to create a token like {1,2,3,4} which contains parenthesis,elements and comma. Then from yacc, i'll convert this token to array by user defined function , which i have already coded. And then i'll pass the array to function of STL.

What do u think. Isn't it a good idea?

I think, you got what i meant.

Hope to get a reply soon. Thanks.

ankit4u1 · 06-03-2006, 04:15 AM

Hey friends. I cleared up the doubt. There's no way out to give opening brace i.e. { as input or token in lex/flex, when ever a scanner scans the opening brace {, it takes it as an opening of an action and not as a token. So for this reason, we cannot give opening brace as input i.e. in rules section of LEX/FLEX.

I was wondering, why ain't I getting reply. But then i found it out and conformed it.

Thanks any ways... c ya !!

Cheers,
Ankit

elsheikhmh · 06-03-2006, 06:54 AM

Quote:

There's no way out to give opening brace i.e. { as input or token in lex/flex

why not

Code:

"{"

elsheikhmh · 06-03-2006, 07:01 AM

moreover, i do think that yacc is more suitable, conside

Code:

%token ITEM
%start union
set : "{" set_contents "}" { $$ = $2; }
set_contents : ITEM { $$ = newSet($1); } 
             | set_contents "," ITEM  { $$ = addToSet($1, $3); }
unoin : set "+" set { performUnoinAction($1, $3); }

with utilities, newSet(), addToSet(), and performUnoinAction() performing proper set building, and attaching item to set, and performing union action using your own set data structure representation.
ITEM may be recognized using lex, or just yylex() in .yacc source

i don't think that lex is suitable of recognizing the set. however if u prefer:

Code:

item [^,] /*anything that's not a comma*/
set "{" {item} ("," {item})* "}" /*signle item followed by zero or more comma-item*/
unoin {set} "+" {set}

happy lexing,,
Mustafa

ankit4u1 · 06-04-2006, 11:48 AM

Well, yes this is a good idea. But the problem is that I am using STL which computes all the set operations : union, intersection, set difference and set symmetric difference.

For this purpose, I need to have a token in the form of a set with brace: e.g. {1,2,3,4}

Code:

set : "{" set_contents "}" { $$ = $2; }

This code means that, $$ won't be able to pass braces to the function. Althouth,i can modify the set and add braces in the user defined function.

I came up with another idea. Instead of braces, i'll keep some other symbols like A,E..watever. e.g. A1,2,3,4E

Then, i'll pass this character pointer to a user defined function,where i'll convert it into an array and then pass this array to STL function (union,intersection,etc), as these functions take array as an parameter. The following is the code i have initially thought to convert the character pointer to an array.

Code:

#include<stdio.h>
#include<conio.h>
#include<string.h>
void main()
{
	char *c="{a,b,c,d,e,f,1,2,3}";
	char a[10];
	int i,j;

	for(i=1,j=0;i<strlen(c);i=i+2,j++)
	{
		a[j]=c[i];
	}
	for(i=0;i<j;i++)
	printf("a[%d]= %c\n",i,a[i]);
	getch();
}

This code is just a logic. I'll use this logic to convert the two sets into array and pass it to the union or xyz function respectively.
Again convert it into character pointer after computation of the sets by the function and return. so $$ will get a char pointer back.

If that doesn't work out, i'll try your option. You have given me a good idea. Thank for the help.

Hope to get a reply soon. Lemme know if i have made any mistake. Thanks.

Regards,
Ankit.

ankit4u1 · 06-14-2006, 01:40 PM

hey i have worked out on this problem and have finally solved it !!! Thanks for you inputs and help !!!!

Instead of comma and starting brace i.e. {, i used the set in this way,

aABCD1234e

where,
a=starting element of the set
e=ending element of the set
all other elements in between are members of the set. Here they are A B C D 1 2 3 4

The only constraint is that i have to take the element as a character i.e. single char. Can't take it like A1,B1....Will soon try to overcome that. Once my project is successful, i'll try to improve it.

Once again to all my friends !!! thanks for extending you help !!!

Regards,
Ankit.

pragnya · 06-28-2006, 03:49 AM

Hi,
I want to give input to the yacc file from a QT dialog box. Is there a way where I can pass a string directly to yyparse() or should I write the string from the dialog into a file and then read the input from this file?
How do I read the contents of a file as input to the parser? Also can I rename the main function in the yacc file as something else and call this function from my application program while passing the input string from the dialog box to be parsed?
Please help!! I am desperate!

ankit4u1 · 06-30-2006, 12:45 AM

hi,

Quote:

I want to give input to the yacc file from a QT dialog box.

I think you are using JAVA !!! If that is the case, I am sorry bcoz i am not good at Java.

But yes, if you pass yr input to a file from the dialog box and then to parser, I'll tell you how.

Quote:

How do I read the contents of a file as input to the parser?

In the LEX file, you have to define a main in this fashion:

Code:

main(argc,argv)
int argc;
char **argv;
      {
               ++argv,--argc;  /* skip over program name; */
               if(argc>0)
               { 
                        yyin=fopen(argv[0],"r"); /* will read a file named argv[0], i.e. the name of the file which you pass as yr 1st argument*/

               else

                        yyin = stdin;
   
                yylex();
        }

Here, in the lex file, yr tokens will be created according to the regular expression you have defined.

Now, when the yyparse function will be called from you main() in YACC file, it will call yylex(), which will pass this tokens to yyparse, and as per the grammar rules you have defined, respective actions wil be taken.

Code:

Also can I rename the main function in the yacc file as something else and call this function from my application program while passing the input string from the dialog box to be parsed?

I coun't get this question of yours. Please be more specific with an example.

By the way, what are you designing with lex and yacc?

SOME USEFUL BOOKS and links for LEX and YACC.

Unix programming tools: LEX & YACC: John Levine,Tony Mason, Doug Brown

For lex, refer manual page of flex. Its worth reading and will cover all the required topics.

For BISON(latest version of yacc, which also supports c++ interface): www.gnu.org/software/bison/manual/pdf/bison.pdf

Lex and Yacc online manual: http://dinosaur.compilertools.net/

All The Best !!!!

Cheers,
Ankit.

ankit4u1 · 06-30-2006, 01:41 AM

Quote:

Also can I rename the main function in the yacc file as something else and call this function from my application program while passing the input string from the dialog box to be parsed?

Do you mean that u want to override main()?

If yes, then do refer to a thread , which describes how to override main. Just search in the search thread "override main()". U'll get the thread.

You would get everything there. But again, I don't understand y do u want to rename main()? Intriguing question !!!

Cheers,
Ankit.

pragnya · 06-30-2006, 03:55 AM

Oops sorry for the error. What I meant was, do i have to call yyparse() from main or can i call it from some other function? I guess I got the answer for this. I think all i have to do is declare yyparse as extern in the file where I am calling yyparse.
And I am doing this in C++.
As i understand the lexer reads the input from the file pointed to by yyin, but is there any way that i can pass a string to be parsed to lexer directly without going thru yyin?
I have another question... Lets keep it simple...
I give an input string from the console/command prompt, i press enter--->parsing begins. I get the final result.
Now What should i do if i want to give the input again to the parser whithout pressing ctrl-c and then doing ./a.out again?
i.e I want to give another input string continously to parse after every successful parse!

PS: Thanks for the reply earlier, helped a lot

ankit4u1 · 07-02-2006, 02:28 PM

I am glad that it helped u a lot....

Quote:

do i have to call yyparse() from main or can i call it from some other function?

Well, as i told u in my previous reply also that i havn't tried this part. Perhaps u try out the same part by executing the some running program i.e. ready made. I'll tell u but i do not have time to do all these stuff as i don't get free time from my job. I can reply u next sunday when i'll b free regarding this question and querry. I am replying now to your queries bcoz i have been through this phase of learning and i know how much curious one is.

Quote:

but is there any way that i can pass a string to be parsed to lexer directly without going thru yyin?

Well, u yourself have answered to this question in yr thread. Here it is.

I give an input string from the console/command prompt

Quote:

What should i do if i want to give the input again to the parser whithout pressing ctrl-c and then doing ./a.out again?

Lemme explain u with an example of calculater.

Once u do ./calc, where calc is the name if executable file in simple terms or output file in linux terms, u'll give yr input on terminal like this

1+2

Now u'll get output like this ( as per the calc prog. designed):

Result = 3

Now, again it will ask you and wait until u give some input. U do not need to press cntl c or cntl z or cntl d and then do a ./calc. Just type your regular expression(input) again as shown above like this

1+4+5

And this way, it goes on until u press cntl c or cntl z !!! I hope this was the answer to your question.

All the best !!!

Cheers,
Ankit.

pragnya · 07-03-2006, 04:15 AM

Hi Ankit,
I finally got the multiple input thing resolved. You are right, it does work as you told, i.e wait for another input after every parse.

But I still dunno how i can pass an input string directly from the dialog box to the parse function without going thru another file/console.

Regards,
Pragnya