LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 10-25-2021, 03:21 PM   #31
schmitta
Member
 
Registered: May 2011
Location: Blacksburg VA
Distribution: UBUNTU, LXLE
Posts: 352

Original Poster
Rep: Reputation: Disabled

I call my program outline as I used it to produce outlines and write letters and text (in office memo and documentation for our state labs project in out VPI&SU computer lab back in 1975 to 81. I call this an automatic editor. I first wrote it in 1972 in PL/1 for an IBM mainframe and wrote my thesis with it. Then when I was separated from Virginia Tech in 1981 I rewrote it in bill gates Microsoft basic. I produced a document with it outlining how to use it but that only exists in paper form as the Microsoft basic version is on a 486 that lost it bios battery that was not a coin cell so the computer is trash until I reengineer it for coin cell batteries. Several years ago I tried to write it in C and have a fist class tokenizer that uses an array of structures to define each token with its own structure and that works. It also has a command processor for the .lm etc. commands using pointers and a structure that is an array for each command and its number that identifies it in an switch statement. I have written most of a compiler with the tokenizer, finite state machines to parse floats and expression analyzer and an infix to post fix stack interpreter for handling equations. I took a compiler course in 1973 in graduate school and have written a number of symbol tables.
I know how to implement the commands in C they are just left margin, right margin top margin bottom margin page number , page number on off etc. .
It is way easier to write this in Lex and Yacc it is just knowing all the nuances for when a $1 is a number vs a string etc. that I finally found out using union for intValue stringValue etc. to take the $1 ($<intValue>1) which I did not know about or that "." is not the same as \. etc. My tokenizing separator is the blank which separates the words and commands which start in column one.

So My problem is I do not know Lex and Yacc and all the available regular expressions tokens like -s which means all character except \n so you don't get trapped by . even though a word can consist of any printable character.
 
Old 10-26-2021, 08:06 PM   #32
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_12{.0|.1}
Posts: 5,777
Blog Entries: 23

Rep: Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785
Without some more complete description of how the application should actually work I and others will not be able to help much with the grammar, so I'll refrain from further suggestions about that and just see how the thread developes those ideas further.

Thinking of it as an outline generator does stir quite a few thoughts about how that would be very useful! One train of thought along those lines suggests to me that you may not need the parser at all, much of that level of function could be implemented as a Flex based filter application... I think I'll explore that a little further based on my own ideas and will post anything useful that may result - or not.

As for the nuances of Flex and Bison, I can mostly just encourage you that if you already know the underlying ideas of what they do, and can work with regular expressions and context-free grammars, they are really pretty easy to work with and powerful.

For regular expressions I would suggest you continue your online searches. I know there are a lot of resources out there but I do not have a favorite to point you towards. Flex has robust regular expression processing and there isn't really anything mysterious or fragile about it. You really shouldn't need much in the way of "advanced" regular expressions I think, so if you can work out the necessary patterns, consider where they might overlap or conflict and sort those, you should be well on your way.

One thing I would suggest is that if all your control expressions begin with a '.' as the first character on a line, and are not embedded within lines containing other text, look into using Flex's start states. Using that you can switch to a set of rules which only handles those commands and does not conflict with the text handling rules. Then revert to the default state at the corresponding newline character. Here is a very simplistic example (off the top of my head, use with care!):

Code:
%option stack case-insensitive
%x COMMAND (Declares exclusive start state)
...
%% (start of rules section)
<COMMAND>{
^[.]"lm" { return LM; }
^[.]"rm" { return RM; }
...
\n { yy_pop_state(); }
}
^[.] { unput(yytext); yy_push_state(COMMAND); }

<INITIAL,COMMAND>{
[0-9]+ { yylval.intValue = atoi(yytext); return NUM; }
[ \t]+ { /* Eat whitespace*/ }
}

[a-z][a-z0-9_-]+ { yylval.stringValue = strdup(yytext); return WORD; }
\n { return EOL; }
A dot as the first character switches to COMMAND state and processes only the rules found in that state. The next newline returns to INITIAL state. The number rule matches in either state, as does whotespace, and the word rule only works in INITIAL state (the default).

Flex allows you to define exclusive and non-exclusive states and they can be very useful and powerful.

There are a few Flex manuals available online, but I think there is not a fully current one based on my last search for one. The definitive manual is the info page so in a terminal type info flex and enjoy a good read! It is complete and well organized, and well indexed. If not familiar with the info system, type info info.

And the same thing goes for Bison, type info bison and learn your way around - ultimately, all the answers are there!

Hope that helps, all for now!

Last edited by astrogeek; 10-26-2021 at 08:18 PM.
 
Old 10-26-2021, 09:52 PM   #33
schmitta
Member
 
Registered: May 2011
Location: Blacksburg VA
Distribution: UBUNTU, LXLE
Posts: 352

Original Poster
Rep: Reputation: Disabled
Thank you astrogeek. Was thinking that I may need bison for the page definition but I can see that flex could also be used there. Pages is top margin (.tm <num> sets the top margin; .bm <num> sets the bottom margin with the last line having the page number centered if it appears) and lines are counted. .pl <num> is the number of lines per page (.pl is page length)(lines in this case would be the definition according to the printer) where <printable lines> = pl - (tm + bm); So a similar communication would need to occur between flex and bison to end the page making rule with PAGEFINISHED. It would be set by flex and transmitted to bison by {return PAGEFINISHED;} For the dot command rules I use
Quote:
[ |\t]+ {return BLANKS;}
^\.lm|LM {return LM;}
[0-9]+ {yylval.intValue = atoi(yytext); return NUM;}
and the bison command is
command : PA
| TITLE BLANKS words '\n'
| LM BLANKS NUM '\n' {lm=$<intValue>3;fprintf(stdout,"here:%d",lm);}
which now works.

Last edited by schmitta; 10-26-2021 at 09:54 PM.
 
Old 11-05-2021, 02:06 AM   #34
schmitta
Member
 
Registered: May 2011
Location: Blacksburg VA
Distribution: UBUNTU, LXLE
Posts: 352

Original Poster
Rep: Reputation: Disabled
Snowed under. Thank you all for your help. Need to work on my system on my own to know what to ask next.
 
Old 11-15-2021, 04:03 AM   #35
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_12{.0|.1}
Posts: 5,777
Blog Entries: 23

Rep: Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785
I have written a simple application which uses Flex only (no grammar and no Bison parser), and implements at least a subset of what you are trying to accomplish, at least as I currently understand it.

I have tried to use the same control codes that you have used in your own examples where appropriate (.pl, .tm, .bm, etc.), and have added a few others that seemed appropriate to my own implementation.

Rather than post a wall of text here, I have posted the source code and an example input file to my LQ blog here, Flex Based Outline Generator. It is fairly simple, but was written and tested in an afternoon so please pardon, and advise me of, any problems you may encounter.

When compiled and passed the example file as input, it should produce the self-referencing output shown below.

Although it is not likely to meet all your needs, I hope it will be a useful example to further your own project! It was a fun exercise!

Code:
                         Outline Generator Outline

        Following is a brief outline of the major features and basic
        operation of an outline generator application constructed
        within a Flex lexer specification.

        This outline has been generated using the application
        described.

        1) Basic operational features
            a) Generates formatted output as indented outlines from
            plain text source with simple markup
            b) Automatically orders indented outline prefixes
            c) Provides top, bottom, left, right page margins
            d) Prints centered page number on last line of page
            e) Supports shell style comments (^#...$)
            f) Supports Flex debug trace at runtime (-d opt)
        2) Source markup rules, apply to all following lines
            a) Page size and margins (one per line)
                1) .pl [0-9]+ - Set page length lines
                2) .pw [0-9]+ - Set page width chars
                3) .tm [0-9]+ - Set top margin lines
                4) .bm [0-9]+ - Set bottom margin lines
                5) .lm [0-9]+ - Set left margin spaces
                6) .rm [0-9]+ - Set right margin spaces
                7) .is [0-9]+ - Set indent spaces per level
            b) Control specifiers (one per line)
                1) .si - Shift in one level
                2) .so - Shift out one level
                3) .sl - Shift out to left margin
                4) .it - Toggle ordering prefixes off/on (def on)
                (Prints lines or paragraphs wrapped to current
                indentation level when toggled off)
            c) Title lines (Single lines, no wrap)
                1) .tl[TEXT] - Print text aligned to left margin
                2) .tc[TEXT] - Print text aligned to page center
                3) .tl/.tc - Without trailing text prints empty line
        3) Examples of use (program name is ol)
            a) ./ol infile
            b) ./ol -d infile (enable debug trace)
            c) ./ol < infile > outfile
            d) ./ol -h (Show help and exit)








                          Page 1
 
Old 11-15-2021, 07:42 AM   #36
schmitta
Member
 
Registered: May 2011
Location: Blacksburg VA
Distribution: UBUNTU, LXLE
Posts: 352

Original Poster
Rep: Reputation: Disabled
Thank you astrogeek. It is fantastic. I am a little snowed under with other things right now. Thank you for your help.
 
Old 11-15-2021, 07:44 AM   #37
schmitta
Member
 
Registered: May 2011
Location: Blacksburg VA
Distribution: UBUNTU, LXLE
Posts: 352

Original Poster
Rep: Reputation: Disabled
Note that I was going to sell mine as shareware. don't know if i can sell yours but I guess I could extend it.
 
Old 11-15-2021, 03:51 PM   #38
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_12{.0|.1}
Posts: 5,777
Blog Entries: 23

Rep: Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785
Quote:
Originally Posted by schmitta View Post
Note that I was going to sell mine as shareware. don't know if i can sell yours but I guess I could extend it.
That should have been "noted" more up front when asking for help in a Free Software oriented forum, I would think. But now noted for future reference.

My license terms are clearly stated in the source code I posted and are not changed by your use, so you must decide how that applies to your own activities. I ask that you please respect and prominently include those terms when distributing my own Freely given works.

You might also consider some applicable guidance I once read, "Freely you have received, freely give".

Good luck!

Last edited by astrogeek; 11-15-2021 at 04:01 PM. Reason: Added source license link
 
Old 11-15-2021, 05:09 PM   #39
schmitta
Member
 
Registered: May 2011
Location: Blacksburg VA
Distribution: UBUNTU, LXLE
Posts: 352

Original Poster
Rep: Reputation: Disabled
I am 70 and my SS is $516 a month and my annuity is $220.69 a month. I do not know what your pay is but do you freely give your money away? I need to make money from my software. read mythical man month. Software is one of the most expensive things on earth.
 
Old 11-17-2021, 10:55 PM   #40
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_12{.0|.1}
Posts: 5,777
Blog Entries: 23

Rep: Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785Reputation: 3785
I am genuinely sorry if my comments seemed harsh in some way. They were intended to make a point, but not to cause injury!

Quote:
Originally Posted by schmitta View Post
Note that I was going to sell mine as shareware. don't know if i can sell yours but I guess I could extend it.
I wrote it and posted it here in order to be helpful, to yourself and anyone else who may find it of use. My intent was more to provide a simple but useful teaching example for those like yourself unfamiliar with Flex, than to write a finished application.

You may do with it as best meets your needs, I did in fact Freely give the time and effort it took to write it. If that helps you to learn Flex well enough to significantly extend it in some way useful to you, then it will have served it's purpose! And if it makes your world better in some small way then it makes my world better, too!

Best of luck with your project!
 
Old 11-17-2021, 11:33 PM   #41
schmitta
Member
 
Registered: May 2011
Location: Blacksburg VA
Distribution: UBUNTU, LXLE
Posts: 352

Original Poster
Rep: Reputation: Disabled
I have decided to continue my efforts with my C program. I have a superb parser that is an array of structures where I only deal with the token's index number to extract if it is a command, word, number, etc. and can put back a token just using its index. My reason for going to YACC was I was having difficulty expanding the line, the outline, the title and keeping track of everything which problem I did not have in 1972-1975 but more difficulty in 1986 when I did it in bill gates Microsoft BASIC and even more difficulty in c (been using c at least 20 years including pointers) but maybe it is due to my age 70. But I realized that I was not using enough states or a descriptive state diagram so I will use my old friend the finite state machine to better understand what I am trying to achieve and its diagram to fully understand my system. Thank you for your help - I want to implement BASIC (probably commodore BASIC without the line numbers ) with YACC which may prove useful to me but mainly because I want to do it. Most of basic is easy to parse as each line starts with a verb, command , operation that sets the effect for the rest of the line. I will not use line numbers but labels. Thank you again for your kind help in my endeavors. I will probably be back when I write a compiler.
 
  


Reply

Tags
yacc


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Is there any support for bison-bridge and bison-locations in flex on windows systems? rami alkhateeb Linux - Software 0 12-29-2010 10:10 AM
Newbie Yacc/Bison shift reduce question IncendiaryProgrammer Programming 5 07-02-2006 12:15 PM
Yacc/Bison question russinoff Programming 1 06-17-2006 02:54 PM
Bison and Yacc Compatibility Problem oulevon Programming 1 10-23-2005 11:56 PM
yacc, Bison Config Linux - General 6 02-21-2002 03:18 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 10:11 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration