LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-18-2018, 09:16 PM   #1
SomeDumbGuy
LQ Newbie
 
Registered: Sep 2018
Posts: 7

Rep: Reputation: Disabled
Using Command Syntax to Create a List of Commands


Hello, everyone. I'm new here, and I've been working with shell scripts (mostly using Cygwin) for almost 2 years, yet I still feel pretty new.

Anyhow, I'm trying to create a list of all the possible commands using command syntax. I've been struggling with this for a really long time, so I'm hoping someone can help me. To help clarify what I'm trying to do, here's an example:

I have a list of commands in syntax form that look like this:


Quote:
{ opt1 main-command <option-123> [num <num>] {avoid | allowable} <SomeVal> { [num1 <number_1>] | [num2 <number_2] [num3 <number_3>]} [ hide <hide> ]} | { no opt1 main-command {<option-123> | <option-123>} [num <num>] [{avoid | allowable} <SomeVal> { [num1 <number_1>] | [num2 <number_2] [num3 <number_3>]} [ hide <hide> ]] }
I need to use the command syntax to create a list of all possible command combinations. As an example, the script should use the command syntax example above to create the list below (I don't think this list is 100% complete, or correct, but hopefully helps clarify what I'm trying to do):

Quote:
opt1 main-command <option-123> [num <num>] avoid <SomeVal>
opt1 main-command <option-123> [num <num>] avoid <SomeVal> [hide <hide>]
opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num1 <number_1>]
opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num1 <number_1>] [hide <hide>]
opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num2 <number_2>] [num3 <number_3>]
opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
opt1 main-command <option-123> avoid <SomeVal>
opt1 main-command <option-123> avoid <SomeVal> [hide <hide>]
opt1 main-command <option-123> avoid <SomeVal> [num1 <number_1>]
opt1 main-command <option-123> avoid <SomeVal> [num1 <number_1>] [hide <hide>]
opt1 main-command <option-123> avoid <SomeVal> [num2 <number_2>] [num3 <number_3>]
opt1 main-command <option-123> avoid <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
no opt1 main-command <option-123> [num <num>] avoid <SomeVal>
no opt1 main-command <option-123> [num <num>] avoid <SomeVal> [hide <hide>]
no opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num1 <number_1>]
no opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num1 <number_1>] [hide <hide>]
no opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num2 <number_2>] [num3 <number_3>]
no opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
no opt1 main-command <option-123> avoid <SomeVal>
no opt1 main-command <option-123> avoid <SomeVal> [hide <hide>]
no opt1 main-command <option-123> avoid <SomeVal> [num1 <number_1>]
no opt1 main-command <option-123> avoid <SomeVal> [num1 <number_1>] [hide <hide>]
no opt1 main-command <option-123> avoid <SomeVal> [num2 <number_2>] [num3 <number_3>]
no opt1 main-command <option-123> avoid <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
opt1 main-command <option-123> [num <num>] allowable <SomeVal>
opt1 main-command <option-123> [num <num>] allowable <SomeVal> [hide <hide>]
opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num1 <number_1>]
opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num1 <number_1>] [hide <hide>]
opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num2 <number_2>] [num3 <number_3>]
opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
opt1 main-command <option-123> allowable <SomeVal>
opt1 main-command <option-123> allowable <SomeVal> [hide <hide>]
opt1 main-command <option-123> allowable <SomeVal> [num1 <number_1>]
opt1 main-command <option-123> allowable <SomeVal> [num1 <number_1>] [hide <hide>]
opt1 main-command <option-123> allowable <SomeVal> [num2 <number_2>] [num3 <number_3>]
opt1 main-command <option-123> allowable <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
no opt1 main-command <option-123> [num <num>] allowable <SomeVal>
no opt1 main-command <option-123> [num <num>] allowable <SomeVal> [hide <hide>]
no opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num1 <number_1>]
no opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num1 <number_1>] [hide <hide>]
no opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num2 <number_2>] [num3 <number_3>]
no opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
no opt1 main-command <option-123> allowable <SomeVal>
no opt1 main-command <option-123> allowable <SomeVal> [hide <hide>]
no opt1 main-command <option-123> allowable <SomeVal> [num1 <number_1>]
no opt1 main-command <option-123> allowable <SomeVal> [num1 <number_1>] [hide <hide>]
no opt1 main-command <option-123> allowable <SomeVal> [num2 <number_2>] [num3 <number_3>]
no opt1 main-command <option-123> allowable <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
<...> etc...
I think the syntax is standard, but just in case, I believe it works something like this:

Some commands are in braces. For example: { Command_1}
and some are in square brackets to indicate they are optional. For example: [ some_val_1 ]

More than one separated by a pipe indicates a choice. For example: [ some_val_1 | some_val_2 | some_val_3 ] or { command_1 | command_2 }. They can also be mixed up like this: {command_1 | command_2 [ val_1 | val_2 ]}

I'm trying to get all possibilities for each command, which seems to be beyond my abilities. Does anyone have any suggestions?

Thanks in advance,

SomeDumbGuy

Last edited by SomeDumbGuy; 09-19-2018 at 02:38 PM. Reason: Changed syntax example
 
Old 09-19-2018, 02:23 PM   #2
scasey
Senior Member
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.6
Posts: 3,629

Rep: Reputation: 1204Reputation: 1204Reputation: 1204Reputation: 1204Reputation: 1204Reputation: 1204Reputation: 1204Reputation: 1204Reputation: 1204
What you're trying to do already exists as the SYNOPSIS entry in the man page for a command.
Example from man ls
Code:
SYNOPSIS
       ls [OPTION]... [FILE]...
of course, the OPTIONs are listed further down in the man page. There are more than 50 such for the ls command.
So, with your suggested syntax, the "main-command" of ls would have more than 50 entries in the pipe delimited list.

Not sure what you're trying to accomplish, exactly, but if you want to go forward, you'll need to extract information from the man page(s) for each command to build the list you describe. You could use grep to find and extract the SYNOPSIS information, but building the OPTION list would be extremely complicated.
 
Old 09-19-2018, 03:10 PM   #3
SomeDumbGuy
LQ Newbie
 
Registered: Sep 2018
Posts: 7

Original Poster
Rep: Reputation: Disabled
First off, thanks for the reply!

It sounds like I'm not being clear (this is kind of complicated). I'm not trying to get the syntax for Linux commands or trying to fill in the values. These are commands used on a networking device. What I want to do is take the syntax for a command and create a list of all possible combinations. So using a simpler example:

This:
Quote:
[no] keyword {choice_1 | choice_2 <some-name>}
should create this list:
Quote:
keyword choice_1
keyword choice_2 <some-name>
no keyword choice_1
no keyword choice_2 <some-name>

Last edited by SomeDumbGuy; 09-19-2018 at 03:12 PM. Reason: Error in list
 
Old 09-19-2018, 03:13 PM   #4
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 19.1 MATE
Posts: 8,016
Blog Entries: 5

Rep: Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859
Sorry for asking, SomeDumbGuy, but why?
 
1 members found this post helpful.
Old 09-19-2018, 03:35 PM   #5
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_12{.0|.1}
Posts: 5,170
Blog Entries: 11

Rep: Reputation: 3091Reputation: 3091Reputation: 3091Reputation: 3091Reputation: 3091Reputation: 3091Reputation: 3091Reputation: 3091Reputation: 3091Reputation: 3091Reputation: 3091
I think the only way would be as you have begun... collect the synopsis variations into a list, then expand the list by writing the various combinations. You could automate it in various ways using any of the various scripting languages available, in particular awk would look like a good place to start.

But if you are looking for an existing application to do this, or a simple one-liner, I think you are probably out of luck!

There is a further complication in writing all possible combinations, that is whether the ordering of command args is important or not. That will vary with command and argument as well.

If this is something important to you, as opposed to a passing thing of interest, you should start with the basis of language parsing, generative grammars and in particular the notation used to represent the rules of a grammar: Backus-Naur Form and Extended Backus–Naur Form which is the basis of the usual synopsis notation.

Essentially what you have described is a kind of reversal of that notation, expanding the concise notation into every possible case it describes - not always possible! The essential useful property of a generative grammar is that it can represent an infinite number of sentences of a given language.

The first two sentences from the above refs:

Quote:
Parsing, syntax analysis or syntactic analysis is the process of analysing a string of symbols, either in natural language, computer languages or data structures, conforming to the rules of a formal grammar.

AND

Generative grammar is a linguistic theory that regards grammar as a system of rules that generates exactly those combinations of words that form grammatical sentences in a given language.
You are trying to perform a syntactic analysis of some command, according to the rules of the command's "language", and then generate exactly all those combinations of words of that language which form valid sentences, or commands according to the rules of the command's "language".

Sounds like what you need!

Last edited by astrogeek; 09-19-2018 at 08:01 PM. Reason: tyops, topsy, typos
 
1 members found this post helpful.
Old 09-19-2018, 03:39 PM   #6
SomeDumbGuy
LQ Newbie
 
Registered: Sep 2018
Posts: 7

Original Poster
Rep: Reputation: Disabled
Well, that's some interesting information!

Last edited by SomeDumbGuy; 09-19-2018 at 03:45 PM. Reason: Reply appearing under different post
 
Old 09-19-2018, 03:43 PM   #7
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 19.1 MATE
Posts: 8,016
Blog Entries: 5

Rep: Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859
Quote:
Originally Posted by SomeDumbGuy View Post
I'm doing this for documentation/tracking purposes.
Fair enough, but what does that mean? What are you hoping to do with the results? I only ask because I feel that the amount of energy you put into this may not match the usefulness of the results - there are some very complex commands out there which would generate hundreds if not thousands of command-line variants.
 
2 members found this post helpful.
Old 09-19-2018, 04:34 PM   #8
scasey
Senior Member
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.6
Posts: 3,629

Rep: Reputation: 1204Reputation: 1204Reputation: 1204Reputation: 1204Reputation: 1204Reputation: 1204Reputation: 1204Reputation: 1204Reputation: 1204
Re: #7
Yes. I was astounded to find that there were 50-ish options to the ls command...and I'm not even going to try to calculate the number of various combinations of those options.

Perhaps the OP is only concerned about some of those combinations...still, the task as described seems massive -- even more so if the documentation/tracking is of a subset of the possibilities.
 
2 members found this post helpful.
Old 09-30-2018, 04:22 PM   #9
SomeDumbGuy
LQ Newbie
 
Registered: Sep 2018
Posts: 7

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by astrogeek View Post
I think the only way would be as you have begun... collect the synopsis variations into a list, then expand the list by writing the various combinations. You could automate it in various ways using any of the various scripting languages available, in particular awk would look like a good place to start.

But if you are looking for an existing application to do this, or a simple one-liner, I think you are probably out of luck!

There is a further complication in writing all possible combinations, that is whether the ordering of command args is important or not. That will vary with command and argument as well.

If this is something important to you, as opposed to a passing thing of interest, you should start with the basis of language parsing, generative grammars and in particular the notation used to represent the rules of a grammar: Backus-Naur Form and Extended Backus–Naur Form which is the basis of the usual synopsis notation.

Essentially what you have described is a kind of reversal of that notation, expanding the concise notation into every possible case it describes - not always possible! The essential useful property of a generative grammar is that it can represent an infinite number of sentences of a given language.

The first two sentences from the above refs:



You are trying to perform a syntactic analysis of some command, according to the rules of the command's "language", and then generate exactly all those combinations of words of that language which form valid sentences, or commands according to the rules of the command's "language".

Sounds like what you need!
I was thinking it would be a matter of running each command through a loop to "tokenize" each word and symbol and then somehow resorting the commands based on the previous symbol. Here's a really bad example that I hope will give you an idea of what I'm trying to explain (I'm writing this off the top of my head and haven't run it as a script):

# tokenize command:

Quote:
while read line
do
for word in $line
do
if [[ $word =~ { ]]
then
echo $word >> OpenBrace.txt
until [[ $word == } ]]
do
echo $word >> Command.txt
done
<...>
And then some how assemble each command. I'm not sure if each command can or should be assembled while the it's being tokenized or after.

Does any of this make sense? Am I on the right track?

In the end, I'm hoping to have an example of each possible command that will be used for documentation, tracking, and testing.

Last edited by SomeDumbGuy; 09-30-2018 at 04:23 PM. Reason: Qouting the example
 
Old 09-30-2018, 04:55 PM   #10
berndbausch
Senior Member
 
Registered: Nov 2013
Location: Tokyo
Distribution: Redhat/Centos, Ubuntu, Raspbian, Fedora, Alpine, Cirros, OpenSuse/SLES
Posts: 3,279

Rep: Reputation: 854Reputation: 854Reputation: 854Reputation: 854Reputation: 854Reputation: 854Reputation: 854
The term “tokenize” is correct. Instead of trying to implement this in shell code, use the right tool: lex or flex.

Not sure if you then need yacc (also known as bison) to complete your task.

A primer: https://www.ibm.com/developerworks/a...acc/index.html.
 
1 members found this post helpful.
Old 10-01-2018, 01:14 AM   #11
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 12,304
Blog Entries: 9

Rep: Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309
sounds like an x-y-problem to me, in other words, maybe op should take a step back and tell us what the problem is (that they think the topic of this thread might be a solution to)?
what are you really trying to achieve?
 
Old 10-04-2018, 12:15 PM   #12
SomeDumbGuy
LQ Newbie
 
Registered: Sep 2018
Posts: 7

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by berndbausch View Post
The term “tokenize” is correct. Instead of trying to implement this in shell code, use the right tool: lex or flex.

Not sure if you then need yacc (also known as bison) to complete your task.

A primer: https://www.ibm.com/developerworks/a...acc/index.html.
Thanks for the info, I'll definitely look into this.
 
Old 10-04-2018, 12:27 PM   #13
SomeDumbGuy
LQ Newbie
 
Registered: Sep 2018
Posts: 7

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by ondoho View Post
sounds like an x-y-problem to me, in other words, maybe op should take a step back and tell us what the problem is (that they think the topic of this thread might be a solution to)?
what are you really trying to achieve?
Here's the issue:

I have hundreds of commands (many of them have several possible options). I need to enter each command in a device to create example documentation. Instead of entering each command manually, I thought I could use the syntax for each command to create the command list that includes every possible option so that I can run these commands using another script.

I'm sorry for not going into much detail. I'm just trying to stick to what's relevant.
 
Old 10-05-2018, 10:55 AM   #14
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 12,304
Blog Entries: 9

Rep: Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309Reputation: 3309
^ i see.
but:
each command belong to a software package and these packages most likely already contain documentation. so that's where you look first.
most likely there's a man page for each command.
or there's something in /usr/share/doc/command_package.
doc stands for documentation.
failing that, the option you're looking for is one of:
Code:
command -h
command --help
man command
# and very rarely
command -?
but still:
i fail to understand why you have to re-create this documentation when it's already there. i think you're just missing something here.
 
Old 10-05-2018, 02:45 PM   #15
individual
Member
 
Registered: Jul 2018
Posts: 234

Rep: Reputation: 176Reputation: 176
Quote:
Originally Posted by scasey View Post
Re: #7
Yes. I was astounded to find that there were 50-ish options to the ls command...and I'm not even going to try to calculate the number of various combinations of those options.

Perhaps the OP is only concerned about some of those combinations...still, the task as described seems massive -- even more so if the documentation/tracking is of a subset of the possibilities.
I counted 57 options for ls. The combinations generated for that is a lot more than any user is going to want to read through.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
From the command line What is the syntax syntax to import some data into mysql table Glenn D. Linux - Software 1 03-13-2015 09:26 AM
what is the command to store a list of commands and their outputs in a file ursuniel Linux - Newbie 9 01-30-2014 03:29 PM
Create a list of date pairings based on a list of dates budgie26 Linux - General 4 05-09-2013 06:41 AM
Where do I go to get a list of the command prompt commands. dickcdavis Linux - Newbie 7 10-14-2011 09:23 AM
how execute a command in a list of commands bkcreddy17 Programming 4 06-24-2009 08:21 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 10:04 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration