LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Using Command Syntax to Create a List of Commands (https://www.linuxquestions.org/questions/linux-newbie-8/using-command-syntax-to-create-a-list-of-commands-4175638682/)

SomeDumbGuy 09-18-2018 09:16 PM

Using Command Syntax to Create a List of Commands
 
Hello, everyone. I'm new here, and I've been working with shell scripts (mostly using Cygwin) for almost 2 years, yet I still feel pretty new.

Anyhow, I'm trying to create a list of all the possible commands using command syntax. I've been struggling with this for a really long time, so I'm hoping someone can help me. To help clarify what I'm trying to do, here's an example:

I have a list of commands in syntax form that look like this:


Quote:

{ opt1 main-command <option-123> [num <num>] {avoid | allowable} <SomeVal> { [num1 <number_1>] | [num2 <number_2] [num3 <number_3>]} [ hide <hide> ]} | { no opt1 main-command {<option-123> | <option-123>} [num <num>] [{avoid | allowable} <SomeVal> { [num1 <number_1>] | [num2 <number_2] [num3 <number_3>]} [ hide <hide> ]] }
I need to use the command syntax to create a list of all possible command combinations. As an example, the script should use the command syntax example above to create the list below (I don't think this list is 100% complete, or correct, but hopefully helps clarify what I'm trying to do):

Quote:

opt1 main-command <option-123> [num <num>] avoid <SomeVal>
opt1 main-command <option-123> [num <num>] avoid <SomeVal> [hide <hide>]
opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num1 <number_1>]
opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num1 <number_1>] [hide <hide>]
opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num2 <number_2>] [num3 <number_3>]
opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
opt1 main-command <option-123> avoid <SomeVal>
opt1 main-command <option-123> avoid <SomeVal> [hide <hide>]
opt1 main-command <option-123> avoid <SomeVal> [num1 <number_1>]
opt1 main-command <option-123> avoid <SomeVal> [num1 <number_1>] [hide <hide>]
opt1 main-command <option-123> avoid <SomeVal> [num2 <number_2>] [num3 <number_3>]
opt1 main-command <option-123> avoid <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
no opt1 main-command <option-123> [num <num>] avoid <SomeVal>
no opt1 main-command <option-123> [num <num>] avoid <SomeVal> [hide <hide>]
no opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num1 <number_1>]
no opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num1 <number_1>] [hide <hide>]
no opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num2 <number_2>] [num3 <number_3>]
no opt1 main-command <option-123> [num <num>] avoid <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
no opt1 main-command <option-123> avoid <SomeVal>
no opt1 main-command <option-123> avoid <SomeVal> [hide <hide>]
no opt1 main-command <option-123> avoid <SomeVal> [num1 <number_1>]
no opt1 main-command <option-123> avoid <SomeVal> [num1 <number_1>] [hide <hide>]
no opt1 main-command <option-123> avoid <SomeVal> [num2 <number_2>] [num3 <number_3>]
no opt1 main-command <option-123> avoid <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
opt1 main-command <option-123> [num <num>] allowable <SomeVal>
opt1 main-command <option-123> [num <num>] allowable <SomeVal> [hide <hide>]
opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num1 <number_1>]
opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num1 <number_1>] [hide <hide>]
opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num2 <number_2>] [num3 <number_3>]
opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
opt1 main-command <option-123> allowable <SomeVal>
opt1 main-command <option-123> allowable <SomeVal> [hide <hide>]
opt1 main-command <option-123> allowable <SomeVal> [num1 <number_1>]
opt1 main-command <option-123> allowable <SomeVal> [num1 <number_1>] [hide <hide>]
opt1 main-command <option-123> allowable <SomeVal> [num2 <number_2>] [num3 <number_3>]
opt1 main-command <option-123> allowable <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
no opt1 main-command <option-123> [num <num>] allowable <SomeVal>
no opt1 main-command <option-123> [num <num>] allowable <SomeVal> [hide <hide>]
no opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num1 <number_1>]
no opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num1 <number_1>] [hide <hide>]
no opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num2 <number_2>] [num3 <number_3>]
no opt1 main-command <option-123> [num <num>] allowable <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
no opt1 main-command <option-123> allowable <SomeVal>
no opt1 main-command <option-123> allowable <SomeVal> [hide <hide>]
no opt1 main-command <option-123> allowable <SomeVal> [num1 <number_1>]
no opt1 main-command <option-123> allowable <SomeVal> [num1 <number_1>] [hide <hide>]
no opt1 main-command <option-123> allowable <SomeVal> [num2 <number_2>] [num3 <number_3>]
no opt1 main-command <option-123> allowable <SomeVal> [num2 <number_2>] [num3 <number_3>] [hide <hide>]
<...> etc...
I think the syntax is standard, but just in case, I believe it works something like this:

Some commands are in braces. For example: { Command_1}
and some are in square brackets to indicate they are optional. For example: [ some_val_1 ]

More than one separated by a pipe indicates a choice. For example: [ some_val_1 | some_val_2 | some_val_3 ] or { command_1 | command_2 }. They can also be mixed up like this: {command_1 | command_2 [ val_1 | val_2 ]}

I'm trying to get all possibilities for each command, which seems to be beyond my abilities. Does anyone have any suggestions?

Thanks in advance,

SomeDumbGuy

scasey 09-19-2018 02:23 PM

What you're trying to do already exists as the SYNOPSIS entry in the man page for a command.
Example from man ls
Code:

SYNOPSIS
      ls [OPTION]... [FILE]...

of course, the OPTIONs are listed further down in the man page. There are more than 50 such for the ls command.
So, with your suggested syntax, the "main-command" of ls would have more than 50 entries in the pipe delimited list.

Not sure what you're trying to accomplish, exactly, but if you want to go forward, you'll need to extract information from the man page(s) for each command to build the list you describe. You could use grep to find and extract the SYNOPSIS information, but building the OPTION list would be extremely complicated.

SomeDumbGuy 09-19-2018 03:10 PM

First off, thanks for the reply!

It sounds like I'm not being clear (this is kind of complicated). I'm not trying to get the syntax for Linux commands or trying to fill in the values. These are commands used on a networking device. What I want to do is take the syntax for a command and create a list of all possible combinations. So using a simpler example:

This:
Quote:

[no] keyword {choice_1 | choice_2 <some-name>}
should create this list:
Quote:

keyword choice_1
keyword choice_2 <some-name>
no keyword choice_1
no keyword choice_2 <some-name>

hydrurga 09-19-2018 03:13 PM

Sorry for asking, SomeDumbGuy, but why?

astrogeek 09-19-2018 03:35 PM

I think the only way would be as you have begun... collect the synopsis variations into a list, then expand the list by writing the various combinations. You could automate it in various ways using any of the various scripting languages available, in particular awk would look like a good place to start.

But if you are looking for an existing application to do this, or a simple one-liner, I think you are probably out of luck!

There is a further complication in writing all possible combinations, that is whether the ordering of command args is important or not. That will vary with command and argument as well.

If this is something important to you, as opposed to a passing thing of interest, you should start with the basis of language parsing, generative grammars and in particular the notation used to represent the rules of a grammar: Backus-Naur Form and Extended Backus–Naur Form which is the basis of the usual synopsis notation.

Essentially what you have described is a kind of reversal of that notation, expanding the concise notation into every possible case it describes - not always possible! The essential useful property of a generative grammar is that it can represent an infinite number of sentences of a given language.

The first two sentences from the above refs:

Quote:

Parsing, syntax analysis or syntactic analysis is the process of analysing a string of symbols, either in natural language, computer languages or data structures, conforming to the rules of a formal grammar.

AND

Generative grammar is a linguistic theory that regards grammar as a system of rules that generates exactly those combinations of words that form grammatical sentences in a given language.
You are trying to perform a syntactic analysis of some command, according to the rules of the command's "language", and then generate exactly all those combinations of words of that language which form valid sentences, or commands according to the rules of the command's "language".

Sounds like what you need!

SomeDumbGuy 09-19-2018 03:39 PM

Well, that's some interesting information!

hydrurga 09-19-2018 03:43 PM

Quote:

Originally Posted by SomeDumbGuy (Post 5905462)
I'm doing this for documentation/tracking purposes.

Fair enough, but what does that mean? What are you hoping to do with the results? I only ask because I feel that the amount of energy you put into this may not match the usefulness of the results - there are some very complex commands out there which would generate hundreds if not thousands of command-line variants.

scasey 09-19-2018 04:34 PM

Re: #7
Yes. I was astounded to find that there were 50-ish options to the ls command...and I'm not even going to try to calculate the number of various combinations of those options.

Perhaps the OP is only concerned about some of those combinations...still, the task as described seems massive -- even more so if the documentation/tracking is of a subset of the possibilities.

SomeDumbGuy 09-30-2018 04:22 PM

Quote:

Originally Posted by astrogeek (Post 5905461)
I think the only way would be as you have begun... collect the synopsis variations into a list, then expand the list by writing the various combinations. You could automate it in various ways using any of the various scripting languages available, in particular awk would look like a good place to start.

But if you are looking for an existing application to do this, or a simple one-liner, I think you are probably out of luck!

There is a further complication in writing all possible combinations, that is whether the ordering of command args is important or not. That will vary with command and argument as well.

If this is something important to you, as opposed to a passing thing of interest, you should start with the basis of language parsing, generative grammars and in particular the notation used to represent the rules of a grammar: Backus-Naur Form and Extended Backus–Naur Form which is the basis of the usual synopsis notation.

Essentially what you have described is a kind of reversal of that notation, expanding the concise notation into every possible case it describes - not always possible! The essential useful property of a generative grammar is that it can represent an infinite number of sentences of a given language.

The first two sentences from the above refs:



You are trying to perform a syntactic analysis of some command, according to the rules of the command's "language", and then generate exactly all those combinations of words of that language which form valid sentences, or commands according to the rules of the command's "language".

Sounds like what you need!

I was thinking it would be a matter of running each command through a loop to "tokenize" each word and symbol and then somehow resorting the commands based on the previous symbol. Here's a really bad example that I hope will give you an idea of what I'm trying to explain (I'm writing this off the top of my head and haven't run it as a script):

# tokenize command:

Quote:

while read line
do
for word in $line
do
if [[ $word =~ { ]]
then
echo $word >> OpenBrace.txt
until [[ $word == } ]]
do
echo $word >> Command.txt
done
<...>
And then some how assemble each command. I'm not sure if each command can or should be assembled while the it's being tokenized or after.

Does any of this make sense? Am I on the right track?

In the end, I'm hoping to have an example of each possible command that will be used for documentation, tracking, and testing.

berndbausch 09-30-2018 04:55 PM

The term “tokenize” is correct. Instead of trying to implement this in shell code, use the right tool: lex or flex.

Not sure if you then need yacc (also known as bison) to complete your task.

A primer: https://www.ibm.com/developerworks/a...acc/index.html.

ondoho 10-01-2018 01:14 AM

sounds like an x-y-problem to me, in other words, maybe op should take a step back and tell us what the problem is (that they think the topic of this thread might be a solution to)?
what are you really trying to achieve?

SomeDumbGuy 10-04-2018 12:15 PM

Quote:

Originally Posted by berndbausch (Post 5909524)
The term “tokenize” is correct. Instead of trying to implement this in shell code, use the right tool: lex or flex.

Not sure if you then need yacc (also known as bison) to complete your task.

A primer: https://www.ibm.com/developerworks/a...acc/index.html.

Thanks for the info, I'll definitely look into this.

SomeDumbGuy 10-04-2018 12:27 PM

Quote:

Originally Posted by ondoho (Post 5909631)
sounds like an x-y-problem to me, in other words, maybe op should take a step back and tell us what the problem is (that they think the topic of this thread might be a solution to)?
what are you really trying to achieve?

Here's the issue:

I have hundreds of commands (many of them have several possible options). I need to enter each command in a device to create example documentation. Instead of entering each command manually, I thought I could use the syntax for each command to create the command list that includes every possible option so that I can run these commands using another script.

I'm sorry for not going into much detail. I'm just trying to stick to what's relevant.

ondoho 10-05-2018 10:55 AM

^ i see.
but:
each command belong to a software package and these packages most likely already contain documentation. so that's where you look first.
most likely there's a man page for each command.
or there's something in /usr/share/doc/command_package.
doc stands for documentation.
failing that, the option you're looking for is one of:
Code:

command -h
command --help
man command
# and very rarely
command -?

but still:
i fail to understand why you have to re-create this documentation when it's already there. i think you're just missing something here.

individual 10-05-2018 02:45 PM

Quote:

Originally Posted by scasey (Post 5905478)
Re: #7
Yes. I was astounded to find that there were 50-ish options to the ls command...and I'm not even going to try to calculate the number of various combinations of those options.

Perhaps the OP is only concerned about some of those combinations...still, the task as described seems massive -- even more so if the documentation/tracking is of a subset of the possibilities.

I counted 57 options for ls. The combinations generated for that is a lot more than any user is going to want to read through.


All times are GMT -5. The time now is 06:56 AM.