Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
By louigi600 at 2016-09-09 13:46
I use bash scripts to do all sorts of things, both in my spare time and for my job, and many of them require creating and reading a configuration file or parsing some options or both.
I used to end up writing very similar code over and over again and dealing with creating config files, parsing options and usage message used to take up a big part in the script. I wanted to do something about this to make my life simpler and minimize the amount of code I'd need to write for dealing with such issues and possibly stop rewriting each time nearly the same thing.
Let's deal with the first issue: minimizing the amount of code to get the script to prompt you with the values to put into it's configuration file. Yes I like my scripts to prompt me for their configuration if the is not one yet and I like to keep it in a separate file so I can update the script without messing up the configuration.
For me the config files would generally contain the declaration of some variables that I would then source into the script itself. Something like this:
Ok the single quotes aren't necessary here but this was automatically created by a script so that's why they all have single quotes in case the variable has blank spaces in them.
Let's have a look at how I get the script to prompt me for each one of those variables, only if no config file is found, without writing specific code for every single configuration option.
First of all I declare a variable that contains all the names of the variables that will go in the configuration file
Then early in the script execution I would issue something like this (which implies that the scripts config file has the same name as the the script with ".conf" appended to it)
All the magic is done in the make_config function, along with some other functions used in there. Let's have a look at that code:
Code:
make_config ()
{ ACCEPT=0
while [ $ACCEPT -eq 0 ]
do
form_dialog "Basic config for $NAME" $SCRIPT_PARAMS && IFS=';' read -r -d '\;' $SCRIPT_PARAMS < $DIALOG_OUTPUT
accept_dialog "Accept Configuration ?" $SCRIPT_PARAMS
ACCEPT=$?
done
( for PARAM in $SCRIPT_PARAMS
do
echo "${PARAM}='$(eval echo "\$$PARAM")'"
done
) > $CWD/${NAME}.conf
chmod 600 $CWD/${NAME}.conf
sleep 1
}
I used to have make_config to prompt for each variable by writing text on the terminal and subsequently print all of them back asking if it was ok to write the to the config file but I now prefer to have dialog do that for me all in one go with the form_dialog function and then prompt me again with accept_dialog function to acknowlege that it's all ok to go into the config file:
Code:
form_dialog ()
{ TITLE=$1
shift
unset STRING
MAX=0
for A in $*
do
[ $((${#A} +3)) -gt $MAX ] && MAX=$((${#A} +3))
done
i=0
for A in $*
do
STRING[$i]="${A}: $(($i + 1)) 1 \"$(eval echo "\$$A")\" $(($i + 1)) $MAX 40 0"
((i++))
done
eval $(echo dialog --title \"$TITLE\" --separator \'\;\' --form \"\" 0 0 0 ${STRING[*]} \2\>$DIALOG_OUTPUT )
}
accept_dialog ()
{ TITLE=$1
shift
unset STRING
MAX=0
for A in $*
do
[ $((${#A} +3)) -gt $MAX ] && MAX=$((${#A} +3))
done
i=0
for A in $*
do
STRING[$i]="${A} = $(eval echo "\$$A") \n"
((i++))
done
eval $(echo dialog --title \"$TITLE\" --yesno \" ${STRING[*]}\" 0 0 ) && return 1 || return 0
}
You may be thinking that's a lot of code for just 6 options in the config file ... but if you had 30 options in the config file all that would change is the SCRIPT_PARAMS variable, no extra code would need to be written: I love that !
Now let's look at how I deal with minimizing the code for the script parameter parsing and usage message generation. I use getopt to parse the script flags and options which in turn does reduce code but still the GETOPT_STRING variable needs to be created according to the flags and options you intend to parse and the code needs to be written to output a usage help message. Like the config file thing I work around the problem by having a variable with all the information required ... but wait a second here it's a little more complicated because each needs to know if it's a flag or parameter and the help message describing what it does: I use an array for this:
Code:
MYOPTIONS=(
"a,,\t\t:Do not require association to AP ,$ASSOCIATE"
"c,:,\t:Wpa supplicant config (default $WCFG),$WCFG"
"d,,\t\t:Do not require IP via DHCP,$DHCP"
"f,,\t\t:Do not activate basic Firewallroles,$FIREWALL"
"h,,\t\t:Show this help message,0"
"i,:,\t:Use wireless specific interface,$NIC_NAME"
"m,:,\t:Set the interface to a specific mode (default $MODE),$MODE"
"M,:,\t:Use interface with this MAC (eg -M $MAC_EG),$MAC"
"r,:,\t:Set REGION (default $REG),$REG"
"s,:,\t:Register to AP with this essid,$ESSID"
"w,,\t\t:Do not Start wpa_supplicant for AP association,$WPA_SUPP"
)
where each element contains all the information for each flag/option separated by colons:
the flag/parameter (I don't use long options in my scripts or things get too complicated)
":" means it requires a parameter id empty it's a flag
the text message describing what it does for usege (the extra tabs help allign properly)
the value stored for the option
Modern bash has now associative arrays (that would make it easier to access an array element named after it's flag name) but I thaught this up prior to the existence of bash4 (nothing against ksh but did not want to deal with making sure it was installed on all the systems). With bash4 or ksh the MYOPTIONS could look like this:
Code:
MYOPTIONS=(
[a]=",\t\t:Do not require association to AP ,$ASSOCIATE"
[c]=":,\t:Wpa supplicant config (default $WCFG),$WCFG"
[d]=",\t\t:Do not require IP via DHCP,$DHCP"
[f]=",\t\t:Do not activate basic Firewallroles,$FIREWALL"
[h]=",\t\t:Show this help message,0"
[i]=":,\t:Use wireless specific interface,$NIC_NAME"
[m]=":,\t:Set the interface to a specific mode (default $MODE),$MODE"
[M]=":,\t:Use interface with this MAC (eg -M $MAC_EG),$MAC"
[r]=":,\t:Set REGION (default $REG),$REG"
[s]=":,\t:Register to AP with this essid,$ESSID"
[w]=",\t\t:Do not Start wpa_supplicant for AP association,$WPA_SUPP"
)
And you could do without the part of the code required to find the element of the array for the relevant flag/parameter.
Here's how the usage function gets reduced to containing no specific code:
Now let's see how I produce the GETOPT_STRING variable and how I use getopt itself:
Code:
GETOPT_STRING=$(I=0
while [ $I -lt ${#MYOPTIONS[*]} ]
do
OPTION=$(echo "${MYOPTIONS[$I]}" |cut -d "," -f 1)
PARM=$(echo "${MYOPTIONS[$I]}" |cut -d "," -f 2)
echo -en "${OPTION}$PARM"
I=$(expr $I + 1)
done
)
getopt $GETOPT_STRING $* >/dev/null 2>/dev/null || \
usage unknown option or insufficient parameters: $*
GPO=$(getopt $GETOPT_STRING $* 2>/dev/null |tr -d "'")
set -- $GPO
while [ $# -ge 1 ]
do
case $1 in
-a) set_myoptions_value a 0; shift ;;
-c) set_myoptions_value c $2; shift 2;;
-d) set_myoptions_value d 0; shift ;;
-f) set_myoptions_value f 0; shift ;;
-i) set_myoptions_value i $2; shift 2 ;;
-m) set_myoptions_value m $2; shift 2;;
-M) set_myoptions_value M $2; shift 2;;
-r) set_myoptions_value r $2; shift 2;;
-s) set_myoptions_value s $2; shift 2;;
-w) set_myoptions_value w 0; shift ;;
-h) usage ;;
--) shift ; break ;;
*) usage $1 unknown option ;;
esac
done
As you can see the only specific code is in the case where I use a function (set_myoptions_value) to store the values in the correct MYOPTIONS array element: that's not much code for handling 11 flags/parameters and the good thing that it's not going to grow much even if I had many more.
There is however some other functions I have to make it easy for me to set and recall values for each flag/parameter but this remains the same and I just copy it into each script that uses MYOPTIONS array:
Similarly, by using associative array instead of a list, for the SCRIPT_PARAMS one could do without the fiddly recall the contents of a variable whose name is in another variable and code would be shorter and simpler. I'll leave that as an exercise for the reader.
It is also possible to replace the case where all the values for the flags/options are parsed with a loop on the MYOPTIONS array wich in turn would require even less specific code for the options ... but you can't get rid of it all because at some point there will be some specific action to be taken for each flag/option.
I'm sorry, but I need to say the code you presented is inefficient. You need to eliminate $( .. ), because that will invoke a new shell and usually can be solved without that (which will need less resources, run faster...)
by louigi600 on Thu, 2016-09-22 04:47
Yes surely but the article is on minimizing code not making it more efficient. It's focus is on reducing the amount of code necessary to deal with input flags/parameters and configuration file creation.
I'll see if I can review the code so that it's not a bad example of bash coding ... can I edit my article ?
Help me find the contents of a variable whose name is in another variable ... I know what I did is a mess but I still can't think of an alternative.
by pan64 on Thu, 2016-09-22 06:09
Quote:
Help me find the contents of a variable whose name is in another variable
Code:
A=B
B=12
echo ${A}
echo ${!A}
by louigi600 on Thu, 2016-09-22 07:02
Quote:
Originally Posted by pan64
Code:
A=B
B=12
echo ${A}
echo ${!A}
Yes I guess I should have read better the bash man page ... No how do I edit the article to make it better ?
by pan64 on Thu, 2016-09-22 07:06
probably there is an edit button, but actually you can post an improved version too.
by louigi600 on Thu, 2016-09-22 07:13
Quote:
Originally Posted by pan64
probably there is an edit button, but actually you can post an improved version too.
For posts there is but I can't find it for article, I wanted to avoid rewriting it just make the current version better.
But if it's not possible I guess a rewrite will be scheduled.
by louigi600 on Thu, 2016-09-22 09:42
Quote:
Originally Posted by pan64
You need to eliminate $( .. ), because that will invoke a new shell ...
No $(command) and/or `command` is command substitution and does not invoke a shell unless the command is a shell.
It's the plain ( list ) that invokes a subshell to run the list.
Ok but still you have a point: there are some places where I can do without command substitution and I can declare som variables as integer and do without some of the ((expression)). It will be an exercise for me to write cleaner bash scripts
by MadeInGermany on Fri, 2017-05-05 15:59
The $( ) or ( ) is a sub shell.
That is another process, or at least it behaves like one.
An example should demonstrate this
Nothing is returned to the main shell.
My attempt for an efficient coding
Code:
get_myoptions_value ()
{ I=0
#SEARCH_OPTION=$(echo "$1" |tr -d "-")
SEARCH_OPTION=${1//-/}
while [ $I -lt ${#MYOPTIONS[*]} ]
do
#OPTION=$(echo "${MYOPTIONS[$I]}" |cut -d "," -f 1)
OPTION=${MYOPTIONS[$I]%%,*}" # the first comma-separated value
#VALUE=$(echo "${MYOPTIONS[$I]}" |cut -d "," -f 4)
VALUE=${MYOPTIONS[$I]##*,} # the last one (this might not be 4)
#[ "$OPTION" = "$SEARCH_OPTION" ] && (echo $VALUE ; break)
[ "$OPTION" = "$SEARCH_OPTION" ] && { echo $VALUE ; break ; }
#I=$(expr $I + 1)
(( I++ ))
done
}
The { ;} is efficient because it stays in the current shell, just like a simple if-then-else.
BTW nothing speaks against
Code:
if [ "$OPTION" = "$SEARCH_OPTION" ] ; then echo $VALUE ; break ; fi
by louigi600 on Sun, 2017-05-07 04:22
To my understanding, which may be wrong, $(command) forks the command so if it's a shell it will run a subshell if it's an executable it will execute the command obviously in another process in any case.
$(command) and `command` do the same thing I just prefer the newer $(command) syntax.
As I said before this not about optimizing code for performance but all about minimizing the amount of code necessary to deal with scripts with several options and configuration parameters.
Although I appreciate comments regarding attempting to get petter performance I do thing that they're missing the whole point of the article.
In any case a reedit or improved version is pending
MYOPTIONS is an associative array you don't need to walk the whole looking for the looking for the right element.
Code:
get_myoptions_value ()
{ I=0
#SEARCH_OPTION=$(echo "$1" |tr -d "-")
SEARCH_OPTION=${1//-/}
while [ $I -lt ${#MYOPTIONS[*]} ]
do
#OPTION=$(echo "${MYOPTIONS[$I]}" |cut -d "," -f 1)
OPTION=${MYOPTIONS[$I]%%,*}" # the first comma-separated value
#VALUE=$(echo "${MYOPTIONS[$I]}" |cut -d "," -f 4)
VALUE=${MYOPTIONS[$I]##*,} # the last one (this might not be 4)
#[ "$OPTION" = "$SEARCH_OPTION" ] && (echo $VALUE ; break)
[ "$OPTION" = "$SEARCH_OPTION" ] && { echo $VALUE ; break ; }
#I=$(expr $I + 1)
(( I++ ))
done
}
That's not more efficient it's doing a ueless array walk and it's less readable, but yes instead of forking cut I could have removed the longest matching prefix ... that might go in the revised version.
by pan64 on Sun, 2017-05-07 04:31
for example:
Code:
eval $(echo dialog --title \"$TITLE\" --yesno \" ${STRING[*]}\" 0 0 ) && return 1 || return 0
# can be simplified by:
dialog --title "$TITLE" --yesno " ${STRING[*]}" 0 0
# you can put a return $? if you want at the end, but not required
in general you should eliminate eval, which will not only speed up the execution, but as you can see makes the code more readable (and also shorter)
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
I'll see if I can review the code so that it's not a bad example of bash coding ... can I edit my article ?
Help me find the contents of a variable whose name is in another variable ... I know what I did is a mess but I still can't think of an alternative.
But if it's not possible I guess a rewrite will be scheduled.
It's the plain ( list ) that invokes a subshell to run the list.
Ok but still you have a point: there are some places where I can do without command substitution and I can declare som variables as integer and do without some of the ((expression)). It will be an exercise for me to write cleaner bash scripts
That is another process, or at least it behaves like one.
An example should demonstrate this
My attempt for an efficient coding
BTW nothing speaks against
$(command) and `command` do the same thing I just prefer the newer $(command) syntax.
As I said before this not about optimizing code for performance but all about minimizing the amount of code necessary to deal with scripts with several options and configuration parameters.
Although I appreciate comments regarding attempting to get petter performance I do thing that they're missing the whole point of the article.
In any case a reedit or improved version is pending
MYOPTIONS is an associative array you don't need to walk the whole looking for the looking for the right element.