LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 02-24-2019, 10:25 AM   #1
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 11,272
Blog Entries: 8

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
How to correctly pass command line options to a program from within a script


please refer to the much simpler example in post #7!!!

I dabble in bash scripting, mostly.
Of course I know how to pass "$1", "$2", "$@" on to the next script/program, but I have been stumbling over a particular problem many times over the years: Quoting and spaces and variable expansion.

let me give you an example:
The script reads a line from a config file that contains options to be passed on to xplanet.
The script itself adds fixed options that are not supposed to be changed by the casual user who adds lines to the config file.
The options are then passed on to xplanet (and also to feh to demonstrate that this is not an xplanet quirk).

Problems arise with the background image.
How can I make the script accept and correctly pass on a file path surrounded by quotes, with the $HOME variable expanded, and spaces in its name?
I thought an array was the way to go.
Then I thought I need to use eval.
Neither works in all scenarios.

so, here's a test script (it is just a working example and doesn't make any sense otherwise):
Code:
#!/bin/bash

# the scripts directory will be created by this script!
scenes="$HOME/test/scenes"

options=( -num_times 1 )

# pass chosen scene 
chosen=$1

set_bg() {
# if -background is found in options (read from current scene) set it as root window background before xplanet even starts
for ((i=0;i<${#options[@]};i++)); do
  [[ "${options[i]}" == "-background" ]] && background="${options[$((i+1))]}" && break
done
echo "--- feh says:"
feh --no-fehbg --bg-fill "$background"
}

# VARIATION 1 - add sed output to array
options=( $(sed -n ${chosen}p "$scenes") "${options[@]}" )
echo -e "All options:\n${options[@]}"
set_bg
echo "--- xplanet says:"
xplanet ${options[@]}
read
# VARIATION 2 - eval sed output and add to array
options=( "$(eval echo $(sed -n ${chosen}p "$scenes"))" "${options[@]}" )
echo -e "All options:\n${options[@]}"
set_bg
echo "--- xplanet says:"
xplanet ${options[@]}
and here's the accompanying "scenes" file:
Code:
-target moon -glare 0 -background "$HOME/images/wall/xplanet/grey111 with spaces.png"
-target moon -separation earth:0.4 -background $HOME/images/wall/xplanet/space-1680x1050-grey111.png
-target earth -labelpos +30+60 -background /home/ondoho/images/wall/xplanet/space-1680x1050-grey111.png
of course the image files exist on my system.

so i can access lines 1, 2 or 3 of the scenes file by passing 1, 2 or 3 to the script.
the output is lengthy so i uploaded it here.
  1. neither xplanet nor feh even start
  2. xplanet does not find the background, but starts with a black background. feh fails.
  3. both applications set the background correctly

Last edited by ondoho; 02-25-2019 at 01:25 AM.
 
Old 02-24-2019, 02:04 PM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 12,460

Rep: Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869
what I used to suggest is to use shellcheck to check your script. It will definitely give you some hints.
I would rather try getopts to process options. And I would definitely use " here:
Code:
xplanet "${options[@]}"
And it looks like $HOME is not always evaluated.
 
Old 02-24-2019, 04:26 PM   #3
l0f4r0
Member
 
Registered: Jul 2018
Location: Paris
Distribution: macOS, Slackware
Posts: 803

Rep: Reputation: 281Reputation: 281Reputation: 281
Weird. From my side, bash can retrieve the whole parameter and expand $HOME:
Code:
$ options=( -target moon -glare 0 -background "$HOME/images/wall/xplanet/grey111 with spaces.png" )
$ for ((i=0;i<${#options[@]};i++)); do [[ "${options[i]}" == "-background" ]] && background="${options[$((i+1))]}" && break; done
$ echo $background
/Users/l0f4r0/images/wall/xplanet/grey111 with spaces.png
Maybe it has to do with the following line (I admit I don't really understand it, can you please explain?):
Code:
options=( -num_times 1 )
NB: btw, you don't need double quotes around variables inside bash [[ (unlike [)

Last edited by l0f4r0; 02-25-2019 at 12:41 AM.
 
Old 02-24-2019, 06:39 PM   #4
michaelk
Moderator
 
Registered: Aug 2002
Posts: 18,318

Rep: Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596Reputation: 2596
Code:
options=( -num_times 1 )
The preceding line creates an array called options with the first element as -num_times and the second as 1.

I'm not sure why you need to use an array. Appending additional options to the beginning or end of your "options" string would work as well. Any arguments you pass to a command with spaces needs to be escaped or enclosed with quotes.
 
1 members found this post helpful.
Old 02-24-2019, 10:45 PM   #5
scasey
Senior Member
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.6
Posts: 2,730

Rep: Reputation: 933Reputation: 933Reputation: 933Reputation: 933Reputation: 933Reputation: 933Reputation: 933Reputation: 933
I'm sure I don't completely understand, but...

Passing command line options to a program would seem to me to depend on what command line options are programmed to be read/accepted. Is what you're passing to xplanet consistent with what it is programmed to read?

I don't see -num_times as an option in the man pages for either feh or xplanet, for example.

...or am I missing the point?

Last edited by scasey; 02-24-2019 at 10:47 PM.
 
Old 02-25-2019, 12:52 AM   #6
l0f4r0
Member
 
Registered: Jul 2018
Location: Paris
Distribution: macOS, Slackware
Posts: 803

Rep: Reputation: 281Reputation: 281Reputation: 281
@michaelk: thank you. Actually I didn't understand how options array could be equal to -target moon -glare 0 -background "$HOME/images/wall/xplanet/grey111 with spaces.png" -num_times 1 but it seems I just missed the sed/eval part

Ok, now I have reproduced the OP's issue...

Code:
$ cat <<'EOF' >scenes
> -target moon -glare 0 -background "$HOME/images/wall/xplanet/grey111 with spaces.png"
> -target moon -separation earth:0.4 -background $HOME/images/wall/xplanet/space-1680x1050-grey111.png
> -target earth -labelpos +30+60 -background /home/ondoho/images/wall/xplanet/space-1680x1050-grey111.png
> EOF

$ options=( -num_times 1 )

$ echo ${options[@]}
-num_times 1

$ options=( $(sed -n 1p "scenes") "${options[@]}" )

$ echo ${options[@]}
-target moon -glare 0 -background "$HOME/images/wall/xplanet/grey111 with spaces.png" -num_times 1

$ for ((i=0;i<${#options[@]};i++)); do [[ "${options[i]}" == "-background" ]] && background="${options[$((i+1))]}" && break; done

$ echo $background 
"$HOME/images/wall/xplanet/grey111

$ echo ${options[4]}
-background

$ echo ${options[5]}
"$HOME/images/wall/xplanet/grey111

Last edited by l0f4r0; 02-25-2019 at 12:56 AM.
 
Old 02-25-2019, 01:15 AM   #7
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 11,272

Original Poster
Blog Entries: 8

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
thanks for taking the time.

please don't get hung up on either xplanet or feh or the weirdness of the script; i have encountered this problem many times, and it really boils down to "how can i read in options and pass them on to other programs, preserving quoted arguments with spaces".
i thought eval would help to avoid the problem where quotes lose their special role, and i start seeing output like
Code:
could not recognize option `"/path/to/some`
could not recognize option `file with spaces"`
- but it doesn't.

expanding (environment) variables is optional (would be nice though).

anyhow, i removed the confusing feh portion of the script (i only added it to show that the problem is not program-specific) and ran it through shellcheck. apart from the usual quoting stuff it gave me the tip to use mapfile or read -a to split command output. i am testing it right now, but it doesn't seem to be doing what i want either.

really, the problem can be reduced to this:
Code:
cat <<EOF > test
-background "/path/to/file with spaces.ext"
EOF
xplanet $(<test)
xplanet "$(<test)"
cat <<EOF > test
--bg-fill "/path/to/file with spaces.ext"
EOF
feh $(<test)
feh "$(<test)"
...and the intention is to have a file where an user unconcerned with the logic of my script can just store some command line options.

Last edited by ondoho; 02-25-2019 at 01:26 AM.
 
Old 02-25-2019, 01:16 AM   #8
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 12,460

Rep: Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869
I would try to implement a short app, something like this:
Code:
#!/bin/bash

# just to save original args
ARGS=( "${@}" )
for ((i=0;i<${#ARGS[@]}; i++))
do
    printf "%2i: '%s'\n" $i "${ARGS[$i]}"
done
you can use it to print args line by line and you can see what will be passed to any subprocess (instead of echoing that array).

$HOME will not be evaulated if it was read from a file and just part of a string. eval may help on this, but eval easily can cause [other] troubles.

Again, don't forget " everywhere, use it [almost] unconditionally around variables.
 
Old 02-25-2019, 01:24 AM   #9
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 11,272

Original Poster
Blog Entries: 8

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
^ please see my previous post.
 
Old 02-25-2019, 01:56 AM   #10
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 12,460

Rep: Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869
Code:
#!/bin/bash

function print_args {
# just to save original args
ARGS=( "${@}" )
for ((i=0;i<${#ARGS[@]}; i++))
do
    printf "%2i: '%s'\n" $i "${ARGS[$i]}"
done
}

cat <<EOF > test
-background "/path/to/file with spaces.ext"
EOF
print_args $(<test)
print_args "$(<test)"
cat <<EOF > test
--bg-fill "$HOME/path/to/file with spaces.ext"
EOF
print_args $(<test)
print_args "$(<test)"
eval set -- $(<test)
print_args "${@}"
eval set -- "$(<test)"         # <<< this is what you need
print_args "${@}"
this what I wanted to explain. In such cases the bash parser was not in use, therefore " and $HOME and similars were not evaluated. You need to use eval for that.
But it can have side effects if the line contains strange things, like:
Code:
sgf ar; sudo rm -rf /  # do not try it at home

Last edited by pan64; 02-25-2019 at 07:21 AM. Reason: highlight important part
 
2 members found this post helpful.
Old 02-25-2019, 07:12 AM   #11
GazL
Senior Member
 
Registered: May 2008
Posts: 4,952
Blog Entries: 15

Rep: Reputation: 2559Reputation: 2559Reputation: 2559Reputation: 2559Reputation: 2559Reputation: 2559Reputation: 2559Reputation: 2559Reputation: 2559Reputation: 2559Reputation: 2559
deleted: sorry, I just realized I was only repeating what Pan has posted.

Last edited by GazL; 02-25-2019 at 07:14 AM.
 
Old 02-25-2019, 07:45 AM   #12
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,673

Rep: Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006
So my first note would be:
Quote:
Originally Posted by l0f4r0
NB: btw, you don't need double quotes around variables inside bash [[ (unlike [)
I fundamentally disagree with the above and when teaching bash to anyone I suggest quoting ALL variables with double quotes until you are 100% sure you know what the outcome is.

As for the problem at hand, I have worked with the test input but only to the point of printing to the screen as I do not have the mentioned commands available.
So my default file is:
Code:
$ cat f1
-target moon -glare 0 -background "$HOME/images/wall/xplanet/grey111 with spaces.png"
-target moon -separation earth:0.4 -background $HOME/images/wall/xplanet/space-1680x1050-grey111.png
-target earth -labelpos +30+60 -background /home/ondoho/images/wall/xplanet/space-1680x1050-grey111.png
And my test code looks like:
Code:
#!/usr/bin/env bash

options=( -num_times 1 )

mapfile -O 2 -t options < <(awk -vline="$1" 'BEGIN{FPAT = "(\"[^\"]+\")|([^ ]+)"}$1=$1 && NR==line' OFS='\n' f1)

echo "${#options[*]}"
printf "%s\n" "${options[@]//\$HOME/$HOME}"
And my testing:
Code:
$ ./d.sh 1
8
-num_times
1
1
moon
-glare
0
-background
"/home/grail/images/wall/xplanet/grey111 with spaces.png"
$ ./d.sh 2
8
-num_times
1
1
moon
-separation
earth:0.4
-background
/home/grail/images/wall/xplanet/space-1680x1050-grey111.png
$ ./d.sh 3
8
-num_times
1
1
earth
-labelpos
+30+60
-background
/home/ondoho/images/wall/xplanet/space-1680x1050-grey111.png
I left my own HOME results in to show the difference

I will be interested to know of solutions where it does not work?
I can see one in advance that you would need to know ahead of time the variables that might be used.
 
Old 02-25-2019, 07:56 AM   #13
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 12,460

Rep: Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869Reputation: 3869
although this awk is safe the solution can not really simulate the behavior of the original eval (which is definitely unsafe).
I would rather try a config file (which has different syntax)
 
2 members found this post helpful.
Old 02-25-2019, 08:07 AM   #14
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,673

Rep: Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006Reputation: 3006
Ok, a little bit of thinking for the variables, and:
Code:
#!/usr/bin/env bash

options=( -num_times 1 )

mapfile -O 2 -t options < <(awk -vline="$1" 'BEGIN{FPAT = "(\"[^\"]+\")|([^ ]+)"}$1=$1 && NR==line' OFS='\n' f1)

reg='\$([^\/]+)'

for item in "${options[@]}"
do
	if [[ "$item"  =~ $reg ]]
	then
		old="${BASH_REMATCH[1]}"
		new="${!old}"
	fi
done

echo "${#options[*]}"
printf "%s\n" "${options[@]//\$$old/$new}"
 
2 members found this post helpful.
Old 02-25-2019, 01:36 PM   #15
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 11,272

Original Poster
Blog Entries: 8

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
thanks, grail & pan64!

both solutions (eval/awk, posts #10 and #14) work; it's just up to me if i want to use the insecure, but 100% authentic shell parsing through eval, or the safer awk...

somehow i'm not 100% satisfied; i guess i'd hoped for some bash magic that would make this possible.
well, obviously eval is that magic, but it's also got very obvious implications.
at least i got to the bottom of the issue.

thanks again to all!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
question: 'onclick' within 'onmouseover' within 'form' within 'table' - how is it possible? rblampain Programming 4 04-25-2017 08:49 PM
[SOLVED] Why would a command work from the command line but not within a script? RandomTroll Programming 14 09-22-2016 11:49 PM
How to pass command line arguments from one shell script to another shell script VijayaRaghavanLakshman Linux - Newbie 5 01-20-2012 09:12 PM
Shell script - Pass command options with equal sign fabdog Linux - Newbie 5 02-13-2009 06:32 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 10:31 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration