LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 05-31-2017, 12:46 AM   #1
andrew.comly
Member
 
Registered: Dec 2012
Distribution: Trisquel-Mini 7.0, Lubuntu 14.04, Debian lxde 8.0
Posts: 294
Blog Entries: 2

Rep: Reputation: 16
Unhappy [ vs [[: filename expands between [[ ... ]] when it is not supposed to


BACKGROUND
I am reading Advanced Bash Scripting by Mendel Cooper. In this eBook there is a Chapter 7 Tests, and between "Example 7-2. Equivalence of test" and "Example 7-3. Arithmetic Tests using (( ))" there is an enclosed section "The [[ ]] construct is the more versatile Bash version of [ ]."

TEXTBOOK
In this section it states: No filename expansion or word splitting takes place between [[ and ]], but there is parameter expansion and command substitution.


CONTRADICTION
However, the following example contradicts this.
Code:
$ foo=[a-z]* name=lhunath
$ [[ ${name} = "$foo" ]] && echo OK || echo NO
--> NO
$ [[ ${name} = $foo ]] && echo OK || NO
--> OK
PREMISES
Clearly the variable "foo", which equals "[a-z]*", contains a wildcard. And wildcards are used for filename expansion.

QUESTION
Can someone explain the contradiction between the former statement "No filename expansion ... takes place between [[ and ]]" and latter example where filename expansion does in fact take place between "[[" and "]]"?

REFLECTION
Looking back, I can see how the two statements in the text "No filename expansion or word splitting takes place between [[ and ]]" and "The [[ ]] construct is the more versatile Bash version of [ ]." aren't exactly parallel, so I'll understand if the author was understaffed (as many volunteers are) and had to hurry through this part to get to something more important.
 
Old 05-31-2017, 12:55 AM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,457

Rep: Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778
no, you missed a very importand point:
Code:
foo=[a-z]*
here will occur filename expansion, the value of foo is not the string [a-z]*.

(Sorry it is not true. The real explanation is in my next post)

Last edited by pan64; 05-31-2017 at 11:30 AM.
 
3 members found this post helpful.
Old 05-31-2017, 01:03 AM   #3
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_10{.0|.1|.2}
Posts: 4,414
Blog Entries: 6

Rep: Reputation: 2373Reputation: 2373Reputation: 2373Reputation: 2373Reputation: 2373Reputation: 2373Reputation: 2373Reputation: 2373Reputation: 2373Reputation: 2373Reputation: 2373
Ah! But clearly "foo" does not equal "[a-z]*" within the [[..]]!

Code:
foo=[a-z]*
echo ${foo}
What does that do on your system?
 
1 members found this post helpful.
Old 05-31-2017, 01:15 AM   #4
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,457

Rep: Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778
additionally you can use set -xv which will switch on debugging and you will see what's really happening
Code:
user@host:/tmp$ foo=[a-z]* name=lhunath
foo=[a-z]* name=lhunath
+ foo='[a-z]*'
+ name=lhunath
user@host:/tmp$ [[ ${name} = $foo ]] && echo OK || echo NO
[[ ${name} = $foo ]] && echo OK || echo NO
+ [[ lhunath = [a-z]* ]]
+ echo OK
OK
usr@host:/tmp$ [[ ${name} = "$foo" ]] && echo OK || echo NO
[[ ${name} = "$foo" ]] && echo OK || echo NO
+ [[ lhunath = \[\a\-\z\]\* ]]
+ echo NO
NO
Need to correct myself, that was not true. see man bash:
Code:
       [[ expression ]]
              ...
              Word  splitting  and  pathname expansion are not performed on the words between the [[ and ]]; tilde expansion, parameter and variable expansion, arithmetic expansion, command
              substitution, process substitution, and quote removal are performed

When the == and != operators are used, the string to the right of the operator is considered a pattern
and matched according to the rules described below under Pattern Matching, as if the extglob shell option were enabled.
The = operator is equivalent to ==. If the shell option nocasematch is enabled, the match is performed without regard to the case of alphabetic characters.
The return value is 0 if the string matches (==) or does not match (!=) the pattern, and 1 otherwise.
Any part of the pattern may be quoted to force the quoted portion to be matched as a string.
by the way you missed an echo before NO
 
1 members found this post helpful.
Old 05-31-2017, 09:06 AM   #5
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 3,469

Rep: Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523
Quote:
Originally Posted by pan64 View Post
no, you missed a very importand point:
Code:
foo=[a-z]*
here will occur filename expansion, the value of foo is not the string [a-z]*.
Hmmm ...
Code:
$ set -x
$ foo=[a-z]*
+ foo='[a-z]*'
$ echo "$foo"
+ echo '[a-z]*'
[a-z]*
$ echo expands to: $foo
+ echo expands to: bin Desktop doc Documents Downloads Externals Financial icons mail Music Pictures Public soffice src Templates tmp Videos
expands to: bin Desktop doc Documents Downloads Externals Financial icons mail Music Pictures Public soffice src Templates tmp Videos
Indeed, variable foo is the literal string '[a-z]*'.

To the OP: There is no contradiction. When unquoted within the "[[ ... ]]' test, variable foo is interpreted as a pattern which means "begins with any single character 'a' through 'z' then followed by any number of any characters". Clearly, the string "lhunath" statsfies that condition.

Last edited by rknichols; 05-31-2017 at 09:16 AM.
 
1 members found this post helpful.
Old 05-31-2017, 11:30 AM   #6
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,457

Rep: Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778
Quote:
Originally Posted by rknichols View Post
Hmmm ...
Code:
$ set -x
$ foo=[a-z]*
+ foo='[a-z]*'
$ echo "$foo"
+ echo '[a-z]*'
[a-z]*
$ echo expands to: $foo
+ echo expands to: bin Desktop doc Documents Downloads Externals Financial icons mail Music Pictures Public soffice src Templates tmp Videos
expands to: bin Desktop doc Documents Downloads Externals Financial icons mail Music Pictures Public soffice src Templates tmp Videos
Indeed, variable foo is the literal string '[a-z]*'.
yes, I tried to correct it in the next post.
 
1 members found this post helpful.
Old 05-31-2017, 12:33 PM   #7
NevemTeve
Senior Member
 
Registered: Oct 2011
Location: Budapest
Distribution: Debian/GNU/Linux, AIX
Posts: 3,194

Rep: Reputation: 961Reputation: 961Reputation: 961Reputation: 961Reputation: 961Reputation: 961Reputation: 961Reputation: 961
Perhaps it depends on the content of the current directory: if it is empty, then [a-z]* won't be resolved to filenames. Also it might depend on the shell. Or the shell's current settings.

Edit: Nope, I misunderstood it.

Last edited by NevemTeve; 06-01-2017 at 01:48 PM.
 
1 members found this post helpful.
Old 06-02-2017, 09:18 AM   #8
MadeInGermany
Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 390

Rep: Reputation: 174Reputation: 174
Post#5 is right. The = operator in [[ ]] treats the right side as a pattern, just like in a case-esac.
For this reason I use == within [[ ]] (pattern!) and = within [ ] (simple equivalence), while of course the latter does the (argument-)expansions on both sides.
 
1 members found this post helpful.
Old 06-03-2017, 11:32 AM   #9
andrew.comly
Member
 
Registered: Dec 2012
Distribution: Trisquel-Mini 7.0, Lubuntu 14.04, Debian lxde 8.0
Posts: 294
Blog Entries: 2

Original Poster
Rep: Reputation: 16
Exclamation Main problem solved

VARIABLES
When I echo with quotes (literal value), I see rknichols's point that when setting the value of foo, the value is literally [a-z]*. When I echo w/o quotes (non-literal value/file expansion), unquoted variables containing wildcards expand.

QUOTES
Code:
set -x
$ echo "${foo} ${name}"
+ echo '[a-z]* lhunath'
[a-z]* lhunath
Thus the literal value is indeed [a-z]*.

NO QUOTES:
Code:
$ echo ${foo}; echo; echo ${name}
+ echo Desktop Documents Downloads ... Pictures Public 
Desktop Documents Downloads ... Pictures Public
+ echo

+ echo lhunath
lhunath
Above we see that without quotes, the variable expands.



CONDITIONAL STATEMENT
Code:
$ [[ ${name} == ${foo} ]] && echo Y || echo N
+ [[ lhunath == [a-z]* ]]
+ echo Y
Y
$ [ ${name} = ${foo} ] && echo Y || echo N
+ '[' lhunath = Desktop Documents Downloads ... Pictures Public ']'
bash: [: too many arguments
+ echo N
N
NOW I kind of feel that I am right back where I started from, facing the contradiction of 1) "No filename expansion ... takes place between [[ and ]]", and 2) the latter example.

Then it suddenly hits me, "[a-z]*" is a pattern, and when it expands within the "[[...]]", the variable is expanding, and no file expansion anywhere; The key process involved here is variable expansion!! Thus there is absolutely no contradiction. This problem stems from my miscategorization of [a-z]*. Thanks a lot rknichols and MadeInGermany!!

It's the damnest thing, but I always forget to utilize "set -x". Thanks Pan64 and rknichols for your kind reminders. Sincere regrets!

And yes, I forgot the echo before the last NO, thanks for the reminder pan64!

Last edited by andrew.comly; 06-06-2017 at 03:21 AM. Reason: grammar
 
Old 06-03-2017, 11:33 AM   #10
andrew.comly
Member
 
Registered: Dec 2012
Distribution: Trisquel-Mini 7.0, Lubuntu 14.04, Debian lxde 8.0
Posts: 294
Blog Entries: 2

Original Poster
Rep: Reputation: 16
Question Any examples of file expansion AND work splitting?

QUESTION:
I still have to wonder, can anyone raise an example of
  1. file expansion
  2. word splitting?

I just can't find any examples on the net. What exactly is a file expansion??
 
Old 06-03-2017, 12:55 PM   #11
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 3,469

Rep: Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523Reputation: 1523
You have an example of file expansion (pathname expansion) above when you typed "echo ${foo}" and saw that "[a-z]*" was expanded to "Desktop Documents Downloads ... Pictures Public".

For an example of word splitting,
Code:
Dirs="Desktop Documents Downloads"
ls -ld $Dirs
The expansion of $Dirs is word-split into separate arguments to ls. Compare that to what happens for
Code:
ls -ld "$Dirs"
where the quotes prevent the word splitting, and the ls command doesn't find anything with the 29-character name, "Desktop Documents Downloads".

Last edited by rknichols; 06-03-2017 at 12:56 PM.
 
1 members found this post helpful.
Old 06-03-2017, 01:28 PM   #12
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,457

Rep: Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778Reputation: 2778
filename expansion is for example: ls -ld D*
(try with and without " )
 
1 members found this post helpful.
Old 06-04-2017, 10:53 AM   #13
MadeInGermany
Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 390

Rep: Reputation: 174Reputation: 174
File name expansion and word splitting occur in a for loop (after the in) and in command arguments.
Commands are for example ls, cp, echo, printf, test.
Also [ is a command, indeed it is an alias of test - in [ mode it expects an additional last argument ].
file name expansion means the given word (argument) is taken as a pattern, and matches are done with file names in the current directory. The matching file names replace the given word. It usually becomes longer and can become many words --> "expansion".

A word splitting occurs when an unquoted variable is substituted by its value and if the value contains spaces.
Example
Code:
var="two words"
printf "%s\n" $var
BTW during file name expansion, if a file name contains spaces it is not split into words.
Example
Code:
touch "oneword" "two words"
var="two*"
printf "%s\n" $var
And what happens if there were
Code:
var="two *"
? Variable substitution and word splitting happen first!

Last edited by MadeInGermany; 06-07-2017 at 10:39 AM. Reason: corrected a= to var=
 
1 members found this post helpful.
  


Reply

Tags
conditionals, expansion, pattern, test, wildcard


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
to extract all the part of the filename before a particular word in the filename aealexanderraj Programming 1 08-27-2012 11:08 AM
[SOLVED] Please help script loop into the /tmp/filename.txt out put with filename and wc. dotran Linux - Newbie 10 06-08-2012 05:02 PM
Change name of backup file in ext3 from filename~ to .filename~ Libu Linux - General 2 07-21-2008 09:29 PM
Convert static library (Filename.a) to dynamic shared object (filename.so) afx2029 Linux - Software 4 08-17-2007 06:07 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 09:17 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration