LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 08-21-2009, 10:02 AM   #1
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
bash: case and command substitution--what gives?


I've been trying to embed a simple filtering statement into another command, but I keep getting a syntax error.

I start with something like this to grab files with a specified extension $ext:
Code:
for i; do
  case $i in
     *$ext) echo $i;;
  esac
done
This works perfectly on its own. But then when I try to embed it to say, set a variable:
Code:
files=$(
for i; do
  case $i in
     *$ext) echo $i;;
  esac
done
)
It always gives me "syntax error near unexpected token `;;'"

However, I can change the case structure to something else, like:
Code:
files=$(
for i; do
  if [[ "$i" = *$ext ]]; then
     echo $i
  fi
done
)
...with no errors.

So why doesn't case work here? What am I overlooking?
 
Old 08-21-2009, 10:20 AM   #2
ilikejam
Senior Member
 
Registered: Aug 2003
Location: Glasgow
Distribution: Fedora / Solaris
Posts: 3,109

Rep: Reputation: 96
Hi.

The ')' in the case statement is matching the '(' from files=$(

Try using files=`blah blah blah` instead of files=$(blah blah blah)

Dave

Last edited by ilikejam; 08-21-2009 at 10:21 AM.
 
Old 08-21-2009, 11:14 AM   #3
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Original Poster
Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
Well, whaddaya know. That works. I hadn't thought about trying the backticks.

But I had considered the idea that the ')' was causing it. I decided that that couldn't be it, however, because I can surround it with parentheses to create a subshell.
Code:
(for i; do
  case $i in
     *$ext) echo $i;;
  esac
done)
This gives no errors. It's only when I enclose it in $() that it breaks. Odd.
 
Old 08-21-2009, 11:58 AM   #4
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,486

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
Here is the way your original code should be written:

Code:
files=$(
for i; do
  case $i in
     (*$ext) echo $i;;
  esac
done
)
 
Old 08-21-2009, 08:41 PM   #5
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Original Poster
Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
What, you can do that? I've never seen anything anywhere that said you could completely enclose the matching conditions in a case construct.

In any case, could this be a bug? I'd certainly consider it one. I don't think it should be necessary to materially alter the syntax of the commands inside of $(), particularly if it's not well-documented.



Edit: Did a little google searching, now that I know what's going on. Apparently it's a long-standing bug that's difficult to handle at the compiler level. And that appears to be the reason the fully-enclosed tests were introduced.

It would be nice if it were better-documented though. At least the ABSG could've mentioned it.

Edit two: This page has a chart that lists which shells can and can't handle various "unbalanced" parentheses statements inside $().

Last edited by David the H.; 08-21-2009 at 09:02 PM.
 
Old 08-22-2009, 02:12 AM   #6
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,486

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
I wouldn't tell it's a shell bug as it was identified when the $() syntax was introduced by the korn shell a couple of decade ago, just a well known incompatibility between an old syntax and a newer one with a documented workaround.

I have been using the extra left parenthesis in all my case/esac statements for ages. It looks to me neater and more convenient than the original unbalanced design anyway. Despite being accepted by the standard, the newer shells accepting unbalanced parenthesis looks an ugly hack to me.

It is worth noticing the backquote syntax was considered as archaic in 1992 in the ksh documentation but is still widely used nowadays.
 
Old 09-01-2009, 11:57 AM   #7
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Original Poster
Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
Got a follow-up to this topic.

Debian has just included Bash v.4 in it's repositories (at least in unstable), and now the problem is gone. They've fixed this little error.

I did come across some new problems, however. One of my scripts uses an embedded here document, and it started spitting out an error after the upgrade. I was using something like this:
Code:
output=$(cat <<-FOOBAR

Sample text.
Sample text.
Sample text.

FOOBAR)

echo "$output"
This worked fine in bash 3, but breaks in bash 4. Apparently it now chokes if there's anything else on the same line as the here statement's terminating string. Even comments mess it up. Moving the ")" to the next line lets it run.

Unfortunately though, there also seems to be a new, different bug. Any and all trailing newlines, in any kind of text output, now appear to be removed when inside $(). They not only disappear in the above code, but even when using something like echo.
Code:
#!/bin/bash

output="$(
echo "Sample text"
echo "Sample text"
echo "Sample text"
echo
echo -e "\n\n\n"
)"
echo "$output"
echo "======="

### The output ###

$ ./test.sh
Sample text
Sample text
Sample text
=======
$
Just about every test I've tried with text inside $() has shown the same effect. All trailing newlines disappear. Spaces and tabs do show up though.

I suppose I'll have to switch to using functions for generating large, formatted blocks of text instead.

All in all, though, bash 4 does seem to have some neat new features. I'm looking forward to playing with them.

Last edited by David the H.; 09-01-2009 at 11:59 AM. Reason: fixed a mistake
 
Old 09-02-2009, 07:59 PM   #8
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,486

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
These aren't two new bugs but actually two bug fixes.

Both removing trailing newlines and having the here document alone in its line are requirement to comply with the POSIX standard.

Quoted from http://www.opengroup.org/onlinepubs/...cu_chap02.html
The shell shall expand the command substitution by executing command in a subshell environment (see Shell Execution Environment) and replacing the command substitution (the text of command plus the enclosing "$()" or backquotes) with the standard output of the command, removing sequences of one or more <newline>s at the end of the substitution.

...

The here-document shall be treated as a single word that begins after the next <newline> and continues until there is a line containing only the delimiter and a <newline>, with no <blank>s in between.
 
Old 09-03-2009, 02:03 AM   #9
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Original Poster
Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
Thank you for that information. To be fair, I didn't say that the first one was a bug, only that the behavior was different than before.

The removing all trailing newlines thing is a bit annoying though. It's my opinion that command substitution should, you know, substitute, the complete output of the embedded commands, no matter what they contain. Editing shouldn't be part of the process, even if it's just a few extra newlines. It should be up to the scripter to ensure that the substitution produces the desired output.

I was used to depending on the former behavior. Now I'm going to have to rewrite a couple of my old scripts.
 
Old 09-03-2009, 02:08 AM   #10
jlliagre
Moderator
 
Registered: Feb 2004
Location: Outside Paris
Distribution: Solaris10, Solaris 11, Mint, OL
Posts: 9,486

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
Yes, I agree that can be annoying. Actually, I'm missing what the rationale might be about removing trailing newlines.
 
  


Reply

Tags
bash, case, command substitution


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Bash scripting - find command with variable substitution and quoting bgeddy Programming 3 12-05-2008 07:00 AM
bash command substitution performed "as encountered"? ta0kira Programming 13 04-24-2008 01:16 AM
bash script command substitution and quoting brian4xp Linux - Software 8 02-05-2008 11:43 AM
BASH command substitution that starts with a pipe | Kristofer Programming 4 02-27-2007 05:52 PM
Bash Command Substitution dakensta Programming 5 11-30-2006 03:10 PM


All times are GMT -5. The time now is 10:16 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration