LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 04-13-2007, 01:37 AM   #1
geoff_f
Member
 
Registered: May 2003
Location: Canberra, Australia
Distribution: openSUSE 11.3
Posts: 445

Rep: Reputation: 31
Bash - double quotes don't protect Exclamation marks


In bash, I thought that the only characters that retained their meta-character status inside double quotes were the dollar sign ($), the back-tick (`) and the backslash (\). Yet I get this:
Code:
geoff@home:~/tmp> echo "Exclamation!"
bash: !": event not found
geoff@home:~/tmp>
After that, hitting the 'up' arrow does not retrieve the echo "Exclamation!" command, but the one issued before it. Obviously, something is going on with the bash history function, but it shouldn't be. You'd think that the double quotes would protect the exclamation from being interpreted as a meta-character.

It even happens with an example from the Bash Beginners Guide (http://www.tldp.org/LDP/Bash-Beginne...ect_03_03.html section 3.3.4:
Code:
geoff@home:~> echo "I'd say: \"Go for it!\""
bash: !\"": event not found
geoff@home:~>
whereas the Guide says that it should result in:

I'd say: "Go for it!"

Does anyone know what's going on here?

This is with bash version: 3.00.16(1)-release (SuSE 10.0). The same behaviour is observed in openSUSE 10.2 (bash version 3.1.17(1)-release) and Knoppix 5.1.1 2007-01-04 (bash version 3.1.17(1)-release).
 
Old 04-13-2007, 01:38 AM   #2
b0uncer
LQ Guru
 
Registered: Aug 2003
Distribution: CentOS, OS X
Posts: 5,131

Rep: Reputation: Disabled
Try this:
Code:
echo something!
worked for me, without quotes. Dunno if it's echo-specific (shouldn't be though), but it seems it works
 
Old 04-13-2007, 02:26 AM   #3
geoff_f
Member
 
Registered: May 2003
Location: Canberra, Australia
Distribution: openSUSE 11.3
Posts: 445

Original Poster
Rep: Reputation: 31
Thanks b0uncer, that's very interesting; though it's the opposite of what you'd expect.

It doesn't help me though. In a bash script, I was trying to evaluate the output of the pppoe-start command. When it connects successfully, it outputs to stdout Connected!. My script was falling over trying to process the variable that received the connection result. In the end, I had to hack the pppoe-start script and remove the offending ! character so that my script could evaluate whether the connection succeeded.

I can't explain what's happening with 'echo' (it's the same whether the builtin or /bin/echo). If you enter:
Code:
!something

or

echo !something

or

echo "!something"
you get:
Code:
bash: !something: event not found
So it seems that regardless of the double quotes, bash is being triggered by the exclamation mark to look to the history function.

From the example given in the Bash Beginners Guide, I'd say it used to work as they explain, but that something has changed in the meantime. It seems that we can add the exclamation mark as a meta-character that retains its special meaning within double quotes.
 
Old 04-13-2007, 02:36 AM   #4
omnio
Member
 
Registered: Feb 2007
Location: $HOME
Distribution: Hardened Gentoo
Posts: 66
Blog Entries: 1

Rep: Reputation: 16
Ok, try these too:
Code:
echo 'Exclamation!'
Code:
echo Exclamation\!
Quote:
My script was falling over trying to process the variable that received the connection result. In the end, I had to hack the pppoe-start script and remove the offending ! character so that my script could evaluate whether the connection succeeded.
You can post that evaluation part of your script; hopefully there will be some other ways?

Last edited by omnio; 04-13-2007 at 02:45 AM.
 
Old 04-13-2007, 08:48 PM   #5
geoff_f
Member
 
Registered: May 2003
Location: Canberra, Australia
Distribution: openSUSE 11.3
Posts: 445

Original Poster
Rep: Reputation: 31
Suspect this is a bash bug

Thanks omnio, your examples have prompted me to look at how I was trying to develop my script. The script looks like this:
Code:
# pppoe-reconnect: stop and start a pppoe connection to refresh the connection.
Result=""		# Initialise $Result variable
pppoe-stop
sleep 10s
echo "Connecting..."
Result=$(pppoe-start)
# sed decode: print the line if it has 'Connected' in it. If not, the result will be Null, and the 'until' loop will continue
until [ -n "$(echo "$Result" | sed -ne '/Connected/p')" ]; do
	echo "Reconnect failed: trying again in 30s."
	sleep 30s
	Result=$(pppoe-start)
done
echo "Connected."
Instead of actually running my 'pppoe-start' command (which would have needed 'pppoe-stop' to have been run beforehand), I wanted to check the accuracy of the 'sed' instruction by simulating that output of 'pppoe-start' by an echo statement directly in the bash shell. This is what I entered:
Code:
Result="... Connected!"; echo "$Result" | sed -ne '/Connected/p'
which resulted in the error messages mentioned in my previous posts.

If I entered:
Code:
Result="TIMED OUT"; echo "$Result" | sed -ne '/Connected/p'
I got a Null output, which is what I expected. I then hacked 'pppoe-start' to remove the exclamation character as a quick way of achieving what I wanted.

On reading your post, I decided to investigate this further. I could see that 'pppoe-start' had no difficulty outputting the exclamation character, so instead of using the 'echo' command in my test, I created this small script called 'pppoe-connected':
Code:
#!/bin/bash
echo "... Connected!"
and used the following as the test directly in the bash shell:
Code:
Result=$(./pppoe-connected); echo "$Result" | sed -ne '/Connected/p'
This worked, which means that I can now go back to my 'pppoe-start' script and put the exclamation character back in.

It also means that the problem I experienced was not with the script I was developing, but with the method of testing in an interactive shell, which behaviour differs from that of a non-interactive shell that the scripts run in. I still think that the behaviour in the interactive shell is a bug, as it doesn't obey the rules specifying which special characters are protected by double quotes. It also used to behave according to the rules in the past, which also points to some recently introduced bug. Also, in the interactive shell, the exclamation character is only supposed to have special character status to invoke the history function when it's the first character on the line. But if you enter:
Code:
echo "something!"
the familiar error is produced. Two things are happening here. Firstly, the double quotes, far from protecting the exclamation character from shell interpretation, actually turn it into its metacharacter form. It should behave identically to your example:
Code:
echo 'Exclamation!'
where the single quotes do protect the exclamation character.

Secondly, the text appearing before the exclamation character is being ignored, as if the exclamation character is the first one in the line, which sends bash on a search for a command from the history function. I assume this is a bug, which I will report to the bash developers.

The problem doesn't show in the non-interactive shell because the history function is not available there.

Many thanks to b0uncer and omnio, your suggestions have helped me greatly - problem solved (for my purposes, at least).
 
Old 04-13-2007, 09:47 PM   #6
cfaj
Member
 
Registered: Dec 2003
Location: Toronto, Canada
Distribution: Mint, Mandriva
Posts: 221

Rep: Reputation: 31
Quote:
Originally Posted by geoff_f
In bash, I thought that the only characters that retained their meta-character status inside double quotes were the dollar sign ($), the back-tick (`) and the backslash (\). Yet I get this:
Code:
geoff@home:~/tmp> echo "Exclamation!"
bash: !": event not found
geoff@home:~/tmp>
...
It even happens with an example from the Bash Beginners Guide (http://www.tldp.org/LDP/Bash-Beginne...ect_03_03.html section 3.3.4:
Code:
geoff@home:~> echo "I'd say: \"Go for it!\""
bash: !\"": event not found
geoff@home:~>
whereas the Guide says that it should result in:

I'd say: "Go for it!"

Does anyone know what's going on here?

Read the bash man page:

HISTORY EXPANSION
... Only backslash (\) and single quotes can quote the history expansion character.

Several characters inhibit history expansion if found immediately following the history expansion character, even if it is unquoted: space, tab, newline, carriage return, and =. If the extglob shell option is enabled, ( will also inhibit expansion.
You can turn off this annoying behaviour by setting histchars to an empty string:
Code:
histchars=
 
Old 04-13-2007, 10:20 PM   #7
geoff_f
Member
 
Registered: May 2003
Location: Canberra, Australia
Distribution: openSUSE 11.3
Posts: 445

Original Poster
Rep: Reputation: 31
Thanks for that cfaj, I missed that snippet. I won't bother with a bug report.
 
Old 04-14-2007, 04:55 AM   #8
omnio
Member
 
Registered: Feb 2007
Location: $HOME
Distribution: Hardened Gentoo
Posts: 66
Blog Entries: 1

Rep: Reputation: 16
I'm glad you solved it; I had my hard times with PPPOE which used to behave strangely. For example, I see that in your script you check only the pppoe-start output, but I also saw some cases when it said "connected" but there was no working internet connection and no traffic, so you will probably want to add a ping test or whatever, just to be sure. And sometimes it takes a while till the connection is actually working, so you should probably not execute the ping test immediately.

I don't know how exactly PPPOE works, but some ISPs provide a dynamic IP (via DHCP) *before* connecting with PPPOE, and if the client fails to receive that IP, PPPOE will fail too; in this case, just issuing "pppoe-stop; pppoe-start" would not suffice, the network interface needs to be reconfigured to get that dynamic IP first. This is a mess, if you ask me.

Quote:
Originally Posted by geoff_f
This worked, which means that I can now go back to my 'pppoe-start' script and put the exclamation character back in.
I hope there are always other ways than that. Let's say you wanted to give the script to your friends, mates etc. You would tell them to alter the pppoe-start script?
 
Old 04-14-2007, 08:45 AM   #9
geoff_f
Member
 
Registered: May 2003
Location: Canberra, Australia
Distribution: openSUSE 11.3
Posts: 445

Original Poster
Rep: Reputation: 31
Omnio, you raise a good point on testing. I have only implemented testing the output of 'pppoe-start' because in the four years I have been using this ISP, I have not had one instance of a 'Connected!' message associated with a failed connection. The only failed connections have had 'TIMED OUT' in the logs. The communication medium is fibre-to-the-node, then Cat 5 cable to the house, which I've found to be very reliable. But you're quite right that if I were to pass the script onto my friends with, say, ADSL, then better testing would be called for.

Note that the 'pppoe-start' script is only establishing a connection to the equipment at the ISP. Ie, you get a line in to their system. If they have problems with other equipment that is needed to give you a connection to the Internet (DNS servers, routers, etc), then you will fail to get a line out to the Internet. Doing a 'pppoe-stop'/'pppoe-start' will not fix that problem. I have had a few occasions when the connection via 'pppoe-start' was successful, but had no service to email or the Internet. The symptoms are that you can ping the 'inet addr' and 'P-t-P' addresses (both listed against 'ppp0' from 'ifconfig'), but can't ping their DNS addresses, nor any other TCP/IP address in the Internet. You also can't ping a plain-language address (eg, www.google.com) as their DNS server is not working. That needed a phone call to the ISP to get them to fix their servers. As soon as you mention that you can ping the 'inet addr' and 'P-t-P', but not their DNS servers, they know the problem is at their end, or at the carrier's. On a couple of occasions, it has been the carrier's equipment, which includes the ethernet modem in my house (physically here, but owned by them). All that needed was a reboot of the modem - something to do with 'clearing the cache' at both ends of the connection with the carrier (a separate company to the ISP in my case).

Quote:
I hope there are always other ways than that. Let's say you wanted to give the script to your friends, mates etc. You would tell them to alter the pppoe-start script?
You misunderstand me on this. I originally hacked the 'pppoe-start' script to take out the exclamation character to get my script to run (ie, I made it non-standard). When I say that I can now put the exclamation character back into the script, that is only to restore it to its original state, which is the same as what other users would have on their system.
 
Old 04-14-2007, 09:14 AM   #10
geoff_f
Member
 
Registered: May 2003
Location: Canberra, Australia
Distribution: openSUSE 11.3
Posts: 445

Original Poster
Rep: Reputation: 31
Quote:
I don't know how exactly PPPOE works, but some ISPs provide a dynamic IP (via DHCP) *before* connecting with PPPOE, and if the client fails to receive that IP, PPPOE will fail too; in this case, just issuing "pppoe-stop; pppoe-start" would not suffice, the network interface needs to be reconfigured to get that dynamic IP first. This is a mess, if you ask me.
I forgot to make mention of this. Unless your contract includes a static IP address, all ISP's will allocate you a dynamic IP address as part of the process of connecting with 'pppoe-start'. Since the ISP provides a dynamic IP address via DHCP, your network interface card should be configured to get an IP address from their DHCP server. In this case, reconfiguring the interface is not required. Whenever you attempt a connection (with a 'pppoe-start'), your network interface card is primed to receive an address via DHCP. This process is automatic and requires no reconfiguration on your part. In this case, the 'pppoe-start' command sent to your ISP is the equivalent of 'ifup' on your local network. If you are having problems connecting, maybe it's because there is not enough time between the 'pppoe-stop' and 'pppoe-start' commands. For example, my previous 'reconnection' script was a simple:
Code:
pppoe-stop
sleep 5s
pppoe-start
This worked for a long time, and only recently began to result in 'TIMED OUT' errors. I then developed the new version that included error checking. With that, I increased the 'sleep' time to 10 seconds. I'm now thinking that I should increase that to 30 seconds or so, to give the ISP's servers enough time to fully log me out before I come back knocking on their door. If the server is delayed for any reason in logging me out, then it might refuse a connection, on the basis that I'm still 'logged in' as far as they're concerned.

[edit: minor change to expression]

Last edited by geoff_f; 04-14-2007 at 09:16 AM.
 
Old 04-14-2007, 02:50 PM   #11
omnio
Member
 
Registered: Feb 2007
Location: $HOME
Distribution: Hardened Gentoo
Posts: 66
Blog Entries: 1

Rep: Reputation: 16
Quote:
Originally Posted by geoff_f
I forgot to make mention of this. Unless your contract includes a static IP address, all ISP's will allocate you a dynamic IP address as part of the process of connecting with 'pppoe-start'. Since the ISP provides a dynamic IP address via DHCP, your network interface card should be configured to get an IP address from their DHCP server. In this case, reconfiguring the interface is not required.
Yep. To restart it, this is what I meant. It stays "configured" to get a dynamic IP, but I saw many cases when it fails to receive it and when a restart of the interface (and of the DHCP client apparently) is needed.
Quote:
Originally Posted by geoff_f
I have only implemented testing the output of 'pppoe-start' because in the four years I have been using this ISP, I have not had one instance of a 'Connected!' message associated with a failed connection.
Wow, you're lucky.
Quote:
Originally Posted by geoff_f
You misunderstand me on this. I originally hacked the 'pppoe-start' script to take out the exclamation character to get my script to run (ie, I made it non-standard). When I say that I can now put the exclamation character back into the script, that is only to restore it to its original state, which is the same as what other users would have on their system.
Well, I was half joking.
 
Old 04-14-2007, 02:58 PM   #12
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 657Reputation: 657Reputation: 657Reputation: 657Reputation: 657Reputation: 657
I hope this wasn't already mentioned; I may have scanned the previous messages too quickly. The history expansion occurs in an interactive shell. If you have a shell running from a script, then a filename with an exclamation point in it shouldn't present the same problems. E.G. evaluating "$file" if the variable value contains an exclamation point won't cause surprises.

Last edited by jschiwal; 04-15-2007 at 05:30 AM.
 
Old 04-14-2007, 09:21 PM   #13
geoff_f
Member
 
Registered: May 2003
Location: Canberra, Australia
Distribution: openSUSE 11.3
Posts: 445

Original Poster
Rep: Reputation: 31
jschiwal said:
Quote:
The history expansion occurs in an interactive shell. If you have a shell running from a script, then a filename with an exclamation point in it should present the same problems.
Yes, this was mentioned a few posts before:
Quote:
It also means that the problem I experienced was not with the script I was developing, but with the method of testing in an interactive shell, which behaviour differs from that of a non-interactive shell that the scripts run in.
I suppose I could have been a bit more explicit and said that scripts run in a non-interactive shell, and thus won't be subjected to history expansion, and therefore not be affected by the problem I was seeing. The mistake I made was to conduct testing directly in an interactive shell, assuming the effect would be the same as the non-interactive shell. Lots of bash tutorial sites recommend running commands in the shell to test that your code delivers what you want. Had I instead created a small script to replicate what I wanted (a trivial exercise in this case) instead of launching off at a tangent chasing false shadows, I would not have wasted a lot of time. That in itself has been an excellent lesson.

Quote:
If you have a shell running from a script, then a filename with an exclamation point in it should present the same problems.
Yes, it probably would occur with a filename, but this thread has been about exclamation points in variables. For the sake of completeness and accuracy for Googlers, an exclamation point in a variable in a bash script will not present the same problem discussed here.
 
Old 04-15-2007, 05:44 AM   #14
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 657Reputation: 657Reputation: 657Reputation: 657Reputation: 657Reputation: 657
I had a typo, which I corrected. I changed should to shouldn't.

I had previously made the same discovery when including an exclamation point in an echo command where the string was in double quotes.

I just tried an interactive command where I had a variable with an exclamation point, so my memory was wrong on that point.

Since you can use single quotes or the backslash to escape it, then perhaps getting in the habit of doing so even if you are writing a script would be a good practice. Then your interactive testing would work.

echo Good Luck\!
salutation='Best Wishes'
echo "and $salutation"'!'

Last edited by jschiwal; 04-15-2007 at 05:55 AM.
 
Old 04-15-2007, 07:49 PM   #15
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 657Reputation: 657Reputation: 657Reputation: 657Reputation: 657Reputation: 657
I hope it isn't too late to add to this thread. There is something else to watch for when testing code by entering lines in interactively.
If you change the IFS variable, as in
IFS='
'
Then you can run into a problem with some commands because the space character in the aliases doesn't separate the command from the arguments because space isn't in IFS. I usually back up IFS and restore it after the command I need it for.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
dead keys, setxkbmap and upside down exclamation marks! Komakino Linux - Software 2 08-03-2010 11:16 AM
Problems with quotes and double quotes Andruha Slackware 6 01-02-2010 04:44 PM
Using single quotes vs double quotes in PHP strings vharishankar Programming 6 07-11-2005 11:41 AM
In BASH shell, what is the difference in usage between single and double quotes? davidas Linux - Newbie 2 04-05-2004 03:00 AM
Shell scripting: exclamation marks in strings Dark_Helmet Programming 9 06-16-2003 06:32 PM


All times are GMT -5. The time now is 02:24 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration