LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-30-2010, 09:51 AM   #1
ghantauke
Member
 
Registered: Nov 2010
Posts: 114

Rep: Reputation: 6
using the special RE codes \1,...,\9 in bash sed


I'm using bash. Assume I have the following text in file test.
Code:
Hey there. This is a sample text.
Testing *cough*... Very harsh testing don't.
Lets say I want to replace each word starting with s and ending with e to be replaced by a 2 at the front. I use the following code.
Code:
sed 's/ s.*e / 2\1 /g' test
I get the error
Code:
sed: -e expression #1, char 16: invalid reference \1 on `s' command's RHS
Seems like i can't use \1 to return the first pattern matched. I can do it in perl but not in bash. Need help. Thanks in advance.
 
Old 11-30-2010, 09:58 AM   #2
GrapefruiTgirl
LQ Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551
Things being referenced with back-references must be found on the LHS inside \(...\) in order to be referenced. Here's an example:
Code:
root@reactor: echo "word1 word2 word3" | sed 's/\(w.*1\)\(.*\)/Q\1Q\2/'
Qword1Q word2 word3
root@reactor:
Hopefully you are not color blind - I have colored the above. The red stuff is the first of two \(...\) I have used. The second refers to the remainder of the input line, after the match. So, on the RHS of the expression, \1 refers to the red stuff on the LHS, and \2 refers to the green stuff.

Note that this is just a simple example - play around a bit with it in your terminal to see what you can discover.
 
Old 11-30-2010, 10:03 AM   #3
GrapefruiTgirl
LQ Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551
PS - Have a look at this page for a very good, more comprehensive (and more convoluted perhaps) instruction about back-references, and browse the remainder of that site for lots of good regex instruction:

http://www.regular-expressions.info/brackets.html
http://www.regular-expressions.info/tutorial.html
 
Old 11-30-2010, 10:03 AM   #4
ghantauke
Member
 
Registered: Nov 2010
Posts: 114

Original Poster
Rep: Reputation: 6
Quote:
Originally Posted by GrapefruiTgirl View Post
Things being referenced with back-references must be found on the LHS inside \(...\) in order to be referenced. Here's an example:
Code:
root@reactor: echo "word1 word2 word3" | sed 's/\(w.*1\)\(.*\)/Q\1Q\2/'
Qword1Q word2 word3
root@reactor:
Hopefully you are not color blind - I have colored the above. The red stuff is the first of two \(...\) I have used. The second refers to the remainder of the input line, after the match. So, on the RHS of the expression, \1 refers to the red stuff on the LHS, and \2 refers to the green stuff.

Note that this is just a simple example - play around a bit with it in your terminal to see what you can discover.
Oh bloody 'ell. Forgot the brackets haha. Thanks for reminding me that.
 
Old 11-30-2010, 10:08 AM   #5
ghantauke
Member
 
Registered: Nov 2010
Posts: 114

Original Poster
Rep: Reputation: 6
Quote:
Originally Posted by GrapefruiTgirl View Post
Things being referenced with back-references must be found on the LHS inside \(...\) in order to be referenced. Here's an example:
Code:
root@reactor: echo "word1 word2 word3" | sed 's/\(w.*1\)\(.*\)/Q\1Q\2/'
Qword1Q word2 word3
root@reactor:
Hopefully you are not color blind - I have colored the above. The red stuff is the first of two \(...\) I have used. The second refers to the remainder of the input line, after the match. So, on the RHS of the expression, \1 refers to the red stuff on the LHS, and \2 refers to the green stuff.

Note that this is just a simple example - play around a bit with it in your terminal to see what you can discover.
Don't think it matters anymore but I think you should remove the backslash before the brackets in \(w.*1\) and \(.*\). Anyway thanks again
 
Old 11-30-2010, 10:13 AM   #6
GrapefruiTgirl
LQ Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551
Code:
root@reactor: echo "word1 word2 word3" | sed 's/(w.*1)(.*)/Q\1Q\2/'
sed: -e expression #1, char 20: invalid reference \2 on `s' command's RHS
root@reactor: echo "word1 word2 word3" | sed 's/(w.*1)(.*)/Q\1Q\2/'
sed: -e expression #1, char 20: invalid reference \2 on `s' command's RHS
root@reactor: echo "word1 word2 word3" | sed 's/(w.*1)(.*)/Q\1Q/'
sed: -e expression #1, char 18: invalid reference \1 on `s' command's RHS
root@reactor:
It seems to matter here, in this case. But, it will depend on the tool, the regex style, and the quoting in use, whether you need to escape, or not, or double-escape, certain things in regex's. I tend to always put escapes in situations like this as a habit - usually there's no harm. Maybe if I actually sat down and studied every possible combination of quotes and regex styles, I might really learn when to escape and when not to, but probably I won't invest the time. I'll just keep escaping things, and if testing poves it to be bad in a certain case, I fix it then.
 
Old 11-30-2010, 10:35 AM   #7
ghantauke
Member
 
Registered: Nov 2010
Posts: 114

Original Poster
Rep: Reputation: 6
Quote:
Originally Posted by GrapefruiTgirl View Post
Code:
root@reactor: echo "word1 word2 word3" | sed 's/(w.*1)(.*)/Q\1Q\2/'
sed: -e expression #1, char 20: invalid reference \2 on `s' command's RHS
root@reactor: echo "word1 word2 word3" | sed 's/(w.*1)(.*)/Q\1Q\2/'
sed: -e expression #1, char 20: invalid reference \2 on `s' command's RHS
root@reactor: echo "word1 word2 word3" | sed 's/(w.*1)(.*)/Q\1Q/'
sed: -e expression #1, char 18: invalid reference \1 on `s' command's RHS
root@reactor:
It seems to matter here, in this case. But, it will depend on the tool, the regex style, and the quoting in use, whether you need to escape, or not, or double-escape, certain things in regex's. I tend to always put escapes in situations like this as a habit - usually there's no harm. Maybe if I actually sat down and studied every possible combination of quotes and regex styles, I might really learn when to escape and when not to, but probably I won't invest the time. I'll just keep escaping things, and if testing poves it to be bad in a certain case, I fix it then.
Code:
bash-4.1$ sed -r 's/ (s.*e) / 2\1 /g' test
Hey there. This is a 2sample text.
Testing *cough*... Very harsh testing don't.

bash-4.1$ sed -r 's/ \(s.*e\) / 2\1 /g' test
sed: -e expression #1, char 20: invalid reference \1 on `s' command's RHS
It seems to not to require \ before brackets here.

Code:
bash-4.1$ sed 's/([^a-zA-Z])/ \1 /g' test
sed: -e expression #1, char 20: invalid reference \1 on `s' command's RHS

bash-4.1$ sed 's/\([^a-zA-Z]\)/ \1 /g' test
Hey   there .    This   is   a   sample   text . 
Testing    * cough *  .  .  .    Very   harsh   testing   don ' t .
It seems to require \ before brackets here.

Its probably has to do with the fact that I'm using [] in the second code. Well I'm a bit confused but I don't think its gonna be a big problem since I can just use trial and error method when needed. Anyway thank you all for helping me out.
 
Old 11-30-2010, 10:43 AM   #8
GrapefruiTgirl
LQ Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551Reputation: 551
Yes, notice how -r changes the behavior of the regex!
 
Old 11-30-2010, 10:45 AM   #9
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910Reputation: 910
Quote:
Originally Posted by ghantauke View Post

Its probably has to do with the fact that I'm using [] in the second code. Well I'm a bit confused but I don't think its gonna be a big problem since I can just use trial and error method when needed. Anyway thank you all for helping me out.

Actually it's the -r
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Problem getting Sed to work with special char inputs. shaselai Programming 8 10-23-2009 01:10 PM
sed special sysslack Programming 3 05-06-2009 04:01 AM
Help w/ sed parsing special characters clem_c_rock General 8 08-31-2007 04:06 PM
Bash scripting problem with exit codes Jeiku Programming 2 05-15-2006 01:22 AM
using sed to insert lines with special characters disorderly Linux - Software 26 04-20-2006 05:30 PM


All times are GMT -5. The time now is 02:45 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration