LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 04-09-2008, 06:28 AM   #1
tomerbd1
LQ Newbie
 
Registered: Apr 2008
Posts: 3

Rep: Reputation: 0
sed search replace


Hi,

I have the following file:
[root@comp tmp]# cat test.txt
skfs user[someuser][skjfks
slfkjasklfa
fasfkjasfsa

I'm running the following command:
# sed -i 's/user\[.*?\]/user\[newuser\]/g' test.txt
the file is not changed
when i run the following
# sed -i 's/user\[.*\]/user\[newuser\]/g' test.txt
the file does change to have content of
# cat test.txt
skfs user[newuser][skjfks
slfkjasklfa
fasfkjasfsa

can anyone explain to me why the .*? didnt work? after all i do want it to be .*?

Thanks
 
Old 04-09-2008, 06:34 AM   #2
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,245
Blog Entries: 15

Rep: Reputation: 233Reputation: 233Reputation: 233
perhaps the problem is within sed's internals... maybe *? is illegal.
have you tried it with grep already?
 
Old 04-09-2008, 06:44 AM   #3
choogendyk
Senior Member
 
Registered: Aug 2007
Location: Massachusetts, USA
Distribution: Solaris 9 & 10, Mac OS X, Ubuntu Server
Posts: 1,189

Rep: Reputation: 105Reputation: 105
Quote:
Originally Posted by tomerbd1 View Post
can anyone explain to me why the .*? didnt work? after all i do want it to be .*?
What are you trying to do?

The .* gives you a repetition of any number of any character, including zero. The ? would be to either include or not include the preceding character. So the syntax .*? makes no sense as far as I can tell.
 
Old 04-09-2008, 06:57 AM   #4
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
Quote:
Originally Posted by konsolebox View Post
perhaps the problem is within sed's internals... maybe *? is illegal.
have you tried it with grep already?
How do you do search and replace with grep??

tomerbd1;

I got it to work using sed -r or by escaping the "?"

eg:
sed 's/a.*\?b/xxx/' oldfile > newfile
matches ab, or a + any number of characters--optional + b

As already noted, the "?" in the above is redundant

Apparently, sed needs one of these options when using "+" (one or more occurences) or "?" (optional). I have not completely tested, however.
 
Old 04-09-2008, 09:35 AM   #5
tomerbd1
LQ Newbie
 
Registered: Apr 2008
Posts: 3

Original Poster
Rep: Reputation: 0
Quote:
The .* gives you a repetition of any number of any character, including zero. The ? would be to either include or not include the preceding character. So the syntax .*? makes no sense as far as I can tell.
But please have a look at:

Quote:
* (star) Repeats the previous item zero or more times. Greedy, so as many items as possible will be matched before trying permutations with less matches of the preceding item, up to the point where the preceding item is not matched at all. ".*" matches "def" "ghi" in abc "def" "ghi" jkl
*? (lazy star) Repeats the previous item zero or more times. Lazy, so the engine first attempts to skip the previous item, before trying permutations with ever increasing matches of the preceding item. ".*?" matches "def" in abc "def" "ghi" jkl
This is taken from http://www.regular-expressions.info/reference.html

This says that .*? has a meeting it means non greedy.
Note also that the same regular expression works fine for me with grep its just not working with sed...

so the following is working fine and as expected for me:
Quote:
VWdebian:/tmp# perl -pi -w -e 's/user\ \[.*?\]/user\ \[stam\]/g;' file.txt
(replace all usernames with ‘stam’)

Last edited by tomerbd1; 04-09-2008 at 09:56 AM.
 
Old 04-09-2008, 01:38 PM   #6
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,974
Blog Entries: 11

Rep: Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879
Code:
sed -r 's/user\[[^\]+\]/user\[newuser\]/g' test.txt

Cheers,
Tink
 
1 members found this post helpful.
Old 04-09-2008, 08:34 PM   #7
choogendyk
Senior Member
 
Registered: Aug 2007
Location: Massachusetts, USA
Distribution: Solaris 9 & 10, Mac OS X, Ubuntu Server
Posts: 1,189

Rep: Reputation: 105Reputation: 105
Cool construction, tink. That does it.

Getting back to the question, why doesn't .*? work in sed -- the various tools in the unix/linux toolbox use different extensions and variations of regular expressions. If you look at the O'Reilly book "Unix in a Nutshell", it gives specifics for the different tools: grep, sed, awk, etc. I think that's where I saw a table listing all the various syntax options and whether they were supported in each of the tools.
 
Old 04-09-2008, 08:51 PM   #8
choogendyk
Senior Member
 
Registered: Aug 2007
Location: Massachusetts, USA
Distribution: Solaris 9 & 10, Mac OS X, Ubuntu Server
Posts: 1,189

Rep: Reputation: 105Reputation: 105
Quote:
Originally Posted by tomerbd1 View Post
I had a look at this web site. Interesting. But, I think it does somewhat of a disservice, because it implies that regular expressions are the same from one tool or environment to another. It even says "whether that code is written in Perl, PHP, Java, a .NET language or a multitude of other languages." Unfortunately, this is not precisely true. Thus, people asking, "why doesn't this work in sed?"

The example they give right on their lead page doesn't work in standard grep. The braces need to be escaped. So \{2,4\} gives a repetition of 2 to 4 of the preceding character. But simply {2,4} doesn't work.
 
Old 04-09-2008, 09:33 PM   #9
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,974
Blog Entries: 11

Rep: Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879Reputation: 879
Quote:
Originally Posted by choogendyk View Post
Cool construction, tink. That does it.

Getting back to the question, why doesn't .*? work in sed
Because the old posix sed doesn't know about ?, I don't
think, just as it doesn't know about + ... to enable those
use -r (in GNU sed)


Cheers,
Tink
 
Old 04-10-2008, 04:31 AM   #10
tomerbd1
LQ Newbie
 
Registered: Apr 2008
Posts: 3

Original Poster
Rep: Reputation: 0
Thanks a lot for the replies
I have learned from it and now i know exactly whats going on
Thanks
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How do I replace ' with sed.... @ngelot Linux - Newbie 2 11-02-2007 07:04 PM
sed question for search and replace jakev383 Linux - General 8 05-05-2007 05:40 AM
Bit complicated sed search & replace sharathkv25 Programming 6 04-07-2007 01:53 PM
sed search & replace sharathkv25 Programming 2 03-07-2007 10:16 AM
problem in perl replace command with slash (/) in search/replace string ramesh_ps1 Red Hat 4 09-10-2003 01:04 AM


All times are GMT -5. The time now is 03:52 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration