LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 10-05-2005, 06:16 PM   #1
Wynd
Member
 
Registered: Jul 2001
Distribution: Slackware 12
Posts: 511

Rep: Reputation: 32
Using sed to replace a lot of text


I have some HTML files that I want to run through sed and replace a block of text with another block. I'm doing this:

cat index.html.old | sed 's@`cat find.txt`@`cat replace.txt`@' > index.html

However, index.html remains unchanged. Is there a better way to be doing this?
 
Old 10-05-2005, 06:55 PM   #2
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 374Reputation: 374Reputation: 374Reputation: 374
That's an interesting idea, and it would probably work except for your choice of quotes. Use double quotes instead of single quotes:
Code:
cat index.html.old | sed "s@`cat find.txt`@`cat replace.txt`@" > index.html
The shell treats anything inside single quotes as literal text, whereas the command actually wants the shell to do command substitution inside the sed expression.

EDIT:
Just an example of what I'm getting at:
Code:
$ echo "$DISPLAY"
:0.0
$ echo '$DISPLAY'
$DISPLAY

Last edited by Dark_Helmet; 10-05-2005 at 06:58 PM.
 
Old 10-06-2005, 01:57 PM   #3
Wynd
Member
 
Registered: Jul 2001
Distribution: Slackware 12
Posts: 511

Original Poster
Rep: Reputation: 32
I get this:

sed: -e expression #1, char 26: Unterminated `s' command
 
Old 10-06-2005, 02:22 PM   #4
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 374Reputation: 374Reputation: 374Reputation: 374
It works for me. Here's a simple run-through:

./input_file
Code:
This is a test
This is another test
Another line is a test
The fourth line is a test
Lastly, the fifth line contains a test
./find.txt
Code:
is a test
./replace.txt
Code:
replaced
Command:
Code:
$ cat input_file | sed "s@`cat find.txt`@`cat replace.txt`@" > processed_output
$ cat processed_output
This replaced
This is another test
Another line replaced
The fourth line replaced
Lastly, the fifth line contains a test
You will run into problems if your find or replacement text contains special characters. For instance, since the 's' command to sed uses that '@' symbol to denote the pattern match, the replacement, and end of the sed command, the find.txt and replace.txt files can't contain '@' unless they are escaped: \@

Similarly, neither find.txt or replace.txt can have a double-quote ( " ) in them, because that will end the sed command (it matches/closes the double-quote right before s@). You can use them, but again, they need to be escaped: \"

You also need to be aware that sed will try to interpret find.txt's contents as a regular expression. That might have undesired results.
 
Old 10-06-2005, 02:32 PM   #5
Wynd
Member
 
Registered: Jul 2001
Distribution: Slackware 12
Posts: 511

Original Poster
Rep: Reputation: 32
Yeah, I figured that out. I'm trying to replace some actual HTML, which I think was throwing it off. I came up with this:

Code:
cat index.html.old | sed -e '/<!-- START -->/,/<!-- END -->/c\`cat replace.txt`'
The problem with this is that it literally prints `cat replace.txt` where the new HTML should go. Is there a way I can get it to put the results of the cat command there?
 
Old 10-06-2005, 02:37 PM   #6
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 374Reputation: 374Reputation: 374Reputation: 374
Not while it's in single quotes

The shell will do no interpretation at all to text within a pair of single quotes. No variable expansion, no command substitution, no file globbing, nada.
 
Old 10-06-2005, 02:49 PM   #7
Wynd
Member
 
Registered: Jul 2001
Distribution: Slackware 12
Posts: 511

Original Poster
Rep: Reputation: 32
Every other way I try I get:

-bash: !--: event not found
 
Old 10-06-2005, 02:55 PM   #8
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 374Reputation: 374Reputation: 374Reputation: 374
Yeah, the exclamation mark is special to bash. You'll need to escape it too.

In case you're curious, it's the history operator (or something like that). It executes the last command that starts with the text immediately after it. So, in this case, it was looking for a command in the shell history that started with two dashes.
 
Old 10-06-2005, 04:09 PM   #9
Dark_Helmet
Senior Member
 
Registered: Jan 2003
Posts: 2,786

Rep: Reputation: 374Reputation: 374Reputation: 374Reputation: 374
I guess I should have provided this before, but here's another example:

./experiment.html
Code:
Here's some text
Some more
<!-- START -->
Text to be cut
Some more text
<!-- END -->
Text after cut section
./replace.txt
Code:
replaced
command:
Code:
$ cat experiment.html | sed -e "/<\!-- START -->/,/<\!-- END -->/c `cat replace.txt`"
Here's some text
Some more
replaced
Text after cut section
All I did was escape the exclamation marks, and removed the backslash after the "/c"

The man page for sed makes it look like there should be one, but it shouldn't be there.
 
Old 10-07-2005, 03:47 AM   #10
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682Reputation: 682
You can't have several lines of text in the regular expression to be searched without adding lines to the line buffer, and you might need to use the Hold buffer also.

You should use a form such as:
sed -f replacehtml.sed index.html.old >index.html
where replacehtml.sed is a file of sed commands.
There may be several cases to consider, such as when the pattern to be replaced is split between several lines, if two patterns to be replaced might occur on the same line, if you have a pattern on one line and the beginning of another pattern on the same line.

After you have developed your sed program, test it out on several files before process all of the files.

You can process them in a bash script like:
for htmlfile in *.html; do
sed -f changehtml.sed $htmlfile ${htmlfile%.html}.new.html
done

If what you are doing is simply replacing html files with references to those files, then a script that takes the names and enters the names into a new document may be a better option. If you use bash, consider using "printf". It is more flexible than using "echo".

Last edited by jschiwal; 10-07-2005 at 04:37 AM.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How can I replace this string with another using sed? dave4545 Programming 7 01-27-2006 10:58 AM
Replace substring with SED marri Programming 2 07-09-2005 05:18 PM
can I replace text with the result of "wc" using sed? BrianK Linux - General 1 04-21-2004 01:15 PM
[sed] replace string? chuanyung Programming 3 03-11-2004 08:42 PM
Sed - suitable to replace CR LF? J_Szucs Programming 3 05-12-2003 06:03 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 10:03 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration