SED limited global substitution help

paullanders · 09-02-2009, 05:28 PM

Hi.

I am learning SED and have happened upon a substitution that I simply cannot figure out. I have a page of HTML with anchor tags and links that contain spaces. I must replace all spaces only within the anchor tags. That is, I need a global substitution that is limited within a string. For example:

originals:

<a href="#So is there more than one Library Committee">So is there more than one Library Committee?</a><br />
<a href="#Where are the Libraries?">Where are the Libraries?</a><br />

substitutions needed:

<a href="#So_is_there_more_than_one_Library_Committee">So is there more than one Library Committee?</a><br />
<a href="#Where_are_the_Libraries?">Where are the Libraries?</a><br />

original:

<a name="So is there more than one Library Committee" id="So is there more than one Library Committee"></a>
<h4 align="center">So is there more than one Library Committee?</h4>

substitution needed:

<a name="So_is_there_more_than_one_Library_Committee" id="So is there more than one Library Committee"></a>
<h4 align="center">So is there more than one Library Committee?</h4>

Any help would be appreciated!

thanks!

Kenhelm · 09-02-2009, 07:12 PM

This uses GNU sed -r so the '( ) |' characters don't need escaping, which makes the code more readable.
It uses a loop ':a.......ta' which replaces one space on each loop cycle.
"[^"]* limits 'greedy matching' so that replacements are only made within the double quotes.

Code:

sed -r ':a s/(<a (href|name)="[^"]*) /\1_/;ta'

paullanders · 09-03-2009, 09:50 AM

Thank you so much!!! You are a hero for taking the time to invest in someone else's learning. It works perfectly!

As I studied the regex I was initially confused by "[^"]* limits 'greedy matching' because I was incorrectly assuming that as we looped we would be replacing the first space encountered until we eventually exhausted all matches. Now I see that we are actually replacing the last space encountered, then working backwards until all matches are exhausted.

I re-tested again, but this time I made the * reluctant:

sed -r ':a s/(<a (href|name)="[^"]*?) /\1_/;ta'

It still works perfectly, but I am assuming the regex engine now replaces the first space matched, continuing onward to the last match. Is my assumption correct?

Thanks again!

Kenhelm · 09-03-2009, 08:31 PM

Greedy matching can be limited in a Perl regex with *? but not in GNU sed.
Removing the loop ':a......ta' shows that in spite of using *? instead of * it still replaces the last space first.
sed -r 's/(<a (href|name)="[^"]*?) /\1_/'

*? seems to be the same as * in GNU sed -r
For example
echo 'abcXdefXg' | sed -r 's/a.*?X//'
g

The output is not 'defXg' so matching has not been limited to the first 'X'.

There's some more on this topic at
http://www.linuxforums.org/forum/lin...ngle-line.html

paullanders · 09-04-2009, 11:47 AM

I have just discovered that there were commas that needed replacing, and also a few instances of multiple characters that needed substitution.

Example: <a name="Education, Careers, and Outreach">

I modified the regex to be:

sed -r ':a s/(<a (href|name)="[^"]*)( |\,)+/\1_/';ta'

But the result is not as I thought:

Expected: <a name="Education__Careers__and_Outreach">
Actual: <a name="Education, Careers, and_Outreach">

Why is + not matching 1 or more of the alternation ( |\,)

Thanks!

Kenhelm · 09-04-2009, 07:38 PM

Code:

# Your method works for me
echo '<a name="Education, Careers, and Outreach">' |
sed -r ':a s/(<a (href|name)="[^"]*)( |\,)+/\1_/;ta'
<a name="Education__Careers__and_Outreach">

# I get your output if the loop is removed
echo '<a name="Education, Careers, and Outreach">' |
sed -r 's/(<a (href|name)="[^"]*)( |\,)+/\1_/'
<a name="Education, Careers, and_Outreach">

# The \ and + in ( |\,)+ aren't necessary.
# [ ,] can be used instead of ( |\,)+
sed -r ':a s/(<a (href|name)="[^"]*)[ ,]/\1_/;ta'

paullanders · 09-05-2009, 10:44 AM

:-( It's the silly mistakes that get me. When I had pasted the code I failed to select the loop. Augh!

Thank you for the guidance on using a character class rather than the alternation. I like it!

Paul