ok so im using osx 10.6.8 but i have something with linux on it somewhere I can try this on if that is the problem

I am attempting to take certain parts out of a html file I have figured out how to write a regular expression to specify this data I want to delete using the search funciton in the text editor TextWrangler:

(?<=<div id="right_col">)[\s\S]*(?=</body>)

This works in text wrangler but when I try to use it with sed it gives me errors like this one:

sed: 1: "(?<=<div id="right_col" ...": invalid command code (

I know to use -e to avoid problems with the unix version OSX is based on. The sed is a bit different because its based on an older unix variant but i did that and its still giving me an error. I assume there is something basic I am missing about formatting your regular expression for using with commandline and what characters you can use. I bet it has something to do with the backslashes or the round brackets. normally I would just keep searching till I find the answer but I am bored of this problem. I need to think about something else for awhile. maybe in the meantime someone else can chime in and help me out.

can you post the exact sed command you run?

Can you post the exact sed command you've issued and post a representative small chunk of strings you wish to process and the desired outcome? Also please post the output of:

as above post the actual command you are using, also what shell, ( bash,ash etc) but at first glance you may have to escape the brackets '()' as they have special meaning in some ( all ? ) shells.

That regular expression uses Positive lookbehind assertion (?<=...) and Positive lookahead assertion (?=...), sed doesn't support these. Also, sed only matches one line at a time, unless you jump through some hoops.

For sed, you need to use the regular expression as part of a command, eg:

which means replace whatever some-regular-expression matches with nothing.

ok thanks guys i'm new to regular expressions as well as sed. It looks like sed may not be the right tool for the job im deleteing hundreds of lines of code from the top and bottom of a webpage. stripping off the header and footer and all adds. all of this data lies in the top and bottom parts of the page and can be deleted in a big chunk im doing this so I can archive the online copy of the articles in my old wired magizenes. I will then scan what little of the magazine isn't on the site and the throw away the magizenes. Text wranger is working well so far. It would be nice to automate more of the task because it still takes time and I have a lot to go through but maybe I will try using applescript or perhaps attempt to learn pearl.

here is the command I tried for the bottom part of the page:
sed -i -e '(?<=<div id="right_col">)[\s\S]*(?=</body>)' The_Cold_Hard_Data_of_Soda_Ice.html > soda.html

as noted before positive lookbehind and lookahead assertion is not supported so this expresison would have to be pretty much redone from the ground up.

the page looks like this:

<p>

</p>



</div>


</div>

<div id="right_col">
<div id="search">

<form action="http://archive.wired.com/search" id="nav_search" name="search" onsubmit="return validateSearch(this)">
<div class="title">
Search Wired
</div>
<input type="text" class="input_text" name="query">
<select class="search_filter" id="art_filter" name="siteAlias">
<option value="noblog" class="opt" id="art_top_stories" name="noblog" default="true">Top Stories
</option>

......536 lines later:


<embed type="application/x-shockwave-flash" src="http://s.moatads.com/swf/MessageSenderV2.swf" quality="high" flashvars="r=MoatSuperV5.yh.zb&amp;s=MoatSuperV5.yh.zc&amp;e=MoatSuperV5.yh.zd&amp;td=afs.moatads.co m" bgcolor="#ffffff" width="1" height="1" id="moatMessageSenderEmbed" align="middle" allowscriptaccess="always" allowfullscreen="false"></object></div><iframe id="stSegmentFrame" name="stSegmentFrame" src="./Star Power_ Why Fusion Proves Elusive_files/getSegment.html" frameborder="0" scrolling="no" width="0px" height="0px" style="display:none;"></iframe></body></html>