NOTE: This post may be longer than it needed to be, or not long enough; I can't be sure. Hopefully it is clear enough. I think if you read it through, it should all make perfect sense . But I make no apologies
The subject of regular expressions is large enough to fill a book
, but I'll see if I can't give you the gist of it here.
Regular expressions are (sometimes complicated-looking) chunks of ascii characters. An example is the one which jim provided: 's/^#/*/'. The actual regular expression ("regex") in question here is the "^#" (the entire expression 's/^#/*/' is called a "substition"). Well, what are regexes for?
The reason regexes exist is to allow us to identify interesting portions of text within some larger block (or line) of text. There are a couple good reasons for wanting to "identify interesting portions of text"; we may want to pull interesting portions out, for use elsewhere
or alter certain portions of text, in place
. The text in question could be the manuscript for a book, an HTML page, or even the example you provided (8 or so lines of #'s, and some human english words).
Clearly your goal here is to alter certain portions, in place
. You want to "change the first and last columns from # to *". Well, how do we do this? We need some way to identify the "first column" and the "last column" and then alter them somehow. Luckily, we have regexes. Regexes identify the portion of text, and programs like sed provide operations to alter, or otherwise use that text. The 's/regex/newtext' operation is one example, which sed provides (as do other programs: perl, awk, many more I can't think of atm).
Before we get to the regex, lets be clear on the syntax for the substition command:
The 's' is for "substitution", the regex is our regex, and the newtext is our new text.
So, I'm going to start with a simple example using grep (grep loves regexes):
$ echo "Not a speck of cereal" | grep "spalien acecraft"
Nothing happened! Why not? grep gobbled up our original text, compared it with our regex ("spalien acecraft") noted that it didn't match, and gave us the cold shoulder. Compare that with:
$ echo "Not a speck of cereal" | grep "t a sp"
Not a speck of cereal
Something happened! grep, being clever, noticed that "t a sp" exists in our original text: "Not a sp
eck of cereal", so it spat the line back at us. What we've learned here is that, in a regex, normal characters match themselves (ie. an 'a' in our regex will match an 'a' in our original text, 'b' will match a 'b', etc.). That's simple enough.
Ok, play with that. It's pretty obvious (extremely useful though!). Now, what if we want to match something more complicated? We need magic. Magic is confusing. I can't teach you magic here! But I can explain the spell that Jim cast.
The '^' is magic; it matches "beginning of a line". So, by saying '^#', we're saying "identify any line which starts with '#'". Just to confuse things, here is another example of this:
grep "^#include" /usr/include/net/*.h
That will match all lines starting with "#include", in every header file in the "net" include directory. Massively interesting, I know...
So, getting back to our regex and our substition command to sed. 's/^#/*/'. Brilliant! Should be clear by now. What you need to do now, is find out how you can match the "end of a line". Now, depending on whether you want to match the end of each line, or just the first and last lines, you'll have to work some magic of your own. Think about the regex. What makes the first and last lines unique? How can you craft a spell to match only the endings of the first and last lines?
It takes a lot of trial and error. You may even think you've found an answer sometimes, but always carefully double-check it. You may be gobbling up more characters than you intended, or who knows what.
p.s. you can of course use multiple seds:
cat somefile |sed 's/regex/newtext/' |sed 's/regex/newtext/'
but a cleaner way (or not, depending on personal taste) is to use the -e option to sed. It works like this:
cat somefile |sed -e 's/regex/newtext/' -e 's/regex/newtext'
which allows multiple edits.