Linux sed command

kml100 · 05-28-2004, 12:54 PM

Please Help, I am a student at Herzing college. We just started learning Red Hat Linux 7.0 (The version my teacher wants us to use.) I have been understanding everything up until this one command:

sed???

This is what we have to do:

#################################
# #
# #
# #
# #
# This is a test #
# #
#################################

We have to use sed to change the first column from # to *
Then change the last column from # to * using sed.

Please someone help. Thank You.

rkef · 05-28-2004, 01:18 PM

I don't want to just post an answer; what have you tried so far?

Are you at all familiar with sed(1), or regexes in general? Please give more info!

jim mcnamara · 05-28-2004, 01:19 PM

Code:

#  somefile has the text to change   
# this does the first part, you do the second part 
# pipe the first sed command to your new one then 
# redirect to a file
cat somefile | sed 's/^#/*/' > newfile
#

kml100 · 05-28-2004, 01:36 PM

I am not familiar with regexes, our teacher gave us a lab manual to work out of, we are learning most of the basic Linux text commands (cat, head, tr, ls, sed, etc.) It is a networking course, but we have to learn the basics first. The book we are using is: exam 101/102 LPIC 1 Certification Bible. I am on Chapter 4 P. 175. It gives a very vague description of the sed command. This is only our first week with Linux (Red Hat 7.0) We just finished learning text processing today (Friday May 28th, 2004) with the shell, we are not using the GUI at all. Thanks for your help, I hope this is enough info.

rkef · 05-28-2004, 03:27 PM

NOTE: This post may be longer than it needed to be, or not long enough; I can't be sure. Hopefully it is clear enough. I think if you read it through, it should all make perfect sense . But I make no apologies

.

The subject of regular expressions is large enough to fill a book, but I'll see if I can't give you the gist of it here.

Regular expressions are (sometimes complicated-looking) chunks of ascii characters. An example is the one which jim provided: 's/^#/*/'. The actual regular expression ("regex") in question here is the "^#" (the entire expression 's/^#/*/' is called a "substition"). Well, what are regexes for?

The reason regexes exist is to allow us to identify interesting portions of text within some larger block (or line) of text. There are a couple good reasons for wanting to "identify interesting portions of text"; we may want to pull interesting portions out, for use elsewhere or alter certain portions of text, in place. The text in question could be the manuscript for a book, an HTML page, or even the example you provided (8 or so lines of #'s, and some human english words).

Clearly your goal here is to alter certain portions, in place. You want to "change the first and last columns from # to *". Well, how do we do this? We need some way to identify the "first column" and the "last column" and then alter them somehow. Luckily, we have regexes. Regexes identify the portion of text, and programs like sed provide operations to alter, or otherwise use that text. The 's/regex/newtext' operation is one example, which sed provides (as do other programs: perl, awk, many more I can't think of atm).

Before we get to the regex, lets be clear on the syntax for the substition command:

Code:

s/regex/newtext/

The 's' is for "substitution", the regex is our regex, and the newtext is our new text.

So, I'm going to start with a simple example using grep (grep loves regexes):

Code:

$ echo "Not a speck of cereal" | grep "spalien acecraft"
$

Nothing happened! Why not? grep gobbled up our original text, compared it with our regex ("spalien acecraft") noted that it didn't match, and gave us the cold shoulder. Compare that with:

Code:

$ echo "Not a speck of cereal" | grep "t a sp"
Not a speck of cereal
$

Something happened! grep, being clever, noticed that "t a sp" exists in our original text: "Not a speck of cereal", so it spat the line back at us. What we've learned here is that, in a regex, normal characters match themselves (ie. an 'a' in our regex will match an 'a' in our original text, 'b' will match a 'b', etc.). That's simple enough.

Ok, play with that. It's pretty obvious (extremely useful though!). Now, what if we want to match something more complicated? We need magic. Magic is confusing. I can't teach you magic here! But I can explain the spell that Jim cast.

The '^' is magic; it matches "beginning of a line". So, by saying '^#', we're saying "identify any line which starts with '#'". Just to confuse things, here is another example of this:

Code:

grep "^#include" /usr/include/net/*.h

That will match all lines starting with "#include", in every header file in the "net" include directory. Massively interesting, I know...

So, getting back to our regex and our substition command to sed. 's/^#/*/'. Brilliant! Should be clear by now. What you need to do now, is find out how you can match the "end of a line". Now, depending on whether you want to match the end of each line, or just the first and last lines, you'll have to work some magic of your own. Think about the regex. What makes the first and last lines unique? How can you craft a spell to match only the endings of the first and last lines?

It takes a lot of trial and error. You may even think you've found an answer sometimes, but always carefully double-check it. You may be gobbling up more characters than you intended, or who knows what.

g/l

p.s. you can of course use multiple seds:

Code:

cat somefile |sed 's/regex/newtext/' |sed 's/regex/newtext/'

but a cleaner way (or not, depending on personal taste) is to use the -e option to sed. It works like this:

Code:

cat somefile |sed -e 's/regex/newtext/' -e 's/regex/newtext'

which allows multiple edits.