Remove text from a file

jviola · 03-19-2007, 10:11 AM

How can I remove text from a file from a script?

Here is what I'm trying to do.

Using awk I locate the lines in the text file I want to remove, but I don't know how to actually remove it?

For Example: This a very short version of the actual file. Let's say the file is called test.txt and I want to pull out [fskioski] and the ports assigned ports=252-253. How do I update the file without these 2 lines from within my script?

[bptruck2]
ports=246-247

[bpretail1]
ports=248-249

[bpretail2]
ports=250-251

[fskiosk1]
ports=252-253

[fstruck1]
ports=254-255

[fstruck2]
ports=256-257

Thanks,

pixellany · 03-19-2007, 12:04 PM

SED is what you want for this.....

cat filename|sed '/string/{N;d;}' >newfilename

This reads "filename" and looks for "string"--when found, it deletes that line + the next and writes to "newfilename"

I just learned something new!!! The "N" command brings another line into the pattern space, and then "d" deletes the combo---neat!!

jviola · 03-19-2007, 12:40 PM

Hey that is great. However, when I run it, I get 2 blank lines instead of 1 between where the lines were deleted and the next one is. Is there a way to delete one more line?

thanks,

matthewg42 · 03-19-2007, 12:45 PM

These two produce the same output, the second saving the invocation of one cat process, but it's probably more important to use whichever is easier to understand than worrk about an extra cat (so choose according to your aesthetic preference):

Code:

cat filename|sed '/string/{N;d;}' >newfilename
sed '/string/{N;d}' filename > newfilename

If you don't want to re-direct into a new file and then rename it back to the original, you can use the -i (or --in-place) switch to sed will cause it to modify the files specified on the command line directly. You can also request that the original file is backed up with some suffix, provided as an argument to the -i/--in-place option. With the input you provided, this should fo what you want and backs up the old file to test.txt.original (works with GNU sed 4.1.5):

Code:

sed --in-place=.original '/^\[fskiosk1\]/,/^\[/ d' test.txt

pixellany · 03-19-2007, 01:13 PM

Quote:

Originally Posted by jviola

Hey that is great. However, when I run it, I get 2 blank lines instead of 1 between where the lines were deleted and the next one is. Is there a way to delete one more line?

thanks,

cat filename|sed '/string/{N;N;d;}' >newfilename

deletes a total of 3 lines including the one with the pattern match

cat filename|sed /^$/d >newfilename deletes all blank lines

To combine them:

cat filename| sed -e /string/{N;d;} -e /^$/d >newfilename

deletes two lines beginning with the match, and THEN deletes all blank lines

jschiwal · 03-19-2007, 01:19 PM

You can use a range for the pattern match:
sed -i '/fstruck1/,/^$/d' test.txt

This can be useful to extract a section from a command output such as lspci:

lspci -v | sed -n '/Broadcom/,^$/p'

jviola · 03-19-2007, 01:59 PM

Ok. I need one more thing. Can I substitute a variable in the string? If so how. I'm running a if then statement for a variable "user". It doesn't seem to work when I put in $user.

Thanks

ntubski · 03-19-2007, 02:04 PM

use "" instead of '' you'll probably need to escape any $'s that are part of your sed command.

matthewg42 · 03-19-2007, 02:14 PM

You should read up on the quoting rules in the manual page for your shell. It might seem like an abstract problem right now, but it will save you many hours of confusion in the long run.

jviola · 03-19-2007, 02:31 PM

Quote:

Originally Posted by ntubski

use "" instead of '' you'll probably need to escape any $'s that are part of you're sed command.

As you can already tell, I'm not a programmer. I'm not sure what that means ...need to escape any $'s that are part of you sed command.

It sounds like you are saying it is possible to use a variable?

Thanks,

ntubski · 03-19-2007, 03:17 PM

Quote:

Originally Posted by jviola

As you can already tell, I'm not a programmer. I'm not sure what that means ...need to escape any $'s that are part of you sed command.

It sounds like you are saying it is possible to use a variable?

Thanks,

Yes, for example:

Code:

 
cat filename|sed "/$USER/{N;N;d;}" >newfilename 
becomes (if USER=jviola)
cat filename|sed /jviola/{N;N;d;} >newfilename

As for escaping, you can see some of the suggestions in this thread include sed expressions with $ in them. It signifies end of line in regular expressions. If you use those ones inside "", you have to make sure to put \$ instead of $, or else the shell will try to substitute a variable for that part:

Code:

 sed -i "/$USER/,/^\$/d" test.txt
becomes 
sed -i /jviola/,/^$/d

You might want to read up on shell scripting:
http://www.linuxcommand.org/writing_shell_scripts.php
http://rute.2038bug.com/index.html.gz
are some good sources

cfaj · 03-19-2007, 04:02 PM

Quote:

Originally Posted by jviola

How can I remove text from a file from a script?

Here is what I'm trying to do.

Using awk I locate the lines in the text file I want to remove, but I don't know how to actually remove it?

For Example: This a very short version of the actual file. Let's say the file is called test.txt and I want to pull out [fskioski] and the ports assigned ports=252-253. How do I update the file without these 2 lines from within my script?

[bptruck2]
ports=246-247

[bpretail1]
ports=248-249

[bpretail2]
ports=250-251

[fskiosk1]
ports=252-253

[fstruck1]
ports=254-255

[fstruck2]
ports=256-257

Code:

lines_to_delete=2
string=[fskiosk1]

awk -v s="$string" -v ltd=$lines_to_delete '
 index($0,s) { ++n; next }
 n && ++n > ltd { n = 0 }
 n { next }
   {print}' "$FILE"

binutils · 03-19-2007, 09:21 PM

Hi,

How i can remove below line from file(configure) using sed?

Code:

{ (exit 1); exit 1; };

the line is pasted

TIA

PS: it solved,

Code:

sed -e 's/{ (exit 1); exit 1; };//g' test.txt > result.txt

matthewg42 · 03-20-2007, 12:51 AM

configure files are generated by autoconf - modifying them is usually asking for a world of pain.

kshkid · 03-20-2007, 05:22 AM

one more,

Code:

awk '{ if( match($0, "fskiosk1") ) { getline; getline } else { print } }' filename