LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 03-19-2007, 11:11 AM   #1
jviola
Member
 
Registered: May 2004
Posts: 47

Rep: Reputation: 15
Remove text from a file


How can I remove text from a file from a script?

Here is what I'm trying to do.

Using awk I locate the lines in the text file I want to remove, but I don't know how to actually remove it?

For Example: This a very short version of the actual file. Let's say the file is called test.txt and I want to pull out [fskioski] and the ports assigned ports=252-253. How do I update the file without these 2 lines from within my script?

[bptruck2]
ports=246-247

[bpretail1]
ports=248-249

[bpretail2]
ports=250-251

[fskiosk1]
ports=252-253

[fstruck1]
ports=254-255

[fstruck2]
ports=256-257

Thanks,
 
Old 03-19-2007, 01:04 PM   #2
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738
SED is what you want for this.....

cat filename|sed '/string/{N;d;}' >newfilename

This reads "filename" and looks for "string"--when found, it deletes that line + the next and writes to "newfilename"

I just learned something new!!! The "N" command brings another line into the pattern space, and then "d" deletes the combo---neat!!
 
Old 03-19-2007, 01:40 PM   #3
jviola
Member
 
Registered: May 2004
Posts: 47

Original Poster
Rep: Reputation: 15
Super

Hey that is great. However, when I run it, I get 2 blank lines instead of 1 between where the lines were deleted and the next one is. Is there a way to delete one more line?

thanks,
 
Old 03-19-2007, 01:45 PM   #4
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
These two produce the same output, the second saving the invocation of one cat process, but it's probably more important to use whichever is easier to understand than worrk about an extra cat (so choose according to your aesthetic preference):
Code:
cat filename|sed '/string/{N;d;}' >newfilename
sed '/string/{N;d}' filename > newfilename
If you don't want to re-direct into a new file and then rename it back to the original, you can use the -i (or --in-place) switch to sed will cause it to modify the files specified on the command line directly. You can also request that the original file is backed up with some suffix, provided as an argument to the -i/--in-place option. With the input you provided, this should fo what you want and backs up the old file to test.txt.original (works with GNU sed 4.1.5):
Code:
sed --in-place=.original '/^\[fskiosk1\]/,/^\[/ d' test.txt
 
Old 03-19-2007, 02:13 PM   #5
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738Reputation: 738
Quote:
Originally Posted by jviola
Hey that is great. However, when I run it, I get 2 blank lines instead of 1 between where the lines were deleted and the next one is. Is there a way to delete one more line?

thanks,
cat filename|sed '/string/{N;N;d;}' >newfilename

deletes a total of 3 lines including the one with the pattern match

cat filename|sed /^$/d >newfilename deletes all blank lines

To combine them:

cat filename| sed -e /string/{N;d;} -e /^$/d >newfilename

deletes two lines beginning with the match, and THEN deletes all blank lines
 
Old 03-19-2007, 02:19 PM   #6
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677
You can use a range for the pattern match:
sed -i '/fstruck1/,/^$/d' test.txt

This can be useful to extract a section from a command output such as lspci:

lspci -v | sed -n '/Broadcom/,^$/p'
 
Old 03-19-2007, 02:59 PM   #7
jviola
Member
 
Registered: May 2004
Posts: 47

Original Poster
Rep: Reputation: 15
Getting there

Ok. I need one more thing. Can I substitute a variable in the string? If so how. I'm running a if then statement for a variable "user". It doesn't seem to work when I put in $user.

Thanks
 
Old 03-19-2007, 03:04 PM   #8
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian, Arch
Posts: 3,255

Rep: Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420
use "" instead of '' you'll probably need to escape any $'s that are part of your sed command.

Last edited by ntubski; 03-19-2007 at 04:03 PM.
 
Old 03-19-2007, 03:14 PM   #9
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
You should read up on the quoting rules in the manual page for your shell. It might seem like an abstract problem right now, but it will save you many hours of confusion in the long run.
 
Old 03-19-2007, 03:31 PM   #10
jviola
Member
 
Registered: May 2004
Posts: 47

Original Poster
Rep: Reputation: 15
Ok, as you can already tell.....

Quote:
Originally Posted by ntubski
use "" instead of '' you'll probably need to escape any $'s that are part of you're sed command.

As you can already tell, I'm not a programmer. I'm not sure what that means ...need to escape any $'s that are part of you sed command.


It sounds like you are saying it is possible to use a variable?

Thanks,
 
Old 03-19-2007, 04:17 PM   #11
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian, Arch
Posts: 3,255

Rep: Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420Reputation: 1420
Quote:
Originally Posted by jviola
As you can already tell, I'm not a programmer. I'm not sure what that means ...need to escape any $'s that are part of you sed command.


It sounds like you are saying it is possible to use a variable?

Thanks,
Yes, for example:

Code:
 
cat filename|sed "/$USER/{N;N;d;}" >newfilename 
becomes (if USER=jviola)
cat filename|sed /jviola/{N;N;d;} >newfilename
As for escaping, you can see some of the suggestions in this thread include sed expressions with $ in them. It signifies end of line in regular expressions. If you use those ones inside "", you have to make sure to put \$ instead of $, or else the shell will try to substitute a variable for that part:

Code:
 sed -i "/$USER/,/^\$/d" test.txt
becomes 
sed -i /jviola/,/^$/d
You might want to read up on shell scripting:
http://www.linuxcommand.org/writing_shell_scripts.php
http://rute.2038bug.com/index.html.gz
are some good sources
 
Old 03-19-2007, 05:02 PM   #12
cfaj
Member
 
Registered: Dec 2003
Location: Toronto, Canada
Distribution: Mint, Mandriva
Posts: 221

Rep: Reputation: 31
Quote:
Originally Posted by jviola
How can I remove text from a file from a script?

Here is what I'm trying to do.

Using awk I locate the lines in the text file I want to remove, but I don't know how to actually remove it?

For Example: This a very short version of the actual file. Let's say the file is called test.txt and I want to pull out [fskioski] and the ports assigned ports=252-253. How do I update the file without these 2 lines from within my script?

[bptruck2]
ports=246-247

[bpretail1]
ports=248-249

[bpretail2]
ports=250-251

[fskiosk1]
ports=252-253

[fstruck1]
ports=254-255

[fstruck2]
ports=256-257
Code:
lines_to_delete=2
string=[fskiosk1]

awk -v s="$string" -v ltd=$lines_to_delete '
 index($0,s) { ++n; next }
 n && ++n > ltd { n = 0 }
 n { next }
   {print}' "$FILE"

Last edited by cfaj; 03-19-2007 at 05:38 PM.
 
Old 03-19-2007, 10:21 PM   #13
binutils
Member
 
Registered: Feb 2007
Posts: 59

Rep: Reputation: 15
Hi,

How i can remove below line from file(configure) using sed?
Code:
{ (exit 1); exit 1; };
the line is pasted

TIA

PS: it solved,
Code:
sed -e 's/{ (exit 1); exit 1; };//g' test.txt > result.txt

Last edited by binutils; 03-20-2007 at 12:56 AM.
 
Old 03-20-2007, 01:51 AM   #14
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
configure files are generated by autoconf - modifying them is usually asking for a world of pain.
 
Old 03-20-2007, 06:22 AM   #15
kshkid
Member
 
Registered: Dec 2005
Distribution: RHEL3, FC3
Posts: 383

Rep: Reputation: 30
one more,

Code:
awk '{ if( match($0, "fskiosk1") ) { getline; getline } else { print } }' filename
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how to remove lf from a text file gfem Linux - Software 3 10-19-2006 07:41 PM
remove \r and \n from a text file powah Programming 9 10-02-2006 07:02 AM
remove some text from a file hfawzy Programming 7 07-10-2006 10:22 PM
remove text from file with script paul_mat Linux - Software 3 11-17-2005 01:21 PM
How to remove line of text from file netkepala Linux - General 2 05-23-2003 12:49 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 10:56 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration