LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 01-27-2010, 12:35 AM   #1
Black Sun
LQ Newbie
 
Registered: Jan 2010
Posts: 3

Rep: Reputation: 0
Delete line in file matching string


Quote:
Originally Posted by topcat View Post
Hello,
I would like to know how i can write a shell script to delete a line if a particular pattern exists?

E.g.

I have a text file with multiple lines. Say 1000s. in the following pattern.

username@email.com:149.0.3.4:1
username1@email.com:149.0.3.4:1
username1@email.net:149.0.3.4:1
username1@email.edu:149.0.3.4:1


If the pattern
username@email.com exists then the line "username@email.com:149.0.3.4:1 should be deleted from the file.

thanks!
I have a very similar question but I need to delete one line in a file which matches one very precise instance of a string only.

Let's assume I have a file composed of thousands of lines and let's call the file chap-secrets. Let's take the following sample entries:

Code:
#USERNAME      SERVER         PASSWORD         IP
pp          pptpd             blahblah         *
cpp              pptpd             ppbbbbpp         *
hopp           pptpd             ggmmsasm         *
flopps    pptpd     01234567890abcdef    *
I would now like to delete the line containing the username "pp" only. The following suggestion has been made in a related question in this thread:

Code:
awk 'match($0,"pp") == 0 {print $0}' FILENAME > NEWFILENAME
However, this doesn't work here, for it would delete (or rather not store) all these lines, because all lines with the usernames "cpp", "hopp" and "flopps" are valid matches, all lines with the server name "pptpd" are valid matches and all lines where the password contains a "pp" (such as "ppbbbbpp") are valid matches, too, and these lines all would be deleted (or rather not stored) as well.

Note that the number of spaces between the fields differ as well all the time, so I can't search for a pattern like

Code:
"pp          pptpd"
either.

I suppose this problem is very simple for some of you, but for me it's not, thus every help would be very much appreciated.

Thank you!

Last edited by Black Sun; 01-27-2010 at 12:36 AM. Reason: Spelling.
 
Old 01-27-2010, 12:42 AM   #2
GrapefruiTgirl
Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550
Hi Black Sun, welcome to LQ!

You have resurrected a LONG dead zombie thread here and I have reported it for a possible move onto its own fresh thread instead.

Meanwhile though, assuming the username is always the first entry, and is always followed by some spaces, you could do this:

Code:
sed -i '/^pp .*/ d' chap-secrets
Sasha
 
1 members found this post helpful.
Old 01-27-2010, 01:11 AM   #3
Black Sun
LQ Newbie
 
Registered: Jan 2010
Posts: 3

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by GrapefruiTgirl View Post
Hi Black Sun, welcome to LQ!

You have resurrected a LONG dead zombie thread here and I have reported it for a possible move onto its own fresh thread instead.

Meanwhile though, assuming the username is always the first entry, and is always followed by some spaces, you could do this:

Code:
sed -i '/^pp .*/ d' chap-secrets
Sasha
Thanks for the warm welcome, Sasha!

Yes, I noticed that it's a very old thread, but old or not, the question seemed to fit to the topic, and Google seems to love pages which are stable and exist for a long time. mistake? Sorry in this case.

Your solution works like a charm, by the way.

Thank you very much -- I wouldn't have expected to get an answer so soon.

I still have one related question, maybe you have a solution to this one as well. As you might notice, I try to get the deletion of users automated, because using a text editor all the time to go through half a dozen of files is a real hassle.

My other file, let's call it users.allowed looks like this:

Code:
allow pp
allow ppv
allow cpp
allow hopp
allow flopps
The usernames are unique again, there is no space after the usernames, and always exactly one space between an "allow" and a username. Again, I would like to get the line containing "allow pp" deleted without lynching the other users who happen to have a "pp" in their names, too.

(That's the last file, I promise.)

EDIT: Actually, I think

Code:
awk 'match($0,"allow pp") == 0 {print $0}' FILENAME > NEWFILENAME
should just work fine in this case? Maybe there is a more elegant solution, though? I'll give it a shot, anyway.

2nd EDIT: No, it doesn't work. I had tried this before, doh! It also deletes the line "allow ppv" that's why I had to dump it ... and because there is no space after the usernames I couldn't use "allow pp " (with a space after the username) either.

So I'd still be looking for a solution.

Thank you!

Last edited by Black Sun; 01-27-2010 at 01:50 AM. Reason: Correcting a solution which doesn't work.
 
Old 01-27-2010, 08:45 AM   #4
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 655Reputation: 655Reputation: 655Reputation: 655Reputation: 655Reputation: 655
You can use sed as well to extract information:
sed '/allow pp$/w newfilename' filename

or

sed -n '/allow pp$/p' filename >newfilename

You could even use grep for this one as well.
grep '^allow pp$' filename >newfilename
 
1 members found this post helpful.
Old 01-27-2010, 09:11 AM   #5
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Rep: Reputation: 714Reputation: 714Reputation: 714Reputation: 714Reputation: 714Reputation: 714Reputation: 714
Why not just use grep?

Code:
cat filename | grep -v 'username@email'
 
1 members found this post helpful.
Old 01-27-2010, 10:56 AM   #6
Black Sun
LQ Newbie
 
Registered: Jan 2010
Posts: 3

Original Poster
Rep: Reputation: 0
Hello, jschiwal!

Thanks for your proposal. Based on the sed commands Sasha and you gave me, I rewrote yours and both

Code:
sed -i '/allow pp$/ d' users.allowed
and

Code:
sed -i '/allow pp$/d' users.allowed
seem to be exactly doing what I want: namely to delete the entry called "allow pp" from the file but not entries like "allow ppv" or "allow ppreacher", etc. So I suppose the "$" sign has the function of a "stop", "EOL" or "line must match exactly" mark?

But what's about the (optional?) space character in the two commands I pasted above: "/ d" vs. "/d"? Are the two commands above functionally completely identical or has the space some meaning or function as well?

Quote:
Originally Posted by MTK358 View Post
Why not just use grep?

Code:
cat filename | grep -v 'username@email'
Cuz' I'm really a total newbie when it comes to awk, sed, grep & co.

And I believe that

Code:
cat users.allowed | grep -v 'allow pp' > NEWFILENAME
would have the same shortcoming as

Code:
awk 'match($0,"allow pp") == 0 {print $0}' users.allowed > NEWFILENAME
in that it would delete not only the line "allow pp" but entries like "allow pp2" or "allow ppreacher" as well?
 
Old 01-27-2010, 11:13 AM   #7
GrapefruiTgirl
Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550
I cannot speak about the AWK code at the end of your above post, since I'd need to play with it to see precisely what it's doing. I can say though, that while awk is fabulous, sometimes (like in this case) there are semantically simpler solutions (such as sed).

The space before the d in "/d" or "/ d" is not significant. The regex goes inside the /slashes/ and the "d" is the command for "delete".

the $ sign signifies the end of the line. a "^" carat sign means the beginning of a line.

With grep, the -v switch means "invert the match", so it greps for everything EXCEPT the regex (it gives you everything EXCEPT the item you use -v on) You are correct: grep -v "allow pp" would also toss out "allow pppreacher" so you would need to fine tune the regex to match for ONLY "allow pp". It isn't a shortcoming, it's just a result of not using the correct regex.

To learn more about all this, you really ought to check `man 7 regex` as well as the excellent online tutorials on regular expressions, sed, awk, and grep

Best regards,
Sasha
 
Old 01-28-2010, 04:18 AM   #8
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Quote:
Originally Posted by Black Sun View Post
I would now like to delete the line containing the username "pp" only. The following suggestion has been made in a related question in this thread:
if you want to match exactly, use == or !=. since username is column 1, then use $1 instead of $0
Code:
$ awk 'NR>1&&$1!="pp"' file
cpp              pptpd             ppbbbbpp         *
hopp           pptpd             ggmmsasm         *
flopps    pptpd     01234567890abcdef    *
similar for your allow file

Code:
# awk '$1=="allow" && $NF!="pp"' file
allow ppv
allow cpp
allow hopp
allow flopps

Last edited by ghostdog74; 01-28-2010 at 04:21 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Perl question: delete line from text file with duplicate match at beginning of line mrealty Programming 7 04-01-2009 07:46 PM
SED - Delete line above or below as well as matching line... OldGaf Programming 7 06-27-2008 12:51 AM
C++ text file line by line/each line to string/array Dimitris Programming 15 03-11-2008 09:22 AM
delete a line containing a pattern and the next line of a text file powah Programming 3 01-31-2007 06:34 PM


All times are GMT -5. The time now is 06:13 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration