LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-08-2011, 07:32 AM   #1
ammu
LQ Newbie
 
Registered: Dec 2011
Posts: 3

Rep: Reputation: Disabled
sort the files and delete the duplicate lines that contain a pattern


Hi,

I need to sort a file and delete the duplicate lines that contains the particular pattern.

for ex:

sort output is:

hi
hi
hi
how r u
how r u
how r u

i'm searching for "hi" i need to delete that duplicate alone not how r u

output need to be

hi
how r u
how r u
how r u

thanks...
 
Old 12-08-2011, 08:39 AM   #2
klearview
Member
 
Registered: Aug 2006
Location: London
Distribution: Debian, Kubuntu
Posts: 572

Rep: Reputation: 75
See if this works for you:

Quote:
sort your_file_name | grep -C 1 -v 'hi'
 
Old 12-08-2011, 10:57 PM   #3
ammu
LQ Newbie
 
Registered: Dec 2011
Posts: 3

Original Poster
Rep: Reputation: Disabled
sorry

It's not working. I need to delete the all the duplicate which contains my search pattern

thanks...
 
Old 12-09-2011, 01:54 AM   #4
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
How exactly is it "not working"? What output do you get?
But yeah, I don't think it works as promised.


This awk command should work. It sets a flag after the first match so that subsequent lines with the same pattern won't print:
Code:
awk '( no && $0 ~ pat ) { next } ; $0 ~ pat { no=1 } ; { print }' "pat=hi" <( sort file.txt )
The first argument fed to awk is a variable setting the pattern you want to match ("pat=hi"). You have to quote the whole thing so that it's not treated as a shell variable. The second argument is a process substitution that supplies the pre-sorted file.

Edit: I just realized...the input pattern is treated as a regular expression, so the above will match any line containing that substring. To force exact line matches only, use "pat=^hi$" (or another regex that targets only what you want).


Finally, please use [code][/code] tags around your code and data, to preserve formatting and to improve readability.

Last edited by David the H.; 12-09-2011 at 02:00 AM. Reason: as stated
 
Old 12-09-2011, 02:11 AM   #5
ammu
LQ Newbie
 
Registered: Dec 2011
Posts: 3

Original Poster
Rep: Reputation: Disabled
thank you

hi,

thank you so much... it's working..
thanks once again
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
delete duplicate lines from a file which has one field different jkeertir Linux - Newbie 2 02-15-2011 01:45 AM
How to use sed to delete all lines before the first match of a pattern? C_Blade Linux - Newbie 9 05-01-2010 05:18 AM
[SOLVED] sed: Find pattern and delete 5 lines after it supersoni3 Programming 4 03-24-2010 08:00 AM
Delete Duplicate Lines in a file, leaving only the unique lines left xmrkite Linux - Software 6 01-14-2010 07:18 PM
how do u delete duplicate lines bharatbsharma Programming 4 10-29-2007 07:04 PM


All times are GMT -5. The time now is 12:31 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration