LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-01-2011, 10:11 PM   #1
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,484

Rep: Reputation: 411Reputation: 411Reputation: 411Reputation: 411Reputation: 411
Eliminate singletons?


How does one eliminate singletons?

I have:
Mary
Jane
Jane
Doris
Linda
Linda
Linda
Janice

uniq -d gives this:
Jane
Linda

But I want:
Jane
Jane
Linda
Linda
Linda

Please advise.
 
Old 12-01-2011, 10:26 PM   #2
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Here's one way to do it:

Code:
grep -f <( uniq -d file.txt ) file.txt
The process substitution generates the list of non-unique names, which is then used by grep as if it were a file, to print out all the matches of those names.

BTW, I hope you know that uniq will only work reliably on a file that's already sorted.
 
Old 12-04-2011, 01:07 PM   #3
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,484

Original Poster
Rep: Reputation: 411Reputation: 411Reputation: 411Reputation: 411Reputation: 411
Thank you, David the H, for the prompt response.

You are right, of course, about sorting the file
before using uniq.

My original post was made with the idea that a clever option
of the uniq command would do the trick.
Apparently there is no such thing.

Your combination of uniq and grep is a promising method.
However I simplified (or oversimplified) the task,
hoping to state it succinctly.

I have a sorted file:
Doris Fletcher
Jane Baker
Jane Simmons
Janice Taylor
Linda Archer
Linda Brown
Linda Green
Mary Carter

I want:
Jane Baker
Jane Simmons
Linda Archer
Linda Brown
Linda Green

I am unable to generalize your solution.
Please help.
 
Old 12-04-2011, 01:24 PM   #4
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
Another solution comparing only the first field:
Code:
rev file | uniq -f1 -D | rev
Since there is not an option to compare only the first N fields, but only to skip them, here is explained the usage of the rev command.
 
Old 12-04-2011, 02:24 PM   #5
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,484

Original Poster
Rep: Reputation: 411Reputation: 411Reputation: 411Reputation: 411Reputation: 411
Thank you, colucix, for the quick answer.
It works perfectly and the execution time is sub-second,
even for a large file.

Mark this one Solved!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to eliminate DHCP etc. johnsfine Linux - Networking 2 08-12-2008 11:02 AM
Eliminate NSAlinux in Fedora 8? PatM Linux - Security 6 04-08-2008 12:39 PM
eliminate the necessity of UPS snehitaasai Linux - Newbie 3 11-03-2007 01:09 AM
Eliminate touchpad aeruzcar Linux - Hardware 2 06-15-2006 03:12 PM
How to eliminate the spyware ? emailssent General 32 11-01-2004 10:49 PM


All times are GMT -5. The time now is 12:15 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration