LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Closed Thread
  Search this Thread
Old 03-10-2011, 05:49 PM   #1
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,879

Rep: Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660
unique command - variation on the theme


I have a large file and want to keep certain lines based on the first word in each. The words are blank-delimited character strings.

The first words are not necessarily unique.
I want to keep the uniques, and that's easy enough.
Where duplicates are encountered I want to keep only one
of each, and the kept line must be the *last* of the duplicates.

Sample input file:
ALBERT 54
BENJAMIN 37
BILL 24
BILL 25
BILL 77
CARL 40
CARL 44
CHESTER 59
DAVID 23
DAVID 23
DAVID 28
DAVID 61
EDGAR 33
EDWARD 54
EDWARD 59
EDWIN 30

Desired output file:
ALBERT 54
BENJAMIN 37
BILL 77
CARL 44
CHESTER 59
DAVID 61
EDGAR 33
EDWARD 59
EDWIN 30

I'm a newbie and still learning the basics, so please:
- no awk
- no bash
- no Perl

Let's stick to commands such as uniq, sort, sed, grep, cut, paste, etc.
 
Old 03-10-2011, 08:32 PM   #2
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,879

Original Poster
Rep: Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660Reputation: 660
Quote:
Originally Posted by danielbmartin View Post
I have a large file and want to keep certain lines based on the first word in each. The words are blank-delimited character strings.

The first words are not necessarily unique.
I want to keep the uniques, and that's easy enough.
Where duplicates are encountered I want to keep only one
of each, and the kept line must be the *last* of the duplicates.

Sample input file:
ALBERT 54
BENJAMIN 37
BILL 24
BILL 25
BILL 77
CARL 40
CARL 44
CHESTER 59
DAVID 23
DAVID 23
DAVID 28
DAVID 61
EDGAR 33
EDWARD 54
EDWARD 59
EDWIN 30

Desired output file:
ALBERT 54
BENJAMIN 37
BILL 77
CARL 44
CHESTER 59
DAVID 61
EDGAR 33
EDWARD 59
EDWIN 30

I'm a newbie and still learning the basics, so please:
- no awk
- no bash
- no Perl

Let's stick to commands such as uniq, sort, sed, grep, cut, paste, etc.
Got it!

tac < {InFile} | sort -k1,1 -t' ' -u
 
Old 03-11-2011, 09:07 AM   #3
hda7
Member
 
Registered: May 2009
Distribution: Debian wheezy
Posts: 252

Rep: Reputation: 31
Quote:
Originally Posted by danielbmartin View Post
Got it!

tac < {InFile} | sort -k1,1 -t' ' -u
You may want to add "-s" to your sort command. If I am correct, this ensures that lines that compare equal are never swapped in the sorting process.
 
Old 03-12-2011, 10:27 AM   #4
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Please post your thread in only one forum. Posting a single thread in the most relevant forum will make it easier for members to help you and will keep the discussion in one place. This thread is being closed because it is a duplicate.
 
  


Closed Thread


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Atolm GTK Theme + Clarity Icon Theme = Perfect for Dark Theme Fans LXer Syndicated Linux News 0 02-26-2011 11:50 PM
openoffice force default theme / no theme | dark theme fix H_TeXMeX_H LinuxQuestions.org Member Success Stories 1 03-08-2010 12:56 PM
Scripts Variation shipon_97 Linux - Newbie 2 03-14-2007 12:10 PM
Post Your Noteworthy Or Unique Linux Command Line Hints Here AwesomeMachine Linux - General 1 10-24-2005 01:33 AM
copying files and give new unique names to each file by using xargs command gnim66 Programming 6 06-22-2005 08:29 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 02:36 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration