LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-25-2008, 10:34 AM   #1
johnpaulodonnell
Member
 
Registered: Jun 2006
Location: Galway
Distribution: Suse 10.1
Posts: 114

Rep: Reputation: 15
awk command to remove repeats...


Hi.

I have a large data file, the first column of which looks something like:

5
9
9
13
14
20
20
20
23
24
26
27

These entries are just an index, not consecutive and some repeat. I'd like to remove the repeat entries, to get:

5
9
13
14
20
23
24
26
27

- but can't find an appropriate awk command. Any suggestions would be welcome.

Thanks.
 
Old 06-25-2008, 10:42 AM   #2
homey
Senior Member
 
Registered: Oct 2003
Posts: 3,057

Rep: Reputation: 59
how about uniq file.txt or sort -u file.txt
 
Old 06-25-2008, 10:43 AM   #3
johnpaulodonnell
Member
 
Registered: Jun 2006
Location: Galway
Distribution: Suse 10.1
Posts: 114

Original Poster
Rep: Reputation: 15
excellent.

thank you!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
awk - remove lines between AAAA and BBBB mangoo Programming 13 07-07-2011 01:55 AM
shell command using awk fields inside awk one71 Programming 6 06-26-2008 04:11 PM
awk to remove first 3 lines and print remaining $1, $2 fields phyx Linux - General 1 01-10-2007 05:21 PM
awk: remove similar lines from logfile peos Programming 7 06-19-2006 07:13 AM
When switched to command line, repeats "sr0: CD-ROM device not ready" over and over paradoxdruid Linux - General 7 10-21-2003 05:44 PM


All times are GMT -5. The time now is 06:40 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration