How to Uniquify w/o sorting??

adroit_91 · 01-31-2008, 11:37 PM

Hello everyone.

I need to uniquify a file which contains only strings, but not in a sorted order. What I basically mean to achieve is that without touching the order of the contents, I should be able to do away with repetitions! Although i can easily write a PERL script for the same, I want to achieve the same through the linux prompt.

Eg:
A file containing the following:

Zeta
Apple
Bear
Gum
Apple
Gum
Tiger
Beargum

Should become:

Zeta
Apple
Bear
Gum
Tiger
Beargum

Is it possible through the command prompt?

fargie_s · 02-01-2008, 12:27 AM

Hello,

I think you should just use sort :

Code:

$ echo 'Zeta
Apple
Bear
Gum
Apple
Gum
Tiger
Beargum' | sort -u

adroit_91 · 02-01-2008, 12:37 AM

Sort -u will uniquify as well as sort, although, I do not want it sorted at all. I want simply the output uniquified, but the remaining content's order should be untouched!

Thanks anyway.

yongitz · 02-01-2008, 03:03 AM

On a command line.. you can invoke perl one-line command..

try this: perl -ane 'print unless $seen{$F[0]}++' testfile

adroit_91 · 02-01-2008, 03:39 AM

Thanks a lot!! Works like a charm... [

]