LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (http://www.linuxquestions.org/questions/linux-software-2/)
-   -   Delete Duplicate Lines in a file, leaving only the unique lines left (http://www.linuxquestions.org/questions/linux-software-2/delete-duplicate-lines-in-a-file-leaving-only-the-unique-lines-left-782245/)

xmrkite 01-14-2010 04:13 PM

Delete Duplicate Lines in a file, leaving only the unique lines left
 
Hello. i've seen several simple commands used to delete duplicate lines from a file, but I've yet to see a command that can leave a file with only the unique lines.

So, for example, here is the file:

1
2
3
1
2
3
4
5
6

I want to have it so that after the command is run on the file, it looks like this:

4
5
6

Any ideas?

-Thanks for the help.

GrapefruiTgirl 01-14-2010 04:16 PM

try the `uniq` command :)

Sasha

xmrkite 01-14-2010 04:22 PM

how does that delete the lines that are 1 2 or 3? I think using unique i can get a file that reads:

1
2
3
4
5
6

but not one that reads only

4
5
6

Am i wrong?
--Thanks

GrapefruiTgirl 01-14-2010 04:25 PM

Check the man page; observe the -u option, to print only unique lines.

Sasha

xmrkite 01-14-2010 04:28 PM

Perfect.

Here is the command i use: cat filename | sort | uniq -u

The issue i had is that uniq can do it, but only if the file is first sorted.

-Thanks for the quick help.

chrism01 01-14-2010 06:59 PM

UUOC (useless use of cat), use

sort file|uniq -u

xmrkite 01-14-2010 07:18 PM

Even Better. Thanks.


All times are GMT -5. The time now is 06:38 AM.