Hi

Can any one help me for the following query.

I have one file which has the entry like as follows

Mar 24 09 abcd
Mar 24 09 abcd
Mar 24 11 pqrs
Mar 24 13 wxyz
Mar 24 13 wxyz
Mar 24 22 abcd
Mar 25 01 abcd
Mar 25 02 abcd
Mar 25 03 wxyz
Mar 25 06 wxyz
Mar 25 06 wxyz
Mar 25 07 pqrs
Mar 25 07 pqrs
Mar 25 07 pqrs
Mar 25 08 pqrs

My requirement is to find out identical entry and keep only one entry and remove other entries from this file.

for example The below are identical

Mar 24 09 abcd
Mar 24 09 abcd

i want to remove one entry from this and keep one entry

another set

Mar 25 07 pqrs
Mar 25 07 pqrs
Mar 25 07 pqrs

i want to remove two entry from this and keep one entry

So requirement is to remove duplicate entries from the file.
Is there any linux command to achive the same?

If the file is called file.txt do:

#> sort file.txt | uniq > file2.txt

file2.txt will only contain single instances of all the entries.

If the file is already sorted, uniq alone will work...

you could also use `sort -u`

`man sort` should help you out.

try:
sort -u oldfile > newfile

See