LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (http://www.linuxquestions.org/questions/linux-software-2/)
-   -   how to delete duplicates entries in xml file using sed/awk/sort ? (http://www.linuxquestions.org/questions/linux-software-2/how-to-delete-duplicates-entries-in-xml-file-using-sed-awk-sort-377358/)

catzilla 10-27-2005 06:02 AM

how to delete duplicates entries in xml file using sed/awk/sort ?
 
I want to use sed/awk/sort (or any other unix commands) to use in shell script to delete duplicate entries (except the original one) from xml file like this example


from

######################################

<?xml version="1.0" encoding="ISO-8859-1"?><markers>

<marker
defacer="TroJaN"
system="Win 2003"
/>

<marker
defacer="TroJaN"
system="Win 2003"
/>

<marker
defacer="TroJaN"
system="Win 2003"
/>

<marker
defacer="TroJaN"
system="Linux"
/>

<marker
defacer="Team-evil"
system="Linux"
/>

</markers>

###################################

to

###################################

<?xml version="1.0" encoding="ISO-8859-1"?><markers>

<marker
defacer="TroJaN"
system="Win 2003"
/>

<marker
defacer="TroJaN"
system="Linux"
/>

<marker
defacer="Team-evil"
system="Linux"
/>

</markers>

####################################

any approach please ? thanks in advance

nilleso 10-28-2005 02:57 PM

Have you tried uniq?
Code:

NAME
    uniq - report or filter out repeated lines in a file

SYNOPSIS
    uniq [-c | -d |  -u]  [-f fields]  [-s char]  [  input_file
    [output_file]]

    uniq  [-c  |  -d  |  -u]  [-n]  [  +  m]  [  input_file
    [output_file]]

DESCRIPTION
    The uniq utility will read an input file comparing  adjacent
    lines,  and write one copy of each input line on the output.
    The second and succeeding copies of repeated adjacent  input
    lines will not be written.

cheers :)


All times are GMT -5. The time now is 11:52 AM.