[SOLVED] remove duplicated stringd in a line...

masavini · 05-16-2012, 08:58 AM

this version works, but with a different behaviour with delimiters and multi-lines variables...

i mean:

Code:

$ echo -e "bubu gaga bubu\ndede dudu-dede" | awk '{ORS=RT}!_[$0]++' RS='[-\n ]'
bubu gaga dede dudu-

the output should be:
bubu gaga
dede dudu

i guess the first proposed solution is still the best one...

Quote:

Code:

awk 'BEGIN{FS = "[ -]"}{ while(++i<=NF) printf (!a[$i]++) ? $i " " : ""; i=split("",a); print "" }'

thank you all anyway!

grail · 05-16-2012, 11:08 AM

Well if you have it, ruby is probably cleaner:

Code:

echo -e "bubu gaga bubu-dede\ndede dudu-dede" | ruby -F"[- \n]" -ane 'puts $F.uniq.join(" ")'

masavini · 05-16-2012, 07:34 PM

and which command do you think it's faster? awk or ruby?
i use it thousands times every day...

ps: i just hope this question is not too stupid...

grail · 05-17-2012, 12:17 AM

No question is stupid when asked honestly

I would have to say that I am not 100% sure. In the trivial cases we have been using I would say whichever takes your fancy, which for me would be the ruby only due to less typing

If you have a decent amount of data to run them both on, simply pre-pend the time function prior to call and it will give you results, ie:

Code:

time ruby -F"[- \n]" -ane 'puts $F.uniq.join(" ")' file