That's brilliant thank you. I wondered about incorporating
cat set*.mp3 | sort | uniq
md5sum * | sort | uniq -d -w32 | cut -d' ' -f3 | xargs rm
somehow to try and filter uniques but not sure how to implement. Could be great for hunting many things.
Last time I tried to use md5sum to find unique files it became slow and gave an error. Obviously large files would be hell for md5sum but good for .txt
Then the uniq command should do this but I'm rusty on how unique this gets. Thanks for your notes above.