App to find and remove duplicate images?
 

Hi all,

I am looking for a Linux app that can find and remove duplicate images (with different filenames if that's at all possible).

Someone suggested fdupes but I'm interested in alternatives.

Thanks.

fslint!
 

Hi, I use fslint, but it works by name and size,

Hope this helps, Regards Glenn

That looks like it'll do the job for some of the duplicates. Thanks! Does anyone have a tool that works via EXIF data?

Cheers.

Any reason why you can't use the find and diff commands?
Two nested loops using find (restrict to jpg files) then just do a diff on the two files in the inner-most loop. Make sure that they aren't the same file of course.
Whilst it may not be the most efficient algorithm, it uses tools already available to you.
Something like:

The other option may be that you create a hash of each file and then compare the hashes. I guess it depends on how big your files are and how many there are as to which you choose to do.

szboardstretcher described a versatile script to do things like that in this LQ post.

If the files themselves are identical, you could create an md5sum list, sort the list and use "uniq -w32 -D" to list the originals and duplicates.

find pictures/ -type f -iname "*.jpg" -exec md5sum '{}' \; >md5sumlist
sort md5sumlist >md5sumlist.s
uniq -w32 -D md5sumlist.s