Latest LQ Deal: Linux Power User Bundle
Go Back > Forums > Linux Forums > Linux - General
User Name
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.


  Search this Thread
Old 12-13-2010, 06:17 PM   #1
Registered: Sep 2010
Posts: 57

Rep: Reputation: 10
App to find and remove duplicate images?

Hi all,

I am looking for a Linux app that can find and remove duplicate images (with different filenames if that's at all possible).

Someone suggested fdupes but I'm interested in alternatives.

Old 12-13-2010, 07:40 PM   #2
Senior Member
Registered: Apr 2004
Location: Brisbane, Australia
Distribution: pclinuxos slackware64 tails kali
Posts: 3,372
Blog Entries: 33

Rep: Reputation: 217Reputation: 217Reputation: 217
Smile fslint!

Hi, I use fslint, but it works by name and size,

Hope this helps, Regards Glenn
Attached Thumbnails
Click image for larger version

Name:	snapshot1.jpg
Views:	33
Size:	159.9 KB
ID:	5516  
Old 12-13-2010, 09:03 PM   #3
Registered: Sep 2010
Posts: 57

Original Poster
Rep: Reputation: 10
Originally Posted by GlennsPref View Post
Hi, I use fslint, but it works by name and size,

Hope this helps, Regards Glenn
That looks like it'll do the job for some of the duplicates. Thanks! Does anyone have a tool that works via EXIF data?

Old 12-14-2010, 06:20 AM   #4
Registered: Jan 2010
Location: Lancashire
Distribution: Slackware Stable
Posts: 553

Rep: Reputation: 116Reputation: 116
Any reason why you can't use the find and diff commands?
Two nested loops using find (restrict to jpg files) then just do a diff on the two files in the inner-most loop. Make sure that they aren't the same file of course.
Whilst it may not be the most efficient algorithm, it uses tools already available to you.
Something like:

The other option may be that you create a hash of each file and then compare the hashes. I guess it depends on how big your files are and how many there are as to which you choose to do.
Old 12-14-2010, 06:28 AM   #5
LQ 5k Club
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,576
Blog Entries: 31

Rep: Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195
szboardstretcher described a versatile script to do things like that in this LQ post.
Old 12-14-2010, 06:34 AM   #6
LQ Guru
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671
If the files themselves are identical, you could create an md5sum list, sort the list and use "uniq -w32 -D" to list the originals and duplicates.

find pictures/ -type f -iname "*.jpg" -exec md5sum '{}' \; >md5sumlist
sort md5sumlist >md5sumlist.s
uniq -w32 -D md5sumlist.s
1 members found this post helpful.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] How to remove duplicate files from Two Folders? guessity Linux - Newbie 19 09-09-2013 01:52 PM
imagemagick to find duplicate images leupi Linux - Software 1 11-18-2010 02:54 PM
Remove duplicate entries on a row sebelk Programming 2 11-01-2010 09:43 AM
[SOLVED] uniq -u : does not seem to remove duplicate lines boxb29 Linux - General 7 08-15-2009 06:34 PM
need app to remove duplicate files from the filesystem Ynot Irucrem Linux - Software 2 05-19-2005 12:35 PM > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 04:14 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration