Help answer threads with 0 replies.
Go Back > Forums > Linux Forums > Linux - General
User Name
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.


Search this Thread
Old 12-13-2010, 07:17 PM   #1
Registered: Sep 2010
Posts: 57

Rep: Reputation: 10
App to find and remove duplicate images?

Hi all,

I am looking for a Linux app that can find and remove duplicate images (with different filenames if that's at all possible).

Someone suggested fdupes but I'm interested in alternatives.

Old 12-13-2010, 08:40 PM   #2
Senior Member
Registered: Apr 2004
Location: Brisbane, Australia
Posts: 3,339
Blog Entries: 33

Rep: Reputation: 213Reputation: 213Reputation: 213
Smile fslint!

Hi, I use fslint, but it works by name and size,

Hope this helps, Regards Glenn
Attached Images
File Type: jpg snapshot1.jpg (159.9 KB, 10 views)
Old 12-13-2010, 10:03 PM   #3
Registered: Sep 2010
Posts: 57

Original Poster
Rep: Reputation: 10
Originally Posted by GlennsPref View Post
Hi, I use fslint, but it works by name and size,

Hope this helps, Regards Glenn
That looks like it'll do the job for some of the duplicates. Thanks! Does anyone have a tool that works via EXIF data?

Old 12-14-2010, 07:20 AM   #4
Registered: Jan 2010
Location: Lancashire
Distribution: Slackware Stable
Posts: 547

Rep: Reputation: 115Reputation: 115
Any reason why you can't use the find and diff commands?
Two nested loops using find (restrict to jpg files) then just do a diff on the two files in the inner-most loop. Make sure that they aren't the same file of course.
Whilst it may not be the most efficient algorithm, it uses tools already available to you.
Something like:

The other option may be that you create a hash of each file and then compare the hashes. I guess it depends on how big your files are and how many there are as to which you choose to do.
Old 12-14-2010, 07:28 AM   #5
LQ 5k Club
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,563
Blog Entries: 29

Rep: Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179Reputation: 1179
szboardstretcher described a versatile script to do things like that in this LQ post.
Old 12-14-2010, 07:34 AM   #6
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 655Reputation: 655Reputation: 655Reputation: 655Reputation: 655Reputation: 655
If the files themselves are identical, you could create an md5sum list, sort the list and use "uniq -w32 -D" to list the originals and duplicates.

find pictures/ -type f -iname "*.jpg" -exec md5sum '{}' \; >md5sumlist
sort md5sumlist >md5sumlist.s
uniq -w32 -D md5sumlist.s
1 members found this post helpful.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] How to remove duplicate files from Two Folders? guessity Linux - Newbie 19 09-09-2013 02:52 PM
imagemagick to find duplicate images leupi Linux - Software 1 11-18-2010 03:54 PM
Remove duplicate entries on a row sebelk Programming 2 11-01-2010 10:43 AM
[SOLVED] uniq -u : does not seem to remove duplicate lines boxb29 Linux - General 7 08-15-2009 07:34 PM
need app to remove duplicate files from the filesystem Ynot Irucrem Linux - Software 2 05-19-2005 01:35 PM

All times are GMT -5. The time now is 02:55 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration