LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 05-25-2010, 08:47 PM   #1
ski_phreak
LQ Newbie
 
Registered: May 2010
Location: Utah
Distribution: Debian 5
Posts: 1

Rep: Reputation: 0
Script to remove duplicate jpg files


Thanks y'all for the great script and explanation. This helped a lot in my own project. I thought I'd share the efforts.

The project is this: I've got lots of duplicate JPGs from all the family members who've named the same photo with different names. Since md5sum generates a "fingerprint" based on the file contents, not the name, I want to use the md5sum of each jpg to uniquely name each photo and also remove exact duplicates.

It has the following flaws:
0) it doesn't handle certain non-alphanumerics
1) it keeps both photo-shopped and unaltered photos (different md5s)
2) it (currently) doesn't preserve descriptive filenames.

(For me, removal of duplicates is more important than keeping the filenames. I may change that to concatenate the md5 and the filename.)

Please note that the commented "rename" command should be used to strip non-aphanumerics from the file names, and the script should be launched with the commented "find" command.

Code:
#!/bin/bash
### prepare files for renaming by removing alphanumerics
# rename -vf 's/[^a-zA-Z0-9]//g' *.jpg   

# then launch this script find . -type f -maxdepth 1 -name "*jpg" -exec ./md5rename.sh {} \;


if [ "$1x" != "x" ] ; then
  filename=$1
  new_filename=`(/usr/bin/md5sum "$filename") | cut -f 1 -d ' '`
  jpg_filename="${new_filename}.jpg"
  echo "mv ${filename} $new_filename"
  mv $filename $jpg_filename
fi

Last edited by pixellany; 05-26-2010 at 09:21 AM.
 
Old 05-26-2010, 09:22 AM   #2
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
Welcome to LQ!!

Please don't jump into 6-year old threads---I moved this into its own thread
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Script to copy specific directory based on variable to folder with that variable name fluxburn Programming 7 01-07-2010 08:59 PM
Problem with bash script - variable name within variable name steven.c.banks Linux - Newbie 3 03-10-2009 04:08 AM
Calling perl script and passing variable from php script hosea Programming 5 10-21-2008 09:01 AM
Help: removing a variable substring from a string variable in sh script gnparsons Programming 2 06-04-2008 06:21 PM
setting a variable variable in a script... this works, but could it be more elegant? pwc101 Programming 3 08-18-2006 12:23 PM


All times are GMT -5. The time now is 03:57 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration