LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 05-25-2010, 07:47 PM   #1
ski_phreak
LQ Newbie
 
Registered: May 2010
Location: Utah
Distribution: Debian 5
Posts: 1

Rep: Reputation: 0
Script to remove duplicate jpg files


Thanks y'all for the great script and explanation. This helped a lot in my own project. I thought I'd share the efforts.

The project is this: I've got lots of duplicate JPGs from all the family members who've named the same photo with different names. Since md5sum generates a "fingerprint" based on the file contents, not the name, I want to use the md5sum of each jpg to uniquely name each photo and also remove exact duplicates.

It has the following flaws:
0) it doesn't handle certain non-alphanumerics
1) it keeps both photo-shopped and unaltered photos (different md5s)
2) it (currently) doesn't preserve descriptive filenames.

(For me, removal of duplicates is more important than keeping the filenames. I may change that to concatenate the md5 and the filename.)

Please note that the commented "rename" command should be used to strip non-aphanumerics from the file names, and the script should be launched with the commented "find" command.

Code:
#!/bin/bash
### prepare files for renaming by removing alphanumerics
# rename -vf 's/[^a-zA-Z0-9]//g' *.jpg   

# then launch this script find . -type f -maxdepth 1 -name "*jpg" -exec ./md5rename.sh {} \;


if [ "$1x" != "x" ] ; then
  filename=$1
  new_filename=`(/usr/bin/md5sum "$filename") | cut -f 1 -d ' '`
  jpg_filename="${new_filename}.jpg"
  echo "mv ${filename} $new_filename"
  mv $filename $jpg_filename
fi

Last edited by pixellany; 05-26-2010 at 08:21 AM.
 
Old 05-26-2010, 08:22 AM   #2
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Mint
Posts: 17,809

Rep: Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743
Welcome to LQ!!

Please don't jump into 6-year old threads---I moved this into its own thread
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Script to copy specific directory based on variable to folder with that variable name fluxburn Programming 7 01-07-2010 07:59 PM
Problem with bash script - variable name within variable name steven.c.banks Linux - Newbie 3 03-10-2009 03:08 AM
Calling perl script and passing variable from php script hosea Programming 5 10-21-2008 08:01 AM
Help: removing a variable substring from a string variable in sh script gnparsons Programming 2 06-04-2008 05:21 PM
setting a variable variable in a script... this works, but could it be more elegant? pwc101 Programming 3 08-18-2006 11:23 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 09:21 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration