LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 04-07-2013, 10:05 AM   #1
davholla
Member
 
Registered: Jun 2003
Location: London
Distribution: Mandriva 2008 Spring
Posts: 652

Rep: Reputation: 31
Script to move duplicate files


Due to reasons too boring and embarrassing to relate I have lots of duplicate files. I found this script on line
Code:
find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate| cut -f3-100 -d ' ' | tr '\n.' '\t.' | sed 's/\t\t/\n/g' | cut -f2-100 | tr '\t' '\n' | perl -i -pe 's/([ (){}-])/\\$1/g' | perl -i -pe 's/'\''/\\'\''/g' | xargs -pr rm -v
rm -v ./test2.txt ?...y
However my natural caution means that I want to copy them t

I tried
Code:
find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate| cut -f3-100 -d ' ' | tr '\n.' '\t.' | sed 's/\t\t/\n/g' | cut -f2-100 | tr '\t' '\n' | perl -i -pe 's/([ (){}-])/\\$1/g' | perl -i -pe 's/'\''/\\'\''/g' | xargs -pr rm -v
rm -v ./test2.txt ?...y
but I got

Code:
mv: cannot overwrite non-directory `./test2.txt' with directory `duplicates/'


---------- Post added 04-07-13 at 04:05 PM ----------

Solved it I needed to put the full path to the duplicate directory!!!
 
Old 04-08-2013, 03:44 AM   #2
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 4,838

Rep: Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287
this is an extremely inefficient solution, there are tools to do it, for example: http://www.junauza.com/2010/01/how-t...duplicate.html
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Script to find the duplicate files Tekken Linux - Server 6 03-30-2013 11:29 AM
[SOLVED] Duplicate picutes - move and remane large number of files smithy2010 Linux - Newbie 4 02-15-2012 01:11 PM
find duplicate files using bash script? b-RAM Linux - Newbie 4 06-08-2010 07:05 AM
Script to remove duplicate jpg files ski_phreak Programming 1 05-26-2010 08:22 AM
Script to find duplicate files within one or more directories peter88 Linux - General 6 12-10-2006 05:17 AM


All times are GMT -5. The time now is 12:12 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration