LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices



Reply
 
Search this Thread
Old 04-07-2013, 11:05 AM   #1
davholla
Member
 
Registered: Jun 2003
Location: London
Distribution: Mandriva 2008 Spring
Posts: 652

Rep: Reputation: 31
Script to move duplicate files


Due to reasons too boring and embarrassing to relate I have lots of duplicate files. I found this script on line
Code:
find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate| cut -f3-100 -d ' ' | tr '\n.' '\t.' | sed 's/\t\t/\n/g' | cut -f2-100 | tr '\t' '\n' | perl -i -pe 's/([ (){}-])/\\$1/g' | perl -i -pe 's/'\''/\\'\''/g' | xargs -pr rm -v
rm -v ./test2.txt ?...y
However my natural caution means that I want to copy them t

I tried
Code:
find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate| cut -f3-100 -d ' ' | tr '\n.' '\t.' | sed 's/\t\t/\n/g' | cut -f2-100 | tr '\t' '\n' | perl -i -pe 's/([ (){}-])/\\$1/g' | perl -i -pe 's/'\''/\\'\''/g' | xargs -pr rm -v
rm -v ./test2.txt ?...y
but I got

Code:
mv: cannot overwrite non-directory `./test2.txt' with directory `duplicates/'


---------- Post added 04-07-13 at 04:05 PM ----------

Solved it I needed to put the full path to the duplicate directory!!!
 
Old 04-08-2013, 04:44 AM   #2
pan64
Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 5,159

Rep: Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364
this is an extremely inefficient solution, there are tools to do it, for example: http://www.junauza.com/2010/01/how-t...duplicate.html
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Script to find the duplicate files Tekken Linux - Server 6 03-30-2013 12:29 PM
[SOLVED] Duplicate picutes - move and remane large number of files smithy2010 Linux - Newbie 4 02-15-2012 02:11 PM
find duplicate files using bash script? b-RAM Linux - Newbie 4 06-08-2010 08:05 AM
Script to remove duplicate jpg files ski_phreak Programming 1 05-26-2010 09:22 AM
Script to find duplicate files within one or more directories peter88 Linux - General 6 12-10-2006 06:17 AM


All times are GMT -5. The time now is 12:18 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration