LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 10-14-2018, 04:21 AM   #1
grumpyskeptic
Member
 
Registered: Apr 2016
Posts: 291

Rep: Reputation: Disabled
Look for duplicates in folder tree A and folder tree B, then delete the duplicates only from A.


I would like to find identical files which are in folder or directory tree A and in folder or directory tree B. (The tree might consist of just one folder). Then I want to delete the duplicated files from tree A only, and not delete anything from tree B.

What is the best way to do this please?

The FS program lists duplicates, but there is no method that I am aware of for getting it to highlight the duplicate files in tree A only. You would have to go through the list manually and manually select those files in tree A, which would be too big a job to do if a large number of files are involved.

Thanks.

Edit: I should have typed FSlint, not FS.

Last edited by grumpyskeptic; 10-14-2018 at 05:09 AM.
 
Old 10-14-2018, 04:35 AM   #2
fatmac
Senior Member
 
Registered: Sep 2011
Location: Upper Hale, Surrey/Hants Border, UK
Posts: 3,347

Rep: Reputation: Disabled
Maybe something like this will work, (not tested).
Code:
ls -a A > A.lst;ls -a B > B.lst; cmp A.lst B.lst > duplist
Plus,maybe.
Code:
grep A

Last edited by fatmac; 10-14-2018 at 04:38 AM.
 
Old 10-14-2018, 04:57 AM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 18,492

Rep: Reputation: 3099Reputation: 3099Reputation: 3099Reputation: 3099Reputation: 3099Reputation: 3099Reputation: 3099Reputation: 3099Reputation: 3099Reputation: 3099Reputation: 3099
I know not of this "FS" program of which you speak, but I refuse to allow dup finders to automatically delete files. It will delete things you don't want it to. Maybe not this time, but it will happen.

fdupes and fslint are probably the best known CLI ones, but I use duff as it allows me to put decent regex into th mix. But I always generate a list I can script later into a "rm" loop.
Much safer, and a trivial to knock up.
 
Old 10-15-2018, 03:08 AM   #4
l0f4r0
Member
 
Registered: Jul 2018
Location: Paris
Distribution: Debian
Posts: 895

Rep: Reputation: 288Reputation: 288Reputation: 288
Quote:
Originally Posted by fatmac View Post
Maybe something like this will work, (not tested).
Code:
ls -a A > A.lst;ls -a B > B.lst; cmp A.lst B.lst > duplist
Plus,maybe.
Code:
grep A
I think it could be dangerous to implement such an algo based on filenames only...except if this is OP's requirement (but the OP didn't define what he called "duplicates/identical files" though...is it same filenames and/or same contents?).
Maybe a safer approach would be to go with hashes? Otherwise with a specialized tool but I have no experience with any...

Last edited by l0f4r0; 10-15-2018 at 03:11 AM.
 
Old 10-15-2018, 03:35 AM   #5
descendant_command
Senior Member
 
Registered: Mar 2012
Posts: 1,716

Rep: Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554Reputation: 554
fdupes is my preferred tool.
It can delete directly or produce a list or further inspection/processing.

Pay attention to syg00's warning too.
 
Old 10-27-2018, 04:26 AM   #6
grumpyskeptic
Member
 
Registered: Apr 2016
Posts: 291

Original Poster
Rep: Reputation: Disabled
Do any Linux file managers include being able to find duplicate files? If they do, then using them may be the simplest option.

Thanks.

Last edited by grumpyskeptic; 10-27-2018 at 04:39 AM.
 
Old 10-27-2018, 09:51 PM   #7
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 1,293

Rep: Reputation: 595Reputation: 595Reputation: 595Reputation: 595Reputation: 595Reputation: 595
With find and a while loop.
By cd'ing into one of the folders one works with relative file names and can simply prepend the other folder name.
Code:
A=A
B=B
(
cd "$A" &&
find . -type f
) |
while IFS= read f
do
  if [ -f "$B/$f" ] && cmp "$f" "$B/$f"
  then
    echo rm "$f"
  fi
done

Last edited by MadeInGermany; 10-29-2018 at 03:35 PM. Reason: Put cd && find in a (subshell) so the relative pathes work
 
Old 10-27-2018, 11:23 PM   #8
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: Somewhere in my head.
Distribution: Slackware FreeBSD Win10 MX
Posts: 9,623

Rep: Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072Reputation: 2072
rdfind??

I'm going to check out MadeInGermany script, that looks interesting.

Last edited by BW-userx; 10-27-2018 at 11:57 PM.
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
can't seem to create an ISO from a folder tree SaintDanBert Linux - Software 7 02-24-2017 07:12 PM
bash find prune duplicates in directory tree wroom Programming 8 11-06-2015 07:22 AM
Chemistry problem: Identify duplicates and non-duplicates within TWO sdf files robertselwyne Programming 5 12-09-2011 07:20 AM
Copy an entire folder tree pshankland Linux - Newbie 3 09-28-2010 04:48 AM
Makefile to resursively compile a folder tree yapp Programming 5 06-14-2005 08:51 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 10:15 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration