LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-31-2019, 11:03 AM   #1
Nick-us
Member
 
Registered: Feb 2019
Distribution: Slackware64-Current
Posts: 65

Rep: Reputation: 8
Question Help comparing 2 directories recursively


I will try to explain what I need to do.
I have 2 folders that are backups from different months of the same Linux:
LinuxA
LinuxB

I want to know what was modified in LinuxB compared to LinuxA

I read on the Internet that people indicate the diff command, But reading the diff manual doesn't seem right
"GNU diff - compare files line by line"

The Comparison I believe I need is to compare the files by date modified and if different, that file has certainly changed. Of course Check files that exist in one folder and do not exist in the other.

Does anyone have any idea how I can do this?
 
Old 10-31-2019, 11:11 AM   #2
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,309
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
You can use the --dry-run option with rsync to check without changing anything.

Code:
rsync --dry-run -av /dirA/ /dirB/
Otherwise, diff can work recursively.
 
Old 10-31-2019, 11:49 AM   #3
Nick-us
Member
 
Registered: Feb 2019
Distribution: Slackware64-Current
Posts: 65

Original Poster
Rep: Reputation: 8
Unhappy

Quote:
Originally Posted by Turbocapitalist View Post
You can use the --dry-run option with rsync to check without changing anything.

Code:
rsync --dry-run -av /dirA/ /dirB/
Otherwise, diff can work recursively.
Does not work!
I tested 2 folders with fewer files to check.

Code:
bash-4.4$ rsync --dry-run -av a/ b/
sending incremental file list
./
androidpit.com.br.htx
cboard.cprogramming.com.htx

sent 2,033 bytes  received 29 bytes  4,124.00 bytes/sec
total size is 2,302,586  speedup is 1,116.68 (DRY RUN)
PROBLEM
androidpit.com.br.htx = OK, this file is different, but it does not report it

cboard.cprogramming.com.htx = OK, this file does not exist in B, but it does not report it

archive.org.htx = This file doesn't exist in A and it doesn't even say anything about it.
 
Old 10-31-2019, 12:08 PM   #4
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,309
Blog Entries: 3

Rep: Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721Reputation: 3721
What about something like this?

Code:
diff --recursive --brief ./A/ ./B/ | sed 's/^Files //; s/ and .*//; s/^Only in //; s/: *//;'
 
1 members found this post helpful.
Old 10-31-2019, 12:13 PM   #5
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,794

Rep: Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201Reputation: 1201
deleted - you want to compile file contents not file names.

Last edited by MadeInGermany; 10-31-2019 at 12:17 PM.
 
Old 10-31-2019, 12:42 PM   #6
Nick-us
Member
 
Registered: Feb 2019
Distribution: Slackware64-Current
Posts: 65

Original Poster
Rep: Reputation: 8
Quote:
Originally Posted by Turbocapitalist View Post
What about something like this?

Code:
diff --recursive --brief ./A/ ./B/ | sed 's/^Files //; s/ and .*//; s/^Only in //; s/: *//;'
Code:
diff --recursive --brief ./b/ ./a/ | sed 's/^Files //; s/ and .*//; s/^Only in //; s/: *//;'
./b/androidpit.com.br.htx
./b/archive.org.htx
./a/cboard.cprogramming.com.htx
Or using simpler: diff -rq a/ b/
Code:
Files a/androidpit.com.br.htx and b/androidpit.com.br.htx differ
Only in b/: archive.org.htx
Only in a/: cboard.cprogramming.com.htx
The diff command seems to work, but it is quite slow with a large directory or files that are 30Gb in size.

And the diff command has problems with shortcuts / links he can't understand that the 2 informed links are the same.
source is a link only

Code:
diff: /mnt/dados/07.10.2019/lib/modules/4.19.75/source: No such file or directory
diff: /run/media/root/sda1/lib/modules/4.19.75/source: No such file or directory





 
Old 11-06-2019, 02:36 PM   #7
rnturn
Senior Member
 
Registered: Jan 2003
Location: Illinois (SW Chicago 'burbs)
Distribution: openSUSE, Raspbian, Slackware. Previous: MacOS, Red Hat, Coherent, Consensys SVR4.2, Tru64, Solaris
Posts: 2,803

Rep: Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550
Quote:
Originally Posted by Nick-us View Post
I will try to explain what I need to do.
I have 2 folders that are backups from different months of the same Linux:
LinuxA
LinuxB

I want to know what was modified in LinuxB compared to LinuxA

I read on the Internet that people indicate the diff command, But reading the diff manual doesn't seem right
"GNU diff - compare files line by line"

The Comparison I believe I need is to compare the files by date modified and if different, that file has certainly changed. Of course Check files that exist in one folder and do not exist in the other.

Does anyone have any idea how I can do this?
To compare the files in both trees you can
use "find" to walk down each tree and obtain the checksum for each file and redirect it into a file.
Code:
LinuxA=<path-to-some-tree>
LinuxB=<path-to-some-other-tree>

for TREE in ${LinuxA} ${LinuxB}
do
  find ${TREE} -type f -exec md5sum {} \; 
done > checksum.lis
Each record will look like "checksum filepath".

To see which files are the same, extract the checksums from the checksum file:
Code:
$ cat checksum.lis | awk '{print $1}' | sort | uniq -c | grep -v ' 1 ' | awk '{print $2}' > duplicate.checksums
which would eliminate all the checksums that are singletons. Locate these files using
Code:
$ grep -f duplicate.checksums checksum.list
If you're interested in finding the files that are different -- or do not exist in one of the trees -- you can use:
Code:
$ cat checksum.lis | awk '{print $1}' | sort | uniq -c | grep ' 1 ' | awk '{print $2}' > lonely.checksums
$ grep -f lonely.checksums checksum.lis
If you're looking for files that exist in LinuxA but not in LinuxB, you combine some of the above code and do something like:
Code:
LinuxA=<path-to-some-tree>
LinuxB=<path-to-some-other-tree>

for TREE in ${LinuxA} ${LinuxB}
do
  find ${TREE} -type f -exec md5sum {} \; 
done > checksum.lis

cat checksum.lis | grep ${LinuxA} | while read FP
do
   FN=$( basename ${FP} )
   if [ ! -e ${LinuxB}/${FP} ]; then
      echo "${FP} does not exist under ${LinuxB}."
   fi
done
(Note: the above should work but I haven't tested it.)

You could add tests in the above to grep for and compare checksums for the cases where the files is found under both trees and determine whether the file has been modified. Obviously, if you want to check if a file in the "LinuxB" tree doesn't exist in "LinuxA", you'd swap the names of the trees in the code.

HTH...
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Using chmod to recursively change directories / files [GOD]Anck Linux - Software 6 11-10-2008 06:16 PM
Remove directories recursively with same name cboyd Linux - Newbie 5 01-14-2007 09:48 PM
recursively copy php.ini to www directories osio Programming 3 07-01-2005 04:24 AM
tar, excluding directories recursively towlie Linux - Newbie 3 06-17-2004 12:32 PM
recursively listing directories full paths kubicon Linux - Newbie 2 02-22-2004 03:55 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 10:46 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration