LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-02-2019, 12:18 AM   #1
catiewong
Member
 
Registered: Aug 2018
Posts: 190

Rep: Reputation: Disabled
how to compare two directory , the files are the same


I have two directory , there are many files inside it , if I would like to compare the files if all files creation date , size are the same , would advise how to do it ?
 
Old 01-02-2019, 01:59 AM   #2
ehartman
Senior Member
 
Registered: Jul 2007
Location: Delft, The Netherlands
Distribution: Slackware
Posts: 1,674

Rep: Reputation: 888Reputation: 888Reputation: 888Reputation: 888Reputation: 888Reputation: 888Reputation: 888
Quote:
Originally Posted by catiewong View Post
I have two directory , there are many files inside it , if I would like to compare the files if all files creation date , size are the same , would advise how to do it ?
Simple solution #1:
Code:
cd <dir1>
ls -l > /tmp/dir1
cd <dir2>
ls -l > /tmp/dir2
diff /tmp/dir? | less
Another solution could be rsync in dry-run mode:
Code:
rsync -avn --del <dir1>/ <dir2> | less
Note the / after dir1 as you want the contents of that directory and not the dir itself.
As rsync compares files on date/time and size everything it wants to COPY is either newer in dir1 or non-existent in dir2, everything it wants to remove doesn't exist in dir1 but _does_ in dir2.
 
Old 01-02-2019, 02:04 AM   #3
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,768

Rep: Reputation: 1193Reputation: 1193Reputation: 1193Reputation: 1193Reputation: 1193Reputation: 1193Reputation: 1193Reputation: 1193Reputation: 1193
You mean the filenames and the attributes are the same?
Then compare the contents/data
Code:
if cd /path/to/dir1
then
  for f in *
  do
    cmp "$f" /path/to/dir2/"$f"
  done
fi
 
Old 01-02-2019, 02:41 AM   #4
catiewong
Member
 
Registered: Aug 2018
Posts: 190

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by MadeInGermany View Post
You mean the filenames and the attributes are the same?
Then compare the contents/data
Code:
if cd /path/to/dir1
then
  for f in *
  do
    cmp "$f" /path/to/dir2/"$f"
  done
fi

it it have sub-directory , what can I do ?
 
Old 01-02-2019, 05:19 AM   #5
ehartman
Senior Member
 
Registered: Jul 2007
Location: Delft, The Netherlands
Distribution: Slackware
Posts: 1,674

Rep: Reputation: 888Reputation: 888Reputation: 888Reputation: 888Reputation: 888Reputation: 888Reputation: 888
Quote:
Originally Posted by catiewong View Post
it it have sub-directory , what can I do ?
The rsync will handle that automatically, for the directory listing example you need to add the -R option (so '$ ls -lR > /tmp/dir1' etc).
 
Old 01-02-2019, 06:24 AM   #6
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,692

Rep: Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274
What I do not really recommend (but I have checked):
Code:
cd <dir1>
find . -type f -ls | sort > /tmp/dir1.txt
cd <dir2>
find . -type f -ls | sort > /tmp/dir2.txt
# and finally you can
diff /tmp/dir1.txt /tmp/dir2.txt
 
Old 01-02-2019, 06:57 AM   #7
l0f4r0
Member
 
Registered: Jul 2018
Location: Paris
Distribution: Debian
Posts: 900

Rep: Reputation: 290Reputation: 290Reputation: 290
Quote:
Originally Posted by catiewong View Post
I have two directory , there are many files inside it , if I would like to compare the files if all files creation date , size are the same , would advise how to do it ?
AFAIK, it all depends on the filesystem eventually but generally creation dates are not systematic at all. Linux is rather used to deal with last access, modification and status change (permissions...) times.
NB: rsync in dry-run doesn't seem to show the differences file by file. It just builds a file listing and general stats about the transfer to be run.
 
Old 01-02-2019, 10:16 AM   #8
ehartman
Senior Member
 
Registered: Jul 2007
Location: Delft, The Netherlands
Distribution: Slackware
Posts: 1,674

Rep: Reputation: 888Reputation: 888Reputation: 888Reputation: 888Reputation: 888Reputation: 888Reputation: 888
Quote:
Originally Posted by l0f4r0 View Post
NB: rsync in dry-run doesn't seem to show the differences file by file. It just builds a file listing and general stats about the transfer to be run.
rsync doesn't compare files (unless you use the -c option, which makes it MUCH slower).
It only compares date/time and size of the two, which, I believe, is what the OP requested.
Comparing directory entries (with ls or find) goes on date/time and size too, you never compare the files themselves.

The one thing to watch out with rsync is that it doesn't treat both directories equivalent.
The first one is the "source" directory, which is treated as the master, the 2nd is "destination".
The difference is most striking in
a) the treatment of a trailing / (contents vs directory itself)
Only important for the source, trailing slash in destination is ignored
b) equal size files that do NOT have the same date/time
if "source" is newer, the file is copied, but if
"destination" is newer, the date/time OF that file is only adjusted, the files
are assumed to be equal, with a clock difference.

One final note: Linux file systems do NOT have a "creation" date, the one you normally see is the "last modified" one.

Last edited by ehartman; 01-02-2019 at 10:20 AM.
 
Old 01-02-2019, 11:13 AM   #9
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,258
Blog Entries: 3

Rep: Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713Reputation: 3713
If the files are only plain text then diff is another tool to try. It can do the checkin recursively:

Code:
diff -q -r /directory01/ /directory02/
diff --brief --recursive /directory01/ /directory02/
It will compare contents but not timestamps.
 
Old 01-02-2019, 05:37 PM   #10
MadeInGermany
Senior Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 2,768

Rep: Reputation: 1193Reputation: 1193Reputation: 1193Reputation: 1193Reputation: 1193Reputation: 1193Reputation: 1193Reputation: 1193Reputation: 1193
Quote:
Originally Posted by catiewong View Post
it it have sub-directory , what can I do ?
The generic recursion tool is find
Code:
if cd /path/to/dir1
then
  find . -type f |
  while IFS= read -r f
  do
    cmp "$f" /path/to/dir2/"$f"
  done
fi
 
Old 01-03-2019, 03:15 AM   #11
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,692

Rep: Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274
yes, it this. Just some comments: fd can be faster than find (which is more or less irrelevant here).
Code:
find . -type f -print0 | while IFS= read -r -d '' file; do
is more generic, can be important if you have "strange" chars in your filenames.
Also you might add some logging (something similar):
Code:
...
do
  cmp "$f" /path/to/dir2/"$f" || echo "$f" is not the same
done >> logfile
and also you may try to use background processes if you want to speed it up.
Code:
function check()
{
    cmp "$1" /path/to/dir2/"$1" || echo "$f" is not the same     # << put this into background
}
cd /path/to/dir1 && find . -type f -exec check "{}" \;
 
Old 01-03-2019, 03:28 AM   #12
l0f4r0
Member
 
Registered: Jul 2018
Location: Paris
Distribution: Debian
Posts: 900

Rep: Reputation: 290Reputation: 290Reputation: 290
Quote:
Originally Posted by pan64 View Post
and also you may try to use background processes if you want to speed it up.
Code:
function check()
{
    cmp "$1" /path/to/dir2/"$1" || echo "$f" is not the same     # << put this into background
}
cd /path/to/dir1 && find . -type f -exec check "{}" \;
Are you sure there is no need of final "&" somewhere in the function?

Last edited by l0f4r0; 01-03-2019 at 03:31 AM.
 
Old 01-03-2019, 04:54 AM   #13
l0f4r0
Member
 
Registered: Jul 2018
Location: Paris
Distribution: Debian
Posts: 900

Rep: Reputation: 290Reputation: 290Reputation: 290
@pan64: ^ Additionally, does your suggestion really work from your side?
find doesn't seem to manage in retrieving the shell function this way ("find: check: No such file or directory")... You probably need the following instead:
Code:
function check()
{
    cmp "$1" /path/to/dir2/"$1" || echo "$f" is not the same     # << put this into background
}
export -f check
cd /path/to/dir1 && find . -type f -exec bash -c 'check "$0"' "{}" \;

Last edited by l0f4r0; 01-03-2019 at 04:55 AM.
 
Old 01-03-2019, 05:51 AM   #14
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,692

Rep: Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274Reputation: 7274
Quote:
Originally Posted by l0f4r0 View Post
@pan64: ^ Additionally, does your suggestion really work from your side?
find doesn't seem to manage in retrieving the shell function this way ("find: check: No such file or directory")... You probably need the following instead:
Code:
function check()
{
    cmp "$1" /path/to/dir2/"$1" || echo "$f" is not the same     # << put this into background
}
export -f check
cd /path/to/dir1 && find . -type f -exec bash -c 'check "$0"' "{}" \;
No, I did not check it and also I did not complete that. It was just an idea.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] How to compare a list of files in two directories: compare content and print size Batistuta_g_2000 Linux - Newbie 9 03-24-2013 07:05 AM
How to compare two files line by line and print the line which is same. nancypriyanjali Programming 9 05-30-2011 10:04 PM
[SOLVED] cannot compare two directories, and print files that have same name and content apanimesh061 Fedora 3 09-17-2010 10:17 AM
compare two files and print the same text kariagekun Linux - Newbie 4 10-14-2009 07:24 AM
compare/diff two directory in two different linux/unix machine chuikingman Linux - Server 6 09-24-2009 08:09 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 06:28 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration