LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
 
Search this Thread
Old 03-23-2013, 10:18 AM   #1
Batistuta_g_2000
Member
 
Registered: Oct 2011
Posts: 85
Blog Entries: 1

Rep: Reputation: Disabled
How to compare a list of files in two directories: compare content and print size


Hi

I have a script which unzips a list of files from a tar to a temp directory.

I now need to compare/diff on each unzipped file with the one in its original source directory.

Not sure where to start even, can find the files withe same file name in the source directory but not sure what to do next.

for i in `ls`;do find $IMPORTANT_FILES_DIRECTORY -name $i; done

Can i do an && and something like below? How would I do it with two lists of files in two directories?

for i in `ls`;
do
[[ find $IMPORTANT_FILES_DIRECTORY -name $i ]] | if diff $1 file2 >/dev/null ; then
echo Same
else
echo Different
fi;

done




Code:
#!/bin/bash

BASEDIR=$(cd `dirname $0` && pwd)
CONFILE=$BASEDIR/confile.cfg
BACKUP_FILES=$HOME/assignment1/backups
RESTORE_COMP_DIRECTORY=$HOME/assignment1/temp
FULL_LOGGING=false
LOGFLAG=false
LOGS=$HOME/assignment1/logs
LOGERR=$LOGS/log.err
LOGTMP=$LOGS/log.tmp
LOGOUT=$LOGS/log.out
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
ACTION=backup

cd "$BASEDIR"
source "$CONFILE"

 rm -R $RESTORE_COMP_DIRECTORY
if [[ ! -d "$RESTORE_COMP_DIRECTORY" ]]; then
           mkdir -p "$RESTORE_COMP_DIRECTORY"
    fi

# if dir is not empty exit

if [[ "$(ls -A $RESTORE_COMP_DIRECTORY)" ]]; then
    echo "Dir "$RESTORE_COMP_DIRECTORY" is not empty exiting"               
    exit   
fi

# take filenames into array
cd $BACKUP_FILES_DIRECTORY

RESTORES="${@}"
        for f in $RESTORES
        do
        f=`echo "$f" | sed 's/^[ \t]*//;s/[ \t]*$//'` # trailing whitespaces
        if [[ -f ${f}.tar.gz ]]
        then
        tar -xzvf $BACKUP_FILES_DIRECTORY/$f.tar.gz -C $RESTORE_COMP_DIRECTORY
        echo "SUCCESS"
        sleep 3s
        else
        echo -e "File not found for "$f" please make sure is present. ...."
         sleep 3s
         exit
         fi
        done

     cd $RESTORE_COMP_DIRECTORY
     for i in `ls`;do find $IMPORTANT_FILES_DIRECTORY -name $i; done
 
Old 03-23-2013, 10:57 AM   #2
Batistuta_g_2000
Member
 
Registered: Oct 2011
Posts: 85
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
Am pretty close with this but am getting permission denied on the files (am root)
Also how would I echo the "files are the same" or files are different from something like this?

for file in $(ls "$RESTORE_COMP_DIRECTORY"); do echo -e "\n$file";
find "$IMPORTANT_FILES_DIRECTORY" -name "$($RESTORE_FILES_DIRECTORY/$file)" -print -exec diff "$file" {} \; ; done

sleep 2s
 
Old 03-23-2013, 11:23 AM   #3
Batistuta_g_2000
Member
 
Registered: Oct 2011
Posts: 85
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
This one shows the actual difference in the files and which are identical:

for file in $RESTORE_COMP_DIRECTORY/*; do diff -s "$file" "$IMPORTANT_FILES_DIRECTORY/${file##*/}"; done

but how do I get more information on both files - display both paths/names/file sizes

So run my restore - untar - check filenames - do diff - if diff show info if same echo same
 
Old 03-23-2013, 11:59 AM   #4
unSpawn
Moderator
 
Registered: May 2001
Posts: 28,346
Blog Entries: 54

Rep: Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215
Quote:
Originally Posted by Batistuta_g_2000 View Post
I now need to compare/diff on each unzipped file with the one in its original source directory.
Wouldn't it be easier to compare hashes as in 'md5sum -c /file/containing/hashes'?
 
Old 03-23-2013, 01:31 PM   #5
Batistuta_g_2000
Member
 
Registered: Oct 2011
Posts: 85
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by unSpawn View Post
Wouldn't it be easier to compare hashes as in 'md5sum -c /file/containing/hashes'?
hey unspawn, thanks but my mind is at meltdown as it is trying to figure out why basic things will not work for me....such as:

cd $RESTORE_COMP_DIRECTORY
for i in `ls`;do find $IMPORTANT_FILES_DIRECTORY -name $i;
if [ "$?" = "0" ]; then
echo -e "Found file "$i" in source directory"
else
echo -e "Did not find file "$i" source directory"
fi
done

giving all success even if file is not there??!!! will check out md5sum though thanks.
 
Old 03-23-2013, 02:14 PM   #6
unSpawn
Moderator
 
Registered: May 2001
Posts: 28,346
Blog Entries: 54

Rep: Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215
Code:
# This may be an easier way to find and unpack tar balls:
find "${BACKUP_FILES_DIRECTORY}" -type f -iname \*.tar.gz | while read ITEM; do
 tar -C "${RESTORE_COMP_DIRECTORY}" -xf "${ITEM}" || { echo "Unpack failed on \"${ITEM}\"."; break; }
done

Code:
# When testing existence this asserts the relative directory structure in
# RESTORE_COMP_DIRECTORY is similarly rooted in IMPORTANT_FILES_DIRECTORY:
cd "${RESTORE_COMP_DIRECTORY}"; find . -type f | while read ITEM; do
 echo -en ""${ITEM:2}" in source directory: "
 if [ -f "${IMPORTANT_FILES_DIRECTORY}/${ITEM:2}" ]; then
  echo -en "yes "
  cmp -s "${ITEM}" "${IMPORTANT_FILES_DIRECTORY}/${ITEM:2}"
  case $? in
   0) echo "...and they are similar.";;
   1) echo "...and they differ.";;
   *) echo "...but you better call Houston right now ;-p";;
  esac
 else
  echo "no"
 fi
done
 
Old 03-23-2013, 08:10 PM   #7
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 2,755

Rep: Reputation: 700Reputation: 700Reputation: 700Reputation: 700Reputation: 700Reputation: 700Reputation: 700
Quote:
Originally Posted by Batistuta_g_2000 View Post
hey unspawn, thanks but my mind is at meltdown as it is trying to figure out why basic things will not work for me....such as:

cd $RESTORE_COMP_DIRECTORY
for i in `ls`;do find $IMPORTANT_FILES_DIRECTORY -name $i;
if [ "$?" = "0" ]; then
echo -e "Found file "$i" in source directory"
else
echo -e "Did not find file "$i" source directory"
fi
done

giving all success even if file is not there??!!! will check out md5sum though thanks.
Because the exit status of find does not indicate anything about whether the file is there or not - it only indicates a command error. If there is no command error then it always exits with a success return.
 
Old 03-24-2013, 06:31 AM   #8
Batistuta_g_2000
Member
 
Registered: Oct 2011
Posts: 85
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by unSpawn View Post
Code:
# This may be an easier way to find and unpack tar balls:
find "${BACKUP_FILES_DIRECTORY}" -type f -iname \*.tar.gz | while read ITEM; do
 tar -C "${RESTORE_COMP_DIRECTORY}" -xf "${ITEM}" || { echo "Unpack failed on \"${ITEM}\"."; break; }
done

Code:
# When testing existence this asserts the relative directory structure in
# RESTORE_COMP_DIRECTORY is similarly rooted in IMPORTANT_FILES_DIRECTORY:
cd "${RESTORE_COMP_DIRECTORY}"; find . -type f | while read ITEM; do
 echo -en ""${ITEM:2}" in source directory: "
 if [ -f "${IMPORTANT_FILES_DIRECTORY}/${ITEM:2}" ]; then
  echo -en "yes "
  cmp -s "${ITEM}" "${IMPORTANT_FILES_DIRECTORY}/${ITEM:2}"
  case $? in
   0) echo "...and they are similar.";;
   1) echo "...and they differ.";;
   *) echo "...but you better call Houston right now ;-p";;
  esac
 else
  echo "no"
 fi
done

This is incredibly helpful - learning so much now, thanks a lot, some people on here go out of their way to help it's really appreciated.
 
Old 03-24-2013, 06:33 AM   #9
Batistuta_g_2000
Member
 
Registered: Oct 2011
Posts: 85
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by jpollard View Post
Because the exit status of find does not indicate anything about whether the file is there or not - it only indicates a command error. If there is no command error then it always exits with a success return.
Thanks jpollard, just spotted after the post, now testing for:

for i in `ls`;do find $IMPORTANT_FILES_DIRECTORY -name $i;
if [[ -f $IMPORTANT_FILES_DIRECTORY/$i ]]; then
### changed the above from bad::::if [ "$?" = "1" ]; then
echo -e "Found file "$i" in source directory"
 
Old 03-24-2013, 07:05 AM   #10
unSpawn
Moderator
 
Registered: May 2001
Posts: 28,346
Blog Entries: 54

Rep: Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215Reputation: 3215
Quote:
Originally Posted by Batistuta_g_2000 View Post
This is incredibly helpful - learning so much now, thanks a lot, some people on here go out of their way to help it's really appreciated.
It's just basic scripting. Here's some BASH scripting guides if you didn't know them already:

http://www.tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
http://www.tldp.org/LDP/Bash-Beginne...tml/index.html
http://www.gnu.org/software/bash/man...ode/index.html
http://www.grymoire.com/Unix/Sh.html
http://www.tldp.org/LDP/abs/html/
http://mywiki.wooledge.org/BashFAQ
http://mywiki.wooledge.org/BashPitfalls

Just don't try to grok them all at once ;-p
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] compare two files and print the common lines sasanthi Linux - Newbie 7 07-26-2011 01:18 PM
[SOLVED] cannot compare two directories, and print files that have same name and content apanimesh061 Fedora 3 09-17-2010 10:17 AM
compare two files and print the same text kariagekun Linux - Newbie 4 10-14-2009 07:24 AM
compare two files in C and print the line when they do not match calorie712 Programming 7 04-13-2006 01:56 AM
I'm looking for a perlscript to compare size of 2 files cccc Programming 5 02-28-2004 04:45 PM


All times are GMT -5. The time now is 04:19 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration