[SOLVED] Script to compare files in directories and if matches copy into third directory

srinietrx · 05-06-2013, 08:36 AM

I have some 100 files in 1st directory and 60 files in 2nd directory.I want to compare files and if it matches then file of 1st directory should be copied to 3rd directory.

I tried using diff command but not successful.Help me with bash script

shivaa · 05-06-2013, 09:21 AM

Welcome to LQ!

What you've tried so far? And how do you want to compare files - on basis of their content or file names?

konsolebox · 05-06-2013, 10:32 AM

Basically it could be something like this:

Code:

#!/bin/bash
for A in directory1/*; do
    [[ -f $A ]] || continue
    for B in directory2/*; do
        cmp "$A" "$B" &>/dev/null && cp "$A" directory3/
    done
done

chrism01 · 05-06-2013, 07:45 PM

Help with bash
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/

srinietrx · 05-07-2013, 12:14 AM

@Konsolebox I will put my question in better manner with example

Dir2 contains -- libmount.so.1, libblkid.so.1
Dir1 contains -- libfuse.so.2, libmount.so.1, libblkid.so.1, libpcre.so.1

Now the filename in above directories should compare.If matches

Dir3 must contain libmount.so.1, libblkid.so.1(These two files should copy from Dir1)

@shivaa I tried diff <(cd dir1 && find | sort) <(cd dir2 && find | sort)
But i don't know how to redirect to dir3

@chrism01 Thankyou for links

konsolebox · 05-07-2013, 01:47 AM

Perhaps this.

Code:

#!/bin/bash
for A in directory1/*; do
    B=directory2/${A##*/}
    [[ -f $A && -f $B ]] && cmp "$A" "$B" &>/dev/null && cp "$A" directory3/
done

Or if it's just about the filename:

Code:

#!/bin/bash
for A in directory1/*; do
    B=directory2/${A##*/}
    [[ -f $A && -f $B ]] && cp "$A" directory3/
done

fortran · 05-07-2013, 02:59 AM

Are there two single directories or there are sub-directories in the directories and you want common files of directories and sub-directories?
If there are two directories only and no sub directories in these two directories. Now you can do this.

To list the common files between two directories, if there are no sub-directories in these two directories :

Code:

ls /path/of/first/directory /path/of/second/directory | awk '{ c[$1]++; } END { for( x in c ) if( c[x] == 2 ) print x; }'

These are the common files between two directories.
Now save this list into any file.

Code:

ls /path/of/first/directory /path/of/second/directory | awk '{ c[$1]++; } END { for( x in c ) if( c[x] == 2 ) print x; }' > list.txt

This is little tricky part,Actually the list is not with the absolute path of the files, it is just file name so when you will try to copy using list, it will not be copied, it will not be able to find the file and give error.
So add absolute path in the file.

Code:

ls /path/of/directory1 /path/of/directory2 | awk '{ c[$1]++; } END { for( x in c ) if( c[x] == 2 ) print x; }' > list.txt | sed -i 's/^/\/path\/of\/directory1\//g' list.txt

Now copy the files into another directory.

Code:

cp `cat list.txt` /path/of/destination/directory

So bottom of the line, you have to run these two commands

Code:

$ ls /path/of/directory1 /path/of/directory2 | awk '{ c[$1]++; } END { for( x in c ) if( c[x] == 2 ) print x; }' > list.txt | sed -i 's/^/\/path\/of\/directory1\//g' list.txt
$ cp `cat list.txt` /path/of/destination/directory

and your common files will be copied into your destination directory.

srinietrx · 05-07-2013, 09:33 AM

Both Konsolebox and pavi kanetkar answer solved my problem

@konsolebox what is meaning of A##*/.I know ## is used for concatenation but here not undestanding the usage.

@pavi kantekar I replaced pipe(|) operator with && operator in code(..... >list.txt && sed -i 's.....) and worked.Otherwise everything was ok.

David the H. · 05-07-2013, 02:58 PM

If you truly want to locate identical files, you might check out fdupes. It can search through a tree and compare all the files by size and md5sum. You could probably use it get a list of files to move.

unSpawn · 05-07-2013, 03:20 PM

...or 'md5deep -r': run it on one directory to create the hashes, run it on the other to find (mis)matches and then pipe the output so you have a list of files to move.
*And if you want to retain the directory structure while copying see tar or cpio.

chrism01 · 05-07-2013, 06:22 PM

Re '##' See param expansion
http://mywiki.wooledge.org/BashFAQ/073

srinietrx · 05-07-2013, 11:45 PM

I will try fdupes and md5deep commands later.Thank you all who replied and helped me