LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-06-2013, 09:36 AM   #1
srinietrx
Member
 
Registered: May 2013
Posts: 86

Rep: Reputation: Disabled
Script to compare files in directories and if matches copy into third directory


I have some 100 files in 1st directory and 60 files in 2nd directory.I want to compare files and if it matches then file of 1st directory should be copied to 3rd directory.

I tried using diff command but not successful.Help me with bash script
 
Old 05-06-2013, 10:21 AM   #2
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
Welcome to LQ!

What you've tried so far? And how do you want to compare files - on basis of their content or file names?
 
Old 05-06-2013, 11:32 AM   #3
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
Basically it could be something like this:
Code:
#!/bin/bash
for A in directory1/*; do
    [[ -f $A ]] || continue
    for B in directory2/*; do
        cmp "$A" "$B" &>/dev/null && cp "$A" directory3/
    done
done
 
Old 05-06-2013, 08:45 PM   #4
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,258

Rep: Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328
Help with bash
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/
 
Old 05-07-2013, 01:14 AM   #5
srinietrx
Member
 
Registered: May 2013
Posts: 86

Original Poster
Rep: Reputation: Disabled
@Konsolebox I will put my question in better manner with example

Dir2 contains -- libmount.so.1, libblkid.so.1
Dir1 contains -- libfuse.so.2, libmount.so.1, libblkid.so.1, libpcre.so.1

Now the filename in above directories should compare.If matches

Dir3 must contain libmount.so.1, libblkid.so.1(These two files should copy from Dir1)

@shivaa I tried diff <(cd dir1 && find | sort) <(cd dir2 && find | sort)
But i don't know how to redirect to dir3

@chrism01 Thankyou for links

Last edited by srinietrx; 05-07-2013 at 01:16 AM.
 
Old 05-07-2013, 02:47 AM   #6
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
Perhaps this.
Code:
#!/bin/bash
for A in directory1/*; do
    B=directory2/${A##*/}
    [[ -f $A && -f $B ]] && cmp "$A" "$B" &>/dev/null && cp "$A" directory3/
done
Or if it's just about the filename:
Code:
#!/bin/bash
for A in directory1/*; do
    B=directory2/${A##*/}
    [[ -f $A && -f $B ]] && cp "$A" directory3/
done
 
1 members found this post helpful.
Old 05-07-2013, 03:59 AM   #7
fortran
Member
 
Registered: Nov 2011
Location: Cairo, Egypt
Distribution: CentOS, RHEL, Fedora
Posts: 300
Blog Entries: 2

Rep: Reputation: 51
Are there two single directories or there are sub-directories in the directories and you want common files of directories and sub-directories?
If there are two directories only and no sub directories in these two directories. Now you can do this.

To list the common files between two directories, if there are no sub-directories in these two directories :
Code:
ls /path/of/first/directory /path/of/second/directory | awk '{ c[$1]++; } END { for( x in c ) if( c[x] == 2 ) print x; }'
These are the common files between two directories.
Now save this list into any file.
Code:
ls /path/of/first/directory /path/of/second/directory | awk '{ c[$1]++; } END { for( x in c ) if( c[x] == 2 ) print x; }' > list.txt
This is little tricky part,Actually the list is not with the absolute path of the files, it is just file name so when you will try to copy using list, it will not be copied, it will not be able to find the file and give error.
So add absolute path in the file.
Code:
ls /path/of/directory1 /path/of/directory2 | awk '{ c[$1]++; } END { for( x in c ) if( c[x] == 2 ) print x; }' > list.txt | sed -i 's/^/\/path\/of\/directory1\//g' list.txt
Now copy the files into another directory.
Code:
cp `cat list.txt` /path/of/destination/directory
So bottom of the line, you have to run these two commands
Code:
$ ls /path/of/directory1 /path/of/directory2 | awk '{ c[$1]++; } END { for( x in c ) if( c[x] == 2 ) print x; }' > list.txt | sed -i 's/^/\/path\/of\/directory1\//g' list.txt
$ cp `cat list.txt` /path/of/destination/directory
and your common files will be copied into your destination directory.
 
1 members found this post helpful.
Old 05-07-2013, 10:33 AM   #8
srinietrx
Member
 
Registered: May 2013
Posts: 86

Original Poster
Rep: Reputation: Disabled
Both Konsolebox and pavi kanetkar answer solved my problem

@konsolebox what is meaning of A##*/.I know ## is used for concatenation but here not undestanding the usage.

@pavi kantekar I replaced pipe(|) operator with && operator in code(..... >list.txt && sed -i 's.....) and worked.Otherwise everything was ok.

Last edited by srinietrx; 05-07-2013 at 10:45 AM.
 
Old 05-07-2013, 03:58 PM   #9
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958
If you truly want to locate identical files, you might check out fdupes. It can search through a tree and compare all the files by size and md5sum. You could probably use it get a list of files to move.
 
1 members found this post helpful.
Old 05-07-2013, 04:20 PM   #10
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,332
Blog Entries: 55

Rep: Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533Reputation: 3533
...or 'md5deep -r': run it on one directory to create the hashes, run it on the other to find (mis)matches and then pipe the output so you have a list of files to move.
*And if you want to retain the directory structure while copying see tar or cpio.
 
1 members found this post helpful.
Old 05-07-2013, 07:22 PM   #11
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,258

Rep: Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328
Re '##' See param expansion
http://mywiki.wooledge.org/BashFAQ/073
 
Old 05-08-2013, 12:45 AM   #12
srinietrx
Member
 
Registered: May 2013
Posts: 86

Original Poster
Rep: Reputation: Disabled
I will try fdupes and md5deep commands later.Thank you all who replied and helped me
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Copy files from multiple directories into one directory MadRabbit Linux - Newbie 8 02-07-2014 08:56 PM
Copy Directories and its files to a new directory just as it is arundhyoti Linux - Newbie 7 07-13-2012 04:50 AM
Copy a directory into another directory while ignoring specific directories or files wh33t Linux - Newbie 14 05-16-2012 09:13 PM
[SOLVED] shell script to copy files into folder that matches the file name daveant Linux - Newbie 5 12-07-2011 08:28 AM


All times are GMT -5. The time now is 02:27 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration