LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices



Reply
 
Search this Thread
Old 10-21-2011, 10:16 AM   #1
metallica1973
Senior Member
 
Registered: Feb 2003
Location: Washington D.C
Posts: 2,120

Rep: Reputation: 58
Using Find with an exclude/exclude file


I am familiar with using tar and exclude/include files:

Code:
tar zcf backup.dirs.tgz --files-from=include.mydirs --exclude-from=exclude.mydirs --no-recursion
but was wondering if I could use find in the same way. I know that you can just specify the directories to exclude but my list is large. Tar is specific which I like:

Code:
find . -depth -ipath './.*' -prune -o -ipath './Downloads*' -prune -o -print
I have to backup specific files from several location and exclude the residual garbage. In my script I have it structure to use find and cpio together and wanted to minimally modify it. So what I want is:

Code:
find (read from include file) (read from exclude file) | cpio -oavc > somedir
I am trying to avoid using several find statements for querying specific directories. Thanks

Code:
find /home/testuser/ \( ! -iname "testdir1*" ! -iname "testdir2*"\)
find /usr/var/ \( ! -iname "testdir3*" ! -iname "testdir4*"\)
find /usr/lib/ \( ! -iname "libssl*" ! -iname "libperl*"\)

or 
find /home/testuser/ -depth -ipath './testdir1*' -o -prune -ipath './testdir2*' -o -prune print

Last edited by metallica1973; 10-22-2011 at 08:15 AM.
 
Old 10-21-2011, 04:59 PM   #2
ShadowCat8
Member
 
Registered: Nov 2004
Location: Arcadia, CA
Distribution: Gentoo, Arch, (RedHat4.x-9.x, FedoraCore 1.x-4.x, Debian Potato-Sarge, LFS 6.0, etc.)
Posts: 209

Rep: Reputation: 43
As a thought, have you looked at using rsync?

HTH. Let us know.
 
Old 10-22-2011, 08:28 AM   #3
metallica1973
Senior Member
 
Registered: Feb 2003
Location: Washington D.C
Posts: 2,120

Original Poster
Rep: Reputation: 58
Many thanks for the reply

It looks promising, do you know if the compression on the archive is as good or better that using tar or cpio? I am concerned about spacing issues.

Regards

Last edited by metallica1973; 10-22-2011 at 08:57 AM.
 
Old 10-23-2011, 10:10 AM   #4
metallica1973
Senior Member
 
Registered: Feb 2003
Location: Washington D.C
Posts: 2,120

Original Poster
Rep: Reputation: 58
it can be done using grep:

Code:
find ... | grep -v -f excludefile | grep -f includefile | cpio ...
regards
 
Old 10-24-2011, 10:11 AM   #5
metallica1973
Senior Member
 
Registered: Feb 2003
Location: Washington D.C
Posts: 2,120

Original Poster
Rep: Reputation: 58
Just an update:

Code:
find ... | grep -f includefile | cpio -oavc > /some/destination
appears to be all that is needed. Using grep in this way, there wasn't a need for an exclude file. It will only use what is in the include-file.
 
Old 10-27-2011, 05:12 PM   #6
metallica1973
Senior Member
 
Registered: Feb 2003
Location: Washington D.C
Posts: 2,120

Original Poster
Rep: Reputation: 58
This worked better

Code:
find / -depth -ipath '/home/testuser/.*' -prune -o -print| grep -f include.mydirs | grep -v -f exclude.mydirs
but it doesnt work if I want to see the differences within 24 hours

Code:
find / -depth -mtime 0 -ipath '/home/testuser/.*' -prune -o -print| grep -f include.mydirs | grep -v -f exclude.mydirs
??
 
Old 10-27-2011, 11:36 PM   #7
metallica1973
Senior Member
 
Registered: Feb 2003
Location: Washington D.C
Posts: 2,120

Original Poster
Rep: Reputation: 58
Wink

it did work. I made an adjustment in my exclude.mydir. i was blocking my test directory.
 
Old 10-28-2011, 01:38 AM   #8
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 655Reputation: 655Reputation: 655Reputation: 655Reputation: 655Reputation: 655
By the way, you can use several directories in find. Including only the directories you want to backup, you may not need a long exclude list

find /home/mydir /var/ ...
 
Old 11-06-2011, 10:39 PM   #9
metallica1973
Senior Member
 
Registered: Feb 2003
Location: Washington D.C
Posts: 2,120

Original Poster
Rep: Reputation: 58
Just wanted to add a note so that other users may have a better understanding of my headaches:

Code:
sudo find / -depth ! -path '*/\.*' -iname '*' -type f | grep -f include_home | grep -v -f exclude_home
excludes "all" hidden directories

Code:
! -path '*/\.*'
my include_home file(files and directories to include) which is all that was needed (no grep -v -f exclude_home) if you are not worried about "find" decending into other directories searching for the same file and directories

ex. just a snippett of the results

Code:
sudo find / -depth ! -path '*/\.*' -iname '*' -type f | grep -f include_home
/etc/network/interfaces
/etc/network/if-up.d/avahi-daemon
/xspace/shit/music/transfer/0a92e46f-a8c1-4e0e-bcfe-c3cd7797a682/etc/network/if-down.d/avahi-autoipd.dpkg-new
/xspace/shit/music/transfer/0a92e46f-a8c1-4e0e-bcfe-c3cd7797a682/etc/network/interfaces
my include_home file(a snippett):

Code:
home/testuser/transfer/
home/testuser/Videos/
home/testuser/VirtualBox VMs/
/etc/lsb-release
/etc/network/
and my exclude_home file

Code:
#home/
usr/
lib/
bin/
etc/
xspace/
boot/
docs/
sbin/
tmp/
sys/
var/
opt/
media/
dev/
proc/
As you can see from above find will ascend down into other directories but if grep -v -f exclude_home files is used, it will not allow that to happen thus filtering out replica files in other-directories except if there are duplicates withing the same directories. Grep are simply the filters:

Code:
sudo find / -depth ! -path '*/\.*' -iname '*' -type f | grep -f include_home | grep -v -f exclude_home
/etc/network/interfaces
/etc/network/if-up.d/avahi-daemon

Last edited by metallica1973; 11-07-2011 at 03:33 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Exclude NFS in find anu_1 Linux - Newbie 11 10-17-2012 03:22 PM
Can we use exclude option in"rm" command to exclude some files/folders? yadav_rk727 Linux - Newbie 1 02-03-2010 11:14 AM
CVS Exclude : Exclude sub directories from check out On Linux from command line shajay12 Linux - Newbie 1 08-03-2009 01:36 AM
"find" exclude file/folder from search da1 *BSD 3 04-12-2009 05:08 PM


All times are GMT -5. The time now is 06:45 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration