LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 08-18-2006, 12:06 PM   #1
hockeyrink
LQ Newbie
 
Registered: Oct 2003
Posts: 3

Rep: Reputation: 0
Solving RSYNC's rather poor directory-exclude ability?


I've read all the previous threads on rsync, but this seems just sooo... strange NOT having a relatively way to tell rsync to "backup everything in this tree, except THIS subtree"!

I'm trying to use rsync to backup a large set of files, and I want to ignore one particulary large subdirectory (which is backed up on a different schedule.

If I have:
Quote:
/mnt/
/mnt/stuff/
/mnt/stuff/ohmygodlots/
/mnt/stuff/restofstuff/
What is my rsync command to backup /mnt/stuff/ and ignore /mnt/stuff/ohmygodlots?

This is what I *think* should work:
Quote:
/usr/bin/rsync -avx --exclude="/mnt/stuff/ohmygodlots/" /mnt/stuff/ /mybackups/foobackup
But the only thing that comes close to working is:
Quote:
/usr/bin/rsync -avx --exclude="/ohmygodlots/" /mnt/stuff/ /mybackups/foobackup
Although it does work, it doesn't preclude that there *may* be another "ohmygodlots" subdir elsewhere that I want to keep. How do I make an explicit path(s) be ignored during an rsync?
 
Old 08-18-2006, 12:35 PM   #2
macemoneta
Senior Member
 
Registered: Jan 2005
Location: Manalapan, NJ
Distribution: Fedora x86 and x86_64, Debian PPC and ARM, Android
Posts: 4,593
Blog Entries: 2

Rep: Reputation: 326Reputation: 326Reputation: 326Reputation: 326
From 'man rsync':

if the pattern contains a / (not counting a trailing /) or a
"**", then it is matched against the full pathname, including
any leading directories. If the pattern doesn’t contain a / or a
"**", then it is matched only against the final component of the
filename. (Remember that the algorithm is applied recursively
so "full filename" can actually be any portion of a path from
the starting directory on down.)


Use:

/usr/bin/rsync -avx --exclude=mnt/stuff/ohmygodlots/ /mnt/stuff/ /mybackups/foobackup
 
Old 08-18-2006, 01:39 PM   #3
hockeyrink
LQ Newbie
 
Registered: Oct 2003
Posts: 3

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by macemoneta
Use:

/usr/bin/rsync -avx --exclude=mnt/stuff/ohmygodlots/ /mnt/stuff/ /mybackups/foobackup
Didn't work.

This works:
Quote:
/usr/bin/rsync -avvx --exclude=/ohmygodlots/ /mnt/stuff/ /mybackups/foobackup
building file list ...
[sender] hiding directory ohmygodlots because of pattern /ohmygodlots/
done
This fails (and any other exclude variation not with a bare "/ohmygodlots/")
Quote:
/usr/bin/rsync -avvx --exclude=stuff/ohmygodlots/ /mnt/stuff/ /mybackups/foobackup
(No report of hiding directory)

Thanks for the effort, tho. Any other ideas?
 
Old 08-18-2006, 02:29 PM   #4
macemoneta
Senior Member
 
Registered: Jan 2005
Location: Manalapan, NJ
Distribution: Fedora x86 and x86_64, Debian PPC and ARM, Android
Posts: 4,593
Blog Entries: 2

Rep: Reputation: 326Reputation: 326Reputation: 326Reputation: 326
What version of rsync are you using? Is /mnt/stuff a symlink?

$ rsync --version
rsync version 2.6.8 protocol version 29
Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others.
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, ACLs, xattrs, symlinks, batchfiles,
inplace, IPv6, 64-bit system inums, 64-bit internal inums

rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
 
Old 08-18-2006, 03:11 PM   #5
hockeyrink
LQ Newbie
 
Registered: Oct 2003
Posts: 3

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by macemoneta
What version of rsync are you using?
Just apt-got (?) the latest during this fiasco. Matches yours.

Quote:
Originally Posted by macemoneta
Is /mnt/stuff a symlink?
You know, that may just be the case... hold on... nope. Yeah. I've created a test directory structure just to figure out this rsync problem.

Here's an actual tree listing of my testdir:
Quote:
sb7:/mnt/stuff# tree
.
|-- ohmygodlots (dir)
| |-- ohdir1 (dir)
| |-- ohlots2 (file)
| `-- ohlots3 (file)
`-- restofstuff (dir)
|-- ohmygodlots (dir)
| |-- ohrest1 (file)
| |-- ohrest2 (file)
| `-- ohrest3 (file)
|-- rest1 (file)
|-- rest2 (file)
`-- rest3 (file)

4 directories, 8 files
sb7:/mnt/stuff#
Did some more tests, and found that:
Quote:
sb7:/mnt/stuff# /usr/bin/rsync -avvxn --exclude=/ohmygodlots /mnt/stuff/ /mybackups/foobackup
building file list ...
[sender] hiding directory ohmygodlots because of pattern /ohmygodlots
done
created directory /mybackups/foobackup
delta-transmission disabled for local transfer or --whole-file
./
restofstuff/
restofstuff/rest1
restofstuff/rest2
restofstuff/rest3
restofstuff/ohmygodlots/
restofstuff/ohmygodlots/ohrest1
restofstuff/ohmygodlots/ohrest2
restofstuff/ohmygodlots/ohrest3
total: matches=0 hash_hits=0 false_alarms=0 data=0

sent 237 bytes received 74 bytes 622.00 bytes/sec
total size is 0 speedup is 0.00
Interesting how the exclude=/ohmygodlots skipped the first "root level" ohmygodlots, but not the one nested under "restofstuff". So as long as the --exclude= is set in reference to the root of the backup path, it seems to work.

To restate it: Putting in the --exclude= as an absolute reference (/mnt/stuff/ohmygodlots) will not work. It seems to only work as referenced from the path to the files to be rsync'd:
/mnt/stuff# /usr/bin/rsync -avvxn --exclude=/ohmygodlots /mnt/stuff/ /mybackups/foobackup

Where:
/usr/bin/rsync - is the application (duh)
-avvxn is archive parameter, with vv for verbose debugging, and n for "don't really do this!" (for testing)
--exclude=/ohmygodlots - is the subtree we want to ignore, which is referenced from the root defined from...
/mnt/stuff - which is the path to start the rsync from
/mybackups/foobackup - the final output file name.

There. Hopefully that helps any other poor schmucks like myself that have been fighting with this.

Last edited by hockeyrink; 08-18-2006 at 03:43 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
exclude directory with cp hampel Linux - Software 3 06-04-2009 11:59 AM
can't understand rsync's behavior at all baronsam Linux - Software 5 08-15-2006 01:46 PM
tar --exclude=directory ampop Linux - Software 7 06-20-2006 01:12 AM
PHP: List files in directory, exclude backups kenneho Programming 2 12-30-2005 04:27 AM
exclude directory from find ebasi Linux - Software 8 07-17-2004 02:12 AM


All times are GMT -5. The time now is 06:43 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration