LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (http://www.linuxquestions.org/questions/linux-software-2/)
-   -   Solving RSYNC's rather poor directory-exclude ability? (http://www.linuxquestions.org/questions/linux-software-2/solving-rsyncs-rather-poor-directory-exclude-ability-475155/)

hockeyrink 08-18-2006 01:06 PM

Solving RSYNC's rather poor directory-exclude ability?
 
I've read all the previous threads on rsync, but this seems just sooo... strange NOT having a relatively way to tell rsync to "backup everything in this tree, except THIS subtree"!

I'm trying to use rsync to backup a large set of files, and I want to ignore one particulary large subdirectory (which is backed up on a different schedule.

If I have:
Quote:

/mnt/
/mnt/stuff/
/mnt/stuff/ohmygodlots/
/mnt/stuff/restofstuff/
What is my rsync command to backup /mnt/stuff/ and ignore /mnt/stuff/ohmygodlots?

This is what I *think* should work:
Quote:

/usr/bin/rsync -avx --exclude="/mnt/stuff/ohmygodlots/" /mnt/stuff/ /mybackups/foobackup
But the only thing that comes close to working is:
Quote:

/usr/bin/rsync -avx --exclude="/ohmygodlots/" /mnt/stuff/ /mybackups/foobackup
Although it does work, it doesn't preclude that there *may* be another "ohmygodlots" subdir elsewhere that I want to keep. How do I make an explicit path(s) be ignored during an rsync?

macemoneta 08-18-2006 01:35 PM

From 'man rsync':

if the pattern contains a / (not counting a trailing /) or a
"**", then it is matched against the full pathname, including
any leading directories. If the pattern doesn’t contain a / or a
"**", then it is matched only against the final component of the
filename. (Remember that the algorithm is applied recursively
so "full filename" can actually be any portion of a path from
the starting directory on down.)


Use:

/usr/bin/rsync -avx --exclude=mnt/stuff/ohmygodlots/ /mnt/stuff/ /mybackups/foobackup

hockeyrink 08-18-2006 02:39 PM

Quote:

Originally Posted by macemoneta
Use:

/usr/bin/rsync -avx --exclude=mnt/stuff/ohmygodlots/ /mnt/stuff/ /mybackups/foobackup

Didn't work.

This works:
Quote:

/usr/bin/rsync -avvx --exclude=/ohmygodlots/ /mnt/stuff/ /mybackups/foobackup
building file list ...
[sender] hiding directory ohmygodlots because of pattern /ohmygodlots/
done
This fails (and any other exclude variation not with a bare "/ohmygodlots/")
Quote:

/usr/bin/rsync -avvx --exclude=stuff/ohmygodlots/ /mnt/stuff/ /mybackups/foobackup
(No report of hiding directory)

Thanks for the effort, tho. Any other ideas?

macemoneta 08-18-2006 03:29 PM

What version of rsync are you using? Is /mnt/stuff a symlink?

$ rsync --version
rsync version 2.6.8 protocol version 29
Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others.
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, ACLs, xattrs, symlinks, batchfiles,
inplace, IPv6, 64-bit system inums, 64-bit internal inums

rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.

hockeyrink 08-18-2006 04:11 PM

Quote:

Originally Posted by macemoneta
What version of rsync are you using?

Just apt-got (?) the latest during this fiasco. Matches yours.

Quote:

Originally Posted by macemoneta
Is /mnt/stuff a symlink?

You know, that may just be the case... hold on... nope. Yeah. I've created a test directory structure just to figure out this rsync problem.

Here's an actual tree listing of my testdir:
Quote:

sb7:/mnt/stuff# tree
.
|-- ohmygodlots (dir)
| |-- ohdir1 (dir)
| |-- ohlots2 (file)
| `-- ohlots3 (file)
`-- restofstuff (dir)
|-- ohmygodlots (dir)
| |-- ohrest1 (file)
| |-- ohrest2 (file)
| `-- ohrest3 (file)
|-- rest1 (file)
|-- rest2 (file)
`-- rest3 (file)

4 directories, 8 files
sb7:/mnt/stuff#
Did some more tests, and found that:
Quote:

sb7:/mnt/stuff# /usr/bin/rsync -avvxn --exclude=/ohmygodlots /mnt/stuff/ /mybackups/foobackup
building file list ...
[sender] hiding directory ohmygodlots because of pattern /ohmygodlots
done
created directory /mybackups/foobackup
delta-transmission disabled for local transfer or --whole-file
./
restofstuff/
restofstuff/rest1
restofstuff/rest2
restofstuff/rest3
restofstuff/ohmygodlots/
restofstuff/ohmygodlots/ohrest1
restofstuff/ohmygodlots/ohrest2
restofstuff/ohmygodlots/ohrest3
total: matches=0 hash_hits=0 false_alarms=0 data=0

sent 237 bytes received 74 bytes 622.00 bytes/sec
total size is 0 speedup is 0.00
Interesting how the exclude=/ohmygodlots skipped the first "root level" ohmygodlots, but not the one nested under "restofstuff". So as long as the --exclude= is set in reference to the root of the backup path, it seems to work.

To restate it: Putting in the --exclude= as an absolute reference (/mnt/stuff/ohmygodlots) will not work. It seems to only work as referenced from the path to the files to be rsync'd:
/mnt/stuff# /usr/bin/rsync -avvxn --exclude=/ohmygodlots /mnt/stuff/ /mybackups/foobackup

Where:
/usr/bin/rsync - is the application (duh)
-avvxn is archive parameter, with vv for verbose debugging, and n for "don't really do this!" (for testing)
--exclude=/ohmygodlots - is the subtree we want to ignore, which is referenced from the root defined from...
/mnt/stuff - which is the path to start the rsync from
/mybackups/foobackup - the final output file name.

There. Hopefully that helps any other poor schmucks like myself that have been fighting with this.


All times are GMT -5. The time now is 06:40 PM.