[SOLVED] rsync backup over network?

dchmelik · 02-18-2021, 02:45 AM

I'm trying to do a complicated rsync backup of a home directory on a local area network (LAN.) I need to exclude some directories but in some cases that doesn't work. Unfortunately some also have spaces (which I learned can still be backed up by escaping the slash twice.) I saw examples on various question & answer sites of how to exclude them but there was a lot of speculation and sloppy writing, so nothing there worked. I'm able to exclude directories one level in like /home/user/directory1 (can exclude directory1) but not multiple levels in like /home/user/directory1/directory2 (however I try to exclude directory2, even if it doesn't have spaces, it always gets copied.) What's the exact way to do that, or is there an easier way to list exclusions in an exclude_file.txt, --exclude-from=exclude_file.txt and what would be the format of such a file (also in the case of directories with spaces?)

My recent command was something like the following (in which /home/user/directory1/directory2/directory3 always was copied) (I did this in several steps.)

Code:

rsync -av --info=progress2 --delete --exclude=directory2/directory3 sourcehost:/home/user/directory1/ /home/user/directory1

berndbausch · 02-18-2021, 06:31 AM

Quote:

Originally Posted by dchmelik

spaces (which I learned can still be backed up by escaping the slash twice.)

Well, by putting a backslash in front of the space, or quotes around the directory name.

Quote:

is there an easier way to list exclusions in an exclude_file.txt, --exclude-from=exclude_file.txt and what would be the format of such a file (also in the case of directories with spaces?)

If you have more than a handful of directories you want to exclude, a file is the better approach. One directory per line, and I don't even think quotes or backslashes are needed (I am not sure; double-check this, please).

Quote:

My recent command was something like the following (in which /home/user/directory1/directory2/directory3 always was copied) (I did this in several steps.)

Code:

rsync -av --info=progress2 --delete --exclude=directory2/directory3 sourcehost:/home/user/directory1/ /home/user/directory1

If you start the rsync at /home/user/directory1, excluding directory2/directory3 has no effect indeed, because no directory with that name would be copied.

My last statement may be incorrect.

HappyTux · 02-18-2021, 09:02 AM

Here is what I use to backup my Pi to another SSD, works flawlessly, for an example of multiple excludes.

Code:

rsync -ahPHAXx --delete --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/home/seeder1/rtorrent/*,/lost+found} / /tmp/ssdroot/

An example of a working exclude file to stop my Apple . files from being copied when using rsync it includes directories with spaces in it.

Code:

MacUser2525:~$ alias rsa
alias rsa="rsync -avP --exclude-from=‘/Users/MacUser2525/Bin/apple_exclude.txt'"

MacUser2525:~$ cat Bin/apple_exclude.txt

.Trashes
._.Trashes
.fseventsd
.Spotlight-V100
.DS_Store
.AppleDesktop
.AppleDB
Network Trash Folder
Temporary Items

dchmelik · 02-18-2021, 03:41 PM

Quote:

Originally Posted by berndbausch

If you start the rsync at /home/user/directory1, excluding directory2/directory3 has no effect indeed, because no directory with that name would be copied.

My last statement may be incorrect.

Yes, it is incorrect, because the example I gave was /home/user/directory1/directory2/directory3 , and in my example command, everything in directory1 is copied (recurses the directories) including directory2 then directory3... which I want to exclude.

berndbausch · 02-18-2021, 10:06 PM

You are right, --exclude=directory2/directory3 should work. And you know what - I tried it, and it did work. This is my directory structure on an Odroid SBC running Ubuntu 16:

Code:

$ tree directory1/
directory1/
├── directory2
│   ├── directory3
│   │   └── passwd
│   └── passwd
├── directory2a
│   └── passwd
├── directory2b
│   └── passwd
├── directory2c
│   └── passwd
└── passwd

Notice that I do have a directory3. Let's see if it gets copied. I run the following on a Centos 8 VM:

Code:

# rsync --version
rsync  version 3.1.3  protocol version 31
... stuff omitted ...
# rsync -av --exclude=directory2/directory3 bbausch@192.168.1.16:/tmp/rsynctest/directory1/ target
bbausch@192.168.1.16's password:
receiving incremental file list
./
passwd
directory2/
directory2/passwd
directory2a/
directory2a/passwd
directory2b/
directory2b/passwd
directory2c/
directory2c/passwd

It's not copied. I don't know what happens on your system.

dchmelik · 11-04-2021, 09:51 PM

I got this working with an exclude.txt.

HappyTux · 11-04-2021, 10:17 PM

Good to read you got it sorted and thanks for marking it solved.