[SOLVED] more data on destination than source after rsync

replica9000 · 11-30-2016, 03:54 PM

Overtime, it seems that syncing the filesystem leaves more data on the destination than the source. This filesystem has been synced about once a month over the past year.

Code:

# df -BG | grep -E "Filesystem|mapper"
Filesystem            1G-blocks  Used Available Use% Mounted on
/dev/mapper/user-data     5545G 5480G       66G  99% /home/replica/user-data
/dev/mapper/backup        5545G 5527G       19G 100% /mnt/backup

The difference in size between the filesystems is less than 100MB. The source has 66GB free, yet rsync is failing due to lack of space on the destination.

Code:

rsync: write failed on "/mnt/backup/Media/.........": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]

This is the rsync command I am using.

Code:

rsync -Phavx --delete /home/replica/user-data/ /mnt/backup/

I had the same thing happen with the same filesystem to a secondary backup of similar size. After a rm -rf of the secondary backup, a fresh sync completed without error.

So my question is: does rsync not always remove old data with the --delete option?

jailbait · 11-30-2016, 04:36 PM

Quote:

Originally Posted by replica9000

So my question is: does rsync not always remove old data with the --delete option?

I use rsync for backup and I use the --delete option. I have never had any problem with files not being deleted when I think that they should be deleted. I just read the rsync man page and looked carefully at the --delete option. I think the man page explanation implies that under some combination of parameters --delete will only be applied to the base directory and --delete will not recurse through the sub-directories.

I always include the recursion parameter (-r) in my rsync backups and I have never had the problem.

--------------------------
Steve Stites

replica9000 · 11-30-2016, 04:55 PM

Quote:

Originally Posted by jailbait

I use rsync for backup and I use the --delete option. I have never had any problem with files not being deleted when I think that they should be deleted. I just read the rsync man page and looked carefully at the --delete option. I think the man page explanation implies that under some combination of parameters --delete will only be applied to the base directory and --delete will not recurse through the sub-directories.

I always include the recursion parameter (-r) in my rsync backups and I have never had the problem.

--------------------------
Steve Stites

The -a option includes the -r option. I might have to compare filesystem to see what's going on.

syg00 · 11-30-2016, 05:00 PM

-P implies --partial
Maybe use --progress explicitly if that's what you really want.

replica9000 · 11-30-2016, 05:20 PM

My understanding of the --partial option is that if the sync operation gets interupted in the middle of a file, the next sync should continue with the rest of the file.

I did try omitting -P and used --progress instead, still fails. I'm going to try the --delete-before option and see what happens.

replica9000 · 11-30-2016, 05:46 PM

The --delete-before option did the trick. rsync was successful, and the filesystems only differ by about 50MB.

It looks like the option --delete is the same as --delete-during for rsync 3.x+, which doesn't work well when low on space.