LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   more data on destination than source after rsync (https://www.linuxquestions.org/questions/linux-software-2/more-data-on-destination-than-source-after-rsync-4175594525/)

replica9000 11-30-2016 03:54 PM

more data on destination than source after rsync
 
Overtime, it seems that syncing the filesystem leaves more data on the destination than the source. This filesystem has been synced about once a month over the past year.

Code:

# df -BG | grep -E "Filesystem|mapper"
Filesystem            1G-blocks  Used Available Use% Mounted on
/dev/mapper/user-data    5545G 5480G      66G  99% /home/replica/user-data
/dev/mapper/backup        5545G 5527G      19G 100% /mnt/backup

The difference in size between the filesystems is less than 100MB. The source has 66GB free, yet rsync is failing due to lack of space on the destination.
Code:

rsync: write failed on "/mnt/backup/Media/.........": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]

This is the rsync command I am using.
Code:

rsync -Phavx --delete /home/replica/user-data/ /mnt/backup/
I had the same thing happen with the same filesystem to a secondary backup of similar size. After a rm -rf of the secondary backup, a fresh sync completed without error.

So my question is: does rsync not always remove old data with the --delete option?

jailbait 11-30-2016 04:36 PM

Quote:

Originally Posted by replica9000 (Post 5636418)
So my question is: does rsync not always remove old data with the --delete option?

I use rsync for backup and I use the --delete option. I have never had any problem with files not being deleted when I think that they should be deleted. I just read the rsync man page and looked carefully at the --delete option. I think the man page explanation implies that under some combination of parameters --delete will only be applied to the base directory and --delete will not recurse through the sub-directories.

I always include the recursion parameter (-r) in my rsync backups and I have never had the problem.

--------------------------
Steve Stites

replica9000 11-30-2016 04:55 PM

Quote:

Originally Posted by jailbait (Post 5636431)
I use rsync for backup and I use the --delete option. I have never had any problem with files not being deleted when I think that they should be deleted. I just read the rsync man page and looked carefully at the --delete option. I think the man page explanation implies that under some combination of parameters --delete will only be applied to the base directory and --delete will not recurse through the sub-directories.

I always include the recursion parameter (-r) in my rsync backups and I have never had the problem.

--------------------------
Steve Stites

The -a option includes the -r option. I might have to compare filesystem to see what's going on.

syg00 11-30-2016 05:00 PM

-P implies --partial
Maybe use --progress explicitly if that's what you really want.

replica9000 11-30-2016 05:20 PM

My understanding of the --partial option is that if the sync operation gets interupted in the middle of a file, the next sync should continue with the rest of the file.

I did try omitting -P and used --progress instead, still fails. I'm going to try the --delete-before option and see what happens.

replica9000 11-30-2016 05:46 PM

The --delete-before option did the trick. rsync was successful, and the filesystems only differ by about 50MB.

It looks like the option --delete is the same as --delete-during for rsync 3.x+, which doesn't work well when low on space.


All times are GMT -5. The time now is 08:59 AM.