LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 11-30-2016, 03:54 PM   #1
replica9000
Senior Member
 
Registered: Jul 2006
Distribution: Debian Unstable
Posts: 1,065
Blog Entries: 2

Rep: Reputation: 227Reputation: 227Reputation: 227
more data on destination than source after rsync


Overtime, it seems that syncing the filesystem leaves more data on the destination than the source. This filesystem has been synced about once a month over the past year.

Code:
# df -BG | grep -E "Filesystem|mapper"
Filesystem            1G-blocks  Used Available Use% Mounted on
/dev/mapper/user-data     5545G 5480G       66G  99% /home/replica/user-data
/dev/mapper/backup        5545G 5527G       19G 100% /mnt/backup
The difference in size between the filesystems is less than 100MB. The source has 66GB free, yet rsync is failing due to lack of space on the destination.
Code:
rsync: write failed on "/mnt/backup/Media/.........": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(393) [receiver=3.1.1]
This is the rsync command I am using.
Code:
rsync -Phavx --delete /home/replica/user-data/ /mnt/backup/
I had the same thing happen with the same filesystem to a secondary backup of similar size. After a rm -rf of the secondary backup, a fresh sync completed without error.

So my question is: does rsync not always remove old data with the --delete option?
 
Old 11-30-2016, 04:36 PM   #2
jailbait
LQ Guru
 
Registered: Feb 2003
Location: Mineral, Virginia
Distribution: Debian 8
Posts: 7,893

Rep: Reputation: 339Reputation: 339Reputation: 339Reputation: 339
Quote:
Originally Posted by replica9000 View Post
So my question is: does rsync not always remove old data with the --delete option?
I use rsync for backup and I use the --delete option. I have never had any problem with files not being deleted when I think that they should be deleted. I just read the rsync man page and looked carefully at the --delete option. I think the man page explanation implies that under some combination of parameters --delete will only be applied to the base directory and --delete will not recurse through the sub-directories.

I always include the recursion parameter (-r) in my rsync backups and I have never had the problem.

--------------------------
Steve Stites
 
Old 11-30-2016, 04:55 PM   #3
replica9000
Senior Member
 
Registered: Jul 2006
Distribution: Debian Unstable
Posts: 1,065

Original Poster
Blog Entries: 2

Rep: Reputation: 227Reputation: 227Reputation: 227
Quote:
Originally Posted by jailbait View Post
I use rsync for backup and I use the --delete option. I have never had any problem with files not being deleted when I think that they should be deleted. I just read the rsync man page and looked carefully at the --delete option. I think the man page explanation implies that under some combination of parameters --delete will only be applied to the base directory and --delete will not recurse through the sub-directories.

I always include the recursion parameter (-r) in my rsync backups and I have never had the problem.

--------------------------
Steve Stites
The -a option includes the -r option. I might have to compare filesystem to see what's going on.
 
Old 11-30-2016, 05:00 PM   #4
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 19,710

Rep: Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548
-P implies --partial
Maybe use --progress explicitly if that's what you really want.
 
Old 11-30-2016, 05:20 PM   #5
replica9000
Senior Member
 
Registered: Jul 2006
Distribution: Debian Unstable
Posts: 1,065

Original Poster
Blog Entries: 2

Rep: Reputation: 227Reputation: 227Reputation: 227
My understanding of the --partial option is that if the sync operation gets interupted in the middle of a file, the next sync should continue with the rest of the file.

I did try omitting -P and used --progress instead, still fails. I'm going to try the --delete-before option and see what happens.
 
Old 11-30-2016, 05:46 PM   #6
replica9000
Senior Member
 
Registered: Jul 2006
Distribution: Debian Unstable
Posts: 1,065

Original Poster
Blog Entries: 2

Rep: Reputation: 227Reputation: 227Reputation: 227
The --delete-before option did the trick. rsync was successful, and the filesystems only differ by about 50MB.

It looks like the option --delete is the same as --delete-during for rsync 3.x+, which doesn't work well when low on space.
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
rsync question. destination larger than source dimitrio Linux - Server 7 11-18-2013 04:17 AM
[SOLVED] match source pattern in destination and substitute in destination usin AWK 123raajesh Linux - Software 7 11-13-2013 04:22 AM
rsync - used incorrect multiple paths format, unsure of resulting source/destination codewaggle Linux - Newbie 6 03-24-2011 12:15 PM
Rsync set permissions destination Abstractt Linux - General 3 09-30-2010 02:05 AM
Incorrect source/destination ports when reading tcpdump data with libpcap Nylex Programming 2 06-21-2007 04:10 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 11:39 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration