rsnapshot doesn't just do incremental backups

nass · 07-24-2011, 04:56 AM

Hello everyone,
I seem to have a bit of a problem with rsnapshot. After the 1st (full) backup, the i am trying to do the 2nd one (back to back really, so not many files will be changed) and it takes a loooong time ( i actually didn't let it finish )

I check the newly created backup a couple of minutes later and it already has copied 150MB of 'altered' data compared to the original full backup. This can't be.

I have checked the time on my 2 systems (the main server and this backup server) and the dates coincide ( I have a time server too for the internal network ). so modification times shouldn't be the reason why rsnapshot gets fooled into backing things up.
ofcourse nothing in the original full bakcup has bee chmod'ed between the 2 backups so thats not the reason either...

I do not know what else could be there reason though... any thoughts?

Thank you for your help
nass

sparkyhall · 07-24-2011, 06:06 AM

Quote:

Originally Posted by nass

I check the newly created backup a couple of minutes later and it already has copied 150MB of 'altered' data compared to the original full backup. This can't be.

nass

Why do think 150MB can't be ? I do a daily incremental backup of my home directory (2 users) and that's usually over 165MB, most of which is firefox and thunderbird related and that excludes the cache directories. I also do a weekly incremental full backup, excluding home directories, and according to my logs that's over 179MB although it only takes 3-4 minutes over wired 100Mb ethernet.

I use my own scripts using rsync and hard links for this so I can't comment on rsnapshot but just thought I would give an indication that 150MB is really not that much data. If I were to include the firefox cache then my backups would be much larger.

nass · 07-24-2011, 04:49 PM

well 150MB of data seem little esp compared to 1.7TB of office data, but among those I find a backup of the /etc/ directory and I know I didn't change any config files between these 2 backups...

granth · 07-25-2011, 04:58 PM

Show us your rsnapshot.conf.

Also, you can use "rsnapshot-diff" to see what changed.

Eg.

Code:

$ sudo rsnapshot-diff /backup/daily.0 /backup/daily.1
Comparing /backup/daily.1 to /backup/daily.0
Between /backup/daily.1 and /backup/daily.0:
  897 were added, taking 1678470607 bytes;
  25028 were removed, saving 2293944092 bytes;

BCarey · 07-26-2011, 06:58 PM

Quote:

Originally Posted by nass

but among those I find a backup of the /etc/ directory and I know I didn't change any config files between these 2 backups...

You do realize that each snapshot is actually a complete backup with all files and directories using hardlinks for unchanged files? (Strictly speaking I should say rsync overwrites hardlinks for changed files, replacing with the new file.)

Brian

Tamadite · 06-06-2014, 04:33 PM

I do not get rsnapshot to do incremental backups. See below:

Code:

 # rsnapshot-diff ./daily.0 ./daily.1
Comparing ./daily.1 to ./daily.0
Between ./daily.1 and ./daily.0:
  220911 were added, taking 604634711656 bytes;
  220911 were removed, saving 604634711656 bytes;

Code:

 # rsnapshot -t daily
echo 19313 > /opt/var/run/rsnapshot.pid 
mv /share/HDB_DATA/Qbackup/rsnapshot/daily.1/ \
    /share/HDB_DATA/Qbackup/rsnapshot/daily.2/ 
mv /share/HDB_DATA/Qbackup/rsnapshot/daily.0/ \
    /share/HDB_DATA/Qbackup/rsnapshot/daily.1/ 
/opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded \
    --exclude=iTunes*/ /share/HDA_DATA/Qmultimedia \
    /share/HDB_DATA/Qbackup/rsnapshot/daily.0/\ 
/opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded \
    --exclude=iTunes*/ /share/HDA_DATA/Private \
    /share/HDB_DATA/Qbackup/rsnapshot/daily.0/\ 
touch /share/HDB_DATA/Qbackup/rsnapshot/daily.0/

Any idea?

Woodsman · 06-06-2014, 06:29 PM

If you are performing the backups manually back-to-back, then I agree there should be almost no time involved with the second backup. In fact, the kernel will have cached much of the files to help improve the time. I saw this on another system I was testing about two weeks ago.

How do you have sync_first configured? If that option is set to 1, then you have to perform the very first backup manually before rsnapshot will start accumulating rotations. If set to 0 then rsnapshot will perform the first backup in its own.

Tamadite · 06-06-2014, 06:35 PM

I have just found the source of the problem. I had:

Code:

# If your version of rsync supports --link-dest, consider enable this.
# This is the best way to support special files (FIFOs, etc) cross-platform.
# The default is 0 (off).
# 
link_dest       0

when it should be:

Code:

#
link_dest       1

Now I get:

Code:

[~] # rsnapshot -t daily
echo 10681 > /opt/var/run/rsnapshot.pid 
mv /share/HDB_DATA/Qbackup/rsnapshot/daily.1/ \
    /share/HDB_DATA/Qbackup/rsnapshot/daily.2/ 
/opt/bin/cp -al /share/HDB_DATA/Qbackup/rsnapshot/daily.0 \
    /share/HDB_DATA/Qbackup/rsnapshot/daily.1 
/opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded \
    --exclude=iTunes*/ /share/HDA_DATA/Qmultimedia \
    /share/HDB_DATA/Qbackup/rsnapshot/daily.0/\ 
/opt/bin/rsync -a --delete --numeric-ids --relative --delete-excluded \
    --exclude=iTunes*/ /share/HDA_DATA/Private \
    /share/HDB_DATA/Qbackup/rsnapshot/daily.0/\ 
touch /share/HDB_DATA/Qbackup/rsnapshot/daily.0/