rsync incremental backups
 

Hello all

How do you get Rsync to do incremental backups rather than full backups? At the moment I have a script that will create a backup folder (if it doesnt already exist) then copy the source files into the backup directory with the command

rsync $VERBOSE --exclude=$TARGET/ $EXCLUDE --exclude '/Ls-wtgl1c8/**' -rt --delete $source/ $TARGET/$source/ >> $LOG_FILE

Target is where the files will be backed up to
Sources is the dir(s) to be backed up
Exclude files is the list of files not to backup
log file is where the output will be saved to.

At the moment it only does full backups, but I would only like to do incremental, how would this be achieved? Am I missing out an option in the Rsync that is required.

Many thanks in advance.

t.

the default action of rsync is to do incremental backups by doing a checksum comparison of source and destination and transferring only the content change in files...

See this is what I thought it should be doing by default - incremental backups. But even when no files have changed it seems to be deleting the files in my backup directory (I can see it in the trash can of my link station and what is being backed up in my log file), and backing up all the files again. So dont understand why a full backup is being performed. I'm going to take a look at my code again, if I cannot see anything obvious will post here fOR some more help!

Thanks centosboy for your rep!

I cant see anything obvious wrong...so heres the code...any help would be greatly appreciated! :)

!/bin/sh
# backup.sh -- backup to a local drive using rsync

# Directories to backup.
SOURCES="/home /etc /root /usr /boot /var /"
#for testing uncomment
#SOURCES="/home/abc"

# Directory to backup to.


BACKUPDIR="dailybackup"
LOG_FILE="/etc/rsyncbackup.txt"



TARGET="/Linkstation/"$BACKUPDIR

#EXCLUDE_FILE tells rsync what NOT to backup.

EXCLUDE_FILE="/bin/exclude_file.txt"

# delete contents of log so it doesnt grow too large
echo > $LOG_FILE

#add date to the first line of the log file
string=`date +%m/%d/%y`
echo $string >> $LOG_FILE


# Comment out the following line to disable verbose output
VERBOSE="-v"
###########################

if [ ! -x $TARGET ]; then
echo "Creating directory"
mkdir $TARGET/$BACKUP_DATE
fi
if [ ! -x $TARGET ]; then
echo "Backup target does not exist or you don't have permission!"
echo "Exiting..."
exit 2
fi

echo "Verifying Sources..."
for source in $SOURCES; do
echo "Checking $source..."
if [ ! -x $source ]; then
echo "Error with $source!"
echo "Directory either does not exist, or you do not have proper permissions."
exit 2
fi
done

if [ -f $EXCLUDE_FILE ]; then
EXCLUDE="--exclude-from=$EXCLUDE_FILE"
fi

echo "Sources verified. Running rsync..."
for source in $SOURCES; do

# Create directories in $TARGET to mimick source directory hiearchy
if [ ! -d $TARGET/$source ]; then
mkdir -p $TARGET/$source
fi

rsync $VERBOSE --exclude=$TARGET/ $EXCLUDE --exclude '/Linkstation/**' -rt --delete $source/ $TARGET/$source/ >> $LOG_FILE
done

exit 0

Think I know what the problem is nothing to do with the code...will know for sure on Tuesday then will update the post if I rememeber.

Back to investigating this problem! It wasnt what i thought the problem was...so does anyone have any ideas??? I have done some expermimenting with different scripts still the same problem. Also I discoverd that If i was backing up a dir such as /home/user incremental backups will work. But if i was to backup a directory like /home/domain/user then it doesnt work...any suggestions as to why this may be???

Many thanks in advacnce

t.

Okay so I have done some more investigating and have found the following:-

It seems that the reason why a full backup is being performed rather than an incremental is because the modification times of the files are different when the files are copied over.

So if i had a file called test.doc on my ubuntu server its file properties may be:- Created 18 Jan 2010 10:45:11
Modified 18 Jan 2010 10:45:11

Rsync copies the file over to the NAS and the file properties for test.doc on there would be:- Created 18th April 2009 17:44
:- Modified 18 Jan 2010 10:45:10

The creation times are completely different but the modification times seem to be out by a second.

Is this simply a case of changing the NAS box date and time settings (I dont have the perms to do this, so cant just simply try this that why posting on here before getting others involved)...

Thanks for any responses in advance!

The fact that the files created by rsync have an mtime < src files indicates you haven't got your systems time-synced; see ntp.
Should be the other way round if anything...
Try adding a few flags eg

-t, --times preserve modification times

http://linux.die.net/man/1/rsync

Hi chrism01 and thanks for your reply!

I have got someone to change the NAS boxes to the correct date and time and tried to synch the time between my NAS boxes and my ubuntu server by making the NTP SERVER - ntp.ubuntu.com for both. But still the problem remains. The only diff is that the
creation time is 1 second out just like the modification time now.

I have used flags in my rsync command - t to preserve mod times and r (recursive)...

I have a feeling that the full backups are being perfomed for when my files are being shared over the samba server so the files stored on my ubuntu server being accessed by windows pc....

Any other suggestions???

t.

FAT fs saves times by 2sec. interval
 

Could be that your NAS boxes a FAT fs?

If this is the case you can see on the man page:
"In particular, when transferring to or from an MS Windows FAT filesystem (which represents times with a 2-second resolution), --modify-window=1 is useful (allowing times to differ by up to 1 second)."


Regards