How do I create an incremental backup?

sibleytr · 08-25-2015, 02:06 PM

How do I create an incremental backup of my windows server?

Currently running the following script for a full back up of our Windows file server. Looking over available documentation I having a hard time wrapping my head around how to accomplish my desired task using linux and/or rsync specifically.

Current Script

Code:

    rsync -arv -h --progress $srcSrvPth $sanBkUpPth/$srcSrvPth

Next part of the script's evolutionary development is to create incremental backups using a Day of Week folder…and previous data is eliminated.

Code:

  #Day of Week
 incDate=”$(date +’%w’)”

    rsync -arv -h --progress <missingCodeHere> $srcSrvPth $sanBkUpPth/_Inc/$incDate/

Being more familiar with Windows ROBOCOPY we would us the "-m" switch to copy and clear the file's "archive" attribute. What would be the superior Linux resolution?

Ihatewindows522 · 08-25-2015, 02:54 PM

As far as I can tell, it seems you're after duplicity.

This should be a good guide to get you started:
https://www.linux.com/learn/tutorial...with-duplicity

joe_2000 · 08-25-2015, 05:42 PM

I am not exactly sure what you mean by day of week folder, but there is a very neat tool called storeBackup.

It allows you to create incremental backups, the frequency of which can be freely configured. So if you ran the script daily, you could configure storebackup to keep one backup for each of the last seven days, then one that is no older than two weeks, another one that is no older than two months, then 6 months, one year etc... You get the picture.

The cool part is that it is using hard links to duplicate files that have not changed between different backups. That means, that if you did not change anything between today's and yesterday's backup, the two backups will not consume more hard disk space than just one. However, each of them are independent, so you can delete either one and the remaining one is still intact.

Have a look at the project homepage for more info, but store backup is also included in many distro's repos.

Ihatewindows522 · 08-26-2015, 09:51 AM

Quote:

Originally Posted by joe_2000

I am not exactly sure what you mean by day of week folder, but there is a very neat tool called storeBackup.

It allows you to create incremental backups, the frequency of which can be freely configured. So if you ran the script daily, you could configure storebackup to keep one backup for each of the last seven days, then one that is no older than two weeks, another one that is no older than two months, then 6 months, one year etc... You get the picture.

The cool part is that it is using hard links to duplicate files that have not changed between different backups. That means, that if you did not change anything between today's and yesterday's backup, the two backups will not consume more hard disk space than just one. However, each of them are independent, so you can delete either one and the remaining one is still intact.

Have a look at the project homepage for more info, but store backup is also included in many distro's repos.

That's what duplicity with Deja Dup does. Welcome to the conversation.

suicidaleggroll · 08-26-2015, 10:25 AM

You can also do it with standard old rsync using the --link-dest flag.

Something like the following will give you incremental backups at whatever frequency you run the script:

Code:

symbackup="/path/to/backup_latest"
prevbackup=$(readlink "$symbackup")
currbackup="/path/to/backup_$(date +Y%m%d_%H%M%S)"

rsync -aAXv --link-dest="$prevbackup" /* $currbackup/ --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/var/run/*,/var/lock/*,/lost+found,/path/to/*}
if [[ $? -eq 0 ]]; then
   rm -f "$symbackup"
   ln -s "$currbackup" "$symbackup"
fi

joe_2000 · 08-26-2015, 11:13 AM

Quote:

Originally Posted by Ihatewindows522

That's what duplicity with Deja Dup does. Welcome to the conversation.

Does duplicity allow to recover data without having duplicity installed?

Ihatewindows522 · 08-26-2015, 11:54 AM

No. Apparently (after some digging) neither does storeBackup.

http://manpages.ubuntu.com/manpages/...Recover.1.html

joe_2000 · 08-26-2015, 02:23 PM

Quote:

Originally Posted by Ihatewindows522

No. Apparently (after some digging) neither does storeBackup.

http://manpages.ubuntu.com/manpages/...Recover.1.html

Not trying to turn this into a "my tool is better" thread, at the end I think it's good to suggest several options to answer a question so the OP is aware of the options.

That said, the fact there is a "storeBackupRecover" does not mean that you actually need it.

I've been using storebackup for years now, and simplicity is my highest priority in the choice of backup tools. Storebackup creates normal directories, and optionally compresses the contents (using standard utilities). storeBackupRecover might provide additional features / convenience in some circumstances, but is not needed anymore than if you had created the backup with rsync. In fact I've never used it so I can't tell exactly what you'd need it for.

I have not used duplicity, so I cannot speak to it, but if it creates a non-standard format that would be a show-stopper for me. The day I loose a file and want to recover it, I just want to open the backup directory and grab it there.

sibleytr · 08-27-2015, 09:15 AM

Thanks everyone for your post. Given that this is an exercise in developing a better understanding of Linux, suicidaleggroll your post is most appreciated. As time and needs develop will probably start looking into the more formal apps.
Thanks again.

Code:

symbackup="/path/to/backup_latest"
prevbackup=$(readlink "$symbackup")
currbackup="/path/to/backup_$(date +Ym%d_%H%M%S)"

rsync -aAXv --link-dest="$prevbackup" /* $currbackup/ --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/var/run/*,/var/lock/*,/lost+found,/path/to/*}
if [[ $? -eq 0 ]]; then
   rm -f "$symbackup"
   ln -s "$currbackup" "$symbackup"
fi

Habitual · 08-27-2015, 09:24 AM

Quote:

Originally Posted by sibleytr

As time and needs develop will probably start looking into the more formal apps.

Have a look at http://www.funkytwig.com/blog/funkybackup