incremental backup in linux

dinakumar12 · 09-18-2010, 07:08 AM

Hi all,

A complete back up using tar takes consumes more time. so is there any way to take incremental backups using tar.

And i also want to take incremental backup dump of my databases too.

Any suggestions and links will be very helpful.i keep on googling for this,but could find any exact for this.

Thanks in advance,
Dinesh.

goldenbarb · 09-18-2010, 08:37 AM

Take a look at this.

catkin · 09-18-2010, 09:42 AM

Have you tried tar's -g, --listed-incremental option?

Regards the database, the answer will be specific to which database software you are using ... ?

dinakumar12 · 09-23-2010, 01:03 AM

Hi all,

Thank you very much for your reply.sorry for the delayed reply.

We are using both postgres and mysql database.so can any one provide me the command for taking the incremental backup of these two databases.

since i don't even have the basic knowledge in shell script. Your reply would be very helpful.

Thanks in advance,
Dinesh.

linuxlover.chaitanya · 09-23-2010, 01:15 AM

rsync can help you with incremental backup.

dinakumar12 · 09-23-2010, 01:43 AM

hi,

Thanks, but can you provide me the exact code using rsync for taking incremental backup of postgresql and mysql databases.

will be happy if anyone provide me the incremental code for "tar" too.

Thanks in advance,
Dinesh.

linuxlover.chaitanya · 09-23-2010, 01:57 AM

Some examples for rsync:
https://calomel.org/rsync_tips.html

http://www.samba.org/rsync/examples.html

http://www.linuxhomenetworking.com/f...-backup-script

http://freshmeat.net/projects/ribs/

b0uncer · 09-23-2010, 03:12 AM

Here are also a few pages on using rsync to create incremental backups. The first one has a few words on the tar method also, you might want to read them to have an opinion on whether to use it or rsync.

http://www.mikerubel.org/computers/rsync_snapshots/
http://blog.interlinked.org/tutorial...e_machine.html
http://www.sanitarium.net/golug/rsync_backups_2010.html

I happened to work on this just a few days ago, altough not with databases.

Edit: for information on MySQL incremental backups, see for example
http://dev.mysql.com/doc/refman/5.1/...p-methods.html

Quote:

MySQL supports incremental backups: You must start the server with the --log-bin option to enable binary logging; see Section 5.2.4, “The Binary Log”. The binary log files provide you with the information you need to replicate changes to the database that are made subsequent to the point at which you performed a backup. At the moment you want to make an incremental backup (containing all changes that happened since the last full or incremental backup), you should rotate the binary log by using FLUSH LOGS. This done, you need to copy to the backup location all binary logs which range from the one of the moment of the last full or incremental backup to the last but one. These binary logs are the incremental backup; at restore time, you apply them as explained in Section 6.5, “Point-in-Time (Incremental) Recovery Using the Binary Log”.

Chuck56 · 09-23-2010, 07:58 AM

Is using tar for full and incremental backups a hard requirement? If not then...

rsnapshot - http://www.rsnapshot.org/

I've been using this script via cron for a couple years now. I maintain 3 monthly backups, 4 weekly backups and 7 daily backups within about 150% disk space of a single, full backup. Rsnapshot eliminates the need for incremental backups by using rsync and "hard links" to perform full backups in a fraction of the space. Hard links are used with unchanged files and are amazing when it comes to saving space with backups.

Here's a script I run daily to backup mysql databases without shutting down the mysql server. It can be included in the rsnapshot config file. I replaced the server ip with "xxx.xxx.xxx.xxx" and the root password with "root_pswd" for security.

Code:

#!/bin/bash

###########################################################
# Back up MySQL databases
###########################################################

# Do not backup these databases
IGNORE="information_schema mysql test"

# Get list of all databases
DB_LIST=`mysql -h xxx.xxx.xxx.xxx -proot_pswd -Bse 'show databases'`

for db in $DB_LIST; do

        # set skip variable
        skip=0

        if [ "$IGNORE" != "" ]; then
                for i in $IGNORE; do
                        [ "$db" == "$i" ] && skip=1 || :
                done
        fi

        if [ "$skip" == "0" ]; then
                mysqldump -h xxx.xxx.xxx.xxx -proot_pswd $db > $db.sql
        fi

done

exit 0

dinakumar12 · 09-27-2010, 02:52 AM

Hi chuck,

The script which you have provided for backing up mysql database works like charm,but will feel very happy if i get a similar one for postgresql database too.

Chuck56 · 09-27-2010, 05:04 AM

Hopefully a script exists for postgresql. I don't use postgresql (yet) but maybe somebody else might know of a similar type script.

linuxlover.chaitanya · 09-27-2010, 05:21 AM

I dont use much critical PostgreSQL myself, but this is what I was reading a while ago for myself

http://www.postgresql.org/docs/8.1/static/backup.html

http://www.wisdombay.com/articles/article000013.htm

ChrisAbela · 09-27-2010, 05:22 AM

You might want to consider the dump/restore applications if you only use ext2/3 file systems.

dinakumar12 · 09-27-2010, 06:51 AM

Hi all,

Thank you very much guys, will definetely try this.

once again thank you very much for your replies.

dinakumar12 · 09-29-2010, 09:24 AM

Hi Chuck,

I have another question,if i need to remove the seven days old database dump,what should be added to the script.

Thanks in advance,
Dinesh.