LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Server (https://www.linuxquestions.org/questions/linux-server-73/)
-   -   how to create daily incremental backups easily? (https://www.linuxquestions.org/questions/linux-server-73/how-to-create-daily-incremental-backups-easily-847210/)

crypted 11-29-2010 07:57 AM

how to create daily incremental backups easily?
 
I've had several HDD crashes on my personal server over the years and it's just gotten to be a real pain in the rear. Crashed again this morning.

Currently, I make monthly tarball backups of the entire filesystem using my script:

Code:

#!/bin/sh

# Removes the tarball from the previous execution.
rm -rf /backup/data/*.tar.gz

# Dates the new tarballs of current builds.
DATE=`date +%m_%d_%Y`

# Directory structure that is being tarballed.
# Have this be what is found in your "/" root
# excluding the /backup/ partition.

tar -pzcf /backup/data/bin.$DATE.tar.gz /bin/
tar -pzcf /backup/data/boot.$DATE.tar.gz /boot/
tar -pzcf /backup/data/dev.$DATE.tar.gz /dev/
tar -pzcf /backup/data/dist.$DATE.tar.gz /dist/
tar -pzcf /backup/data/etc.$DATE.tar.gz /etc/
tar -pzcf /backup/data/kernel.$DATE.tar.gz /kernel*
tar -pzcf /backup/data/lib.$DATE.tar.gz /lib
tar -pzcf /backup/data/root.$DATE.tar.gz /root/
tar -pzcf /backup/data/sbin.$DATE.tar.gz /sbin/
tar -pzcf /backup/data/stand.$DATE.tar.gz /stand/
tar -pzcf /backup/data/usr.$DATE.tar.gz /usr/
tar -pzcf /backup/data/var.$DATE.tar.gz /var/
tar -pzcf /backup/data/emul.$DATE.tar.gz /emul/
tar -pzcf /backup/data/home.$DATE.tar.gz /home/
tar -pzcf /backup/data/selinux.$DATE.tar.gz /selinux/
tar -pzcf /backup/data/srv.$DATE.tar.gz /srv/
tar -pzcf /backup/data/ssl.$DATE.tar.gz /ssl/
tar -pzcf /backup/data/sys.$DATE.tar.gz /sys/
ls -lah > /backup/data/rootmap
cp /quota* /backup/data/

I'd like to get daily backups of everything, but at least of /var/ since it contains my email, websites, and sql databases.

Any free, simple solutions out there to run such automated backups to my secondary hdd?

fordeck 11-29-2010 08:40 AM

You might take a look at rdiff-backup:

http://www.nongnu.org/rdiff-backup/

Regards,

Fordeck

fordeck 11-29-2010 08:41 AM

See previous post

catkin 11-29-2010 09:25 AM

9 out of 10 cat owners who were asked said their cats prefer rsync. Me, I like Bacula with crunchy fish bits but it's not what any sane puss would call simple.

jlcasado 11-29-2010 12:55 PM

rsync...simple and really effective

repo 11-29-2010 01:01 PM

+1 for rsync

Quote:

# Removes the tarball from the previous execution.
rm -rf /backup/data/*.tar.gz
BTW, I would remove the old backups, AFTER the new backup is made.


Kind regards

crypted 11-29-2010 10:25 PM

any suggestions for the rsync thing?

i'd like to do a backup every 24 hrs on my /backup/ harddrive.

1) /var/www/
2) /var/vmail/
3) /var/lib/mysql

not sure how it'd work out...

fbsduser 11-29-2010 11:27 PM

+1 for rsync. Also
Code:

#!/bin/sh

# Dates the new tarballs of current builds.
DATE=`date +%m_%d_%Y`

# Directory structure that is being tarballed.
# Have this be what is found in your "/" root
# excluding the /backup/ partition.

tar -pzcf /backup/data/bin.$DATE.tar.gz /bin/
tar -pzcf /backup/data/boot.$DATE.tar.gz /boot/
tar -pzcf /backup/data/dev.$DATE.tar.gz /dev/
tar -pzcf /backup/data/dist.$DATE.tar.gz /dist/
tar -pzcf /backup/data/etc.$DATE.tar.gz /etc/
tar -pzcf /backup/data/kernel.$DATE.tar.gz /kernel*
tar -pzcf /backup/data/lib.$DATE.tar.gz /lib
tar -pzcf /backup/data/root.$DATE.tar.gz /root/
tar -pzcf /backup/data/sbin.$DATE.tar.gz /sbin/
tar -pzcf /backup/data/stand.$DATE.tar.gz /stand/
tar -pzcf /backup/data/usr.$DATE.tar.gz /usr/
tar -pzcf /backup/data/var.$DATE.tar.gz /var/
tar -pzcf /backup/data/emul.$DATE.tar.gz /emul/
tar -pzcf /backup/data/home.$DATE.tar.gz /home/
tar -pzcf /backup/data/selinux.$DATE.tar.gz /selinux/
tar -pzcf /backup/data/srv.$DATE.tar.gz /srv/
tar -pzcf /backup/data/ssl.$DATE.tar.gz /ssl/
tar -pzcf /backup/data/sys.$DATE.tar.gz /sys/
ls -lah > /backup/data/rootmap
cp /quota* /backup/data/

# Removes the tarball from the previous execution.
rm -rf /backup/data/*.tar.gz

Works more safely than what you had.

repo 11-30-2010 02:18 AM

Quote:

Originally Posted by crypted (Post 4175370)
any suggestions for the rsync thing?

i'd like to do a backup every 24 hrs on my /backup/ harddrive.

1) /var/www/
2) /var/vmail/
3) /var/lib/mysql

not sure how it'd work out...

Start with reading the documentation for rsync.
Then create a script to backup your directories.
Make sure the script works.
Create a cronjob for the script.
Some info
http://www.thegeekstuff.com/2010/09/...mand-examples/
http://www.cyberciti.biz/tips/tag/rsync-examples

Kind regards

choogendyk 11-30-2010 06:54 AM

I like rsync too, and use it regularly. However, the original post did say incremental. That would be rdiff-backup, which uses librsync. See http://www.nongnu.org/rdiff-backup/, or, for the general concept and to roll your own with rsync, see http://www.mikerubel.org/computers/rsync_snapshots/.

I would also note that in the original configuration, rather than deleting all the old tarballs with the *, you could use `find /backup/data/ -name '*.tar.gz' -mtime +7 -exec rm {} \;`. That would remove any tarballs older than 7 days. Of course, make sure you have enough space for the extras, since they are all full backups. You can do incrementals with gnutar, but then you're getting more complicated (scripting which day to run fulls, incrementals on others, removing the older ones, etc.), and you might as well use something off the shelf.

As far as cat owners go, I like Amanda ;) , and it's not too complicated to get running (there is a quick start with backup to disk on the wiki); but, both Amanda and Bacuala come into their own in networked environments with multiple machines being backed up. For a single computer, there are many simpler solutions.

crypted 12-11-2010 04:46 PM

I've tried using the find command above from choogendyk.

It doesn't work.

Code:

my:/backup# find /backup/databases/ -name '*.sql' -mtime +30 -exec rm {} \
>

Just sits there at >

Code:

my:/backup/databases# find /backup/databases/ -name '*.sql' -mtime +30 -exec rm {}
find: missing argument to `-exec'

How could I get this to work?

It'd be very useful to remove backup files whether tar.gz or sql dumps (not incremental) after so many days...

barriehie 12-11-2010 09:02 PM

Try putting a ; on the end. :)
Code:

my:/backup# find /backup/databases/ -name '*.sql' -mtime +30 -exec rm {} \;

crypted 12-11-2010 10:24 PM

Ah yes, correct you are! Thanks for catching that...

barriehie 12-12-2010 02:41 AM

Quote:

Originally Posted by crypted (Post 4188424)
Ah yes, correct you are! Thanks for catching that...

:) Yep, I'll probably do that a few more times too!


All times are GMT -5. The time now is 10:06 PM.