LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
Reload this Page how to create daily incremental backups easily?
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 11-29-2010, 07:57 AM   #1
crypted
Member
 
Registered: Jan 2004
Posts: 59

Rep: Reputation: 15
how to create daily incremental backups easily?

I've had several HDD crashes on my personal server over the years and it's just gotten to be a real pain in the rear. Crashed again this morning.

Currently, I make monthly tarball backups of the entire filesystem using my script:

Code:
#!/bin/sh

# Removes the tarball from the previous execution.
rm -rf /backup/data/*.tar.gz

# Dates the new tarballs of current builds.
DATE=`date +%m_%d_%Y`

# Directory structure that is being tarballed. 
# Have this be what is found in your "/" root
# excluding the /backup/ partition.

tar -pzcf /backup/data/bin.$DATE.tar.gz /bin/
tar -pzcf /backup/data/boot.$DATE.tar.gz /boot/
tar -pzcf /backup/data/dev.$DATE.tar.gz /dev/
tar -pzcf /backup/data/dist.$DATE.tar.gz /dist/
tar -pzcf /backup/data/etc.$DATE.tar.gz /etc/
tar -pzcf /backup/data/kernel.$DATE.tar.gz /kernel*
tar -pzcf /backup/data/lib.$DATE.tar.gz /lib
tar -pzcf /backup/data/root.$DATE.tar.gz /root/
tar -pzcf /backup/data/sbin.$DATE.tar.gz /sbin/
tar -pzcf /backup/data/stand.$DATE.tar.gz /stand/
tar -pzcf /backup/data/usr.$DATE.tar.gz /usr/
tar -pzcf /backup/data/var.$DATE.tar.gz /var/
tar -pzcf /backup/data/emul.$DATE.tar.gz /emul/
tar -pzcf /backup/data/home.$DATE.tar.gz /home/
tar -pzcf /backup/data/selinux.$DATE.tar.gz /selinux/
tar -pzcf /backup/data/srv.$DATE.tar.gz /srv/
tar -pzcf /backup/data/ssl.$DATE.tar.gz /ssl/
tar -pzcf /backup/data/sys.$DATE.tar.gz /sys/
ls -lah > /backup/data/rootmap
cp /quota* /backup/data/
I'd like to get daily backups of everything, but at least of /var/ since it contains my email, websites, and sql databases.

Any free, simple solutions out there to run such automated backups to my secondary hdd?
 
Old 11-29-2010, 08:40 AM   #2
fordeck
Member
 
Registered: Oct 2006
Location: Utah
Posts: 520

Rep: Reputation: 61
You might take a look at rdiff-backup:

http://www.nongnu.org/rdiff-backup/

Regards,

Fordeck
 
Old 11-29-2010, 08:41 AM   #3
fordeck
Member
 
Registered: Oct 2006
Location: Utah
Posts: 520

Rep: Reputation: 61
See previous post
Last edited by fordeck; 11-29-2010 at 08:43 AM. Reason: duplicate
 
Old 11-29-2010, 09:25 AM   #4
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208
9 out of 10 cat owners who were asked said their cats prefer rsync. Me, I like Bacula with crunchy fish bits but it's not what any sane puss would call simple.
 
Old 11-29-2010, 12:55 PM   #5
jlcasado
LQ Newbie
 
Registered: Oct 2010
Location: Madrid - Spain
Distribution: RHEL
Posts: 26

Rep: Reputation: Disabled
rsync...simple and really effective
 
Old 11-29-2010, 01:01 PM   #6
repo
LQ 5k Club
 
Registered: May 2001
Location: Belgium
Distribution: Arch
Posts: 8,529

Rep: Reputation: 899Reputation: 899Reputation: 899Reputation: 899Reputation: 899Reputation: 899Reputation: 899
+1 for rsync

Quote:
# Removes the tarball from the previous execution.
rm -rf /backup/data/*.tar.gz
BTW, I would remove the old backups, AFTER the new backup is made.


Kind regards
Last edited by repo; 11-29-2010 at 01:03 PM.
 
Old 11-29-2010, 10:25 PM   #7
crypted
Member
 
Registered: Jan 2004
Posts: 59

Original Poster
Rep: Reputation: 15
any suggestions for the rsync thing?

i'd like to do a backup every 24 hrs on my /backup/ harddrive.

1) /var/www/
2) /var/vmail/
3) /var/lib/mysql

not sure how it'd work out...
 
Old 11-29-2010, 11:27 PM   #8
fbsduser
Member
 
Registered: Oct 2009
Distribution: Hackintosh, SlackWare
Posts: 267

Rep: Reputation: 30
+1 for rsync. Also
Code:
#!/bin/sh

# Dates the new tarballs of current builds.
DATE=`date +%m_%d_%Y`

# Directory structure that is being tarballed. 
# Have this be what is found in your "/" root
# excluding the /backup/ partition.

tar -pzcf /backup/data/bin.$DATE.tar.gz /bin/
tar -pzcf /backup/data/boot.$DATE.tar.gz /boot/
tar -pzcf /backup/data/dev.$DATE.tar.gz /dev/
tar -pzcf /backup/data/dist.$DATE.tar.gz /dist/
tar -pzcf /backup/data/etc.$DATE.tar.gz /etc/
tar -pzcf /backup/data/kernel.$DATE.tar.gz /kernel*
tar -pzcf /backup/data/lib.$DATE.tar.gz /lib
tar -pzcf /backup/data/root.$DATE.tar.gz /root/
tar -pzcf /backup/data/sbin.$DATE.tar.gz /sbin/
tar -pzcf /backup/data/stand.$DATE.tar.gz /stand/
tar -pzcf /backup/data/usr.$DATE.tar.gz /usr/
tar -pzcf /backup/data/var.$DATE.tar.gz /var/
tar -pzcf /backup/data/emul.$DATE.tar.gz /emul/
tar -pzcf /backup/data/home.$DATE.tar.gz /home/
tar -pzcf /backup/data/selinux.$DATE.tar.gz /selinux/
tar -pzcf /backup/data/srv.$DATE.tar.gz /srv/
tar -pzcf /backup/data/ssl.$DATE.tar.gz /ssl/
tar -pzcf /backup/data/sys.$DATE.tar.gz /sys/
ls -lah > /backup/data/rootmap
cp /quota* /backup/data/

# Removes the tarball from the previous execution.
rm -rf /backup/data/*.tar.gz
Works more safely than what you had.
 
Old 11-30-2010, 02:18 AM   #9
repo
LQ 5k Club
 
Registered: May 2001
Location: Belgium
Distribution: Arch
Posts: 8,529

Rep: Reputation: 899Reputation: 899Reputation: 899Reputation: 899Reputation: 899Reputation: 899Reputation: 899
Quote:
Originally Posted by crypted View Post
any suggestions for the rsync thing?

i'd like to do a backup every 24 hrs on my /backup/ harddrive.

1) /var/www/
2) /var/vmail/
3) /var/lib/mysql

not sure how it'd work out...
Start with reading the documentation for rsync.
Then create a script to backup your directories.
Make sure the script works.
Create a cronjob for the script.
Some info
http://www.thegeekstuff.com/2010/09/...mand-examples/
http://www.cyberciti.biz/tips/tag/rsync-examples

Kind regards
 
Old 11-30-2010, 06:54 AM   #10
choogendyk
Senior Member
 
Registered: Aug 2007
Location: Massachusetts, USA
Distribution: Solaris 9 & 10, Mac OS X, Ubuntu Server
Posts: 1,197

Rep: Reputation: 105Reputation: 105
I like rsync too, and use it regularly. However, the original post did say incremental. That would be rdiff-backup, which uses librsync. See http://www.nongnu.org/rdiff-backup/, or, for the general concept and to roll your own with rsync, see http://www.mikerubel.org/computers/rsync_snapshots/.

I would also note that in the original configuration, rather than deleting all the old tarballs with the *, you could use `find /backup/data/ -name '*.tar.gz' -mtime +7 -exec rm {} \;`. That would remove any tarballs older than 7 days. Of course, make sure you have enough space for the extras, since they are all full backups. You can do incrementals with gnutar, but then you're getting more complicated (scripting which day to run fulls, incrementals on others, removing the older ones, etc.), and you might as well use something off the shelf.

As far as cat owners go, I like Amanda , and it's not too complicated to get running (there is a quick start with backup to disk on the wiki); but, both Amanda and Bacuala come into their own in networked environments with multiple machines being backed up. For a single computer, there are many simpler solutions.
 
Old 12-11-2010, 04:46 PM   #11
crypted
Member
 
Registered: Jan 2004
Posts: 59

Original Poster
Rep: Reputation: 15
I've tried using the find command above from choogendyk.

It doesn't work.

Code:
my:/backup# find /backup/databases/ -name '*.sql' -mtime +30 -exec rm {} \
>
Just sits there at >

Code:
my:/backup/databases# find /backup/databases/ -name '*.sql' -mtime +30 -exec rm {} 
find: missing argument to `-exec'
How could I get this to work?

It'd be very useful to remove backup files whether tar.gz or sql dumps (not incremental) after so many days...
 
Old 12-11-2010, 09:02 PM   #12
barriehie
Member
 
Registered: Nov 2010
Distribution: Debian Lenny
Posts: 136
Blog Entries: 1

Rep: Reputation: 23
Try putting a ; on the end.
Code:
my:/backup# find /backup/databases/ -name '*.sql' -mtime +30 -exec rm {} \;
 
Old 12-11-2010, 10:24 PM   #13
crypted
Member
 
Registered: Jan 2004
Posts: 59

Original Poster
Rep: Reputation: 15
Ah yes, correct you are! Thanks for catching that...
 
Old 12-12-2010, 02:41 AM   #14
barriehie
Member
 
Registered: Nov 2010
Distribution: Debian Lenny
Posts: 136
Blog Entries: 1

Rep: Reputation: 23
Quote:
Originally Posted by crypted View Post
Ah yes, correct you are! Thanks for catching that...
Yep, I'll probably do that a few more times too!
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
rsync incremental backups tqz Linux - Newbie 9 09-15-2012 03:22 AM
incremental and differential backups badkuk Linux - General 1 07-26-2010 03:46 AM
Incremental backups? arashi256 Linux - Newbie 9 07-06-2009 08:46 PM
LXer: Create Incremental Snapshot-style Backups With rSync And SSH LXer Syndicated Linux News 0 08-13-2006 08:21 PM
Are incremental backups possible/safe? imagirlgeek Linux - Newbie 4 08-25-2004 04:11 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 08:10 AM.

Contact Us - Advertising Info - Rules - Privacy - LQ Merchandise - Donations - Contributing Member - LQ Sitemap -
Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration