LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices



Reply
 
Search this Thread
Old 11-28-2006, 05:46 AM   #1
prozac
Member
 
Registered: Oct 2005
Location: Australia
Distribution: slackware 12.1
Posts: 753

Rep: Reputation: 32
an incremental backup strategy confusion


I am planning to backup 6-7 servers using bacula. I have been considering a backup plan whereby I can be "able to access/restore the old files on daily, weekly and monthly basis for 3 months." By which I mean I should be able to access/restore data to any point within this 3 month.

I have decided to go with the GFS scheme on this. My current data-size to backup is around 80 G in total. I expect (i am still confirming in this) a daily increment of 0.5 M and a weekly increment of nearly 30 M.

With the above parameters I have tried to calculate the total size of data I would be backing up for these 3 months.

80Gx3(months)=240G (A full backup every month)
30Mx4(weeks)x3(months)=360M (A weekly incremental backup)
0.5Mx5(days)x4(weeks)x3(months)=30M (Daily incremental backup)
---------------------------
Total data size~=241G
---------------------------

With this strategy, I think If I required to restore to Nov 7 2006 then I would first restore the monthly backup for the month of Oct/Nov. then 1st week incremental backup for the same month and then the daily incremental backup from that day till I reach Nov 7.

I am confused. Is this how it works? Or, Will the above strategy work? I am confused on the weekly incremental section. Am I supposed to take a full backup each week or just an incremental backup?

Any useful comments is appreciated. Basically I am just confused.

TIA,
prozac.
 
Old 11-28-2006, 03:17 PM   #2
MensaWater
Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,028
Blog Entries: 5

Rep: Reputation: 791Reputation: 791Reputation: 791Reputation: 791Reputation: 791Reputation: 791Reputation: 791
Haven't used Bacula but based on the information on its site and other backup products/tools I've used the answer is yes it will work.

According to their definition:
Quote:
Incremental
A backup that includes all files changed since the last Full, Differential, or Incremental backup started. It is normally specified on the Level directive within the Job resource definition, or in a Schedule resource.
I will say that most people don't do it that way (at least for larger environments). Typically they'll do the full on a regular basis (once a week) then do a what bacula calls a differential (backs up everything changed since the last full) on a daily basis. This would make restores a lot faster because you'd only have to restore the most recent full and the most recent differential.

In your scenario you'd have to restore the most recent full and each subsequent incremental. (Not just the weekly incremental but each day's incremental.) "Level" is a keyword in the above quote. If they allow you to do "Levels" as some utilities do and you set your full to highest level then your weekly to next lower level then your dailies to next lower level you can probably do it the way you said. I didn't read their documentation to see if that's how they do levels but its pretty standard in backup solutions.
 
Old 11-28-2006, 11:25 PM   #3
prozac
Member
 
Registered: Oct 2005
Location: Australia
Distribution: slackware 12.1
Posts: 753

Original Poster
Rep: Reputation: 32
Thanks once again for the response. Yes, my backup plan in practice would mean longer restore times and from many disks (or volumes). But the disk size and cost is reduced which is a factor I needed to consider. One thing that still confuses me-just say I did a full backup each week instead of the incremental and keep weekly backups for 3 months, that would mean:

80Gx3(months)=240G (A full backup every month)
80Gx4(weeks)x3(months)=960G (A weekly incremental backup)
0.5Mx5(days)x4(weeks)x3(months)=30M (Daily incremental backup)
---------------------------
Total data size~=1200G
---------------------------

Now when I need to do a full restore, where do I start from? Do I start from restoring Oct/Nov monthly backup then the 1st week full backup and then the incremental ones till I reach Nov 7? I am confused, Look at the sheer size of doing restores, First nearly 80G of monthly full backup, again nearly 80G of the weekly backup and then some MB's of daily incremental backup? Or, start from the weekly one? Which seems the way to do but then how do I justify the monthly backups (I mean when to use them anyway?).

Again TIA,
prozac.
 
Old 11-28-2006, 11:29 PM   #4
prozac
Member
 
Registered: Oct 2005
Location: Australia
Distribution: slackware 12.1
Posts: 753

Original Poster
Rep: Reputation: 32
One more thing though, I didn't find anything related to compression on bacula documentation. I know compression ratio largely depends on the data but since most of the data I would be backing up are databases and simple text files, It would be a help to know in advance the total approx. disk size I would be needing. I've read bacula uses zlib as it compression engine, that would mean the compression provided would roughly be equal to zip compression. What do you think?
 
Old 11-29-2006, 07:31 AM   #5
trickykid
Guru
 
Registered: Jan 2001
Posts: 24,133

Rep: Reputation: 199Reputation: 199
Industry standard average is around 3% change or addition every day that most backup and storage solution enterprises will use to calculate incremental changes to assist in predicting the amount of data to backup.
 
Old 11-29-2006, 11:39 PM   #6
prozac
Member
 
Registered: Oct 2005
Location: Australia
Distribution: slackware 12.1
Posts: 753

Original Poster
Rep: Reputation: 32
Thanks for all the response. I am setting up a test-bed to experiment w/ the plan and I will surely post my findings, whatever and however it be. Meanwhile, I have realized that I was not confused. I was just not confident. Thanks for helping me out.
 
  


Reply

Tags
backup, bacula, calculation, incremental, space, strategy


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
About incremental backup satimis Linux - Software 0 06-15-2006 05:01 AM
incremental backup .... redrobin77 Linux - Software 1 01-31-2006 06:29 AM
rsync incremental backup sachin_keluskar Linux - Software 1 06-29-2005 03:09 AM
How to take incremental Backup in postgres minil Linux - Software 1 05-11-2005 05:47 AM
incremental backup reaky Linux - Software 3 03-10-2004 04:02 PM


All times are GMT -5. The time now is 03:43 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration