LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 01-07-2013, 02:46 AM   #1
hickehacke
LQ Newbie
 
Registered: Jan 2013
Posts: 6

Rep: Reputation: Disabled
offline backup with md5-verification?


I need to create an offline archival backup solution which verifies md5 to protect form bitrot.

The workflow I had in mind works like this.
Material is rsynced from server to backup disk A. Md5 sums is calculated from the original server material.
Backup disk A is rsynced to backup disk B to hopefully have one proper copy of a file if it rots.
The files is deleted from server so backupA is now the main source for the files.
Every time I add new files to backupA from server I also run a check on existing files to see that they match original md5, if not, check backupB and copy files from there to A.

I figure this should keep my files fairly safe for archival.

How should I set up this? Is this the best way for offline archival storage?

I work with film and animation so I do have large amount of data that needs to be backed up. In average each project is about 100-500Gb.
 
Old 01-07-2013, 03:32 AM   #2
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 4,950

Rep: Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308
I think rsync is safe enough, so you can simply try to copy files from B to A if those versions were different.
Also I would try to set those files readonly, so there will be no way to overwrite them (only one exception, the initial copy)
 
Old 01-07-2013, 03:35 AM   #3
hickehacke
LQ Newbie
 
Registered: Jan 2013
Posts: 6

Original Poster
Rep: Reputation: Disabled
Thank you.
But what if the file on B is rotten. How do I know which file is the proper one?
 
Old 01-07-2013, 04:35 AM   #4
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 4,950

Rep: Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308Reputation: 1308
you can store the md5 checksums on another filesystem, you can have raid, but if your storage was not safe enough your data will be lost. So make your server B safe, use raid if you want to be sure and protect it )with firewall, special users and restictions.
 
Old 01-07-2013, 05:37 AM   #5
hickehacke
LQ Newbie
 
Registered: Jan 2013
Posts: 6

Original Poster
Rep: Reputation: Disabled
thanks again.
It will not be raided backup servers but offline harddrives I'll use for archival backup. I'm not afraid of outside user compromises but bitrot, data getting corrupted over time. I've experienced that before when opening old projects. Mostly on old DVD-backups though. Also I've had harddrives stopped working in the past. Wouldn't want that on archival disks...
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
MD5 Sum Verification Procedure Mokoyombi Linux - Newbie 4 07-03-2012 09:19 PM
IPCop Backup Verification jjp001 Linux - Security 0 08-18-2005 07:20 AM
What is MD5 Verification anindyanuri Linux - Networking 7 03-31-2005 09:06 AM
MD5, collision and verification program Mr-TY General 1 11-30-2004 08:59 AM
re: how to md5 for verification ergo_sum Linux - Software 2 11-02-2003 09:18 AM


All times are GMT -5. The time now is 01:27 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration