LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 09-21-2015, 04:35 AM   #1
rblampain
Senior Member
 
Registered: Aug 2004
Location: Western Australia
Distribution: Debian 9.2
Posts: 1,096

Rep: Reputation: 48
backup verification


Does anyone know of a site explaining how to verify a backup? I could not find anything that gives an answer to my problem. Alternatively, does anyone know of a simple backup program (see below) that verifies the backup? The backup programs included in Debian 7 look far too complex for my needs.

I found too many unreadable files on both CD/DVD and USB sticks (sometimes an entire directory) after backing up with "tar" or the archiver provided in the Gnome Debian 7 distro (I do not know what program that is), this is on an Asus laptop about 4/5 years old with text backed up to a single CD or DVD or USB stick, no sound or video.

I have recently discovered that the option
Code:
"--verify"
in tar only
Quote:
"attempt to verify"
which means it could as well be removed.

There are suggestions to make a hash but this is no good in my case because it can only be used when the time to restore files has come and one wants to make sure the contents of the backup have not changed. What I need to do is insure the files that have been backed up are in the backup and I can do a complete restore if necessary. It is not possible to make a hash of the files backed up to compare to a hash of the backup because I have an "xclude" file containing a lot of file-type to ignore. Or may be it's possible but too involved for my level of experience. I have used K3B under Debian 5/6 and just reinstalled it to do my latest backup and it looks (insist: looks) OK.

Thank you for your help.
 
Old 09-22-2015, 05:00 AM   #2
wpeckham
Senior Member
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, Fedora, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, Vsido, tinycore, Q4OS
Posts: 2,667

Rep: Reputation: 1084Reputation: 1084Reputation: 1084Reputation: 1084Reputation: 1084Reputation: 1084Reputation: 1084Reputation: 1084
Ancient sysadmin rant and answer.

There are commands to verify an archive file (details depend upon what kind of archive file) used for backup, but most of those only test that the result is valid and consistent. The best verify backup files against what is on disk. all of those good thing, but ..

The only complete verification of a backup is a restore.
We once maintained DR systems (in another building, three valleys over and about 7 miles away, still to close by today's post-katrina post 9/11 standards) that we used to restore backups from the most critical production systems monthly. We could bring up and entire production network replacement in a few hours, and did once a year. This tested our backups, our DR plan, and our procedure manuals ability for ignorant peasants (usually played by non-technical managers) to get the entire operation functional in the event of a real disaster.

For home backups this might be a little (ok serious) overkill. I still believe that a full restore to different iron, and full function testing, is the only complete and valid backup verification procedure.

We always, after the first time, pulled two backup sets from off-site storage for the DR testing events. We rarely needed the second set, but often enough found that backups that passed every lessor test were not always good enough to give us a working system. Also, media fails: tapes go bad, drives die, there is no backup media that is invulnerable to the ravages of time.

So: how much of that should you take to heart? Do your own risk analysis and business survival analysis of your network. What can you not afford to lose, and what level of risk are you willing to accept. Trim that down or add to that as you test and verify, adjust it as your network or business changes. Then find ways to secure it against such a disaster that your iron (machines), building, and possibly even most technical staff might be unavailable after a disaster.

And test it.
Secure is when you KNOW that a monkey that can read (even a manager) could follow the book and bring up the business in an emergency using your backups.

Does that answer your question, or perhaps the need behind your question?
 
1 members found this post helpful.
Old 09-22-2015, 07:17 AM   #3
rblampain
Senior Member
 
Registered: Aug 2004
Location: Western Australia
Distribution: Debian 9.2
Posts: 1,096

Original Poster
Rep: Reputation: 48
Thank you wpeckham for the good explanation, my understanding of the background is a lot better. If nobody suggests a software that could fall within my needs, I will assume there is none.
 
Old 09-22-2015, 07:52 AM   #4
wpeckham
Senior Member
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, Fedora, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, Vsido, tinycore, Q4OS
Posts: 2,667

Rep: Reputation: 1084Reputation: 1084Reputation: 1084Reputation: 1084Reputation: 1084Reputation: 1084Reputation: 1084Reputation: 1084
Can we flesh this out?

Can you be specific about what you believe your needs are?
That are a LOG of backup and recovery applications available, as well as tools to build your own from pre-existing software.
It is difficult to advise if we are not sure what you are looking for.
Almost every program that does any kind of backup includes SOME kind of verification.

Last edited by wpeckham; 09-22-2015 at 07:54 AM.
 
Old 09-23-2015, 08:45 AM   #5
mabo1
Member
 
Registered: Jul 2015
Location: Nanjing, China
Distribution: Debian
Posts: 48

Rep: Reputation: Disabled
I have been using rsync a command line program.
 
1 members found this post helpful.
Old 09-23-2015, 09:07 AM   #6
rblampain
Senior Member
 
Registered: Aug 2004
Location: Western Australia
Distribution: Debian 9.2
Posts: 1,096

Original Poster
Rep: Reputation: 48
Thank you for your answers.

I need a simple backup I can preferably start from the cli, a backup that can give a warning if some files are not readable or are not copied. This request comes from the fact I have found that files that were backed up using "tar" were not readable on the backup medium, this happened on CD/DVD as well as USB stick (I/O error) and this is new medium created a few weeks before. As far as degradation of the medium is concerned, there is the possibility of it being affected by the local heat getting well over 40C in the shade and substantially more in the car in summer, if that could make a difference although the USB sticks are supposed to be indestructable.

I got the following message:
Code:
Message from syslogd@asus at Sep 19 20:34:38 ...
 kernel:[80844.505795] journal commit I/O error
but this may be unrelated and may have been the delayed result of something else, I must insist on that.

"tar" has an option to verify (--verify) but I have recently found it only "attempt to verify" and I have not been able to find what it does if this attempt fails. May be I should try the tar command with the -v switch but this seems to only list the files processed.

There has never been anything in the logs to attract my attention.

I remember "rsync" as a copy from one machine to another, I'll have a look if I can try it. I missed that possibility.

Last edited by rblampain; 09-23-2015 at 09:24 AM.
 
Old 09-23-2015, 09:36 AM   #7
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,569

Rep: Reputation: 2129Reputation: 2129Reputation: 2129Reputation: 2129Reputation: 2129Reputation: 2129Reputation: 2129Reputation: 2129Reputation: 2129Reputation: 2129Reputation: 2129
What type of backup are you trying to create? Just a backup copy of your files that you can access, or an entire filesystem image that you can restore and boot into?

For the former I usually just use rsync and check the exit status. For the latter, my feeling is full disk clones are painful to maintain and completely unnecessary for a home user, so I avoid them.
 
1 members found this post helpful.
Old 09-23-2015, 09:38 PM   #8
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.10, Centos 7.3
Posts: 17,537

Rep: Reputation: 2420Reputation: 2420Reputation: 2420Reputation: 2420Reputation: 2420Reputation: 2420Reputation: 2420Reputation: 2420Reputation: 2420Reputation: 2420Reputation: 2420
Re your kernel msg: the ext3 & ext4 filesystems come with journalling built-in. An error like that would indicate to me that the HDD is possibly dying.
You could try http://linux.die.net/man/8/smartctl to check.

Re backups: rsync checksums as it goes, but you can also do a post-facto check with it as well - see https://serverfault.com/questions/32...sync-transfers
 
1 members found this post helpful.
Old 09-25-2015, 08:17 AM   #9
rblampain
Senior Member
 
Registered: Aug 2004
Location: Western Australia
Distribution: Debian 9.2
Posts: 1,096

Original Poster
Rep: Reputation: 48
Quote:
Just a backup copy of your files that you can access,
Yes, that is what I want, I do not need ever to do a restore, I usually backup my user files and a few configuration files which I want to be able to recover from a backup, if the problem is more severe, I usually reinstall the distro then the user files (if necessary) and so on.

Last edited by rblampain; 09-25-2015 at 08:18 AM.
 
Old 10-03-2015, 04:07 AM   #10
rblampain
Senior Member
 
Registered: Aug 2004
Location: Western Australia
Distribution: Debian 9.2
Posts: 1,096

Original Poster
Rep: Reputation: 48
I have used rsync with the verify option and it seems to do the job, I also have a new hard drive (SSD) on order since it is a bit pointless to report my result if my hard drive is faulty. The main question turned out to be whether or not the verification system included in any backup could be relied upon and I will definitely come back if this is not the case with
Quote:
rsync
which seems to have a better reputation than
Quote:
tar
.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Backup, shrink backup and modify MBR of backup jps1x2 Linux - General 1 12-17-2013 05:03 AM
offline backup with md5-verification? hickehacke Linux - General 4 01-07-2013 05:37 AM
IPCop Backup Verification jjp001 Linux - Security 0 08-18-2005 07:20 AM
What is MD5 Verification anindyanuri Linux - Networking 7 03-31-2005 09:06 AM
kernel verification.... roofy Linux - Software 4 05-05-2003 02:21 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 02:38 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration