LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 07-17-2016, 10:11 PM   #16
Steve R.
Member
 
Registered: Jun 2009
Location: Morehead City, NC
Distribution: Mint 20.2
Posts: 507

Rep: Reputation: 97

Back-up your data to an external device. Basically your home directory and some configuration files that have been customized. I have a Western Digital USB hard drive that is attached to my router for that purpose. As Astrogeek notes, it is not necessary to back-up the operating system.
I have been using sbackup.

Last edited by Steve R.; 07-17-2016 at 10:15 PM.
 
Old 07-17-2016, 11:27 PM   #17
Doug G
Member
 
Registered: Jul 2013
Posts: 749

Rep: Reputation: Disabled
Quote:
So now we come to the elephant in the room. How are backups "verified" ?
Verify the validity of your backup process process by actually restoring the backup to a testing machine. In the old days, however, when an external system cost hundreds of thousands of dollars, testing your backups wasn't really feasible, so you pretty much had to take it on faith that your backup tapes/disks are in fact usable.

Modern systems make it a lot easier, with external cloud storage, high-reliability hardware and media, built-in read after write verification against the source data, etc.

I have always done a trial restore at least once every time I set up a new backup plan for my stuff. Better safe than sorry seems prudent. Plus, when you do restore a backup, unless it's simply a simple data file backup, it seems "there's always something" that doesn't go according to plan. It's nice to find out the gotchas rather than trying to figure them out when there is a very upset customer looking over your shoulder, asking evey couple minutes "how much longer???'.
 
Old 07-18-2016, 03:40 AM   #18
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,912

Rep: Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513
Quote:
Originally Posted by syg00 View Post
So now we come to the elephant in the room. How are backups "verified" ?.

rsync, while excellent, doesn't check files at the target for validity - not by default anyway.
So those backups are just sitting there waiting for bitrot to occur undetected. Some filesystems can detect this (btrfs and zfs for example), but you need to be using them at the target as well. I also take intermittent CRC protected full filesystem backups.
Actually not.

What the filesystems check is that the data recorded hasn't changed via bitrot. They don't verify that a backup is valid. Storing the data on the same disk (checkpoints and such) is not a backup. You actually have to transfer the data to a different storage media, verify that the data transferred matches the original, AND have a way to transfer the data back for use (and verify again), to have a backup.

Last edited by jpollard; 07-18-2016 at 03:44 AM.
 
Old 07-18-2016, 06:59 AM   #19
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
On the 12th, everything worked.
5 days later "oops".

Like I said "every one has to learn this lesson.".

Not our fault. Not your fault. It's the user's fault.
Who wants cookies?

Last edited by Habitual; 07-18-2016 at 07:04 AM.
 
Old 07-18-2016, 08:24 AM   #20
biosboy4
Member
 
Registered: Aug 2015
Distribution: *DEB, Centos, NXOS
Posts: 242

Rep: Reputation: 38
In my (fairly) youthful opinion, a REAL backup solution must have 2 things:

1: It must increment

2: It must store at least a month's worth of versions via delta algorithms.

Backups haven't always been this robust, but since they are today. It really should be considered the proper way to do it.

Edit:

forgot the third requirement:

3: These days applications are much smaller than data, so your backups should operate as live data when/if needed. The idea behind that is to take the application to the data rather than copying a ton of data to the application. Much faster/more efficient.

Last edited by biosboy4; 07-18-2016 at 08:27 AM.
 
Old 07-18-2016, 09:48 AM   #21
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
4.) "Users" only need to worry about $HOME.
</opinion>
 
Old 07-18-2016, 09:54 AM   #22
Emerson
LQ Sage
 
Registered: Nov 2004
Location: Saint Amant, Acadiana
Distribution: Gentoo ~arch
Posts: 7,231

Rep: Reputation: Disabled
Quote:
Originally Posted by Habitual View Post
4.) "Users" only need to worry about $HOME.
</opinion>
Yea right, you better do it for them, fingers will pointed at YOU.
 
Old 07-18-2016, 09:56 AM   #23
beachboy2
Senior Member
 
Registered: Jan 2007
Location: Wild West Wales, UK
Distribution: Linux Mint 20.2 MATE, MX-19.3, antiX, EndeavourOS
Posts: 3,349

Original Poster
Blog Entries: 17

Rep: Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266
Quote:
"Users" only need to worry about $HOME.
+1.
 
Old 07-18-2016, 10:12 AM   #24
keefaz
LQ Guru
 
Registered: Mar 2004
Distribution: Slackware
Posts: 6,325

Rep: Reputation: 760Reputation: 760Reputation: 760Reputation: 760Reputation: 760Reputation: 760Reputation: 760
5) Backup needs to be stored in another place (break-in, fire?)
 
Old 07-18-2016, 11:40 AM   #25
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by Emerson View Post
Yea right, you better do it for them, fingers will pointed at YOU.
No backups? Now they have two problems.
Yak Shaving is not my thing.

Sucks to be them.

I generally <cough> assume the visitor is unskilled in system maintenance, and I try to keep things
really simple for them.

Experience tells me people will blame anyone but themselves.

Peace.
 
Old 07-18-2016, 11:42 AM   #26
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by keefaz View Post
5) Backup needs to be stored in another place (break-in, fire?)
"Backup, backup, backup" says to me, 3 copies.
  • local
  • off-site
  • remote
or variance of it?
 
Old 07-19-2016, 10:11 AM   #27
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 9,538
Blog Entries: 4

Rep: Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430
Physical device *click* failures are *click, click* fairly uncommon, such that *click click click* you probably mostly need to be able to *clickclickthump* gracefully recover from "a simple user screw-up." *thumpthumpclick* The odds of a physical failure are actually very sm


 
Old 07-19-2016, 10:59 AM   #28
biosboy4
Member
 
Registered: Aug 2015
Distribution: *DEB, Centos, NXOS
Posts: 242

Rep: Reputation: 38
Quote:
Originally Posted by sundialsvcs View Post
Physical device *click* failures are *click, click* fairly uncommon, such that *click click click* you probably mostly need to be able to *clickclickthump* gracefully recover from "a simple user screw-up." *thumpthumpclick* The odds of a physical failure are actually very sm


There absolutely a 100% chance that every piece of hardware in the world will fail. Ask the Lord, he will tell you the same thing.
 
Old 07-19-2016, 05:39 PM   #29
replica9000
Senior Member
 
Registered: Jul 2006
Distribution: Debian Unstable
Posts: 1,065
Blog Entries: 2

Rep: Reputation: 227Reputation: 227Reputation: 227
It wasn't until I started collecting Terabytes of data that I actually started to make regular backups. There was a couple times before that I thought that I had lost my data. Once was when I was still using Win98 with fat32, tried out knoppix, and somehow the table was corrupted. All the files were there, all with complete random names though. Second time was with an external drive with XFS. Went to mount it one day, and it refused to mount. Somehow, the filesystem had 2 UUIDs.

Now I have 2 backups of my 6TB of data.

I want to start doing offsite backups, but I have 2 issues.

1. Online/cloud storage: Who is going to let me upload 6TBs of data without charging me an crazy monthly fee?

2. Where to put physical copies of offsite storage?
- I could put encrypted copies hidden in my vehicle, but I'm afraid that isn't an option with HDDs, with the constant weather changes.
- SSDs would be ok for that, but then there's that high price tag again.
- I don't particularly trust anyone with my HDDs, encrypted or not.
 
Old 07-19-2016, 06:13 PM   #30
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,912

Rep: Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513
Quote:
Originally Posted by replica9000 View Post
It wasn't until I started collecting Terabytes of data that I actually started to make regular backups. There was a couple times before that I thought that I had lost my data. Once was when I was still using Win98 with fat32, tried out knoppix, and somehow the table was corrupted. All the files were there, all with complete random names though. Second time was with an external drive with XFS. Went to mount it one day, and it refused to mount. Somehow, the filesystem had 2 UUIDs.

Now I have 2 backups of my 6TB of data.

I want to start doing offsite backups, but I have 2 issues.

1. Online/cloud storage: Who is going to let me upload 6TBs of data without charging me an crazy monthly fee?
and how long will it take to transfer using up any data limits by your ISP
Quote:

2. Where to put physical copies of offsite storage?
- I could put encrypted copies hidden in my vehicle, but I'm afraid that isn't an option with HDDs, with the constant weather changes.
- SSDs would be ok for that, but then there's that high price tag again.
- I don't particularly trust anyone with my HDDs, encrypted or not.
Depends on how large that backup is (even with hard disks).

A disk (up to about 4) may fit in bank vault deposit boxes (not sure about the smallest deposit boxes though), the medium sized ones should do. I don't think they fit very well with a mounting cartridge though. You would have to take one to the bank to decide the necessary size. Then there is the yearly cost of the deposit box.
 
  


Reply

Tags
backup, basis, data regular


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Make Linux Translate String c:\data\ As /mnt/ntfs/data/ kraml Linux - General 4 01-15-2012 11:48 AM
What is the best backup method to make a backup of a running Ubuntu server ? pmorin Linux - Software 9 09-28-2011 08:12 AM
HOW TO MAKE backup script LINUX CentOS5 copy data to another machine ytd Linux - Newbie 8 09-30-2009 02:54 PM
Backing up ~75GB of data on Redhat rel 4 srvr using backup exec shows ~1.3TB of data? unicorntoo Linux - Software 1 05-01-2008 09:32 AM
Using rsync to backup data. Best way to backup folders? Micro420 Linux - General 2 11-23-2006 02:13 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 11:42 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration