LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 07-17-2016, 10:15 AM   #1
beachboy2
Senior Member
 
Registered: Jan 2007
Location: Wild West Wales, UK
Distribution: Linux Mint 20.2 MATE, MX-19.3, antiX, EndeavourOS
Posts: 3,349
Blog Entries: 17

Rep: Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266
Make sure that you BACKUP your DATA!!


For the umpteenth time on LQ forums, do BACKUP your personal data!

If it is not backed up, it is not a matter of IF you are going to lose it, it is simply a matter of WHEN you are going to lose it.

See post #9 here:

http://www.linuxquestions.org/questi...33#post5577333
 
Old 07-17-2016, 10:51 AM   #2
Emerson
LQ Sage
 
Registered: Nov 2004
Location: Saint Amant, Acadiana
Distribution: Gentoo ~arch
Posts: 7,231

Rep: Reputation: Disabled
Computer users fall into two groups:
0. those that do backups
1. those that have never had a hard drive fail.
 
Old 07-17-2016, 10:53 AM   #3
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,912

Rep: Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513
Quote:
Originally Posted by Emerson View Post
Computer users fall into two groups:
0. those that do backups
1. those that have never had a hard drive fail.
There is actually third...

2. those that have never accidentally deleted their data...

 
Old 07-17-2016, 11:21 AM   #4
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 9,538
Blog Entries: 4

Rep: Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430Reputation: 3430
Ouch ... this sounds like yet-another "voice of recent and painful experience."

If you do a Google DuckDuckGo search on linux automatic backup rsync, you will find hundreds of hits that describe various ways to use the rsync command to make constant backups of your data. (Apple's now-famous Time Machine basically works in just that way.) rsync is an extremely sophisticated command, brimming with useful options.

The most critical consideration is that the backups must occur continuously, sending the data to an external drive (or, better yet, several of them). They should also protect the backed-up data from theft, eavesdropping, or (malicious) interference ... a task that can be accomplished by placing all of the backups into a private folder that only some _backup user owns. [i](A user that, via /bin/nologin, can never "log in" to the system.) And, there should be some way to separate multiple versions of the backed-up material.

But there's another consideration: "a static backup is never good-enough for a database." A database needs replication, and MySQL is very good at it. "Even an hour" is much too long for data from a production database. You need several read-only copies of the data that are always known to be in-sync with one another "right now." (This is also useful for reporting purposes, since reports can be taken from replicas.)
 
Old 07-17-2016, 11:25 AM   #5
Emerson
LQ Sage
 
Registered: Nov 2004
Location: Saint Amant, Acadiana
Distribution: Gentoo ~arch
Posts: 7,231

Rep: Reputation: Disabled
rsnapshot is my favorite, it can use hard links, saves lots of space.
 
Old 07-17-2016, 11:26 AM   #6
DavidMcCann
LQ Veteran
 
Registered: Jul 2006
Location: London
Distribution: PCLinuxOS, Debian
Posts: 5,871

Rep: Reputation: 2169Reputation: 2169Reputation: 2169Reputation: 2169Reputation: 2169Reputation: 2169Reputation: 2169Reputation: 2169Reputation: 2169Reputation: 2169Reputation: 2169
Another tip is to have two sets of backup data and use them alternately. I once found I'd backed-up a corrupted file, but the other backup still had a good copy. I back-up daily: with rsync only the changed files are altered, so it's a quick job.
 
Old 07-17-2016, 11:30 AM   #7
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,912

Rep: Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513
Quote:
Originally Posted by DavidMcCann View Post
Another tip is to have two sets of backup data and use them alternately. I once found I'd backed-up a corrupted file, but the other backup still had a good copy. I back-up daily: with rsync only the changed files are altered, so it's a quick job.
Which is also a recommendation for incremental backups.

I have also seen where some people use a CMS to keep backups - and only archive entire files if they are new, everything else is just incremental deltas to the files.
 
Old 07-17-2016, 11:31 AM   #8
Emerson
LQ Sage
 
Registered: Nov 2004
Location: Saint Amant, Acadiana
Distribution: Gentoo ~arch
Posts: 7,231

Rep: Reputation: Disabled
That's what is great about rsnapshot, it creates daily backups - as many as you want to keep - and won't waste space because it knows how to use hard links.
 
Old 07-17-2016, 01:21 PM   #9
beachboy2
Senior Member
 
Registered: Jan 2007
Location: Wild West Wales, UK
Distribution: Linux Mint 20.2 MATE, MX-19.3, antiX, EndeavourOS
Posts: 3,349

Original Poster
Blog Entries: 17

Rep: Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266
For those who may prefer rsync with a GUI (Graphical User Interface), there is Grsync:

http://www.opbyte.it/grsync/

Review (old but still valid):

http://www.dedoimedo.com/computers/grsync.html

For Debian, Ubuntu, Linux Mint etc, install as follows:

Code:
sudo apt-get install grsync
 
Old 07-17-2016, 05:38 PM   #10
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Backups are Useless if they aren't Tested?

"Users". Seriously, it's a bummer and I feel, but
Everyone has to learn the lesson (partitioning), IMO.
It is inevitable. No backups however, is inexcusable.

Have we not all done this exactly same "oops" (selected the wrong 'one') ?
I did this exact same thing once. I printed http://www.tldp.org/HOWTO/Partition/index.html
and worked it out.

I most certainly wouldn't be responsible for someone's data loss if they have no backup.
I rsync several times a day whether I need it or not.

Last edited by Habitual; 07-17-2016 at 05:41 PM.
 
Old 07-17-2016, 06:48 PM   #11
Doug G
Member
 
Registered: Jul 2013
Posts: 749

Rep: Reputation: Disabled
+1 for testing your backups.

Back when, a large corporate customer had a DEC system with tape backup, and faithfully did daily backups to tape. When their SMD hard drive failed after a couple years, it turned out there was some hardware bug in the tape controller (a DEC compatible) and guess what, they did not have a single good backup tape! This failure equated to a couple managers being fired and the loss of a few man-years of circuit design work, and if I were to guess the loss of multi-millions of dollars.
 
Old 07-17-2016, 07:20 PM   #12
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_12{.0|.1}
Posts: 5,791
Blog Entries: 23

Rep: Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808
If it hasn't been verified, it isn't a backup. How is that not obvious?
 
Old 07-17-2016, 07:32 PM   #13
sgosnell
Senior Member
 
Registered: Jan 2008
Location: Baja Oklahoma
Distribution: Debian Stable and Unstable
Posts: 1,943

Rep: Reputation: 538Reputation: 538Reputation: 538Reputation: 538Reputation: 538Reputation: 538
My personal files aren't worth millions of dollars, and I don't think I would fire myself, but I do keep multiple backups. I have 3 drives, one of them SSD, and I keep them synced with Dropbox, Box, and Google Drive. I use rclone via cronjobs to keep everything in sync. Rsync works, and rclone uses it as a backend, but it doesn't do cloud syncing, AFAIK. All the files that I have any worries about having anyone else access are encrypted with gpg before being put in any folder or drive that is synced to the cloud. I have a local veracrypt volume for storing the unencrypted versions. My KeePass database is also synced so I can access it anywhere, but it's also locally encrypted. That's probably the file that would hurt the most to lose, but the odds of losing every copy of it are very long.
 
Old 07-17-2016, 08:20 PM   #14
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 20,177

Rep: Reputation: 3752Reputation: 3752Reputation: 3752Reputation: 3752Reputation: 3752Reputation: 3752Reputation: 3752Reputation: 3752Reputation: 3752Reputation: 3752Reputation: 3752
So now we come to the elephant in the room. How are backups "verified" ?.

rsync, while excellent, doesn't check files at the target for validity - not by default anyway.
So those backups are just sitting there waiting for bitrot to occur undetected. Some filesystems can detect this (btrfs and zfs for example), but you need to be using them at the target as well. I also take intermittent CRC protected full filesystem backups.

FWIW, it seems the issue mentioned in the initial post is still unresolved - I answered another post last night.
 
Old 07-17-2016, 09:44 PM   #15
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_12{.0|.1}
Posts: 5,791
Blog Entries: 23

Rep: Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808Reputation: 3808
Quote:
Originally Posted by syg00 View Post
So now we come to the elephant in the room. How are backups "verified" ?.
Not so much an elephant as a very good question! But one that has an answer!

Of course it varies with the nature of the thing being backed up, but needs to be done to a level of confidence proportional to the value of the data!

Checksums at source and archive are a good first measure.

As you note, the best, and only way to know for sure is to use the backup, take it live.

This is the method I use for high value, critical data.

My own strategy is to keep at least one live backup of critical sub-systems, and one or more copies of archived offline backups. When I take a backup of some part of my data, I create the archive and then "restore" from it to my live backup platform. In important cases I maintain and run validation test cases against the restored sytems.

This provides the confidence that the checksummed offline backup is valid, and also gives me a fully functional, always available online fallback system.

The live backups are never system-wide backups - I do not think that is a best strategy. Incremental backups of smaller parts, separately managed, separately verified, according to their separate requirements. Like is often said of security, backups are a process, not a program.

Importantly, you should differentiate between data and system backups - data is the all important thing! Systems can be rebuilt, data must be recovered!

I cringe when I see discussion of hard drive images being used as "backups"! That is like taking the express train to oblivion in my experience! (I know! I did that long ago - two backup hard drives mirroring my critical system - both ultimately failing in the same way, at the same time as the main system, for the same reasons...)
 
1 members found this post helpful.
  


Reply

Tags
backup, basis, data regular


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Make Linux Translate String c:\data\ As /mnt/ntfs/data/ kraml Linux - General 4 01-15-2012 11:48 AM
What is the best backup method to make a backup of a running Ubuntu server ? pmorin Linux - Software 9 09-28-2011 08:12 AM
HOW TO MAKE backup script LINUX CentOS5 copy data to another machine ytd Linux - Newbie 8 09-30-2009 02:54 PM
Backing up ~75GB of data on Redhat rel 4 srvr using backup exec shows ~1.3TB of data? unicorntoo Linux - Software 1 05-01-2008 09:32 AM
Using rsync to backup data. Best way to backup folders? Micro420 Linux - General 2 11-23-2006 02:13 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 10:35 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration