LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-14-2020, 11:37 PM   #1
hifi100
Member
 
Registered: Sep 2016
Location: India
Distribution: Arch Linux
Posts: 357

Rep: Reputation: Disabled
Am I doing backups correctly ?


I am using Grsync with this command line

Code:
rsync -r -t -v --progress -c -s
I am using an 32GB USB flash drive as backup media.

I am running Grsync once a week.

Is this enough ? Am I doing backups correctly ?
 
Old 05-14-2020, 11:55 PM   #2
scasey
LQ Veteran
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: Rocky 9.5
Posts: 5,876

Rep: Reputation: 2289Reputation: 2289Reputation: 2289Reputation: 2289Reputation: 2289Reputation: 2289Reputation: 2289Reputation: 2289Reputation: 2289Reputation: 2289Reputation: 2289
The ultimate test of a backup is to recover from it.
That’s not necessarily easy to test...but it pretty much defines “correctly.”

That said, we need a bit more information...what are you backing up? How often does the data you’re backing up change?
 
1 members found this post helpful.
Old 05-15-2020, 02:47 AM   #3
agillator
Member
 
Registered: Aug 2016
Distribution: Mint 19.1
Posts: 419

Rep: Reputation: Disabled
You are the only one who can answer your question. Whether you want to do it by directly copying files or uses a 'system' is up to you. Personally I use rsnapshot which uses rsync and tends to be quite fast after the first time. How often to back up is again up to you. The question to ask yourself is 'how much of your 'stuff' are are you willing to lose (i.e. replace) and is any of it irreplacable?' Some people with critical files backup hourly or more often. Others back up monthly or so. Another consideration is how many backups do you keep? When you back up are you overwriting your last one? If so, what happens if everything goes bad in the middle of one? Usually people will at least alternate so they always have a good one to fall back on. Many keep more than one. It depends on their situation and comfort level. As scasey pointed another major consideration is how difficult will restoration be? Also think about what you are backing up. Do you really want or need to back up everything? Do you really want to spend the time, energy and media space to back up programs you can easily replace? Remember a program is normally not one file but may have modules and configuration files scattered all over the back forty and won't run if you miss just one. Very few programs are not easily reinstalled from repositories or other sources so you need to save them unless you really want to. Data, on the other hand, can be very important and irreplaceable, but not all data is created equal. Do you really need to back up last week's grocery list? What about yesterday's checkbook register, or last hour's for that matter. Once you have answered those, why are you backing up? Do you need to keep a copy offsite or even in a safety deposit box? One thing that may help is to make yourself a detailed checklist of what you would need to do to do a bare metal recovery. That would begin to give you some idea of what you need to have on hand and what you can always get elsewhere. Also remember Murphy. I don't care how careful you are there will always be something you forget about or think is critical and isn't.

These are questions no one else can answer for you. I know it seems like a lot but learning to think this way will save you time, frustration and money in the long run. We will all be glad to give you our opinions, suggestions, and tell you what we do but as scasey said, you need to give us some guidance about what and why, too.
 
1 members found this post helpful.
Old 05-15-2020, 04:24 AM   #4
fatmac
LQ Guru
 
Registered: Sep 2011
Location: Upper Hale, Surrey/Hants Border, UK
Distribution: One main distro, & some smaller ones casually.
Posts: 5,823

Rep: Reputation: Disabled
The average user just needs to keep a back up of personal files, how you back up is immaterial, as long as it can be read back, & put on your disk after a disaster.

(If you are running a server, you should do the child, parent, grandparent type of back ups.)

Pendrives are OK for short term back ups, but I'd use an external HDD myself.
 
2 members found this post helpful.
Old 05-19-2020, 05:06 AM   #5
hifi100
Member
 
Registered: Sep 2016
Location: India
Distribution: Arch Linux
Posts: 357

Original Poster
Rep: Reputation: Disabled
Thanks everyone for the replies.
Quote:
we need a bit more information...what are you backing up? How often does the data you’re backing up change?
I am backing up all of my data which includes audio, video, Libreoffice docs, KeePassXC database, text documents and Veracrypt containers. Leaving the KeePassXC database the other files doesn't change much.

Quote:
Another consideration is how many backups do you keep? When you back up are you overwriting your last one?
At the moment I am keeping only 1 backup. I am using "incremental" backup (rsync) which as you know only changes those files which have changed.

Quote:
Pendrives are OK for short term back ups, but I'd use an external HDD myself
I had purchased a Western Digital My Passports 500GB drive some years back. It failed after just 6 months of usage. A)I don't have the cash right now to buy a new external drive & B)I will never buy another spinning HDD again. When I am financially ready I will buy an external SSD.

Last edited by hifi100; 05-19-2020 at 05:09 AM.
 
Old 05-19-2020, 06:45 AM   #6
wpeckham
LQ Guru
 
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS, Manjaro
Posts: 6,117

Rep: Reputation: 2906Reputation: 2906Reputation: 2906Reputation: 2906Reputation: 2906Reputation: 2906Reputation: 2906Reputation: 2906Reputation: 2906Reputation: 2906Reputation: 2906
First, KUDOS to you for doing backups at ALL! Far too many people just never do.

Second, it looks to me as if you are backing up only data and certain settings. That may serve you well. It is often better to install a system correctly, then recover data than to attempt to recover a full system image on new and posibly incompatible) hardware.

Third: a real backup allows for multi-generational backups. Think about the case where malware encrypts or corrupts your data just prior to you running the backup: could you go back to a previous (uncorrupted) copy?
If the data value is high (or the risk to you of losing it high enough) there is value in having a copy of your data backup off site. This covers the case where you have a fire or flood and your local devices drown or go up in smoke. You may want to give thought to those factors and evaluate your backup plan with these things in mind.
 
1 members found this post helpful.
Old 05-19-2020, 08:13 AM   #7
hifi100
Member
 
Registered: Sep 2016
Location: India
Distribution: Arch Linux
Posts: 357

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by wpeckham View Post
If the data value is high (or the risk to you of losing it high enough) there is value in having a copy of your data backup off site. This covers the case where you have a fire or flood and your local devices drown or go up in smoke. You may want to give thought to those factors and evaluate your backup plan with these things in mind.
The only data that I consider as high value is my KeePassXC database coz it contains a huge number of username/password belonging to different sites. So I use MEGAsync to keep an offsite backup of the KeePassXC database.
 
Old 05-19-2020, 08:52 AM   #8
taylorkh
Senior Member
 
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 2,127

Rep: Reputation: 174Reputation: 174
A key concept in backup strategy which I have followed (since my Osborne Executive running CP/M+ with 2 180 kB floppy drives) is to separate data from the operating system. On my current CentOS 7 workstation I have this layout:

/boot 512 MB
/ 12 GB encrypted for the OS, programs etc.
/var 6 GB encrypted
/home 4 GB encrypted
/data 400+ GB encrypted - the rest of the drive

My data partition is broken up as:

/data/data - this is for files which change frequently such as Office documents, Firefox and Thunderbird profiles etc.

/data/static - files which are important but which do not change such as bank statements, receipts from on-line purchases etc.

/data/reference - files such as downloaded catalogs, manuals etc. - could be re-
downloaded if lost

/data/_vmland - a number of virtual machines which run in VMWare

I keep only some configuration files and whatever is on my Desktop in my /home directory thus it is not very big (and I do not really need 4 GB)

My backup strategy is as follows:

On a monthly basis shut down the computer and do a Clonezilla image of /boot, /, /var and /home. This will let me restore the OS if (when) something, usually a point upgrade or installing a package, hoses things up. This has come in handy MANY times over the years. However, worse comes to worse I could simply restore the OS or move to new hardware.

On a nightly basis I use tar to create a backup of /home and /data/data. I keep 30 iterations of these backups so I can go back day by day for 30 days to look for a missing file.

I also keep a single, cumulative backup of /data/static and /data/reference. This and the incremental backups are done to a hard drive which is not normally powered up. This minimizes the attack surface for ransomware or such to get at my backups.

Also on a nightly basis I backup my virtual machine files, the same stuff as above and some files from other drives on the system to one of two external hard drives which I swap odd and even depending on the date. I happened to have these drives sitting around so I decided to use them.

On a monthly basis I copy /data/data /data/static and /data/reference to one of two flash drives which I again swap odd and even based on the month. These devices are stored "off site" - actually in my storage building at the back of the yard. I had also been burning a copy to a DVD and keeping a years worth of monthly snapshots. I am currently keeping the monthly's on a 64GB micro-SD card which is stored in my workshop building.

This helped me just last week when I noticed that my "Computer" bookmark folder and about 20 sub-folders had disappeared from Firefox. I had to go month by month back to January to find a copy. Obviously I do not access these bookmarks frequently or I would have discovered the loss earlier. I was able to extract the bookmarks and add them back to my "production" Firefox profile.

Finally on an annual basis I burn multiple DVDs of important data and store them in different places. I also copy the annual snapshot to a VeraCrypt conainer which allows for convenient access to annual snapshots going back to 1984.

Most of this is done with scripts so my involvement is limited to plugging and unplugging USB devices and powering on/off the backup hard drive.

Ken

p.s. And then there is the backup process for my 3 servers worth of movies, music, TV shows, books etc. and of course my porno collection
 
1 members found this post helpful.
Old 05-24-2020, 01:46 AM   #9
lewac
LQ Newbie
 
Registered: Apr 2010
Posts: 8

Rep: Reputation: 0
timeshift

Quote:
Originally Posted by hifi100 View Post
I am using Grsync with this command line

Code:
rsync -r -t -v --progress -c -s
I am using an 32GB USB flash drive as backup media.

I am running Grsync once a week.

Is this enough ? Am I doing backups correctly ?
well I use timeshift before every kernel update. I've not had to restore though. but my main backup is more complex. it goes to several 500gb, 2.5" HD's that are "hotshoe" mounted. I've written a bash script file that backs up ONLY the directories that I require. this creates ONE gigantic file (about 20gb) that can be restored to an empty directory using Engrampa. after that I go in there and look at or copy over what I need. quite often I have some code I wrote a few years back but can't remember what I did! but unfortunately I do not use what you're using. however a weekly backup is generally sufficient and its a LOT better than what most people do on a client box!
 
Old 05-24-2020, 01:35 PM   #10
JeremyBoden
Senior Member
 
Registered: Nov 2011
Location: London, UK
Distribution: Debian
Posts: 1,950

Rep: Reputation: 513Reputation: 513Reputation: 513Reputation: 513Reputation: 513Reputation: 513
It is a good idea to have a separate backup containing the backup/restore software required, in case it becomes discontinued.
If you just use things like gzip or rsync etc, these will always be available in all distro's.

http://backup2l.sourceforge.net/ meets these requirements whilst automating the use of these standard utilities.
Easy to install from Debian.
 
Old 05-27-2020, 06:59 AM   #11
taylorkh
Senior Member
 
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 2,127

Rep: Reputation: 174Reputation: 174
And then there is the question of hardware to read the backup media. I have rolled my historical backups forward to newer media over the years (decades). It is hard to find a 5 1/4" floppy drive these days.

A year or so back I had need to read my original version of Norton Commander from the original diskette. I had some 3 1/2" drives in my junk box just in case. I found the drive, and an old PC out in the storage building which had the necessary connection on the motherboard. But when I went looking for a cable... I had cleaned out my accumulation of obsolete cables etc. a while back. I did the Noah's Arc thing. I saved each type of cable two by two and disposed of the excess - I thought. However, I did not manage to save a couple of floppy cables. I purchased a USB connected 3 1/2" drive on evilbay for about the same as the original ribbon cable.

Ken
 
2 members found this post helpful.
Old 06-02-2020, 05:14 PM   #12
Hermani
Member
 
Registered: Apr 2018
Location: Delden, NL
Distribution: Ubuntu
Posts: 261
Blog Entries: 3

Rep: Reputation: 113Reputation: 113
Quote:
Originally Posted by taylorkh View Post
And then there is the question of hardware to read the backup media. I have rolled my historical backups forward to newer media over the years (decades). It is hard to find a 5 1/4" floppy drive these days.
The old 3-2-1 principle applies here. 3 copies, 2 different technologies and 1 of them should be off-site.

The key is to think ahead and still keep it simple. And even then you can get surprised.

For example: For the past 6 months I've been struggling with my data. I thought I had it done right: having a QNAP file server with some 9 TB of data, an older QNAP backup file server backing up about 5 TB of data and used for some misc data and finally the most important data on Blu-ray recordable.

Then my ISP suddenly blocked all internet traffic because of malware activity originating from my account! Turns out QNAP had malware problems and I had to find out how to get multiple TB of data off those servers using sneakernet, slow USB ports and terminal only. In the end I now have 2 servers both with 30 TB of storage using nothing but plain Ubuntu server, NFS networking, backup shell scripts I wrote myself and cron jobs controlling the lot.

What I learned was:
  1. Don't put your trust in the tools from one vendor only (don't put all your eggs in 1 basket)
  2. When shit hits the fan, it splatters all over the place
  3. Simple tools you understand are your best friend
  4. Write once media are golden

If you'd ask me I would recommend you add DVD+R backup to your current solution. Now you use the USB drive once a week and if it works for you, it is OK. If you backup your data e.g. every three months on DVD+R as well you could be just fine for the foreseeable future while not spending a lot of money. Please do use a case of some sort to store those DVDs as they need to be kept in a dry place, preferably not too hot.

Last edited by Hermani; 06-02-2020 at 05:20 PM.
 
1 members found this post helpful.
Old 06-02-2020, 09:28 PM   #13
JJJCR
Senior Member
 
Registered: Apr 2010
Posts: 2,288

Rep: Reputation: 466Reputation: 466Reputation: 466Reputation: 466Reputation: 466
@OP as Scasey suggested, to do a sanity check for your backup. Is to open those files and check whether those files or backup is really a backup and whether it will help when it's really needed. Of course, if it is corrupted and then i guess it's a sign that those backup cannot be trusted and you have to check.

I used to do rsync -av and so far those option has been good. You have to check, don't just follow blindly what is suggested.

Things to consider for your backup:

- Do you just want to mirror backup? means you just need the current data
- Do you need an incremental or differential backup? If yes, how long do you need the data? 1 month, 1 year? Cost storage to be considered.

Basically, you're the one who knows your data and just preempt yourself and ask, if really the data will be lost what would i need?

Good luck!

Last edited by JJJCR; 06-02-2020 at 09:44 PM. Reason: edit
 
1 members found this post helpful.
Old 06-03-2020, 06:03 AM   #14
JeremyBoden
Senior Member
 
Registered: Nov 2011
Location: London, UK
Distribution: Debian
Posts: 1,950

Rep: Reputation: 513Reputation: 513Reputation: 513Reputation: 513Reputation: 513Reputation: 513
Suggest you write down write down (on paper) full instructions on how to do the restore and put it in a safe place where it can never be lost.
Don't trust a computer!
 
1 members found this post helpful.
Old 06-03-2020, 03:59 PM   #15
ceantuco
Member
 
Registered: Mar 2008
Location: New York
Distribution: Debian
Posts: 809

Rep: Reputation: 88
Hi,

I use the following command for daily backups:

Code:
rsync -av --delete /mnt/personal /mnt/usbdrive/ | tee /bklogs/personal_$(date +"%Y%m%d").log

I use the following command for weekly full backups:

Code:
tar -cpvf /mnt/usbdrive/personal_full-$(date +"%Y%m%d").tar /mnt/personal/ | tee /bklogs/personal_full-$(date +"%Y%m%d").log
hope this helps!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
XSIbackup - do backups overwrite existing backups robertkwild Linux - General 3 11-14-2020 09:33 AM
Switching from dump tape directory backups to NAS disk backups on RHEL 6 Jerry_C Red Hat 0 06-05-2013 12:44 AM
Question on rsync script, not doing what it should be doing... JockVSJock Programming 5 01-03-2010 02:07 PM
Tar won't correctly restore from incremental backups therealbeale Linux - Software 1 08-31-2006 10:02 PM
Etrem not doing what it was doing? mrsolo Linux - Software 2 09-26-2004 12:32 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 12:28 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration