Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place! |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
|
05-14-2020, 11:37 PM
|
#1
|
Member
Registered: Sep 2016
Location: India
Distribution: Arch Linux
Posts: 357
Rep: 
|
Am I doing backups correctly ?
I am using Grsync with this command line
Code:
rsync -r -t -v --progress -c -s
I am using an 32GB USB flash drive as backup media.
I am running Grsync once a week.
Is this enough ? Am I doing backups correctly ?
|
|
|
05-14-2020, 11:55 PM
|
#2
|
LQ Veteran
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: Rocky 9.5
Posts: 5,876
|
The ultimate test of a backup is to recover from it.
That’s not necessarily easy to test...but it pretty much defines “correctly.”
That said, we need a bit more information...what are you backing up? How often does the data you’re backing up change?
|
|
1 members found this post helpful.
|
05-15-2020, 02:47 AM
|
#3
|
Member
Registered: Aug 2016
Distribution: Mint 19.1
Posts: 419
Rep: 
|
You are the only one who can answer your question. Whether you want to do it by directly copying files or uses a 'system' is up to you. Personally I use rsnapshot which uses rsync and tends to be quite fast after the first time. How often to back up is again up to you. The question to ask yourself is 'how much of your 'stuff' are are you willing to lose (i.e. replace) and is any of it irreplacable?' Some people with critical files backup hourly or more often. Others back up monthly or so. Another consideration is how many backups do you keep? When you back up are you overwriting your last one? If so, what happens if everything goes bad in the middle of one? Usually people will at least alternate so they always have a good one to fall back on. Many keep more than one. It depends on their situation and comfort level. As scasey pointed another major consideration is how difficult will restoration be? Also think about what you are backing up. Do you really want or need to back up everything? Do you really want to spend the time, energy and media space to back up programs you can easily replace? Remember a program is normally not one file but may have modules and configuration files scattered all over the back forty and won't run if you miss just one. Very few programs are not easily reinstalled from repositories or other sources so you need to save them unless you really want to. Data, on the other hand, can be very important and irreplaceable, but not all data is created equal. Do you really need to back up last week's grocery list? What about yesterday's checkbook register, or last hour's for that matter. Once you have answered those, why are you backing up? Do you need to keep a copy offsite or even in a safety deposit box? One thing that may help is to make yourself a detailed checklist of what you would need to do to do a bare metal recovery. That would begin to give you some idea of what you need to have on hand and what you can always get elsewhere. Also remember Murphy. I don't care how careful you are there will always be something you forget about or think is critical and isn't.
These are questions no one else can answer for you. I know it seems like a lot but learning to think this way will save you time, frustration and money in the long run. We will all be glad to give you our opinions, suggestions, and tell you what we do but as scasey said, you need to give us some guidance about what and why, too.
|
|
1 members found this post helpful.
|
05-15-2020, 04:24 AM
|
#4
|
LQ Guru
Registered: Sep 2011
Location: Upper Hale, Surrey/Hants Border, UK
Distribution: One main distro, & some smaller ones casually.
Posts: 5,823
Rep: 
|
The average user just needs to keep a back up of personal files, how you back up is immaterial, as long as it can be read back, & put on your disk after a disaster.
(If you are running a server, you should do the child, parent, grandparent type of back ups.)
Pendrives are OK for short term back ups, but I'd use an external HDD myself. 
|
|
2 members found this post helpful.
|
05-19-2020, 05:06 AM
|
#5
|
Member
Registered: Sep 2016
Location: India
Distribution: Arch Linux
Posts: 357
Original Poster
Rep: 
|
Thanks everyone for the replies.
Quote:
we need a bit more information...what are you backing up? How often does the data you’re backing up change?
|
I am backing up all of my data which includes audio, video, Libreoffice docs, KeePassXC database, text documents and Veracrypt containers. Leaving the KeePassXC database the other files doesn't change much.
Quote:
Another consideration is how many backups do you keep? When you back up are you overwriting your last one?
|
At the moment I am keeping only 1 backup. I am using "incremental" backup (rsync) which as you know only changes those files which have changed.
Quote:
Pendrives are OK for short term back ups, but I'd use an external HDD myself
|
I had purchased a Western Digital My Passports 500GB drive some years back. It failed after just 6 months of usage. A)I don't have the cash right now to buy a new external drive & B)I will never buy another spinning HDD again. When I am financially ready I will buy an external SSD.
Last edited by hifi100; 05-19-2020 at 05:09 AM.
|
|
|
05-19-2020, 06:45 AM
|
#6
|
LQ Guru
Registered: Apr 2010
Location: Continental USA
Distribution: Debian, Ubuntu, RedHat, DSL, Puppy, CentOS, Knoppix, Mint-DE, Sparky, VSIDO, tinycore, Q4OS, Manjaro
Posts: 6,117
|
First, KUDOS to you for doing backups at ALL! Far too many people just never do.
Second, it looks to me as if you are backing up only data and certain settings. That may serve you well. It is often better to install a system correctly, then recover data than to attempt to recover a full system image on new and posibly incompatible) hardware.
Third: a real backup allows for multi-generational backups. Think about the case where malware encrypts or corrupts your data just prior to you running the backup: could you go back to a previous (uncorrupted) copy?
If the data value is high (or the risk to you of losing it high enough) there is value in having a copy of your data backup off site. This covers the case where you have a fire or flood and your local devices drown or go up in smoke. You may want to give thought to those factors and evaluate your backup plan with these things in mind.
|
|
1 members found this post helpful.
|
05-19-2020, 08:13 AM
|
#7
|
Member
Registered: Sep 2016
Location: India
Distribution: Arch Linux
Posts: 357
Original Poster
Rep: 
|
Quote:
Originally Posted by wpeckham
If the data value is high (or the risk to you of losing it high enough) there is value in having a copy of your data backup off site. This covers the case where you have a fire or flood and your local devices drown or go up in smoke. You may want to give thought to those factors and evaluate your backup plan with these things in mind.
|
The only data that I consider as high value is my KeePassXC database coz it contains a huge number of username/password belonging to different sites. So I use MEGAsync to keep an offsite backup of the KeePassXC database.
|
|
|
05-19-2020, 08:52 AM
|
#8
|
Senior Member
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 2,127
Rep: 
|
A key concept in backup strategy which I have followed (since my Osborne Executive running CP/M+ with 2 180 kB floppy drives) is to separate data from the operating system. On my current CentOS 7 workstation I have this layout:
/boot 512 MB
/ 12 GB encrypted for the OS, programs etc.
/var 6 GB encrypted
/home 4 GB encrypted
/data 400+ GB encrypted - the rest of the drive
My data partition is broken up as:
/data/data - this is for files which change frequently such as Office documents, Firefox and Thunderbird profiles etc.
/data/static - files which are important but which do not change such as bank statements, receipts from on-line purchases etc.
/data/reference - files such as downloaded catalogs, manuals etc. - could be re-
downloaded if lost
/data/_vmland - a number of virtual machines which run in VMWare
I keep only some configuration files and whatever is on my Desktop in my /home directory thus it is not very big (and I do not really need 4 GB)
My backup strategy is as follows:
On a monthly basis shut down the computer and do a Clonezilla image of /boot, /, /var and /home. This will let me restore the OS if (when) something, usually a point upgrade or installing a package, hoses things up. This has come in handy MANY times over the years. However, worse comes to worse I could simply restore the OS or move to new hardware.
On a nightly basis I use tar to create a backup of /home and /data/data. I keep 30 iterations of these backups so I can go back day by day for 30 days to look for a missing file.
I also keep a single, cumulative backup of /data/static and /data/reference. This and the incremental backups are done to a hard drive which is not normally powered up. This minimizes the attack surface for ransomware or such to get at my backups.
Also on a nightly basis I backup my virtual machine files, the same stuff as above and some files from other drives on the system to one of two external hard drives which I swap odd and even depending on the date. I happened to have these drives sitting around so I decided to use them.
On a monthly basis I copy /data/data /data/static and /data/reference to one of two flash drives which I again swap odd and even based on the month. These devices are stored "off site" - actually in my storage building at the back of the yard. I had also been burning a copy to a DVD and keeping a years worth of monthly snapshots. I am currently keeping the monthly's on a 64GB micro-SD card which is stored in my workshop building.
This helped me just last week when I noticed that my "Computer" bookmark folder and about 20 sub-folders had disappeared from Firefox. I had to go month by month back to January to find a copy. Obviously I do not access these bookmarks frequently or I would have discovered the loss earlier. I was able to extract the bookmarks and add them back to my "production" Firefox profile.
Finally on an annual basis I burn multiple DVDs of important data and store them in different places. I also copy the annual snapshot to a VeraCrypt conainer which allows for convenient access to annual snapshots going back to 1984.
Most of this is done with scripts so my involvement is limited to plugging and unplugging USB devices and powering on/off the backup hard drive.
Ken
p.s. And then there is the backup process for my 3 servers worth of movies, music, TV shows, books etc.  and of course my porno collection 
|
|
1 members found this post helpful.
|
05-24-2020, 01:46 AM
|
#9
|
LQ Newbie
Registered: Apr 2010
Posts: 8
Rep:
|
timeshift
Quote:
Originally Posted by hifi100
I am using Grsync with this command line
Code:
rsync -r -t -v --progress -c -s
I am using an 32GB USB flash drive as backup media.
I am running Grsync once a week.
Is this enough ? Am I doing backups correctly ?
|
well I use timeshift before every kernel update. I've not had to restore though. but my main backup is more complex. it goes to several 500gb, 2.5" HD's that are "hotshoe" mounted. I've written a bash script file that backs up ONLY the directories that I require. this creates ONE gigantic file (about 20gb) that can be restored to an empty directory using Engrampa. after that I go in there and look at or copy over what I need. quite often I have some code I wrote a few years back but can't remember what I did! but unfortunately I do not use what you're using. however a weekly backup is generally sufficient and its a LOT better than what most people do on a client box!
|
|
|
05-24-2020, 01:35 PM
|
#10
|
Senior Member
Registered: Nov 2011
Location: London, UK
Distribution: Debian
Posts: 1,950
|
It is a good idea to have a separate backup containing the backup/restore software required, in case it becomes discontinued.
If you just use things like gzip or rsync etc, these will always be available in all distro's.
http://backup2l.sourceforge.net/ meets these requirements whilst automating the use of these standard utilities.
Easy to install from Debian.
|
|
|
05-27-2020, 06:59 AM
|
#11
|
Senior Member
Registered: Jul 2006
Location: North Carolina
Distribution: CentOS 6, CentOS 7 (with Mate), Ubuntu 16.04 Mate
Posts: 2,127
Rep: 
|
And then there is the question of hardware to read the backup media. I have rolled my historical backups forward to newer media over the years (decades). It is hard to find a 5 1/4" floppy drive these days.
A year or so back I had need to read my original version of Norton Commander from the original diskette. I had some 3 1/2" drives in my junk box just in case. I found the drive, and an old PC out in the storage building which had the necessary connection on the motherboard. But when I went looking for a cable... I had cleaned out my accumulation of obsolete cables etc. a while back. I did the Noah's Arc thing. I saved each type of cable two by two and disposed of the excess - I thought. However, I did not manage to save a couple of floppy cables. I purchased a USB connected 3 1/2" drive on evilbay for about the same as the original ribbon cable.
Ken
|
|
2 members found this post helpful.
|
06-02-2020, 05:14 PM
|
#12
|
Member
Registered: Apr 2018
Location: Delden, NL
Distribution: Ubuntu
Posts: 261
Rep: 
|
Quote:
Originally Posted by taylorkh
And then there is the question of hardware to read the backup media. I have rolled my historical backups forward to newer media over the years (decades). It is hard to find a 5 1/4" floppy drive these days.
|
The old 3-2-1 principle applies here. 3 copies, 2 different technologies and 1 of them should be off-site.
The key is to think ahead and still keep it simple. And even then you can get surprised.
For example: For the past 6 months I've been struggling with my data. I thought I had it done right: having a QNAP file server with some 9 TB of data, an older QNAP backup file server backing up about 5 TB of data and used for some misc data and finally the most important data on Blu-ray recordable.
Then my ISP suddenly blocked all internet traffic because of malware activity originating from my account! Turns out QNAP had malware problems and I had to find out how to get multiple TB of data off those servers using sneakernet, slow USB ports and terminal only. In the end I now have 2 servers both with 30 TB of storage using nothing but plain Ubuntu server, NFS networking, backup shell scripts I wrote myself and cron jobs controlling the lot.
What I learned was: - Don't put your trust in the tools from one vendor only (don't put all your eggs in 1 basket)
- When shit hits the fan, it splatters all over the place
- Simple tools you understand are your best friend
- Write once media are golden
If you'd ask me I would recommend you add DVD+R backup to your current solution. Now you use the USB drive once a week and if it works for you, it is OK. If you backup your data e.g. every three months on DVD+R as well you could be just fine for the foreseeable future while not spending a lot of money. Please do use a case of some sort to store those DVDs as they need to be kept in a dry place, preferably not too hot.
Last edited by Hermani; 06-02-2020 at 05:20 PM.
|
|
1 members found this post helpful.
|
06-02-2020, 09:28 PM
|
#13
|
Senior Member
Registered: Apr 2010
Posts: 2,288
|
@OP as Scasey suggested, to do a sanity check for your backup. Is to open those files and check whether those files or backup is really a backup and whether it will help when it's really needed. Of course, if it is corrupted and then i guess it's a sign that those backup cannot be trusted and you have to check.
I used to do rsync -av and so far those option has been good. You have to check, don't just follow blindly what is suggested.
Things to consider for your backup:
- Do you just want to mirror backup? means you just need the current data
- Do you need an incremental or differential backup? If yes, how long do you need the data? 1 month, 1 year? Cost storage to be considered.
Basically, you're the one who knows your data and just preempt yourself and ask, if really the data will be lost what would i need?
Good luck!
Last edited by JJJCR; 06-02-2020 at 09:44 PM.
Reason: edit
|
|
1 members found this post helpful.
|
06-03-2020, 06:03 AM
|
#14
|
Senior Member
Registered: Nov 2011
Location: London, UK
Distribution: Debian
Posts: 1,950
|
Suggest you write down write down (on paper) full instructions on how to do the restore and put it in a safe place where it can never be lost.
Don't trust a computer!
|
|
1 members found this post helpful.
|
06-03-2020, 03:59 PM
|
#15
|
Member
Registered: Mar 2008
Location: New York
Distribution: Debian
Posts: 809
Rep:
|
Hi,
I use the following command for daily backups:
Code:
rsync -av --delete /mnt/personal /mnt/usbdrive/ | tee /bklogs/personal_$(date +"%Y%m%d").log
I use the following command for weekly full backups:
Code:
tar -cpvf /mnt/usbdrive/personal_full-$(date +"%Y%m%d").tar /mnt/personal/ | tee /bklogs/personal_full-$(date +"%Y%m%d").log
hope this helps!
|
|
|
All times are GMT -5. The time now is 12:28 AM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|