LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 10-03-2021, 05:26 PM   #1
kieppie
LQ Newbie
 
Registered: Oct 2021
Posts: 2

Rep: Reputation: Disabled
A better backup solution


Hi folks,
(misplaced my earlier login, so starting anew)

Been using various flavours of Linux for ages, primarily server-side, but coming back to the desktop for a primary workhorse after a significant hiatus.

Rolling with Fedora (34) this time, as I find it better suited to BaU work more than 'fun', which is what I need.

Something that I've noticed is that there doesn't seem to be a good backup mechanism baked into the OS, and this is quite the departure from the W10 mechanisms provided through 'File History' & OneDrive. Though I'm not a big fan of pushing sensitive data to a cloud I don't own, manage or control, I've experienced first-hand how this has pulled critical projects form the fire, and the benefits in the way this facilitates something of a seamless experience across desktops is not trivial.

My setups are:
  • Desktops - Fedora, Ubuntu, etc; each for different purposes; 90% of endpoints are W10p64 desktops, because that what users/clients/family use
  • Servers - baremetal, VM/container hosts & cloud
  • IoT/embedded, mobiles
  • NAS: FreeNAS/TrueNAS, Synology (paired redundency) & offsite backup; running ZFS & btrfs RAID-5's, making daily snapshots so that I can roll back to an arbitrary day in the last month/year. These hosts also pull by online data via rclone.
  • Simple but segmented network with VPN's; no VLAN's or AD/DS

What is really valuable is the way that W10 naively integrates this: users' data backs up to the NAS on the smb://home share, and they can simply right-click on a file/dir of their choice, which correspond to the daily snapshots & "Recover a previous version" with little or no help or intervention on my part.
Because these are server-side generated snapshots, these same snapshots are available to my various POSIX machines over protocol of choice - NFS, iSCSI, SSHFS, etc - but this functionality is not native to the OS AFAICT.

Historically I've managed backups via rsync/clone - typically pulling data (starting with /home & /etc) so that I don't have to install or configure an agent on every client/endpoint - but this is inelegant, and does not effectively use the resources at disposal and introduces complexity that could otherwise be avoided.

This is my ask: what is the more elegant & efficient backup solution?

But I have a few considerations I'm taking into account:
  • needs to be Open Source & (near-)native: in the stable repo's and/or in Snap/Flatpak stores.
  • Support for FS-based snapshots (btrfs, ZFS, LVM) as well as backing/syncing up that (incremental) data to the network. (if/when my disk dies, my backups die with it unless archived elsewhere)
  • needs to be active/current - i.e. git repo still a project under active development/maintenance/activity in the last ~year, not stale
  • needs a usable/functional GUI - desktop but web or TUI/ng preferable. I've set this as an arbitrary bar for where I invest time & effort, as I've found this to be a fair indicator of a user-interacting app stack; especially if/when I don't want to do everything for everyone.

Some of of the solutions I've found may be OK fits, but have some shortcomings:
  • rclone(baseline reference)
  • preferred over rsnapshot & rdiff-backup
  • Duplicati (solid choice, but no FS snapshot)
  • Snapper (fair candidate)
  • Back In Time (stale?)
  • TimeShift (doesn't seem to support networking)
  • DejaDup(not something I would consider 'elegant')
  • based on Duplicity (no FS snapshotting)
  • Bacula ('open core', which just sits weird with me...)
  • Kopia (intriguing...)
  • BackupPC (no FS snapshotting AFAICT)

I know this is a tall order, but if I'm going to invest time & effort into adopting a strategy or solution more widely, I'd prefer it to meet my needs, so that ongoing deployment & use is somewhat uniform, regular & predictable and not needing to contend with numerous exceptions & edge-cases.

I'd appreciate folks insights & comments re this, please.
 
Old 10-04-2021, 01:13 PM   #2
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,784

Rep: Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214
My choice without question is rdiff-backup now that it again in active development (The need to port to Python 3 was the stimulus for that.). It combines a complete mirror of the source with a set of reverse diffs that permit recovery of files from any previous backup. About the only major disadvantages are:
  1. You can't delete intermediate backups (e.g., you want to delete the daily increments and keep only monthly backups for dates more than a year in the past).
  2. Fully verifying a complete backup archive and all of its increments is a long and diffcult process that is both compute and I/O intensive. (Verifying the checksums on the current backup, or for any specific date, is straightforward.)
(FWIW, I manage #1 by keeping a separate monthly backup, and simply deleting anything more than a year old from the daily backup.)
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Meet Proxmox Backup Server, a Debian-Based Open Source Enterprise Backup Solution LXer Syndicated Linux News 0 11-13-2020 04:28 AM
Remote Backup script needed | backup every 1-2 seconds (Online/Hot backup) reda Linux - Newbie 4 04-20-2019 05:02 PM
Backup tools comparison. Choosing a backup solution for various purposes zomane Linux - Server 4 01-29-2019 12:30 AM
[SOLVED] Backup, shrink backup and modify MBR of backup jps1x2 Linux - General 1 12-17-2013 05:03 AM
better drivers for better speed? lukeprog Linux - Hardware 1 03-23-2006 06:24 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 01:04 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration