Good luck with your hardware issues.

When you come back, perhaps we should discuss how you would use a backup system such as you have described, with a full backup that is offline and remote, and incremental backups that are local, internal, and immediately accessible.

I am concerned because the ease of recovery of a file depends on its history. With the local incremental backup, there are two kinds of files you can recover immediately: files that were created after the full backup, and new versions of files that were modified after the full backup. If you need to recover a file which was deleted, or an old version of a file that was modified, you have to wait until the remote backup is put online or physically brought in. Is that inconvenience and delay acceptable to you?

There is also a question of how to merge the incremental and full backups when you periodically bring in the full backup. The easy part is adding files which have been created or modified. It's a little harder to figure out what to delete from the full backup when files have been deleted, renamed, or moved. Rather than try to sort out the incremental backups, it might be easier and more reliable to abandon the incremental backups at that point. Simply rsync the current state of the system to the full backup, then delete the old incrementals and start the incremental cycle over again.

But then I shouldn't be telling you how to use your own backup system. How do you envision using it?

It was a strange set of problems, but after the initial disk died, I got a little worried when the next one showed errors and then another. Testing the sata ports and cables seemed conclusive, but it wasn't. It turned out that with summer coming here the fault was an overheating CPU. It certainly explained the hanging, but never would have guessed buffer IO errors. A little later after some TLC most of the hardware problems resolved. Just some weird issues with the new drive not coming online after suspend / resume cycle.

With regards to the backup ideally I would like to be able to restore the full backup and then restore the latest local backup. The ideal format for the local backup would be a snapshot structure where each incremental backup is hard linked to the previous, which would leave me with a single snapshot to restore after the full backup. Running rsync over the restore of the full backup I would hope would take care of all the versions and get me back to the last place a snapshot was made. If I needed to recover work from a previous day I could use the local snapshot for this.

The reasoning behind this is that I have a large amount of data that is already backed up twice and I do not have the capacity (or ports) to run another full backup locally, yet I want something automated between full backups.