Backup solution using cron and tar
 

Hello all,

I thought I would share my practice on backups. Initially, I was using the backup GUI with Fedora. I then decided I wanted something automated that would run everyday by cron.

I have created a couple of scripts to do the backup.

My general thoughts are

• Use tools that are available on a standard install of my operating system.
• Do a full backup once a week
• Do incremental backups every other day
• Do an offsite backup once a week

The first thing I do is mount another machine. I do this through a cron job that runs every minute. I mount a cifs/samba mount to a Windows XP machine with a big hard drive.
The script, (a kind of long script, but it works) automatically creates a new directory for today's backup and removes directories from 8, 9, and 10 days ago. That way I keep a week of rotating directories and if the machine is off for a couple of days, it blows away the old backups.

The script then runs the tar operations concurrently, and waits until all of the tar operations are done to terminate the script.

I welcome any suggestions for improvements, and let me know if this helps jumpstart your own backup strategy.

I had to utilize this once when I corrupted the filesystem last week when extending my swap partition. It made it relatively easy to restore.

- Raj

That's way too much. You should loop over account names to process, then over the source directories to backup and then loop over dates ("for ((i=0;i<10;i++)); do doSomething; done"). No need to use this much variables. BTW incrementals (and why a 10 day cycle?) use "--newer" AFAIK, you could keep a copy of the weeks tarball as a set of 4 and erase the last one when you do monthly backups. Also have a look at rsync if you've got a spare server or external hard disk around.

If you want to use an archive rather than rsync to ensure that all Linux filesystem properties are retained when stored on Windows, then you might want to reconsider gzip compressed tars because a single corrupt bit near the beginning of the archive means the rest of the file is a write off. This is less of an issue when using an actual disk for backup as opposed to media like DVDs, Blu-ray, etc. but still something to consider. Personally I would either skip compression or use xar, dar or afio instead, all of which can compress files individually as they are added (afio gives you the most compression options, since you can specify any compressor you like). This is safer as any corruptions will mean only losing some of you files. Alternatively (or better yet in addition) look at making parity archive volume sets. Check out the par2cmdline utils, an implementation of PAR v2.0 specification.

Initially I chose tar with compression because I could open up the file on Windows XP with Winzip to inspect the backups.

I will look into rsync and another compression option to see what makes sense.

Thanks for your feedback.

UnSpawn, I was planning on doing some kind of looping to take care of the backups and users and I haven't incorporated this as of yet. Thanks for the ideas.

- Raj

If you want something you can easily inspect on Windows I would go for Afio. By default it makes POSIX compliant cpio archives and only uses it own extensions to the format for any files it adds which push the limitations of that format. For example, if you were to add a multi-GiB file it would likely store that entry with an afio specific "large ASCII" header. This means that most (perhaps all) of the files within your backup will be accessible with any utility that can read POSIX cpio archives, of which there are many on windows (I'd use 7zip). Even the fact that internal compression is used is not a problem. Just extract the file first and then decompress it. If you need strong compression due to space limitations I would go for XZ compression internally. This will likely give you the smallest possible archive of any of the popular compression tools with excellent decompression performance. Alternatively you might want to consider bzip2. The archive won't be as small but it is supported by a wider range of Windows tools. Additionally on Linux it is one of the few common compressors that provides recovery tools. I'd skip gzip altogether, since its major feature these days is speed. However if that was my number one criteria I'd use lzop. It has comparable compression but is much faster, albeit with the downside of very few other tools supporting it. Finally do consider skipping compression altogether if you have the space. It is the lowest risk, and makes for fast backups that are very easy to open.