My backup solution is a custom bash script based around piping the results from find to cpio, so cpio it is.

I use shootsnap, with a restricted keyset and a large AOE array.

http://typinganimal.net/code/textify.php?f=shootsnap.sh

Filesize limitations with the bin, odc or newc file formats would make GNU cpio a poor choice for many. If you like cpio I would use bsdcpio (or heirloom cpio) with pax format or use afio.

You could also get GNU tar to use the pipe to read its file list, e.g. "tar --no-recursion -T- -cvf archive.tar", again with pax file format or even just gnutar file format. Neither have any serious limitations.

Pax is another option but make sure it is heirloom Pax as the pax util provided by most distros cannot actually make pax formatted archives, only ustar, which is really no better than newc.

1 members found this post helpful.

MAS (To use Tim Minchen's suggestion of "mildly amused smirk" rather than LOL ).

Thanks for the technical advice, but for the situation I have, backups of recently acquired files on a suite of Windows machines, I like cpio for the ability to manipulate path names. It has been working fine for years. I do not want to use a packed archive format, as what is required is to be able to read files directly from the Windows machines. It is also easier to demonstrate the recovery process to the occasional external auditor when our quality system is being assessed. The disaster recovery plan for these Windows machines (basically dedicated instrument controllers) is to restore a known good disk image, then restore any needed recently acquired files from backup. The file size issue will not occur due to the way these Windows machines are used.

I think what I am really trying to say is just the truism that the best choice of backup system heavily depends on what taking backups is trying to achieve.

You can do the same with numerous other utils, including those I mentioned (e.g. GNU tar has the --xform switch).

Fair enough but as computing changes and common file sizes increase you may find a time where you hit the individual file size limitations of the old cpio formats (bin = 2GB, odc = 8GB, newc = 4GB). I know that I personally have multimedia files and Linux distro ISO images in these size ranges lying around on my disks. afio extends the odc format (only for entries that need it) past these limits and pax has no real, practical limits right from the get go (9 EB file sizes are possible).

I'm not sure what you mean by this but the pax file format is the POSIX.1-2001 standard file format and an extension of tar. There are numerous utilities available on Windows that will open its contents just fine. Additionally as afio works with odc by default and only extends the header on entries that exceed its limitations its archives should also be readable using common Windows archiving tools, particularly as you state no files currently do exceed the limits.

I fail to see how this would be different if you used pax or afio.

Fair enough, I did not know your specific use case until you just stated it, so it could well have been an issue. In any case you can just take the information (assuming you were not already aware) as something to keep in mind for the future.

Sure, I agree with that. It was just a warning that (IMHO at least) cpio (the GNU implementation in particular) has run out of steam as it is no longer being actively developed to handle the types of files that are ever more common in the modern world. So it was just a heads up, so that you don't get bitten in the future. Like all advice on the internet you are free to just ignore it!

Should inclute tarsnap - defitely my choice.

Having a single backup isn't exactly the best idea for protecting your data. If one of your backup mediums gets damaged or destroyed, it's very difficult to restore from your backup.

I follow the 3-2-1 backup philosophy quite strongly: 3 copies, 2 mediums, 1 off-site.

I'm using CrashPlan for my off-site backup solution and Deja-Dup for my local backup. Having a local backup is immensely useful, since there are many times where you might need to restore your data from a backup but not have the time, patience, or bandwidth to restore from an online backup service; the online backup service comes in handy when something catastrophic occurs, such as a fire, theft, or natural disaster.

I also store most of my media files on a dedicated file server at my friend's house in another state (since he has a fast FiOS connection).

I agree about the value of a local backup. I was thinking about off-site backups when I mentiond tarsnap. I think it's good to have both.

I use rsnapshot as part of my automated plan. I also use grsync on regular basis for selective backups. FSArchiver is also quite useful.

tar

There are many good ones such as: Bacula, Amanda, rsync etc. but I will go for Legendary TAR

Git annex
http://git-annex.branchable.com/

CloneZilla

tar with bzip2

rsync