Continuous / On-change backup of a directory?
 

What is the best way to monitor and back up a directory (and its sub directories) based on changes? Fam and inotify (and Gamin?) are often mentioned in this context, but they provide programming interfaces. I'm rather looking for an application software (that would likely utilize one of those APIs) that I could simply set up as a daemon and point to a directory which to monitor and to duplicate to a target location. It would also preferably have versioning (i.e. e.g. "keep the 25 most recent copies of each file").

On Windows side apps such as ViceVersa with its associated service, VVEngine do the same... but I need similar functionality on my Linux dev server (running CentOS 5.3) and so the Windows apps are of little use there. :)

Thanks for any insights/advise on this!

Solaris ZFS is probably the best tool to do that but Ubuntu TimeVault and some other backup system can also work.

I've been looking into this more. inotify-based iWatch is the most promising setup I've come across thus far. It has the option to execute a command when a change occurs and it provides the command full path to the changed file as an argument. It should be fairly easy to script a versioned copy with that information even as a shell script, and certainly with Python or Perl. Or, perhaps a versioned copy utility exists already (haven't researched it yet) which the output from iWatch could be piped to.

It is what TimeVault and ZFS (mentionned in my earlier post) does. But perhaps you are for a SVN server trigered by your iWatch?

My understanding of TimeVault is that it takes "snapshots" of the filesystem; it might be too "heavy duty" for the intended use. Basically what I'm thinking about is to have a working copy checked out of SVN in a directory that would be monitored by inotify. That way while files are be worked on, several historical copies would be maintained in a separate directory (outside of the development area) so that when, say, I mess up the code I'm currently working on I could step backwards a save or two with little effort simply by picking up a slightly older copy from the backup folder. Commits to SVN would only happen when the code is actually working. With TimeVault my concern here is that obtaining copy of a file that was saved few minutes ago might be too much effort. The versioning for currently worked on items I'm looking for here is pretty casual, and it's mainly intended to prevent situations such as where one spends an hour hunting an inadvertendly introduced extra quote in a large JavaScript file. Many IDEs such as Zend Studio or even UEStudio of course provide "keep x previous copies" which might suffice.. except that I would like to keep the versioning tool-independent.

However, your suggestion for automatic commits to SVN based on an iWatch trigger might do the trick! It would certainly take care of versioning; the in-progress code could be always stored in a fork as not to pollute the trunk. In the past I had a reverse setup where I did a commit to SVN (or at that time to CVS) on every change/save, and a script then updated a working copy in the web directory. But that was a pain to work with as there was always a 4-6 second delay before the change would be reflected in the web folder and so in the browser refresh. If commits to SVN would be triggered by an iWatch event, it would occur independently from server side debugging (I'll use this this with zend server) and browser update. The only consideration would be configuration files and such which I would ultimately not want to commit to the trunk, but perhaps they could be pruned during the fork merge process.

I'll give this a try, and update this thread later. More ideas/thoughts are of course welcome! :)

There's a small package called inotify-tools that can be used in command line. Example:

Thanks! Also incron (from inotify folks) might work as the svn commit trigger. Need to test these alternatives this week.