Question: rsync with dynamic directory determination?
Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Question: rsync with dynamic directory determination?
I am trying to figure out a way to initiate an rsync on LOCAL that connects to a rsync daemon on REMOTE. REMOTE is Linux, LOCAL is Linux initially, but I'm hoping to add some Windows machines as LOCAL later.
With the following functions/features:
The rsync daemon on REMOTE defines a module for "LOCAL" and within that module's directory structure, I want snapshot directories named like this:
These are not full backup directories - I will be using --link-dest to create hard links to previous backups instead. But each backup directory will APPEAR to be a full backup.
The specific directory names don't have to be exactly this, my point being that I want the snapshot directories named by date, not some arbitrary thing like Backup.1, Backup.2, Backup.3, etc. I prefer date-naming for less sophisticated users who might have trouble determining when an arbitrarily named "Backup.3" was actually created.
So I want to calculate --link-dest dynamically. 99% of the time the --link-dest will have a filename based on yesterdays date, but on some occasions maybe the backup didn't occur yesterday (an extended household power failure, etc.) So I want to dynamically figure out the most recent backup directory, whether that be one day ago, three days ago, a week ago, etc. If I were on the REMOTE machine (where rsync daemon runs), I would run a command like "ls -t Backup_* | head -1" to determine this most recent directory.
Is there a way I can dynamically determine --link-dest at run time by running a command on the destination (rsync daemon) machine? I know I could write a script on LOCAL to call ssh to REMOTE before calling rsync, and have that ssh command determine the --link-dest. But I was hoping to do it all within the rsync command itself. (Reason: eventually I hope to port this to Windows too, and I'd have to create a functionally equivalent Windows script to determine --link-dest ... and I hate scripting in Windows!)
An alternative to this would be to always create a symlink to the latest backup dir (the destination dir on REMOTE) and name that symlink "Backup_current". Then I could use --link-dest=Backup_current (I am assuming that --link-dest will follow a symlink, but I haven't tried that yet to verify). So in this alternate strategy, the question becomes, "Is there a way to have rsync daemon automatically update this symlink after completing a backup?" Again, I could write a script to do this on LOCAL, calling ssh to REMOTE after the rsync has competed. But I hope to do this all in a single rsync command. I could kludge this by running a cron job on REMOTE that asynchronously creates the symlink (coordinating that so the symlink isn't changed in the middle of a running backup, although that would probably be OK - would have to test for dure).
My description above is about as clear as mud. But if anybody actually understands what it is I'm trying to do, I'd love to hear any suggestions or alternate approaches. Thanks!
Where localhost is the snapshot of the local server and SCA is the snapshot from the remote server "pulled" over SSH.
This doesn't provide the directory naming you were asking for, but might be a serviceable given the timestamps on the directories.
It is certainly very easy to set up and configure.
Thanks - I am investigating a couple different backup solutions to suppliment what I'm currently doing. One, something I'd craft myself using rsync. Two, something like you mentioned, rsnapshot. Three, a more customizeable client/server backup solution like UrBackup.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.