Utilizing rsync to backup data but symbolic links to included
Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Utilizing rsync to backup data but symbolic links to included
Please forgive me but I'm a little new to Red Hat (RHEL 5). I'm using rysnc to backup critical data and to a second disk; here is what I'm typing at the command line rsync -rvgal /data/disk1/share /data/backup/share. It appears that the softlinks are not transfered to the backup drive and some of the links point to data not located in the source folder (/data/share). After reading the rsync man page I was a little confused about the L option (vs the l option). In order to ensure that the linked files are moved should I type the below:
Using -L means: "transform symlink into referent file/dir". In my opinion, this means that it copies the directories/files that those links refer to, and NOT the symlinks themselves, that's why I'd suggest you use -l (lower-case): "copy symlinks as symlinks".
If the issue relates only to dirlinks, you can try using upper-case -K (--keep-dirlinks), but first try out -l.
This will transfer the files in archive mode and ensure that symbolic links, devices, permissions and ownerships, modification times, ACLs and extended attributes are preserved.
Guys, he's already running with -a, which implies -g, -r, -l and a bunch of others. He's already doing what you're telling him, that's the problem, it's copying the symlinks as symlinks, but they point to targets that are outside of the backup directory.
OP - this is exactly how an archive SHOULD behave. If this is a critical backup, then why are these critical files not in the backup directory? Where are they? Should you be backing up that directory as well? You could remove -a and set the flags as needed to copy the target instead of the symlink, but this can get real messy real fast (think infinite recursion). For a true backup you should re-think what directories you're copying and why if you don't want to bring symlinks over as symlinks.
Last edited by suicidaleggroll; 01-16-2015 at 03:22 PM.
Guys, he's already running with -a, which implies -g, -r, -l and a bunch of others. He's already doing what you're telling him, that's the problem, it's copying the symlinks as symlinks, but they point to targets that are outside of the backup directory.
OP - this is exactly how an archive SHOULD behave. If this is a critical backup, then why are these critical files not in the backup directory? Where are they? Should you be backing up that directory as well? You could remove -a and set the flags as needed to copy the target instead of the symlink, but this can get real messy real fast (think infinite recursion). For a true backup you should re-think what directories you're copying and why if you don't want to bring symlinks over as symlinks.
If -a implies -l, then how does that agree with -L? -L and -l exclude one another, right?
If -a implies -l, then how does that agree with -L?
It doesn't
Quote:
Originally Posted by vincix
-L and -l exclude one another, right?
Yes, I'm not sure where the confusion is.
He's using -a, which implies -l. This means any symlinks in the source are being copied as-is to the destination. This also matches his description of the "problem". His symlinks are being copied to the destination, but the destination does not have the targets where those symlinks are pointing (because the targets are not in the directory being backed up by rsync, so they're not being copied to the destination), so he's just left with a bunch of broken links on the destination.
Instead of copying the symlinks as-is, he wants to copy the targets that are being pointed to. This is not a good backup strategy for multiple reasons, but if he really wants to do that he needs to get rid of -a and use -L and whatever other flags he wants. The better solution is to modify the backup strategy to include the actual location of all critical files. If this is supposed to be a critical backup, but the directory being backed up apparently doesn't contain many of the critical files, then something is wrong. Where are those files? Is anything else missing? Why is their actual location not being included in the backup?
This is what the OP wrote. So it's both -a and -L. That's all I was saying. So after I explained the difference between -L and -l you simply said that I didn't help him. That's the whole point.
On another train of thoughts, why do links break? Shouldn't they have broken only if they referred to relative paths?
That's just the command he asked if he should be running. The one he actually ran, and was having problems with, was "rsync -rvgal", the same thing you told him to run.
Quote:
I'm using rysnc to backup critical data and to a second disk; here is what I'm typing at the command line rsync -rvgal /data/disk1/share /data/backup/share. It appears that the softlinks are not transfered to the backup drive and some of the links point to data not located in the source folder (/data/share).
-aL would be invalid as you pointed out. I imagine rsync would throw an error if you tried it, not sure.
Yes they're probably relative links, either that or they break as soon as the backup drive is removed from the computer and plugged in elsewhere and the absolute path no longer points to anything useful.
Last edited by suicidaleggroll; 01-17-2015 at 08:39 AM.
-aL would be invalid as you pointed out. I imagine rsync would throw an error if you tried it, not sure.
No, the capital L would override the small l from "a". Nice trick to un-switch one option from the "a" option-collection without needing to remember them all.
To the OP: But this is for sure not what you want for backup because you cannot restore it to the same state anyway. You want "rsync -aAX" as already mentioned. And backup the important link targets from their original location.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.