LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-16-2015, 07:26 AM   #1
johnmccarthy
Member
 
Registered: Jul 2010
Posts: 64

Rep: Reputation: 1
Smile Utilizing rsync to backup data but symbolic links to included


Please forgive me but I'm a little new to Red Hat (RHEL 5). I'm using rysnc to backup critical data and to a second disk; here is what I'm typing at the command line rsync -rvgal /data/disk1/share /data/backup/share. It appears that the softlinks are not transfered to the backup drive and some of the links point to data not located in the source folder (/data/share). After reading the rsync man page I was a little confused about the L option (vs the l option). In order to ensure that the linked files are moved should I type the below:

rsync -rvgaL /data/disk1/share /data/backup/share

A million thanks,
Johnny Mac
 
Old 01-16-2015, 08:18 AM   #2
vincix
Member
 
Registered: Feb 2011
Distribution: Centos 6.7, 7
Posts: 586

Rep: Reputation: 52
Using -L means: "transform symlink into referent file/dir". In my opinion, this means that it copies the directories/files that those links refer to, and NOT the symlinks themselves, that's why I'd suggest you use -l (lower-case): "copy symlinks as symlinks".


If the issue relates only to dirlinks, you can try using upper-case -K (--keep-dirlinks), but first try out -l.

Last edited by vincix; 01-16-2015 at 08:20 AM.
 
Old 01-16-2015, 03:14 PM   #3
Head_on_a_Stick
Senior Member
 
Registered: Dec 2014
Location: London, England
Distribution: Arch & Debian
Posts: 1,183

Rep: Reputation: 283Reputation: 283Reputation: 283
For system backups, use:
Code:
rsync -aAXv $source $target
This will transfer the files in archive mode and ensure that symbolic links, devices, permissions and ownerships, modification times, ACLs and extended attributes are preserved.
 
Old 01-16-2015, 03:19 PM   #4
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,396

Rep: Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017
Guys, he's already running with -a, which implies -g, -r, -l and a bunch of others. He's already doing what you're telling him, that's the problem, it's copying the symlinks as symlinks, but they point to targets that are outside of the backup directory.

OP - this is exactly how an archive SHOULD behave. If this is a critical backup, then why are these critical files not in the backup directory? Where are they? Should you be backing up that directory as well? You could remove -a and set the flags as needed to copy the target instead of the symlink, but this can get real messy real fast (think infinite recursion). For a true backup you should re-think what directories you're copying and why if you don't want to bring symlinks over as symlinks.

Last edited by suicidaleggroll; 01-16-2015 at 03:22 PM.
 
Old 01-16-2015, 03:40 PM   #5
vincix
Member
 
Registered: Feb 2011
Distribution: Centos 6.7, 7
Posts: 586

Rep: Reputation: 52
Quote:
Originally Posted by suicidaleggroll View Post
Guys, he's already running with -a, which implies -g, -r, -l and a bunch of others. He's already doing what you're telling him, that's the problem, it's copying the symlinks as symlinks, but they point to targets that are outside of the backup directory.

OP - this is exactly how an archive SHOULD behave. If this is a critical backup, then why are these critical files not in the backup directory? Where are they? Should you be backing up that directory as well? You could remove -a and set the flags as needed to copy the target instead of the symlink, but this can get real messy real fast (think infinite recursion). For a true backup you should re-think what directories you're copying and why if you don't want to bring symlinks over as symlinks.
If -a implies -l, then how does that agree with -L? -L and -l exclude one another, right?
 
Old 01-16-2015, 06:03 PM   #6
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,396

Rep: Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017
Quote:
Originally Posted by vincix View Post
If -a implies -l, then how does that agree with -L?
It doesn't

Quote:
Originally Posted by vincix View Post
-L and -l exclude one another, right?
Yes, I'm not sure where the confusion is.

He's using -a, which implies -l. This means any symlinks in the source are being copied as-is to the destination. This also matches his description of the "problem". His symlinks are being copied to the destination, but the destination does not have the targets where those symlinks are pointing (because the targets are not in the directory being backed up by rsync, so they're not being copied to the destination), so he's just left with a bunch of broken links on the destination.

Instead of copying the symlinks as-is, he wants to copy the targets that are being pointed to. This is not a good backup strategy for multiple reasons, but if he really wants to do that he needs to get rid of -a and use -L and whatever other flags he wants. The better solution is to modify the backup strategy to include the actual location of all critical files. If this is supposed to be a critical backup, but the directory being backed up apparently doesn't contain many of the critical files, then something is wrong. Where are those files? Is anything else missing? Why is their actual location not being included in the backup?
 
Old 01-17-2015, 08:22 AM   #7
vincix
Member
 
Registered: Feb 2011
Distribution: Centos 6.7, 7
Posts: 586

Rep: Reputation: 52
Here's the confusion:
Quote:
rsync -rvgaL /data/disk1/share /data/backup/share
This is what the OP wrote. So it's both -a and -L. That's all I was saying. So after I explained the difference between -L and -l you simply said that I didn't help him. That's the whole point.


On another train of thoughts, why do links break? Shouldn't they have broken only if they referred to relative paths?
 
Old 01-17-2015, 08:34 AM   #8
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,396

Rep: Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017Reputation: 2017
That's just the command he asked if he should be running. The one he actually ran, and was having problems with, was "rsync -rvgal", the same thing you told him to run.

Quote:
I'm using rysnc to backup critical data and to a second disk; here is what I'm typing at the command line rsync -rvgal /data/disk1/share /data/backup/share. It appears that the softlinks are not transfered to the backup drive and some of the links point to data not located in the source folder (/data/share).
-aL would be invalid as you pointed out. I imagine rsync would throw an error if you tried it, not sure.

Yes they're probably relative links, either that or they break as soon as the backup drive is removed from the computer and plugged in elsewhere and the absolute path no longer points to anything useful.

Last edited by suicidaleggroll; 01-17-2015 at 08:39 AM.
 
1 members found this post helpful.
Old 01-17-2015, 09:33 AM   #9
cepheus11
Member
 
Registered: Nov 2010
Location: Germany
Distribution: Gentoo
Posts: 273

Rep: Reputation: 86
Quote:
Originally Posted by suicidaleggroll View Post
-aL would be invalid as you pointed out. I imagine rsync would throw an error if you tried it, not sure.
No, the capital L would override the small l from "a". Nice trick to un-switch one option from the "a" option-collection without needing to remember them all.

To the OP: But this is for sure not what you want for backup because you cannot restore it to the same state anyway. You want "rsync -aAX" as already mentioned. And backup the important link targets from their original location.
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Rsync + hard links backup strategy questions TrainingPeaks Linux - Server 3 07-25-2012 10:59 AM
[SOLVED] Why do symbolic links retain data of deleted files to which they previously linked lityit Linux - General 2 11-05-2011 10:35 PM
Ubuntu backup to USB flash drive. Symbolic links and special files. icekreaman Linux - Newbie 1 12-19-2010 03:16 PM
Backup software that can backup symbolic links Geert86 Linux - Software 2 01-20-2010 06:10 PM
Rsync - Backup vs. Hard Links Only hazmatt20 Linux - Software 2 08-18-2006 07:32 AM


All times are GMT -5. The time now is 12:35 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration