LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 01-22-2013, 11:47 AM   #1
KirkD
Member
 
Registered: Oct 2007
Posts: 58

Rep: Reputation: 15
rsync Invalid cross-device link


I posted the same question over on FedoraForum, but was hoping to widen my net a bit.



I've started using rsync to back up my LINUX systems, and I'm getting some odd behaviors, which maybe are only odd because I've just begun using rsync.

Here's the detail: I have a desktop system running CentOS 5.9 and a laptop running Fedora 17. The desktop is wired to the router and the laptop connects wirelessly. I have a Western Digital MyBook Live Duo wired to the router as well. This drive gives me 2x2 TB set up as a RAID 1 array. Interestingly, the MyBook is running a variant of LINUX on its controller board and has a Public directory shared through NFS. I can easily mount the drive to both the desktop and laptop with a simple line added to fstab. The MyBook is formatted as NTFS, by the way. I do have a couple of Windows machines that need to connect as well, so NTFS will work. [EDIT: doing a little digging, it appears that the MyBook Duo has an ext3 filesystem. I'll have to double check that when I get home, but that eliminates the likliehood that the filesystem doesn't support hardlinks.]

When I run rsync the first time, I get a few errors related to chgrp. As best as I can tell, this is not a problem and I have essentially ignored them. Here's the code I used to make the first backup:


Code:
DATE=`date "+%Y-%m-%dT%H_%M_%S"`
BACKUPDIR=/mnt/MyBook/host/Backups

rsync -azgRP --log-file=$BACKUPDIR/rsync-$DATE.log \
	/home/username/Projects /home/username/Datasets \
	$BACKUPDIR/incomplete_backup-$DATE

mv $BACKUPDIR/incomplete_backup-$DATE $BACKUPDIR/backup-$DATE
rm -f $BACKUPDIR/current
ln -s backup-$DATE $BACKUPDIR/current
After generating the first backup with rsync, I then modified a couple of files - .mp3 files that had incorrect ID3 tags - and then reran rsync just to see how it behaved. Here's the code I used for that second round:

Code:
DATE=`date "+%Y-%m-%dT%H_%M_%S"`
BACKUPDIR=/mnt/MyBook/host/Backups

rsync -azgRP --log-file=$BACKUPDIR/rsync-$DATE.log \
        --link-dest=$BACKUPDIR/current \
	/home/username/Projects /home/username/Datasets \
	$BACKUPDIR/incomplete_backup-$DATE

mv $BACKUPDIR/incomplete_backup-$DATE $BACKUPDIR/backup-$DATE
rm -f $BACKUPDIR/current
ln -s backup-$DATE $BACKUPDIR/current
As this runs, I get numerous errors related to an Invalid cross-link:

Quote:
2013/01/22 06:42:36 [3446] rsync: link "/mnt/MyBook/Backups/incomplete_backup-2013-01-22T06_42_30/home/user/.VirtualBox/Oracle_VM_VirtualBox_Extension_Pack-4.1.12.vbox-extpack" => /mnt/MyBook/Backups/current/home/user/.VirtualBox/Oracle_VM_VirtualBox_Extension_Pack-4.1.12.vbox-extpack failed: Invalid cross-device link (18)
Notice that the links are being made within the same directory on the MyBook. I see this error for a number of files in the .VirtualBox directory and I've seen it on other files that are in non-hidden directories as well. On of them was on an .mp3 file that did not change between backups. Any ideas why is is choking on this?

Also, as I mentioned, only a couple of files were changed between backups, but when I look at the directory sizes and the disk usage, I see that about 60 GB of space has been consumed by the second backup. The first backup was about 127 GB, so 50% more space gets consumed. Shouldn't this be significantly less given than only sets of links are made? Or, is this related to the invalid cross-device link error, above?

Any help is appreciated.

-Kirk
 
Old 01-22-2013, 11:01 PM   #2
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,790

Rep: Reputation: 653Reputation: 653Reputation: 653Reputation: 653Reputation: 653Reputation: 653
Using --link-dest creates hard links to the files that haven't changed, perhaps the filesystem on the MyBook doesn't support it? .. try creating one manually with 'ln'. It's also possible that it doesn't like linking when using the full path, you could try rsync without '-R'.
 
Old 01-23-2013, 01:13 AM   #3
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,642

Rep: Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760
hard links can only be used within one device, files cannot be hardlinked across different filesystems.
 
Old 01-23-2013, 10:52 AM   #4
KirkD
Member
 
Registered: Oct 2007
Posts: 58

Original Poster
Rep: Reputation: 15
Quote:
hard links can only be used within one device, files cannot be hardlinked across different filesystems.
Yes, I realize this. All hard links are being created within the same filesystem.

Quote:
Originally Posted by kbp View Post
Using --link-dest creates hard links to the files that haven't changed, perhaps the filesystem on the MyBook doesn't support it? .. try creating one manually with 'ln'. It's also possible that it doesn't like linking when using the full path, you could try rsync without '-R'.
I checked and I can indeed create a hardlink on the device. My understanding from what I've read is that the MyBook uses ext3, but I see nfs reported on my system using du -T. Regardless, I can create them without any errors.

I will give it a try without -R and report back...
 
Old 01-23-2013, 10:48 PM   #5
KirkD
Member
 
Registered: Oct 2007
Posts: 58

Original Poster
Rep: Reputation: 15
I tried the same without the -R option and it failed with the same error, e.g.:

rsync: link "/mnt/MyBook/Kirk/Penguin/Backups/incomplete_backup-2013-01-23T21_25_46/Aerosmith/Aerosmith/08_-_Walkin_the_Dog.mp3" => /mnt/MyBook/Kirk/Penguin/Backups/current/Aerosmith/Aerosmith/08_-_Walkin_the_Dog.mp3 failed: Invalid cross-device link (18)

I also tried to go in and do it manually with:

ln /mnt/MyBook/Kirk/Penguin/Backups/current/Aerosmith/Aerosmith/08_-_Walkin_the_Dog.mp3 test

and got the same error. I'm wondering if it because it is doing the link on the MyBook but it is referring to it through the system's /mnt/MyBook mount point. I also tried to manually create a link without the fully elaborated path with this while in the containing directory:

ln backup-2013-01-23T07_30_58/Aerosmith/Aerosmith/08_-_Walkin_the_Dog.mp3 test

and got the same error.

If I then create a junk file with vi junk, and then do ln junk test, it works fine. What's more, there's a log file in that directory which was created with the

--log-file=$BACKUPDIR/rsync-$DATE.log

option to rsync, but I cannot even make a link to that file. For some reason, it thinks that the files that were created with rsync are not in the same filesystem.
 
Old 01-24-2013, 12:35 AM   #6
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 14,642

Rep: Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760Reputation: 4760
rsync will not create filesystems and will not select "on demand", it simply uses what was given. So in simple cases you try to create a link between filesystems, in more complex cases you may have a tricky (virtual?) filesystem with some strange solutions...
 
Old 01-24-2013, 03:15 AM   #7
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,790

Rep: Reputation: 653Reputation: 653Reputation: 653Reputation: 653Reputation: 653Reputation: 653
Hmm .. I just checked a rsync snapshot script I wrote a while back and it's using relative paths for the --link-dest argument. I haven't tested it with an NFS backup dir, maybe give it a go and see if it has the same problem?
 
Old 01-24-2013, 11:42 AM   #8
KirkD
Member
 
Registered: Oct 2007
Posts: 58

Original Poster
Rep: Reputation: 15
I tried another round of tests this time using ssh rather than just pushing to a mounted location. That seemed to fix the hard link problem. I'm guessing being logged into the device and in a particular directory is different from being in that same directory location through a mounted point on another device.

Next issue is this:

I backed up a small directory of .mp3 files, verified they were there, modified the ID3 tags and saved a subset of them, reran the script, and when I checked the newest backup directory, the files I expected to be modified were still the old files. I tried this a couple more times - backup, modify, backup, check - and every time zero files were actually copied to the backup. All those that should have been modified were simply hard links to the old ones without the modified ID3 tags.

Ideas??
 
Old 01-24-2013, 03:22 PM   #9
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,790

Rep: Reputation: 653Reputation: 653Reputation: 653Reputation: 653Reputation: 653Reputation: 653
Ok, so let's test this a step at a time.

- wipe everything from the backup directory
- run your script on a small directory
- 'ls -lr $BACKUPDIR/'
- modify and/or add some files in the small directory
- run the script again
- 'ls -lr $BACKUPDIR/'

If you can post the current version of your script and the output of the commands above it will help us to troubleshoot the issue
 
Old 01-24-2013, 11:39 PM   #10
KirkD
Member
 
Registered: Oct 2007
Posts: 58

Original Poster
Rep: Reputation: 15
Sorry for the chaos.

Here's the script I'm using which doesn't have a problem with hardlinks.

Code:
n=30                                       #Number of backups to retain
NAS=MyBookLiveDuo                        #IP address or resolvable hostname of
SrcDir=/Library/Aerosmith           #Directory to be backed up
DestDir=/Backups   #Backup destination on NAS

# Delete the níth backup.
ssh root@$NAS '[ -d '$DestDir'/'$n'.days_ago ] && [ rm -rf '$DestDir'/'$n'.days_ago ]'

# Rename backup directories to free up the 0 day directory.
ssh root@$NAS 'for i in {'$n'..1}; \
do [[ -d '$DestDir'/$(($i-1)).days_ago ]] && \
/bin/mv '$DestDir'/$(($i-1)).days_ago '$DestDir'/${i}.days_ago; done'

nice rsync -av \
--delete \
--link-dest=../1.days_ago \
$SrcDir root@$NAS:$DestDir/0.days_ago/
ls -lr

Quote:
total 50496
-rw-rw-r--. 1 KirkD KirkD 4588463 Nov 14 2010 08_-_Walkin_the_Dog.mp3
-rw-rw-r--. 1 KirkD KirkD 7246056 Nov 14 2010 07_-_Movin_Out.mp3
-rw-rw-r--. 1 KirkD KirkD 6005971 Nov 14 2010 06_-_Write_Me.mp3
-rw-rw-r--. 1 KirkD KirkD 6366461 Nov 14 2010 05_-_Mama_Kin.mp3
-rw-rw-r--. 1 KirkD KirkD 10103643 Nov 14 2010 04_-_One_Way_Street.mp3
-rw-rw-r--. 1 KirkD KirkD 6375865 Nov 14 2010 03_-_Dream_On.mp3
-rw-rw-r--. 1 KirkD KirkD 5409125 Nov 14 2010 02_-_Somebody.mp3
-rw-rw-r--. 1 KirkD KirkD 5266810 Nov 14 2010 01_-_Make_It.mp3
I also verified that the original .mp3s have an error in the ID3 tags. I fixed the ID3 tags on the original files and repeated the backup script.

ls -lr

Quote:
total 50496
-rw-rw-r--. 2 KirkD KirkD 4588463 Nov 14 2010 08_-_Walkin_the_Dog.mp3
-rw-rw-r--. 2 KirkD KirkD 7246056 Nov 14 2010 07_-_Movin_Out.mp3
-rw-rw-r--. 2 KirkD KirkD 6005971 Nov 14 2010 06_-_Write_Me.mp3
-rw-rw-r--. 2 KirkD KirkD 6366461 Nov 14 2010 05_-_Mama_Kin.mp3
-rw-rw-r--. 2 KirkD KirkD 10103643 Nov 14 2010 04_-_One_Way_Street.mp3
-rw-rw-r--. 2 KirkD KirkD 6375865 Nov 14 2010 03_-_Dream_On.mp3
-rw-rw-r--. 2 KirkD KirkD 5409125 Nov 14 2010 02_-_Somebody.mp3
-rw-rw-r--. 2 KirkD KirkD 5266810 Nov 14 2010 01_-_Make_It.mp3
I verified that the .mp3s in the original directory do indeed have the updated ID3 tags, but the files in the backup directory do not have the tags.

I also looked at the inode numbers of the files the two directories that were created on the backup drive. One directory was originally created with the files with errors, and the second was created during back after correcting the errors. I get this which I do ls -i in each directory:

Quote:
total 50496
25067543 -rw-rw-r--. 2 KirkD KirkD 5266810 Nov 14 2010 01_-_Make_It.mp3
25067544 -rw-rw-r--. 2 KirkD KirkD 5409125 Nov 14 2010 02_-_Somebody.mp3
25067545 -rw-rw-r--. 2 KirkD KirkD 6375865 Nov 14 2010 03_-_Dream_On.mp3
25067546 -rw-rw-r--. 2 KirkD KirkD 10103643 Nov 14 2010 04_-_One_Way_Street.mp3
25067547 -rw-rw-r--. 2 KirkD KirkD 6366461 Nov 14 2010 05_-_Mama_Kin.mp3
25067548 -rw-rw-r--. 2 KirkD KirkD 6005971 Nov 14 2010 06_-_Write_Me.mp3
25067549 -rw-rw-r--. 2 KirkD KirkD 7246056 Nov 14 2010 07_-_Movin_Out.mp3
25067550 -rw-rw-r--. 2 KirkD KirkD 4588463 Nov 14 2010 08_-_Walkin_the_Dog.mp3



total 50496
25067543 -rw-rw-r--. 2 KirkD KirkD 5266810 Nov 14 2010 01_-_Make_It.mp3
25067544 -rw-rw-r--. 2 KirkD KirkD 5409125 Nov 14 2010 02_-_Somebody.mp3
25067545 -rw-rw-r--. 2 KirkD KirkD 6375865 Nov 14 2010 03_-_Dream_On.mp3
25067546 -rw-rw-r--. 2 KirkD KirkD 10103643 Nov 14 2010 04_-_One_Way_Street.mp3
25067547 -rw-rw-r--. 2 KirkD KirkD 6366461 Nov 14 2010 05_-_Mama_Kin.mp3
25067548 -rw-rw-r--. 2 KirkD KirkD 6005971 Nov 14 2010 06_-_Write_Me.mp3
25067549 -rw-rw-r--. 2 KirkD KirkD 7246056 Nov 14 2010 07_-_Movin_Out.mp3
25067550 -rw-rw-r--. 2 KirkD KirkD 4588463 Nov 14 2010 08_-_Walkin_the_Dog.mp3
All the inode numbers are the same between the old and new directories.
So, modifying the ID3 tags, then saving the .mp3 does not lead to an updated file in the backup.
 
Old 01-25-2013, 03:25 AM   #11
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,790

Rep: Reputation: 653Reputation: 653Reputation: 653Reputation: 653Reputation: 653Reputation: 653
Hmm .. ok, so try adding some garbage to the end of a file (take a backup first), then sync again.

Code:
echo "just some garbage" >> /Library/Aerosmith/01_-_Make_It.mp3
I suppose it's vaguely possible that the id3 tag application is leaving the files modified date intact, you could try adding '-c' to rsync so it uses the checksum rather than going by the files modified time.
 
Old 01-25-2013, 09:13 PM   #12
KirkD
Member
 
Registered: Oct 2007
Posts: 58

Original Poster
Rep: Reputation: 15
I redid the backup using the -c option and....it worked. The appropriate files were copied and the rest were hard linked.

This seems a valid option apart from the inevitable slow down and CPU costs of validating checksums on every file. A second option would be to touch the modified files so that rsync picks up the date change. Last in line would be to figure out shy EasyTag does alter the size nor the date of the file.

Any other suggestions?

And, thank you very much for your help and patience.

-Kirk
 
Old 01-25-2013, 09:26 PM   #13
KirkD
Member
 
Registered: Oct 2007
Posts: 58

Original Poster
Rep: Reputation: 15
And there you have it:

http://www.redhat.com/archives/rhl-l.../msg03238.html

Apparently, that is expected behaviour from EasyTag. I tested the option they mention - uncheck "preserve modification time" - and it works now.

Wow. That was a convoluted path, eh?

Thank you again!!
 
Old 01-25-2013, 09:28 PM   #14
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,790

Rep: Reputation: 653Reputation: 653Reputation: 653Reputation: 653Reputation: 653Reputation: 653
No worries, we learn something every day
 
Old 01-26-2013, 08:44 PM   #15
KirkD
Member
 
Registered: Oct 2007
Posts: 58

Original Poster
Rep: Reputation: 15
Isn't that the truth!? I would never have guess that the tag changes wouldn't alter the file size nor the modified timestamp. Go figure.

So, continuing the discussion...

The code I'm using right now is below. It works OK, but there are a number of tweaks I would like to accomplish that I don't seem to be able to work out.

1) I have a variable called SrcDirs at the top and I also have two --exclude switches in the rsync command. I would much rather store these in a file which could then be modified as necessary without having to touch the script itself. I tried --include-from and --exclude-from, which seems to work in concept, but I still have to specify a source directory in the rsync command line itself. If I do this, I have to make the --exclude-from directory excessively large or I end up backing up way more than I need. I tried different variations of --exclude=*, but that essentially excludes everything and I get an empty backup.

2) Eventually, I would like to set this up through crontab to perfrom a backup every few hours while the machine is on. I could alternatively have it backup once per day at a certain time, assuming I leave the machine on all the time. Or, can I tie the backup to shutdown?

3) Regardless of how often a backup is done, I will begin to accumulate directories and I'd like to limit those to not more than 60 - see the n=60 variable at the top. The original script created backup directories called "x.days_ago". When a new backup was created, the 30.days_ago directory was deleted and 0.days_ago through 29.days_ago were promoted by 1, and a new 0.days-ago directory was created. I don't much like that system as I may not have this machine on every day. I like the method I have here which puts a date-/time-stamp in the directory title. What I would like to do is delete the oldest one when the number of directories goes over n - 60 in this example.

As always, any advice is greatly appreciated.

-Kirk

Code:
# —– Edit these variables to suit your environment —–

n=60                                       #Number of backups to retain
NAS=192.168.0.185                          #IP address or resolvable hostname of NAS
SrcDirs="/home/keenansm/Projects /home/keenansm/Datasets"
DestDir=/nfs/Public/alfonz/Backups   #Backup destination on NAS
Date=`date "+%Y-%m-%dT%H_%M_%S"`

# —– End of edits —–

echo
echo =========================================================================
echo “Starting backup.sh…..”;
date;

# Delete the oldest backup if there are more than n backup directories.
echo
echo Removing oldest backup.

echo
echo Starting rsync...
# Run the Rsync command. Nice is used to prevent Rysnc from hogging the CPU.
# –link-dest creates hard links so that each backup run appears as a full
# backup even though they only copy changed blocks 
nice rsync -azv \
--link-dest=$DestDir/current \
--delete \
--delete-excluded \
--exclude=*/pics \
--exclude=*/HTML \
$SrcDirs root@$NAS:$DestDir/incomplete_backup-$Date \
&& ssh root@$NAS \
"mv $DestDir/incomplete_backup-$Date $DestDir/Backup-$Date \
&& rm -f $DestDir/current \
&& ln -s $DestDir/Backup-$Date $DestDir/current"
echo
echo =========================================================================
echo “Completed running backup.sh”;
date;

Last edited by KirkD; 01-27-2013 at 10:03 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Invalid cross-device link linuxqueric Linux - Newbie 12 02-22-2020 12:24 PM
DSL modem exploit for Cross Site Request Forgery....a link aus9 Linux - Security 1 10-01-2012 09:23 PM
Invalid cross-device link teonghan Linux - Software 0 10-31-2010 10:52 PM
GNU based c++ cross compiler unable to link c library k_taid Programming 1 06-29-2005 09:38 AM
How to find an invalid symbolic link?? zhaoyoubing Linux - General 3 08-21-2003 09:01 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 01:49 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration