rsync doesn't seem to delete
Hi All,
Still learning and always looking for a source of new answers when I can't find one myself, so I thought I would try here. I've got a script using rsync 2.6.4 to backup a large RAID array to a separate NAS box I have. All seemed to be working fine, however I've just checked the logs to find it isn't working... It seems that the --delete option isn't deleting and the NAS is now full. The command I'm running is: rsync -vrutW --delete /usr/local/graphicsraid/storage/ /mnt/mirror/ the NAS being mounted through samba (it doesn't support rsync natively, well, the current firmware it has doesn't anyway). When I run the command with the --dry-run option, all looks fine - I can see the many files that it intends to delete. I remove the dry-run option, and rather than delete first it starts copying immediately. Of course it runs out of space with a "No space left on device (28)" error soon after. I just can't think of a reason it is doing it like this... Would anyone be able to shed some light on the situation? Thanks! |
Quote:
Sounds pretty normal to me. Are you sure /mnt/mirror has enough space for the files you're copying there? Compare 'du -ks /usr/local/graphicsraid/storage' against 'df -k /mnt/mirror' and confirm there is enough space on your NAS for all the data from your RAID. Or maybe I misunderstand your question, or rsync, or both :) |
Neek, you may misunderstand rsync.
Quote:
Quote:
BTW, accessing a mounted (remote?) file system undoes the benefits of the ingenious rsync protocol, which can bring a file that is already at the recipient in sync by only transferring block changes, sometimes significantly reducing network bandwidth usage. But that may not be an issue for Cooks44. Sorry, I don't have a clear route to troubleshooting. Maybe (A) try a small subtree, (B) use --ignore-errors there, (C) find a rsync dedicated mailing list, (D) look at the source code, (E) strace, (F) pull out your hair? /Quigi |
My guess is that your rsync is dying before it even gets around to deleting the files.
Are you getting errors? |
Yeah it sounds like he needs to put --delete in there
|
See the original post --
LnxAdm: Nope, it doesn't die. It skips deleting, and starts copying (which should happen after). kav: Cooks44 did use --delete. I just had to quote the rsync man page to Neek, who suspected he misunderstood. |
Quote:
Quote:
I like the strace suggestion; try to figure out when the unlink operation is attempted and what the return value is from the system call. I've tested rsync on a tmp source/dest dir, with something simple like "strace <original rsync command line>" but cannot see unlink being performed though the removed source file is removed from the destination dir, but I probably need the strace man page quoted at me :P Maybe someone can suggest a fix to this, probably obviously mistaken, strace usage. |
Yes, the --delete option was set, has always been. I've tried "--delete-before", and a few other combinations to ensure it's using default behaviour.
I'm mirroring a 1TB RAID on my main graphics server to a 1TB JBOD on a NAS I have. It's all running locally on gigabit, so saving bandwidth through rsync algorithms isn't necessary - saving time on the job is. From using rsync in the past it is the perfect tool, so I'll stick with it and figure it out. I have thrown an --ignore-errors into the scripts too, hoping that might show up an I/O error to no avail. As listed in my OP, it's so strange that the behaviour I expect is happening in the dry run, but not when it's actually performed. Either way, you have given me a few avenues to try - I'm leaning towards Samba not liking me (it never does...) so I'll see if I can trace it out and figure out why. Thanks for your help! |
First, doggedly pursuing strace:
Quote:
Code:
$ mkdir orig mirror Code:
26540 execve("/usr/bin/rsync", ["rsync", "-vrutW", "--delete", "orig/", "mirror/"], [/* 20 vars */]) = 0 Second, a different approach, avoiding rather than solving the problem: Cooks44, you mentioned Samba, and that it might not like you. This makes me guess that the destination (JBOD on NAS) lives on a Windows machine. Is this correct? If yes, you could try rsync-ing to that directly, and avoid using the Samba mount. I routinely do that here at home. I initiate the rsync from the Linux side (by cron), so the Windows machines must be running sshd. Rync also works the other way. /Quigi |
y, direct rsync is simpler. Could simply be a permissions issue? ie can recv rsync delete the files and/or file perms are set r--r--r-- or equiv....
|
NAS is a Thecus NS5200, my understanding is it runs a "proprietory" version of linux on it. Unfortunately it doesn't run the rsync protocol, and being designed for windows use it only provides for samba connections (or so I recall - haven't checked what else is available in a while).
Haven't had a chance to revisit it as yet - our ISDN PRI has decided it wants to join the fun and is giving me an immense amount of grief. That takes priority so I'll be back to this issue soon. Thanks all for keeping the possibilities up! Cooks44. |
All times are GMT -5. The time now is 12:16 AM. |