LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 11-15-2007, 08:22 PM   #1
cooks44
LQ Newbie
 
Registered: Nov 2007
Location: Australia
Distribution: Debian
Posts: 3

Rep: Reputation: 0
rsync doesn't seem to delete


Hi All,

Still learning and always looking for a source of new answers when I can't find one myself, so I thought I would try here.

I've got a script using rsync 2.6.4 to backup a large RAID array to a separate NAS box I have. All seemed to be working fine, however I've just checked the logs to find it isn't working... It seems that the --delete option isn't deleting and the NAS is now full.

The command I'm running is:

rsync -vrutW --delete /usr/local/graphicsraid/storage/ /mnt/mirror/

the NAS being mounted through samba (it doesn't support rsync natively, well, the current firmware it has doesn't anyway).

When I run the command with the --dry-run option, all looks fine - I can see the many files that it intends to delete. I remove the dry-run option, and rather than delete first it starts copying immediately. Of course it runs out of space with a "No space left on device (28)" error soon after.

I just can't think of a reason it is doing it like this...

Would anyone be able to shed some light on the situation? Thanks!
 
Old 11-16-2007, 02:10 AM   #2
neek
LQ Newbie
 
Registered: Nov 2007
Posts: 6

Rep: Reputation: 0
Quote:
Originally Posted by cooks44 View Post
The command I'm running is:

rsync -vrutW --delete /usr/local/graphicsraid/storage/ /mnt/mirror/
You're copying a large amount of data from /usr/local/graphicsraid/storage to /mnt/mirror, asking rsync to delete files from /usr/local/graphicsraid/storage when it has finished, and /mnt/mirror is becoming full?

Sounds pretty normal to me. Are you sure /mnt/mirror has enough space for the files you're copying there? Compare 'du -ks /usr/local/graphicsraid/storage' against 'df -k /mnt/mirror' and confirm there is enough space on your NAS for all the data from your RAID.

Or maybe I misunderstand your question, or rsync, or both
 
Old 11-16-2007, 01:48 PM   #3
Quigi
Member
 
Registered: Mar 2003
Location: Cambridge, MA, USA
Distribution: Ubuntu (Dapper and Heron)
Posts: 377

Rep: Reputation: 31
Neek, you may misunderstand rsync.
Quote:
Originally Posted by man rsync
--delete
This tells rsync to delete any files on the receiving side that
arenít on the sending side.
So it should remove files from /mnt/mirror/ that don't (any longer) exist on /usr/local/graphicsraid/storage/. I'd say that's what Cooks44 intended, and verified in a dry run. And, as there's no --delete-after option, we get the default behavior:
Quote:
By default rsync does file deletions on the receiving side
before transferring files to try to ensure that there is suffi-
cient space on the receiving filesystem.
So maybe an I/O error prevents the deletion (which you could ignore with --ignore-errors). Or there may be a bug in rsync?

BTW, accessing a mounted (remote?) file system undoes the benefits of the ingenious rsync protocol, which can bring a file that is already at the recipient in sync by only transferring block changes, sometimes significantly reducing network bandwidth usage. But that may not be an issue for Cooks44.

Sorry, I don't have a clear route to troubleshooting. Maybe (A) try a small subtree, (B) use --ignore-errors there, (C) find a rsync dedicated mailing list, (D) look at the source code, (E) strace, (F) pull out your hair?

/Quigi
 
Old 11-16-2007, 02:20 PM   #4
LnxAdm
LQ Newbie
 
Registered: Nov 2007
Posts: 3

Rep: Reputation: 0
My guess is that your rsync is dying before it even gets around to deleting the files.
Are you getting errors?
 
Old 11-16-2007, 02:45 PM   #5
kav
Member
 
Registered: May 2006
Location: USA
Distribution: FreeBSD Ubuntu Debian
Posts: 137

Rep: Reputation: 15
Yeah it sounds like he needs to put --delete in there
 
Old 11-17-2007, 08:18 AM   #6
Quigi
Member
 
Registered: Mar 2003
Location: Cambridge, MA, USA
Distribution: Ubuntu (Dapper and Heron)
Posts: 377

Rep: Reputation: 31
See the original post --
LnxAdm: Nope, it doesn't die. It skips deleting, and starts copying (which should happen after).
kav: Cooks44 did use --delete. I just had to quote the rsync man page to Neek, who suspected he misunderstood.
 
Old 11-17-2007, 09:57 AM   #7
neek
LQ Newbie
 
Registered: Nov 2007
Posts: 6

Rep: Reputation: 0
Quote:
Originally Posted by Quigi View Post
Neek, you may misunderstand rsync.
Indeed I mis-read my version of the man page, which is possibly the Debian version on my Fedora Core 6 box and reads "--delete delete extraneous files from dest dirs", for some reason I confused this with the --remove-sent-files option on the version of rsync we use on Solaris :P Sorry. My man page does state "--delete-before receiver deletes before transfer (default)" and so with --delete it should be deleting before transfer, as the OP said --dry-run indicated.

Quote:
Originally Posted by Quigi View Post
BTW, accessing a mounted (remote?) file system undoes the benefits of the ingenious rsync protocol, which can bring a file that is already at the recipient in sync by only transferring block changes, sometimes significantly reducing network bandwidth usage. But that may not be an issue for Cooks44.
He used -W, which my man page says "-W, --whole-file copy files whole (without rsync algorithm)" so good point, but Cook44 may already have taken it into account.

I like the strace suggestion; try to figure out when the unlink operation is attempted and what the return value is from the system call. I've tested rsync on a tmp source/dest dir, with something simple like "strace <original rsync command line>" but cannot see unlink being performed though the removed source file is removed from the destination dir, but I probably need the strace man page quoted at me :P Maybe someone can suggest a fix to this, probably obviously mistaken, strace usage.

Last edited by neek; 11-17-2007 at 11:13 AM.
 
Old 11-18-2007, 04:44 PM   #8
cooks44
LQ Newbie
 
Registered: Nov 2007
Location: Australia
Distribution: Debian
Posts: 3

Original Poster
Rep: Reputation: 0
Yes, the --delete option was set, has always been. I've tried "--delete-before", and a few other combinations to ensure it's using default behaviour.

I'm mirroring a 1TB RAID on my main graphics server to a 1TB JBOD on a NAS I have. It's all running locally on gigabit, so saving bandwidth through rsync algorithms isn't necessary - saving time on the job is. From using rsync in the past it is the perfect tool, so I'll stick with it and figure it out.

I have thrown an --ignore-errors into the scripts too, hoping that might show up an I/O error to no avail.

As listed in my OP, it's so strange that the behaviour I expect is happening in the dry run, but not when it's actually performed.

Either way, you have given me a few avenues to try - I'm leaning towards Samba not liking me (it never does...) so I'll see if I can trace it out and figure out why.

Thanks for your help!
 
Old 11-21-2007, 11:34 AM   #9
Quigi
Member
 
Registered: Mar 2003
Location: Cambridge, MA, USA
Distribution: Ubuntu (Dapper and Heron)
Posts: 377

Rep: Reputation: 31
First, doggedly pursuing strace:

Quote:
Originally Posted by neek View Post
I like the strace suggestion; try to figure out when the unlink operation is attempted and what the return value is from the system call. I've tested rsync on a tmp source/dest dir, with something simple like "strace <original rsync command line>" but cannot see unlink being performed though the removed source file is removed from the destination dir, but I probably need the strace man page quoted at me :P Maybe someone can suggest a fix to this, probably obviously mistaken, strace usage.
It needs the -f option to follow children. (Apparently rsync doesn't call fork but clone, which seems similar). Here is a tiny example rsync use.
Code:
$ mkdir orig mirror
$ touch orig/{a,b,c,d}
$ rsync -vrutW --delete orig/ mirror/
building file list ... done
./
a
b
c
d
                                                                                                                          
sent 242 bytes  received 114 bytes  712.00 bytes/sec
total size is 0  speedup is 0.00
$ rm orig/{a,c}
$ strace -f -o rsync.strace rsync -vrutW --delete orig/ mirror/
building file list ... done
deleting c
deleting a
./
 
sent 64 bytes  received 26 bytes  180.00 bytes/sec
total size is 0  speedup is 0.00
$
The deleting worked. Both orig and mirror are on a local file system, Samba is NOT involved. The resulting file rsync.strace has 392 lines. It shows that the process 26540 had a child 26541, which did the unlinking, and a grandchild 26542. Here's an excerpt:
Code:
26540 execve("/usr/bin/rsync", ["rsync", "-vrutW", "--delete", "orig/", "mirror/"], [/* 20 vars */]) = 0
...
26540 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb7e2d708) = 26541
26541 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb7e2d708) = 26542
...
26541 lstat64("c", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
26541 lstat64("d", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
26541 getdents64(0, /* 0 entries */, 4096) = 0
26541 close(0)                          = 0
26541 unlink("c")                       = 0
26541 select(4, [3], [1], NULL, {60, 0}) = 1 (out [1], left {60, 0})
26541 write(1, "\1\0\0lc", 5 <unfinished ...>
26540 <... select resumed> )            = 1 (in [5], left {59, 997000})
26540 read(5, "\1\0\0l", 4)             = 4
26540 select(6, [5], [], NULL, {60, 0}) = 1 (in [5], left {60, 0})
26540 read(5, "c", 1)                   = 1
26540 write(1, "deleting c\n", 11)      = 11
26540 select(6, [5], [], NULL, {60, 0} <unfinished ...>
26541 <... write resumed> )             = 5
26541 time(NULL)                        = 1195601625
26541 unlink("a")                       = 0
26541 select(4, [3], [1], NULL, {60, 0}) = 1 (out [1], left {60, 0})
26541 write(1, "\1\0\0la", 5 <unfinished ...>
26540 <... select resumed> )            = 1 (in [5], left {59, 999000})
26540 read(5, "\1\0\0l", 4)             = 4
26540 select(6, [5], [], NULL, {60, 0}) = 1 (in [5], left {60, 0})
26540 read(5, "a", 1)                   = 1
26540 write(1, "deleting a\n", 11)      = 11
26540 select(6, [5], [], NULL, {60, 0} <unfinished ...>
26541 <... write resumed> )             = 5
...
So much for strace.


Second, a different approach, avoiding rather than solving the problem:

Cooks44, you mentioned Samba, and that it might not like you. This makes me guess that the destination (JBOD on NAS) lives on a Windows machine. Is this correct? If yes, you could try rsync-ing to that directly, and avoid using the Samba mount. I routinely do that here at home. I initiate the rsync from the Linux side (by cron), so the Windows machines must be running sshd. Rync also works the other way.

/Quigi
 
Old 11-21-2007, 07:20 PM   #10
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,289

Rep: Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034Reputation: 2034
y, direct rsync is simpler. Could simply be a permissions issue? ie can recv rsync delete the files and/or file perms are set r--r--r-- or equiv....
 
Old 11-21-2007, 11:57 PM   #11
cooks44
LQ Newbie
 
Registered: Nov 2007
Location: Australia
Distribution: Debian
Posts: 3

Original Poster
Rep: Reputation: 0
NAS is a Thecus NS5200, my understanding is it runs a "proprietory" version of linux on it. Unfortunately it doesn't run the rsync protocol, and being designed for windows use it only provides for samba connections (or so I recall - haven't checked what else is available in a while).

Haven't had a chance to revisit it as yet - our ISDN PRI has decided it wants to join the fun and is giving me an immense amount of grief. That takes priority so I'll be back to this issue soon.

Thanks all for keeping the possibilities up!

Cooks44.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
rsync newbie --delete -b noir911 Linux - General 1 11-23-2006 12:21 AM
Rsync server vs rsync over ssh humbletech99 Linux - Networking 1 10-18-2006 12:10 PM
can rsync change hardlinks instead of delete-recopy? allasso Linux - General 1 09-17-2006 10:46 AM
rsync --delete question Red Squirrel Linux - Software 2 10-10-2005 09:55 PM
Windows Rsync Upload to Linux Rsync - permissions inspleak Linux - Software 0 10-12-2004 02:49 PM


All times are GMT -5. The time now is 08:29 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration