Hi,

I hope this is the right section on the forums, I apologize in advance if it's not. Both of the machines involved below use ubuntu.

I'm trying to copy of a some rather large files from machine A to machine B, but the connection between the two machines is rather slow compared to the file sizes and they can't use bandwidth during certain hours of the day. My idea was to setup a couple cron jobs on machine B to start and stop rsync at the right time of the day.

The script to start the cron job looks like
screen -d -m -S rt rsync --progress --partial -avv -e ssh user@machinea:/remotedir /localdir
where I used screen just to give myself the option of checking up on it. The script to stop rsync for the day uses simply "killall rsync" since there are no other rsync jobs running.

The issue is that rsync seems to write data to file very infrequently and since the files are larger than can be transfered in one session, rsync often looses the entire days worth of work when I kill it this way. I've tried killing rsync with control-c from within screen and it seems a bit better about saving data though still somewhat unreliable. I've also tried replacing killall with "killall -s 15 rsync" to give it time to shut down without effect.

Is there a better way to automatically synchronize the directories which can be halted for part of the day? Is there a different way of invoking rsync which would cause it to save it's partial progress more readily? The need to sync from machine A to machine B is an ongoing one, so I need to find an automated solution instead of just resuming with gftp.

I've given the problem a bit of thought with no solution yet, so I'd be very interested in the communities suggestions. Thanks!

- Ryan

Have you played around with the compression option to rsync. Would these large files compress down?

No, they won't compress down. For a very ad hoc solution I used split to break them into chunks and let rsync go to work, but 1) rather labor intensive, 2) not easily automated, 3) a rather complicated solution to what ought to be a rather simple problem.

It's an old thread, I know. But anyway...

Have you found a solution?