Problem using rsync to back up large files to external harddrive
Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
hangs on large files. By 'large', I only mean files < 100MB. (so not so large). Killing rsync usually works, but then when I do an 'ls' on on mounted file system, even 'ls' hangs. Eventually my whole system gets stuck.
Simply copying the offending files (e.g. data000.hdf5) to the externally mounted disk works instanteonously.
I am copying to the Lacie drive over USB (via a 6 foot extension cable).
Are there any known problems with rsync that might explain this?
not with rsync that I am aware of. I backup files that are over 250 MB up to and including some multi GB .iso images to my External USB/Firewire drive (yes Ii alternate which connection it's using at times.) using rsync and have not had the issue you are describing.
It is by nomeans a fast process, I run it at off hours so I'm not sitting there watching it or waiting for it to finish
I have had problems with LaCie drives (their early ones, I do not have a "Quadra"): They look good, but don't always work well. (I had two - one is reliable, the other NOT - it sometimes hung, like yours does, so the enclosure, but not the disk, eventually went to the dump).
6foot for "high speed" USB may be stretching things. Have you tried using a shorter (or just a different) cable ?
Any useful errors in /var/log/messages ?
rsync has always worked perfectly for me, so I don't think the problem lies there.
(1) the directory 'backup.0' doesn't need to already exist on the mounted disk and so for first time backups, rsync is behaving
essentially like a copy command.
(2) the final backslash doesn't matter.
I am going to double check, but I am fairly sure that just doing a copy of the larger file worked perfectly. And the long cable was another concern I had, so I'll check that out too.
The USB 2.0 specification states the maximum cable length is 5m (16.4ft) but maybe you just had a bad cable, or there were timing issues with the LaCie. The only way to find out is by trying different cables and testing their performance.
And I forgot to mention that I had it connected to the USB cable that came with the Lacie drive and so the two cables together were probably about 10 ft.
The extension appeared to work fine when attached to my USB key, although in that situation, I generally transfered much smaller files. But I'll be wary of using the longer cable from now on.
It's just is such a pain to crawl under the desk to plug in a USB key...
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.