Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I looked around for answers, and I did find a lot of folks bitchin about the speed of rsync, but no real solutions. We have a fileserver, not very big, current usage is ~1.7TB. I made a new fileserver using Debian 12 running Samba .. I mounted the old fileserver in /mnt and cifs, and let rsync copy the files over, it took three days. Now you'd think that after that, I could run rsync and since pretty much everything is here, it would take a few minutes, but it takes about four hours. The command is:
I've really not used rsync much in the past outside of small cron jobs at home, can anyone shed light on what I'm doing wrong, or is rsync just that slow?
As a side note, I also back up that same (old) file server with my windows work computer using robocopy.exe to an external usb spinner, and the backup takes about 7-10 minutes every hour, there's just not that much going on on that particular server.
Rsync is slow, as it's encrypting and decrypting. It's only as fast as the slowest cpu. It takes me just over 5 minutes 10 seconds to copy 1GB (=1024́³) between two boxes here. The slower one is a RazPi 4, 4core A-72 Cortex @1.8 GHz. I've seen it take longer if the thing isn't idle.
That 1GB file was constructed with 'dd if=/dev/random...' so it wasn't a zero string. For your 1.7TB =1700GB disk,, that calculates to a copying time of 527,000 seconds, = 146 hours = 6 days, 2 hours and twenty something minutes. I can hardly think of a slower way to do it.
You'd employ your time much better mounting the disk locally and copying over. Or set up an ftp server, NFS mount, or something that didn't involve heavy lifting on a light cpu.
rsync is initially designed to copy files from a remote location over some network. It loses almost all its power if you want to make a local copy (single host, mounted filesystems), it will work as a single copy command. So the question is: how long does it take to copy those files using cp (or any other command)? Additionally cifs is a relatively slow filesystem, you ought to use something faster. rsync does not need a locally mounted drive, it can directly access the remote host/filesystem, probably that would be faster.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.