LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 02-20-2014, 10:46 AM   #1
MikeyCarter
Member
 
Registered: Feb 2003
Location: Orangeville
Distribution: Fedora
Posts: 492

Rep: Reputation: 31
Question Slow transfer speeds when rsync'ng files over network?


I used to get about 100MBytes/s when transferring files from one computer to another. Something changed but unfortunately it was a long time ago and don't know what.

If I run iperf3 between the two machines I get ~110MBytes/s If I rsync files via rsync/nfs/ssh protocols I get average of 38Mbyte/s. The drives are capable of writing sequentially at about 140-160 Mbytes/s. (and a local rsync goes super sonic) And watching the array, the drives are under 20% utilization when the rsync is happening.

If I scp the file I get 80-100 Mbytes/s.

Any ideas to where I start looking? Is there some global rsync config limiting the speed I'm missing?
 
Old 02-20-2014, 10:57 AM   #2
MikeyCarter
Member
 
Registered: Feb 2003
Location: Orangeville
Distribution: Fedora
Posts: 492

Original Poster
Rep: Reputation: 31
Never mind just found it. Never fails. I'll look at this for hours. As soon as I post this... some brain storm hits me and I figure it out on my own.

It's "rsync -z"! seems if I compress it, it slows down to ~38Mbytes/s (and sites at about 30% CPU while it's doing it, which is why I didn't notice it off the bat.)

Removing the -z and I'm up to 110MBytes/s.
 
Old 07-03-2017, 02:45 PM   #3
thinkdigital
LQ Newbie
 
Registered: Jul 2017
Posts: 1

Rep: Reputation: Disabled
Quote:
Originally Posted by MikeyCarter View Post
Never mind just found it. Never fails. I'll look at this for hours. As soon as I post this... some brain storm hits me and I figure it out on my own.

It's "rsync -z"! seems if I compress it, it slows down to ~38Mbytes/s (and sites at about 30% CPU while it's doing it, which is why I didn't notice it off the bat.)

Removing the -z and I'm up to 110MBytes/s.
Thank you for that. That did it for me too. I thought compression would speed the network transfer speed up. it didn't help when I switched to doing it locally.
 
Old 08-28-2017, 12:57 AM   #4
coreylee
LQ Newbie
 
Registered: Aug 2017
Posts: 1

Rep: Reputation: Disabled
The slow transfer speed, bothered me a lot. I was working on windows platform with cwrsync and same problem existed there too I searched a lot on internet and found that cwrsync is kind of slow, I started to look for other options and found (edited by mod) a commercial product works very well. Like if cwrsync could transfer 8 per second, this commercial product could do 300 per second. All thanks to its multi threaded TCP mode. I am using its enterprise edition, although its paid but it saves a lot of time for me!

Last edited by jefro; 08-28-2017 at 02:46 PM.
 
Old 08-28-2017, 02:46 PM   #5
jefro
Moderator
 
Registered: Mar 2008
Posts: 21,937

Rep: Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619
coreylee, welcome to LQ.

Please try to limit plugs for commercial products.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Network file transfer speeds slow and drop off Jebuswankel Linux - Networking 3 09-12-2012 05:59 AM
Slow transfer speeds lew Linux - Networking 8 09-14-2009 07:52 AM
Slow NTFS Transfer Speeds OpenCircuit Slackware 1 12-11-2005 05:27 PM
Very slow transfer speeds via LAN TBomb Linux - Networking 6 07-26-2005 08:34 AM
Excessively slow network transfer speeds from XP to Linux… MooGLEX Linux - Networking 1 07-04-2004 01:13 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 01:26 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration