LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 05-12-2015, 06:20 AM   #1
Merlyn
Member
 
Registered: Jun 2005
Location: London
Distribution: None - yet
Posts: 54

Rep: Reputation: 15
Question rsync runs slowly between two external hapd drives


I have two external harddrives linked over usb 2.
I'm using rsync to copy data between them:

Quote:
rsync --progress -avcW '/media/merlyn/SAMSUNG/pictures' '/media/merlyn/5B0DFDEB2728C418/'
I have a lot of data to copy and it takes hours to complete. It will appear stuck at "sending incremental file list" for a long time before it starts to work.

Is there a way to improve performance? I do not want to compress the destination data.
Thank you.

Last edited by Merlyn; 05-12-2015 at 10:19 AM.
 
Old 05-12-2015, 09:38 AM   #2
sgosnell
Senior Member
 
Registered: Jan 2008
Location: Baja Oklahoma
Distribution: Debian
Posts: 1,054

Rep: Reputation: 281Reputation: 281Reputation: 281
The rsync command as you posted it won't work. You have unmatched ' characters. If you've copied that incorrectly, and you're really using it correctly, I only have one suggestion. If you're just copying files, use cp, not rsync. Rsync is not designed to do simple copying, it's designed to sync source and destination, and it has to thoroughly check both before it actually does anything. Compressing the data during the operation does not give compressed data at the destination, it just compresses it during the transfer and then decompresses it for writing. Most modern CPUs are much faster than USB2 bandwidth, so you may see some improvement in transfer times by using compression. The actual data on disk will be identical for source and destination. But in almost all cases, cp is faster than rsync if all you need is a copy of the source material. Rsync excels when there is already data in the destination, and all you want to do is transfer any modified files. If the destination is empty, use cp.
 
Old 05-12-2015, 10:20 AM   #3
Merlyn
Member
 
Registered: Jun 2005
Location: London
Distribution: None - yet
Posts: 54

Original Poster
Rep: Reputation: 15
I corrected the '. I will try it with compression. Thanks.
 
Old 05-12-2015, 11:02 AM   #4
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 4,343

Rep: Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989
Compression, or the lack thereof, will not make any difference. Compression applies to network links, and here both source and destination are local.

Your problem is the "-c" option, which requires a match of checksums of files on both the source and destination. That is going to take a long time if there are a lot of large files that already exist at the destination. Without that option, rsync will just compare file modification times and sizes to see what needs to be transferred, and that is a very quick operation which, in most cases**, is sufficient.
** Binary executables modified by prelink retain their original modification times and can have changes to content without affecting the size. I have also seen some database files that somehow get updated without affecting their modification times.
 
1 members found this post helpful.
Old 05-13-2015, 02:28 AM   #5
Merlyn
Member
 
Registered: Jun 2005
Location: London
Distribution: None - yet
Posts: 54

Original Poster
Rep: Reputation: 15
Removing -c was the right thing to do. Thank you for your kind help.
 
  


Reply

Tags
backup, rsync


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Server runs commands slowly javito_pir Linux - Server 1 09-02-2010 02:28 AM
'shred' runs very slowly jefm Linux - Software 1 11-13-2009 04:22 PM
Ubuntu Studio 8 runs extremely slowly MrRandomPerson Ubuntu 3 02-11-2009 09:44 PM
FCE Ultra runs slowly snarfthefurious Linux - Games 1 03-10-2006 06:02 PM
SUSE 9.1 runs very slowly tbfirefox Linux - Software 7 09-10-2004 11:39 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 02:18 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration