LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (http://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Multi-threaded rsync ? (http://www.linuxquestions.org/questions/linux-newbie-8/multi-threaded-rsync-840777/)

permalac 10-27-2010 11:24 AM
Multi-threaded rsync ?
 
Hello,

I want to copy a folder from one server to another, the folder is full of small files and it goes very slow because of that.

I've been looking around for a thing like a multi-treaded rsync to copy faster but I haven't found anything.

Can somebody help me on this?


thanks.

fbobraga 10-27-2010 11:30 AM
it's a job to do only once? If so, there's my tip: zip/tar the files in the source, copy/move the package created, and extract it on the destination

permalac 10-27-2010 11:50 AM
its a 70gb directory in a disc without free space, so there is no space for the resulting tar file. :(

fbobraga 10-27-2010 12:15 PM
Quote:
Originally Posted by permalac (Post 4141162)
its a 70gb directory in a disc without free space, so there is no space for the resulting tar file. :(
You cannot use a external/USB HDD? - it would make the whole thing a lot easier :)

TobiSGD 10-27-2010 04:44 PM
You can use tar in conjunction with netcat to do this job. Have a look here.

permalac 10-27-2010 06:11 PM
Quote:
Originally Posted by fbobraga (Post 4141190)
You cannot use a external/USB HDD? - it would make the whole thing a lot easier :)

What happens if the server is not side by side with the coffee machine?

Sorry, just kidding, probably in 10 hours or so it will be done with rsync, I was just wondering if there is a way to parallelize rsync to use all the cores from a host.

permalac 10-27-2010 06:17 PM
Quote:
Originally Posted by TobiSGD (Post 4141467)
You can use tar in conjunction with netcat to do this job. Have a look here.

In this case I will have to copy all again in case of an updated file. Nice approach anyway.


All times are GMT -5. The time now is 03:48 PM.