using Bittorrent to transfer data between servers?
Linux - ServerThis forum is for the discussion of Linux Software used in a server related context.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
using Bittorrent to transfer data between servers?
Hello,
I need to transfer one large directory from one server to another, where is much network delay in between, using bittorrent, as I can see it's the best data transfer method over bad connections ..
Is there any simple command line way to do this?
Something like:
On server 1;
bittorent tracker create /home/sourcedir sharename start
On server 2:
bittorrent download server1:/sharename /home/distination
I really need this so much, I couldn't get much help through google ..
Thanks a lot
rsync cannot solve the problem .. rsync could only transfer data with 25 KB/s ...
I've made a script that run rsync for each terminal subdirectory , so that I had more than 100 rsync running, reach transfer rate of 1 MB/s ... and then I've to run rsync for the parent dir again to transfer missed files ....
This is really to much headache ..
As the goal is:
1- to achieve high transfer rate
2- to assume data integrity
And that can be really achived by bittorent as it can open many simultaneous connections achieving the best possible transfer rate ..
Why do you think bit torrent will be faster? If you're limited by network bandwidth (which from the sounds of it you are) on the switch or network level, why would you think bit torrent will be faster?
Take for example, if you have a room full of elephants that you need to move to the room next door only linked with one hallway, instead of shoving them through one door, you create a second door to this hallway from each of the rooms to shove them through. You might think they're emptying from the room faster but you still hit the bottle neck in the hallway since all the elephants still have to pass through such hallway to fill the other room through the other two doors. Does that make any sense?
Thanks ,
But I think you didn't read my last reply ... The main problem is the much delay between the 2 servers that doesn't allow the single connection to make best use of the allowed bandwidth ..
I've already made a test , explained in the previous reply, using rsync that showed that multiple connections can achieve too much better transfer rate ..
I dont think bit torrent will solve your problem at all. Yes bit torrent can have multiple connections, in the exact same way rsync will. Basically it would be much more complex IMO.
rsync is great. If you dont want to do it by hand consider writing a script to do it for you. The fact that opening multiple connections increases the download speed seems strange, it must be throttled or something.
This is really not strange at all! This is not the 1st time to meet such case ...
Have you ever tried to rsync 1 TB from west USA to Germany? Really having multiple connections can speed things much ...
Thanks any way for trying to help...
On debian systems (and I'm sure most other linux systems) you can use btmakemetafile to create the torrent file, then use bttrack to setup a tracker for it. Then just set up bittorrent clients on the server and client sides.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.