LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 11-02-2008, 08:56 AM   #1
Amr_not_Amr
LQ Newbie
 
Registered: Jun 2005
Location: Egypt
Distribution: OpenSuSE, CentOS
Posts: 23

Rep: Reputation: 15
using Bittorrent to transfer data between servers?


Hello,
I need to transfer one large directory from one server to another, where is much network delay in between, using bittorrent, as I can see it's the best data transfer method over bad connections ..
Is there any simple command line way to do this?
Something like:
On server 1;
bittorent tracker create /home/sourcedir sharename start
On server 2:
bittorrent download server1:/sharename /home/distination

I really need this so much, I couldn't get much help through google ..
Thanks a lot
 
Old 11-02-2008, 09:19 AM   #2
bmarx
Member
 
Registered: Dec 2004
Distribution: Slackware, Arch, FreeBSD
Posts: 202

Rep: Reputation: 34
Setting up torrents may be complicating the situation more than you need to. Is there any reason you can not use rsync?
 
Old 11-03-2008, 01:46 PM   #3
Amr_not_Amr
LQ Newbie
 
Registered: Jun 2005
Location: Egypt
Distribution: OpenSuSE, CentOS
Posts: 23

Original Poster
Rep: Reputation: 15
rsync cannot solve the problem .. rsync could only transfer data with 25 KB/s ...
I've made a script that run rsync for each terminal subdirectory , so that I had more than 100 rsync running, reach transfer rate of 1 MB/s ... and then I've to run rsync for the parent dir again to transfer missed files ....
This is really to much headache ..
As the goal is:
1- to achieve high transfer rate
2- to assume data integrity
And that can be really achived by bittorent as it can open many simultaneous connections achieving the best possible transfer rate ..
 
Old 11-03-2008, 03:28 PM   #4
trickykid
LQ Guru
 
Registered: Jan 2001
Posts: 24,149

Rep: Reputation: 269Reputation: 269Reputation: 269
Why do you think bit torrent will be faster? If you're limited by network bandwidth (which from the sounds of it you are) on the switch or network level, why would you think bit torrent will be faster?

Take for example, if you have a room full of elephants that you need to move to the room next door only linked with one hallway, instead of shoving them through one door, you create a second door to this hallway from each of the rooms to shove them through. You might think they're emptying from the room faster but you still hit the bottle neck in the hallway since all the elephants still have to pass through such hallway to fill the other room through the other two doors. Does that make any sense?
 
Old 11-03-2008, 05:34 PM   #5
Amr_not_Amr
LQ Newbie
 
Registered: Jun 2005
Location: Egypt
Distribution: OpenSuSE, CentOS
Posts: 23

Original Poster
Rep: Reputation: 15
Thanks ,
But I think you didn't read my last reply ... The main problem is the much delay between the 2 servers that doesn't allow the single connection to make best use of the allowed bandwidth ..
I've already made a test , explained in the previous reply, using rsync that showed that multiple connections can achieve too much better transfer rate ..
 
Old 11-05-2008, 10:57 AM   #6
beiller
LQ Newbie
 
Registered: Nov 2008
Posts: 22

Rep: Reputation: 15
I dont think bit torrent will solve your problem at all. Yes bit torrent can have multiple connections, in the exact same way rsync will. Basically it would be much more complex IMO.

rsync is great. If you dont want to do it by hand consider writing a script to do it for you. The fact that opening multiple connections increases the download speed seems strange, it must be throttled or something.
 
Old 11-05-2008, 02:46 PM   #7
Amr_not_Amr
LQ Newbie
 
Registered: Jun 2005
Location: Egypt
Distribution: OpenSuSE, CentOS
Posts: 23

Original Poster
Rep: Reputation: 15
This is really not strange at all! This is not the 1st time to meet such case ...
Have you ever tried to rsync 1 TB from west USA to Germany? Really having multiple connections can speed things much ...
Thanks any way for trying to help...
 
Old 11-05-2008, 06:44 PM   #8
estabroo
Senior Member
 
Registered: Jun 2008
Distribution: debian, ubuntu, sidux
Posts: 1,126
Blog Entries: 2

Rep: Reputation: 124Reputation: 124
On debian systems (and I'm sure most other linux systems) you can use btmakemetafile to create the torrent file, then use bttrack to setup a tracker for it. Then just set up bittorrent clients on the server and client sides.
 
Old 11-16-2008, 12:26 PM   #9
Amr_not_Amr
LQ Newbie
 
Registered: Jun 2005
Location: Egypt
Distribution: OpenSuSE, CentOS
Posts: 23

Original Poster
Rep: Reputation: 15
Thanks a lot,
I'm going to try this!
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how to transfer a file between 2 servers graziano1968 Linux - Server 15 08-14-2008 03:34 AM
memcpy_toio transfers data in 4 byte chunks, but I need to transfer data in one lump. jbreaka4lyfe Linux - Embedded & Single-board computer 2 06-02-2008 11:25 AM
How to measure data Transfer beween two servers. rajaniyer123 Solaris / OpenSolaris 5 06-05-2007 03:41 PM
Transfer Servers biggdady6998 Linux - General 3 10-02-2003 05:55 PM
transfer DNS data to secondary name servers? Grum Linux - General 0 05-15-2002 03:36 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 03:57 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration