multiple source download tool?

nkoplm · 03-23-2006, 02:47 AM

in all my years on linux, i find it odd that i have never come across a tool for downloading (a single file) concurrently from multiple sources. something akin to getright on windows. does anyone know of one they could suggest?
fedora/gnome is what im runnig if it has a gui/matters, but cli is fine for me

Simon Bridge · 03-23-2006, 03:09 AM

So why is bittorrent not what you are after?
http://linuxreviews.org/software/p2p/bittorrent/

from http://www.dessent.net/btfaq/

Quote:

How does BitTorrent compare to other forms of file transfer?

The most common method by which files are transferred on the Internet is the client-server model. A central server sends the entire file to each client that requests it -- this is how both http and ftp work. The clients only speak to the server, and never to each other. The main advantages of this method are that it's simple to set up, and the files are usually always available since the servers tend to be dedicated to the task of serving, and are always on and connected to the Internet. However, this model has a significant problem with files that are large or very popular, or both. Namely, it takes a great deal of bandwidth and server resources to distribute such a file, since the server must transmit the entire file to each client. Perhaps you may have tried to download a demo of a new game just released, or CD images of a new Linux distribution, and found that all the servers report "too many users," or there is a long queue that you have to wait through. The concept of mirrors partially addresses this shortcoming by distributing the load across multiple servers. But it requires a lot of coordination and effort to set up an efficient network of mirrors, and it's usually only feasible for the busiest of sites.

Another method of transferring files has become popular recently: the peer-to-peer network, systems such as Kazaa, eDonkey, Gnutella, Direct Connect, etc. In most of these networks, ordinary Internet users trade files by directly connecting one-to-one. The advantage here is that files can be shared without having access to a proper server, and because of this there is little accountability for the contents of the files. Hence, these networks tend to be very popular for illicit files such as music, movies, pirated software, etc. Typically, a downloader receives a file from a single source, however the newest version of some clients allow downloading a single file from multiple sources for higher speeds. The problem discussed above of popular downloads is somewhat mitigated, because there's a greater chance that a popular file will be offered by a number of peers. The breadth of files available tends to be fairly good, though download speeds for obscure files tend to be low. Another common problem sometimes associated with these systems is the significant protocol overhead for passing search queries amongst the peers, and the number of peers that one can reach is often limited as a result. Partially downloaded files are usually not available to other peers, although some newer clients may offer this functionality. Availability is generally dependent on the goodwill of the users, to the extent that some of these networks have tried to enforce rules or restrictions regarding send/receive ratios.

Use of the Usenet binary newsgroups is yet another method of file distribution, one that is substantially different from the other methods. Files transferred over Usenet are often subject to miniscule windows of opportunity. Typical retention time of binary news servers are often as low as 24 hours, and having a posted file available for a week is considered a long time. However, the Usenet model is relatively efficient, in that the messages are passed around a large web of peers from one news server to another, and finally fanned out to the end user from there. Often the end user connects to a server provided by his or her ISP, resulting in further bandwidth savings. Usenet is also one of the more anonymous forms of file sharing, and it too is often used for illicit files of almost any nature. Due to the nature of NNTP, a file's popularity has little to do with its availability and hence downloads from Usenet tend to be quite fast regardless of content. The downsides of this method include a baroque set of rules and procedures, and requires a certain amount of effort and understanding from the user. Patience is often required to get a complete file due to the nature of splitting big files into a huge number of smaller posts. Finally, access to Usenet often must be purchased due to the extremely high volume of messages in the binary groups.

BitTorrent is closest to Usenet, in my opinion. It is best suited to newer files, of which a number of people have interest in. Obscure or older files tend to not be available. Perhaps as the software matures a more suitable means of keeping torrents seeded will emerge, but currently the client is quite resource-intensive, making it cumbersome to share a number of files. BitTorrent also deals well with files that are in high demand, especially compared to the other methods.

The bittorrent client comes with fedora.

cs-cam · 03-23-2006, 03:20 AM

Axel is a nice CLI tool that will download from multiple sources, download chunks etc. Less options that wget but ample for me, I'm a fan

nkoplm · 03-23-2006, 03:22 AM

Quote:

Originally Posted by Simon Bridge

So why is bittorrent not what you are after?

i use bit torrent when it is the appropreate tool for the job, but it is not always best suited for the task at hand.

in this case an http and/or ftp multisource downloader is better for me.