LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   wget with multiple connection. (https://www.linuxquestions.org/questions/linux-software-2/wget-with-multiple-connection-4175605900/)

hack3rcon 05-14-2017 02:10 AM

wget with multiple connection.
 
Hello.
How can I set "wget" for download a file with multiple connections?


Thank you.

Laserbeak 05-14-2017 02:21 AM

Why do you need multiple connections, it's not likely to increase your speed?

273 05-14-2017 03:48 AM

I'm not sure that wget can do that -- have you looked at axel instead?

Turbocapitalist 05-14-2017 04:44 AM

Using a torrent will allow use of multiple connections and thus increase speed, but as far as I know wget just does a single connection.

ondoho 05-14-2017 06:12 AM

there's a myth of increasing download speeds through "multiple download slots" or some such technobabble.
i think it comes from certain commercial download sites like megaupload etc., that would limit bandwidth per user.
some addons/software claimed to be able to circumvent that restriction (and maybe did until the providers circumvented THAT again), but i just don't see how wget has anything to do with it.

Turbocapitalist 05-14-2017 06:15 AM

Quote:

Originally Posted by ondoho (Post 5710489)
there's a myth of increasing download speeds through "multiple download slots" or some such technobabble.
i think it comes from certain commercial download sites like megaupload etc., that would limit bandwidth per user.
some addons/software claimed to be able to circumvent that restriction (and maybe did until the providers circumvented THAT again), but i just don't see how wget has anything to do with it.

Even with multiple connections, it would still be using a single source and thus be limited. Since torrents use multiple sources, the throughput is much, much higher and limited only by the destination bandwidth, at least if there are enough seeds available.

273 05-14-2017 06:16 AM

Quote:

Originally Posted by ondoho (Post 5710489)
there's a myth of increasing download speeds through "multiple download slots" or some such technobabble.

Strangely enough in my limited testing I found that axel seems to be able to download quicker than wget from sites where the downkoad is slower than my internet connection. I have been sceptical about the issue so even with some limited evidence I'm not yet prepared to state that multiple connections is definitely faster but it seems there may be some truth in it under certain conditions at least.

ondoho 05-14-2017 06:38 AM

i'm looking at search results for "download multi" from my package manager, and there's about a dozen AUR packages (all of them unsupported, as AUR packages always are).
i'm reading things like
"Xtreme Download Manager is a powerful tool to increase download speed up-to 500%, save videos from video sharing sites and integration with ANY browser."
and
"A full featured, advanced, multi-threaded, multisegment download manager and accelerator."

it all sounds a lot like snake oil to me, but so many coders can't be completely wrong i guess.

i will have a closer look.

273 05-14-2017 06:44 AM

Quote:

Originally Posted by ondoho (Post 5710502)
it all sounds a lot like snake oil to me,

Oh, to me too and I was surprised when axel seemed faster than wget. I think it may depend very much on the reasons why wget isn't maxing out the link.

ondoho 05-14-2017 07:06 AM

ok, this is the one got the most votes on AUR:
https://www.flareget.com/
no source code. go figure.

i now found axel, from the man page:
"Axel is a program that downloads a file from a FTP or HTTP server through multiple connection. Each connection downloads its own part of the file."
just like i thought, this only makes sense when the provider throttles bandwidth (and is stupid enough to not recognize this trick).

273 05-14-2017 08:12 AM

Quote:

Originally Posted by ondoho (Post 5710508)
ok, this is the one got the most votes on AUR:
https://www.flareget.com/
no source code. go figure.

i now found axel, from the man page:
"Axel is a program that downloads a file from a FTP or HTTP server through multiple connection. Each connection downloads its own part of the file."
just like i thought, this only makes sense when the provider throttles bandwidth (and is stupid enough to not recognize this trick).

I think that's roughly how it works, yes, but I think it could also combat unintentional throttling due to server processes not been given enough resources or possibly processes scanning the outbound data. Sadly I don't know enough about this kind of thing so the previous may be nonsense but something like that would make sense to me. A bit like when, sometimes, it seems quicker to copy files from a file sever simultaneously rather than one after the other.

hack3rcon 05-23-2017 01:40 AM

Quote:

Originally Posted by Laserbeak (Post 5710423)
Why do you need multiple connections, it's not likely to increase your speed?

Yes.

hack3rcon 05-23-2017 01:41 AM

I like to use "wget" not "axel" !!!

pan64 05-23-2017 05:58 AM

https://www.tecmint.com/download-managers-for-linux/
in general look for linux download manager - if interested.
wget itself is unable to use multiple connections. If you really want to use wget just do, but there will be only single connections or probably you can try to implement that feature into it.

ondoho 05-23-2017 10:05 AM

Quote:

Originally Posted by hack3rcon (Post 5714047)
I like to use "wget" not "axel" !!!

read the "documentation" !!!
either the feature is there, or it isn't !!!

273 05-23-2017 11:52 AM

Quote:

Originally Posted by pan64 (Post 5714102)
https://www.tecmint.com/download-managers-for-linux/
in general look for linux download manager - if interested.
wget itself is unable to use multiple connections. If you really want to use wget just do, but there will be only single connections or probably you can try to implement that feature into it.

I have to admit that I don't like the look of the linked page at all.
It might just be me but I see no indication of why a download manager is required at all (outside of wget) and I also notice that almost all the suigestions require installing packages outside of the repositories.
As mentioned above, I am not entirely dismissive of multiple-connection downloading and, as it happens, I've nothing against installing third-party software on Linux. However, that Debian doesn't have this type of "download manager" in the repositories makes me doubtful of their utility.
If Gentoo, Arch, Fedora and Slackware are similarly missing them I would suggest that the major Linux developers don't think they're worth bothering with.

pan64 05-24-2017 01:02 AM

first of all I totally agree with you. But actually I do not care about that, I just wanted to show the available software without thinking about their usability. OP will decide....
last but not least I think there can be a reason to use download manager, probably not for increasing download speed but because of auto retry and other features.

273 05-24-2017 01:22 AM

Quote:

Originally Posted by pan64 (Post 5714521)
first of all I totally agree with you. But actually I do not care about that, I just wanted to show the available software without thinking about their usability. OP will decide....
last but not least I think there can be a reason to use download manager, probably not for increasing download speed but because of auto retry and other features.

Fair enough, and I agree about things like auto retry. Funny though that they decided one should add a repository for uget when it's likely included in Ubuntu, Mint etc. as it is in Debian.


All times are GMT -5. The time now is 01:03 AM.