LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 05-14-2017, 02:10 AM   #1
hack3rcon
Senior Member
 
Registered: Jan 2015
Posts: 1,432

Rep: Reputation: 10
Post wget with multiple connection.


Hello.
How can I set "wget" for download a file with multiple connections?


Thank you.
 
Old 05-14-2017, 02:21 AM   #2
Laserbeak
Member
 
Registered: Jan 2017
Location: Manhattan, NYC NY
Distribution: Mac OS X, iOS, Solaris
Posts: 508

Rep: Reputation: 143Reputation: 143
Why do you need multiple connections, it's not likely to increase your speed?
 
Old 05-14-2017, 03:48 AM   #3
273
LQ Addict
 
Registered: Dec 2011
Location: UK
Distribution: Debian Sid AMD64, Raspbian Wheezy, various VMs
Posts: 7,585

Rep: Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351
I'm not sure that wget can do that -- have you looked at axel instead?
 
Old 05-14-2017, 04:44 AM   #4
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,611
Blog Entries: 3

Rep: Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857
Using a torrent will allow use of multiple connections and thus increase speed, but as far as I know wget just does a single connection.
 
Old 05-14-2017, 06:12 AM   #5
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 17,237
Blog Entries: 10

Rep: Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160
there's a myth of increasing download speeds through "multiple download slots" or some such technobabble.
i think it comes from certain commercial download sites like megaupload etc., that would limit bandwidth per user.
some addons/software claimed to be able to circumvent that restriction (and maybe did until the providers circumvented THAT again), but i just don't see how wget has anything to do with it.
 
Old 05-14-2017, 06:15 AM   #6
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,611
Blog Entries: 3

Rep: Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857Reputation: 2857
Quote:
Originally Posted by ondoho View Post
there's a myth of increasing download speeds through "multiple download slots" or some such technobabble.
i think it comes from certain commercial download sites like megaupload etc., that would limit bandwidth per user.
some addons/software claimed to be able to circumvent that restriction (and maybe did until the providers circumvented THAT again), but i just don't see how wget has anything to do with it.
Even with multiple connections, it would still be using a single source and thus be limited. Since torrents use multiple sources, the throughput is much, much higher and limited only by the destination bandwidth, at least if there are enough seeds available.
 
Old 05-14-2017, 06:16 AM   #7
273
LQ Addict
 
Registered: Dec 2011
Location: UK
Distribution: Debian Sid AMD64, Raspbian Wheezy, various VMs
Posts: 7,585

Rep: Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351
Quote:
Originally Posted by ondoho View Post
there's a myth of increasing download speeds through "multiple download slots" or some such technobabble.
Strangely enough in my limited testing I found that axel seems to be able to download quicker than wget from sites where the downkoad is slower than my internet connection. I have been sceptical about the issue so even with some limited evidence I'm not yet prepared to state that multiple connections is definitely faster but it seems there may be some truth in it under certain conditions at least.
 
Old 05-14-2017, 06:38 AM   #8
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 17,237
Blog Entries: 10

Rep: Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160
i'm looking at search results for "download multi" from my package manager, and there's about a dozen AUR packages (all of them unsupported, as AUR packages always are).
i'm reading things like
"Xtreme Download Manager is a powerful tool to increase download speed up-to 500%, save videos from video sharing sites and integration with ANY browser."
and
"A full featured, advanced, multi-threaded, multisegment download manager and accelerator."

it all sounds a lot like snake oil to me, but so many coders can't be completely wrong i guess.

i will have a closer look.
 
Old 05-14-2017, 06:44 AM   #9
273
LQ Addict
 
Registered: Dec 2011
Location: UK
Distribution: Debian Sid AMD64, Raspbian Wheezy, various VMs
Posts: 7,585

Rep: Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351
Quote:
Originally Posted by ondoho View Post
it all sounds a lot like snake oil to me,
Oh, to me too and I was surprised when axel seemed faster than wget. I think it may depend very much on the reasons why wget isn't maxing out the link.
 
Old 05-14-2017, 07:06 AM   #10
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 17,237
Blog Entries: 10

Rep: Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160
ok, this is the one got the most votes on AUR:
https://www.flareget.com/
no source code. go figure.

i now found axel, from the man page:
"Axel is a program that downloads a file from a FTP or HTTP server through multiple connection. Each connection downloads its own part of the file."
just like i thought, this only makes sense when the provider throttles bandwidth (and is stupid enough to not recognize this trick).
 
Old 05-14-2017, 08:12 AM   #11
273
LQ Addict
 
Registered: Dec 2011
Location: UK
Distribution: Debian Sid AMD64, Raspbian Wheezy, various VMs
Posts: 7,585

Rep: Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351Reputation: 2351
Quote:
Originally Posted by ondoho View Post
ok, this is the one got the most votes on AUR:
https://www.flareget.com/
no source code. go figure.

i now found axel, from the man page:
"Axel is a program that downloads a file from a FTP or HTTP server through multiple connection. Each connection downloads its own part of the file."
just like i thought, this only makes sense when the provider throttles bandwidth (and is stupid enough to not recognize this trick).
I think that's roughly how it works, yes, but I think it could also combat unintentional throttling due to server processes not been given enough resources or possibly processes scanning the outbound data. Sadly I don't know enough about this kind of thing so the previous may be nonsense but something like that would make sense to me. A bit like when, sometimes, it seems quicker to copy files from a file sever simultaneously rather than one after the other.
 
Old 05-23-2017, 01:40 AM   #12
hack3rcon
Senior Member
 
Registered: Jan 2015
Posts: 1,432

Original Poster
Rep: Reputation: 10
Quote:
Originally Posted by Laserbeak View Post
Why do you need multiple connections, it's not likely to increase your speed?
Yes.
 
Old 05-23-2017, 01:41 AM   #13
hack3rcon
Senior Member
 
Registered: Jan 2015
Posts: 1,432

Original Poster
Rep: Reputation: 10
I like to use "wget" not "axel" !!!
 
Old 05-23-2017, 05:58 AM   #14
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,490

Rep: Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532
https://www.tecmint.com/download-managers-for-linux/
in general look for linux download manager - if interested.
wget itself is unable to use multiple connections. If you really want to use wget just do, but there will be only single connections or probably you can try to implement that feature into it.
 
Old 05-23-2017, 10:05 AM   #15
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 17,237
Blog Entries: 10

Rep: Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160Reputation: 5160
Quote:
Originally Posted by hack3rcon View Post
I like to use "wget" not "axel" !!!
read the "documentation" !!!
either the feature is there, or it isn't !!!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
wget multiple files with same extension glennbtn Linux - Newbie 1 09-08-2016 12:10 PM
Multiple Connections using wget utw-mephisto Linux - General 8 07-16-2012 02:49 AM
wget, download from multiple servers skuzye Linux - Software 4 05-27-2009 03:09 PM
cron+wget = multiple downloads stokilo Linux - Newbie 1 03-02-2009 11:49 AM
running wget multiple times quickk Linux - Newbie 1 09-16-2004 06:53 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 04:01 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration