LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Desktop
User Name
Password
Linux - Desktop This forum is for the discussion of all Linux Software used in a desktop context.

Notices


Reply
  Search this Thread
Old 07-25-2009, 08:04 PM   #1
pratap.iisc
LQ Newbie
 
Registered: Nov 2006
Posts: 19

Rep: Reputation: 1
Using wget or curl command to download from dynamic urls


Dear all,
I want to use two of the fantastic commands available in linux e.g. wget and curl which help me download files with direct and straightforward links right from the command prompt without going into any gui browser!
But unfortunately, I am not being able to download the files which would be linked with dynamic urls (urls containing "?" in them which is probably used for redirection).
The link to all the the IEEE papers also fall in this category...Eg. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5173739&isnumber=5173690
Also some times some of the useful software also come in this format E.g. "http://www.mozilla.com/products/download.html?product=firefox-3.5.1&os=win&lang=en-US"
While using wget to download files like this I get the error...(I am attaching the complete log...)

Command: user@machine:~> wget "http://www.mozilla.com/products/download.html?product=firefox-3.5.1&os=win&lang=en-US"

--04:33:52-- http://www.mozilla.com/products/down...win&lang=en-US
=> `download.html?product=firefox-3.5.1&os=win&lang=en-US'
Resolving iiscproxy.serc.iisc.ernet.in... aa.bb.cc.dd, aa.bb.cc.ee, aa.bb.cc.fe, ...
Connecting to iiscproxy.serc.iisc.ernet.in|aa.bb.cc.dd|:3128... connected.
Proxy request sent, awaiting response... 302 Moved Temporarily
Location: http://www.mozilla.com/en-US/product...win&lang=en-US [following]
--04:33:53-- http://www.mozilla.com/en-US/product...win&lang=en-US
=> `download.html?product=firefox-3.5.1&os=win&lang=en-US'
Connecting to iiscproxy.serc.iisc.ernet.in|aa.bb.cc.dd|:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: unspecified [text/html]
download.html?product=firefox-3.5.1&os=win&lang=en-US: File name too long

Cannot write to `download.html?product=firefox-3.5.1&os=win&lang=en-US' (File name too long).

Command:
user@machine:~> wget "http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5173739&isnumber=5173690"

--04:58:39-- http://ieeexplore.ieee.org/stamp/sta...number=5173690
=> `stamp.jsp?tp=&arnumber=5173739&isnumber=5173690'
Resolving iiscproxy.serc.iisc.ernet.in... aa.bb.cc.dd, 10.66.70.81, 10.16.40.12, ...
Connecting to iiscproxy.serc.iisc.ernet.in|aa.bb.cc.dd|:3128... connected.
Proxy request sent, awaiting response... 302 Moved Temporarily
Location: http://ieeexplore.ieee.org/stamp/sta...number=5173690 [following]
--04:58:40-- http://ieeexplore.ieee.org/stamp/sta...number=5173690
=> `stamp.jsp?tp=&arnumber=5173739&isnumber=5173690'
Connecting to iiscproxy.serc.iisc.ernet.in|aa.bb.cc.dd|:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: 612 [text/html]
stamp.jsp?tp=&arnumber=5173739&isnumber=5173690: File name too long

Cannot write to `stamp.jsp?tp=&arnumber=5173739&isnumber=5173690' (File name too long).



Can anybody suggest any workaround for these type of links?

Waiting for some positive responses,
-Pratap
 
Old 07-26-2009, 03:52 AM   #2
JulianTosh
Member
 
Registered: Sep 2007
Location: Las Vegas, NV
Distribution: Fedora / CentOS
Posts: 674
Blog Entries: 3

Rep: Reputation: 90
for wget, you could use the -o option to specify the output file...

wget -o /tmp/myDownload "http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5173739&isnumber=517369"
 
Old 07-26-2009, 11:55 PM   #3
archtoad6
Senior Member
 
Registered: Oct 2004
Location: Houston, TX (usa)
Distribution: MEPIS, Debian, Knoppix,
Posts: 4,727
Blog Entries: 15

Rep: Reputation: 234Reputation: 234Reputation: 234
I think you mean the -O option, from the wget man page:
Quote:
-o logfile
--output-file=logfile
Log all messages to logfile. The messages are normally reported to standard error.

-O file
--output-document=file
The documents will not be written to the appropriate files, but all will be concatenated together and written to file. If - is used as file, documents will be printed to standard output, disabling link conversion. (Use ./- to print to a file literally named -.)
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
wget failing to download some URLs moob8 Linux - General 7 07-20-2011 07:50 AM
download only zip files using wget command Fond_of_Opensource Linux - Newbie 1 08-09-2006 04:47 AM
How to download web pages from a website using wget command Fond_of_Opensource Linux - Newbie 5 07-05-2006 10:50 AM
Pros and cons of CLI download tools [wget and curl] Conjurer Linux - Software 1 12-01-2005 03:45 PM
YOU for SUSE 9.1 - curl or wget? djc SUSE / openSUSE 1 02-15-2005 04:26 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Desktop

All times are GMT -5. The time now is 03:15 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration