LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 03-07-2014, 02:49 PM   #1
nondem
LQ Newbie
 
Registered: Mar 2014
Posts: 2

Rep: Reputation: Disabled
Wget isn't cooperating...302 referrals and 404 while a Browser works fine.


I'll try to be brief

I manage a system that collects weather data and it uses wget to capture data from various pages the National Weather Service. They are changing the links for the data and I'm under a deadline to get it pointing to the new links before the scripts break.


This is a link to the information that I need WGET to dump to a file(wget -O filename.txt etc...)
http://forecast.weather.gov/product....F&issuedby=MLB

It works fine from a browser - Please see for yourself - but WGET gets a 404 error ultimately...the actual output is below.

I think it has something to do w/the 302 Redirect but have yet to find a way around it. I tried various options including sending a browsers user agent string but everything ends in a 404.

I'd owe whoever knows what is going on and how to fix it a beer.

_____________________________________________________

This command
wget http://forecast.weather.gov/product....F&issuedby=MLB

Give the following output:
[2] 25677
[3] 25678
--2014-03-07 15:40:48-- http://forecast.weather.gov/product.php?site=NWS
Resolving forecast.weather.gov... [3]- Done product=FWF
nondem@nondem:~/clscraper/temp$ 69.22.154.11, 69.22.154.35
Connecting to forecast.weather.gov|69.22.154.11|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: / [following]
--2014-03-07 15:40:48-- http://forecast.weather.gov/
Reusing existing connection to forecast.weather.gov:80.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: weather.gov [following]
--2014-03-07 15:40:48-- http://forecast.weather.gov/weather.gov
Reusing existing connection to forecast.weather.gov:80.
HTTP request sent, awaiting response... 404 Not Found
2014-03-07 15:40:48 ERROR 404: Not Found.
 
Old 03-08-2014, 09:45 AM   #2
Kenhelm
Member
 
Registered: Mar 2008
Location: N. W. England
Distribution: Mandriva
Posts: 360

Rep: Reputation: 170Reputation: 170
Try putting the URL in quotes; it contains special characters which need to be protected from the shell.
Code:
wget 'http://forecast.weather.gov/product.php?site=NWS&product=FWF&issuedby=MLB'
 
Old 03-08-2014, 10:17 AM   #3
nondem
LQ Newbie
 
Registered: Mar 2014
Posts: 2

Original Poster
Rep: Reputation: Disabled
Holy Crap that was it!

Thankyou upon thank-you


Quote:
Originally Posted by Kenhelm View Post
Try putting the URL in quotes; it contains special characters which need to be protected from the shell.
Code:
wget 'http://forecast.weather.gov/product.php?site=NWS&product=FWF&issuedby=MLB'
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Linux VM , Ping works but wget and browser does not open the page. karais Linux - Networking 2 10-19-2013 07:53 AM
Linux VM , Ping works but wget and browser does not open the page. karais Linux - Newbie 1 10-17-2013 02:10 AM
Installed Opera browser and it really works fine everal Slackware 25 02-14-2009 04:27 AM
Stopping wget to follow 302 codes? northy_ie Programming 8 09-03-2007 01:13 AM
wget gives me 404 when IE works just fine...? ivj Linux - Software 7 07-24-2004 10:13 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 12:52 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration