LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 12-14-2007, 09:05 AM   #1
Jinouchi
Member
 
Registered: Jul 2007
Location: WI, USA
Distribution: Debian, Lenny
Posts: 111

Rep: Reputation: 19
Wget: Error 403- Can I get around this?


Is there a way I can get around an error 403: Forbidden when using wget?
 
Old 12-14-2007, 10:32 AM   #2
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,398
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
If this is a problem related only to wget, but not other browsers, you may be able to spoof the site by using the wget '-U' option, giving it a user-agent description of another browser.
Code:
wget -U 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4' http://yourURL.com
To see what a working browser sends as a user-agent header, you can run netcat on your localhost, and have a browser try to fetch a page from it:
Code:
nc -l -p 8000 -v
Now, in your browser, go to 'http://localhost:8000'. Observe the user-agent header received by netcat. Cut and paste the string into the wget -U argument.
--- rod.
 
Old 12-14-2007, 11:27 AM   #3
Jinouchi
Member
 
Registered: Jul 2007
Location: WI, USA
Distribution: Debian, Lenny
Posts: 111

Original Poster
Rep: Reputation: 19
Ty, I didn't know you could do that. (unfortunatly, it still didn't work)

Ok, so here's the user agent

Code:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.10) Gecko/20071115 Iceweasel/2.0.0.10 (Debian-2.0.0.10-0etch1)
Here's what happened (in case you're wondering, I'm trying to download some wallpapers, don't want to do all x00+ individually)
Code:
***@***:/dir$ wget -U= 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.10) Gecko/20071115 Iceweasel/2.0.0.10 (Debian-2.0.0.10-0etch1)' -r http://www.wizards.com/magic/images/mtgcom/wallpapers
--11:19:07--  http://mozilla/5.0%20(X11;%20U;%20Linux%20i686;%20en-US;%20rv:1.8.1.10)%20Gecko/20071115%20Iceweasel/2.0.0.10%20(Debian-2.0.0.10-0etch1)
           => `mozilla/5.0 (X11'
Resolving mozilla... failed: Name or service not known.
--11:19:07--  http://www.wizards.com/magic/images/mtgcom/wallpapers
           => `www.wizards.com/magic/images/mtgcom/wallpapers'
Resolving www.wizards.com... 64.223.12.31
Connecting to www.wizards.com|64.223.12.31|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://www.wizards.com/magic/images/mtgcom/wallpapers/ [following]
--11:19:08--  http://www.wizards.com/magic/images/mtgcom/wallpapers/
           => `www.wizards.com/magic/images/mtgcom/wallpapers/index.html'
Reusing existing connection to www.wizards.com:80.
HTTP request sent, awaiting response... 403 Forbidden
11:19:08 ERROR 403: Forbidden.


FINISHED --11:19:08--
Downloaded: 0 bytes in 0 files
Did I do it correctly?
(ps if I did get this to work I would of corse use the --limit-rate and -w options.

Last edited by Jinouchi; 12-14-2007 at 11:29 AM.
 
Old 12-14-2007, 12:43 PM   #4
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,398
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
Nope. The -U option does not need a '=' between it and the argument.
--- rod.
 
Old 12-14-2007, 12:56 PM   #5
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,398
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
I was not able to use Mozilla to access the URL you cited, either. Seems to be a different problem.
--- rod.

Last edited by theNbomr; 12-14-2007 at 12:57 PM.
 
Old 12-22-2007, 11:33 PM   #6
Jinouchi
Member
 
Registered: Jul 2007
Location: WI, USA
Distribution: Debian, Lenny
Posts: 111

Original Poster
Rep: Reputation: 19
Well, that worked for a different website. Anyone got any other suggestions?
 
Old 12-23-2007, 01:12 AM   #7
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 64
If you go to the URL you specified using firefox, you get the same error (at least I do). wget won't fix what doesn't work anyway.
 
Old 06-11-2008, 05:36 AM   #8
arivendu
LQ Newbie
 
Registered: Apr 2006
Posts: 7

Rep: Reputation: 0
Unhappy cant get rid of ERROR:403

Quote:
Originally Posted by theNbomr View Post
If this is a problem related only to wget, but not other browsers, you may be able to spoof the site by using the wget '-U' option, giving it a user-agent description of another browser.
Code:
wget -U 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4' http://yourURL.com
To see what a working browser sends as a user-agent header, you can run netcat on your localhost, and have a browser try to fetch a page from it:
Code:
nc -l -p 8000 -v
Now, in your browser, go to 'http://localhost:8000'. Observe the user-agent header received by netcat. Cut and paste the string into the wget -U argument.
--- rod.

I did the same as advised by you, but i am still getting ERROR:403
i used the user agent as-- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4
help needed.
 
Old 06-12-2008, 12:31 PM   #9
Jinouchi
Member
 
Registered: Jul 2007
Location: WI, USA
Distribution: Debian, Lenny
Posts: 111

Original Poster
Rep: Reputation: 19
Well, there was one specific site that wouldn't accept the entire user agreement, like the one u posted. I tried simply
Code:
wget -U firefox http://xxx.xxx.com/
and, lol, surprisingly, it worked! you should try that if you're still stuck. Or maybe i set the user agent as "mozilla" instead of "firefox", can't remember. Just remember, the user agent you're trying to use has to be able to access the website in the first place.
 
Old 06-12-2008, 11:53 PM   #10
arivendu
LQ Newbie
 
Registered: Apr 2006
Posts: 7

Rep: Reputation: 0
Unhappy i am still stuck

Quote:
Originally Posted by Jinouchi View Post
Well, there was one specific site that wouldn't accept the entire user agreement, like the one u posted. I tried simply
Code:
wget -U firefox http://xxx.xxx.com/
and, lol, surprisingly, it worked! you should try that if you're still stuck. Or maybe i set the user agent as "mozilla" instead of "firefox", can't remember. Just remember, the user agent you're trying to use has to be able to access the website in the first place.
Ty, i tried your suggestion, but i am still not able to access the site thru Wget, though the same is working thru my browser, firefox 2.0
 
Old 06-13-2008, 12:16 AM   #11
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,398
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
Since your browser ID contains whitespace characters, you must enclose it in quotes, "double" or 'single'.
--- rod.

EDIT:
Oops. Misread that post, thinking the URL was part of the browser ID. Never mind.

Last edited by theNbomr; 06-13-2008 at 12:18 AM.
 
Old 06-13-2008, 01:46 PM   #12
Jinouchi
Member
 
Registered: Jul 2007
Location: WI, USA
Distribution: Debian, Lenny
Posts: 111

Original Poster
Rep: Reputation: 19
Would you mind sharing the URL? That way other people could work on the same situation. If not, that's ok.
 
Old 09-15-2008, 11:37 AM   #13
ras
LQ Newbie
 
Registered: May 2007
Location: Mountain View, CA
Distribution: GNU
Posts: 1

Rep: Reputation: 0
Thumbs up Thanks!

'Just wanted to say THANKS- I've been having this problems and using -U Mozilla works great.

(I wrote a quick script to update my web site with curl if the version on my disk differs from a cached copy- much nicer to run a command than all these strange gooey programs...)
-r
 
Old 10-26-2009, 09:20 PM   #14
glarrain
LQ Newbie
 
Registered: Oct 2009
Posts: 2

Rep: Reputation: 0
Thanks

Many thanks to those who have helped enormously!
wget -U firefox ...

So simple, so elegant!
 
Old 11-20-2010, 06:10 PM   #15
yeseen
LQ Newbie
 
Registered: Dec 2009
Distribution: Debian lenny
Posts: 6

Rep: Reputation: 0
Well, I agree, that was a very bright answer that helped me: -U mozilla
Very good.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Mysterious 403 error intramaweb Linux - Software 4 06-01-2006 05:42 PM
apache2 403 error eagletalontim Linux - Software 2 01-03-2006 10:39 PM
can't get anything but error 403 kuplo Linux - Software 2 11-02-2005 10:11 AM
Nagios (403 Error) cuiguy Linux - Software 1 04-05-2005 06:27 PM
wget problem - 403 error but I can dl the file in windows Mytherall Linux - Newbie 1 02-27-2004 10:04 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 03:09 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration