LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices



Reply
 
Search this Thread
Old 05-28-2009, 09:04 PM   #1
dedeco
LQ Newbie
 
Registered: Aug 2005
Posts: 6

Rep: Reputation: 0
[SOLVED] "wget -p" problem with PHP page


Hello,

I tried to make wget fetch a complete page (with called page-requisites):

Code:
wget -p "http://projecteuler.net/index.php?section=problems&id=246"

But once it finished, only two files were saved:

Code:
./projecteuler.net/index.php?section=problems&id=246
./projecteuler.net/robots.txt
And there should be more files, as you might see on the page (either the actual page or the downloaded file). The image files and the stylesheet file were not downloaded (despite the "-p").

What should I do for this to work? I guess it is because the file is not ended with ".htm" or ".html", but ends with ".php". (???) Not sure, though.

Dedeco

Last edited by dedeco; 06-04-2009 at 08:00 PM. Reason: Problem solved, editing title and tags.
 
Old 06-02-2009, 03:19 PM   #2
mrog
Member
 
Registered: Jul 2003
Location: Ontario, Canada
Distribution: Debian, Ubuntu
Posts: 39

Rep: Reputation: 15
Look at the robots.txt file that was download. It says "Disallow: /". Therefore the people that created/own the site don't want you to do this. Wget respects the robots.txt file.
 
Old 06-04-2009, 07:58 PM   #3
dedeco
LQ Newbie
 
Registered: Aug 2005
Posts: 6

Original Poster
Rep: Reputation: 0
Thumbs up

Yes, that was it.

Disallowing all robots is probably not the best idea, IMHO.

I have to disrespect this file to do what otherwise would be a pain.

Of course, care should always be taken with the Internet, as I will have in doing what I want. But forbidding everything should not be the spirit.

Wikipedia's article about wget is pretty usefull, by the way.

Thank you.
 
  


Reply

Tags
robots, wget


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Urgent PHP problem with "undefined function: domxml_open_mem()"! Recomplie php? Oskare100 Linux - Server 0 12-27-2006 01:28 PM
Suggestion: for "subscribed threads" & "top of page" buttons Old_Fogie LQ Suggestions & Feedback 7 07-10-2006 06:10 PM
What is this "profile.php" page? manhinli LQ Suggestions & Feedback 3 12-07-2005 01:23 AM
wget fails when i want to download from a URL which contains "=" or "&' noware Linux - General 7 11-13-2005 08:35 AM
my web browser "mozilla fire fox" isn't rendering the page, rather opening the page amolgupta Linux - Software 2 07-26-2005 01:41 AM


All times are GMT -5. The time now is 08:04 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration