LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   is there a linux program that downloads an entire website? (https://www.linuxquestions.org/questions/linux-software-2/is-there-a-linux-program-that-downloads-an-entire-website-273972/)

lemuel 01-04-2005 10:34 PM

is there a linux program that downloads an entire website?
 
is there a linux program that downloads an entire website?

Elomis 01-04-2005 10:48 PM

If you are using firefox (and you should be) there is an extension called Scrapbook that lets you snake a website for offline viewing. Keep in mind the importance of respecting people's intellectual property rights when investigating this sort of thing.

:-)

E

uman 01-04-2005 10:57 PM

wget -r

lemuel 01-04-2005 11:00 PM

cool!
(although something that downloads an entire website in one click would be way cooler)

I'll go check that out.

thanks!

Shade 01-05-2005 01:51 AM

wget is great for this sort of thing. You can also use it to do things like grab all files with a certain extension in a directory on a website, or just mirror an entire site...

man wget

:)

--Shade

lemuel 01-05-2005 03:00 AM

um...

what does it mean to mirror an entire site?

scuzzman 01-05-2005 03:47 AM

To grab an entire site, directory structure and all, and repost it for viewing to alleviate some of the stress from the original server.

lemuel 01-05-2005 04:39 AM

oh ok thanks for the info

I already tried wget using options -r -k -p on a website and found out there are still other directories it didn't download. What's wrong?

MasterC 01-05-2005 04:46 AM

wget -m is for mirroring

Cool

lemuel 01-05-2005 04:56 AM

I also tried wget -m. how come there are other directories within the website that werent downloaded?

MasterC 01-06-2005 02:10 AM

Could be a robot.txt file? Sometimes people will set it up to deliberately stop people from recursively downloading every directory on their website... .htaccess...

Cool


All times are GMT -5. The time now is 05:15 PM.