LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 09-25-2007, 12:22 AM   #1
vargadanis
Member
 
Registered: Sep 2006
Posts: 248

Rep: Reputation: 30
Software to download websites X pages deep?


Hi all,

I would like to download a website XY pages deep.It contains frames and flash. Is there a way to be able to access the website's contents being offline? I am talking about hundreds of pages that are linked together and would be easy to download because there is an index page I just don't know about any software like that.
Thanx
 
Old 09-25-2007, 12:39 AM   #2
gilead
Senior Member
 
Registered: Dec 2005
Location: Brisbane, Australia
Distribution: Slackware64 14.0
Posts: 4,141

Rep: Reputation: 168Reputation: 168
Have you looked at the wget command? It should do most of what you want and is available with most (all?) distros.
 
Old 09-25-2007, 02:23 AM   #3
vargadanis
Member
 
Registered: Sep 2006
Posts: 248

Original Poster
Rep: Reputation: 30
How can I write a script that will send my username and passwd to the server and afterwards I am able to download the site? It is using sessions to auth so not even user side stuff such as cookies. Makes it sort of complicated.
 
Old 09-25-2007, 03:12 AM   #4
Su-Shee
Member
 
Registered: Sep 2007
Location: Berlin
Distribution: Slackware
Posts: 509

Rep: Reputation: 41
Despite of wget or curl, there's always Perl's LWP.
 
Old 09-25-2007, 02:23 PM   #5
gilead
Senior Member
 
Registered: Dec 2005
Location: Brisbane, Australia
Distribution: Slackware64 14.0
Posts: 4,141

Rep: Reputation: 168Reputation: 168
Please read the man page - wget has options like --keep-session-cookies and --save-cookies - do these help you?
 
Old 09-25-2007, 02:46 PM   #6
farslayer
LQ Guru
 
Registered: Oct 2005
Location: Northeast Ohio
Distribution: linuxdebian
Posts: 7,248
Blog Entries: 5

Rep: Reputation: 191Reputation: 191
httrack was simple for me to use to copy websites locally to my PC.. there is also a web front end if you want to bother setting that up.. webhtrack

Code:
it-etch:~# aptitude search httrack
p   httrack            - Copy websites to your computer (Offline browser)
p   webhttrack         - Copy websites to your computer, httrack with a Web interface
 
Old 10-28-2007, 09:23 AM   #7
archtoad6
Senior Member
 
Registered: Oct 2004
Location: Houston, TX (usa)
Distribution: MEPIS, Debian, Knoppix,
Posts: 4,727
Blog Entries: 15

Rep: Reputation: 234Reputation: 234Reputation: 234
I use wget.

It's one of the few programs whose man page could be accused of being too long, i.e. complicated. The flip side is that wget is very powerful & the man page explains it comprehensively. (Can you say "More than I wanted to know." )

Snarf has been recommended to me as being simpler. Using Google Linux, I find this quote: "snarf ... similar to wget, curl, ncftp, lynx, and fetch". So there are some others to investigate.

Please post your results, I am looking for a simpler alternative to wget.
 
Old 10-28-2007, 09:58 AM   #8
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Mint
Posts: 17,808

Rep: Reputation: 742Reputation: 742Reputation: 742Reputation: 742Reputation: 742Reputation: 742Reputation: 742
I was playing with Twill for a while. It seemed promising for what I wanted.
 
Old 03-01-2008, 08:20 AM   #9
frenchn00b
Senior Member
 
Registered: Jun 2007
Location: E.U., Mountains :-)
Distribution: Debian, Etch, the greatest
Posts: 2,546

Rep: Reputation: 57
Quote:
Originally Posted by pixellany View Post
I was playing with Twill for a while. It seemed promising for what I wanted.
as alternative to wget you ve got the link of this Twill you mentioned ?
thanks
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Can't download pages in Firefox 1.5.0.7 and Opera 9 keith11 Mandriva 1 01-06-2007 10:12 AM
Recommended websites for software search66 Linux - Newbie 5 02-23-2005 01:56 PM
any legal music download websites for linux? Keshet General 8 01-07-2005 06:54 AM
How do i download from a websites link....OK I'm pitiful glenn69 Linux - Newbie 3 05-04-2004 04:31 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 02:15 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration