LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 01-19-2010, 06:58 AM   #1
ravi.xolve
Member
 
Registered: Jul 2006
Posts: 89

Rep: Reputation: 15
Exclamation Can't replicate the site using wget


I want to replicate this small howto (http://legos.sourceforge.net/HOWTO) using wget.

I use the following command:
Code:
wget --recursive --tries=100 --timeout=60s --spider --verbose --convert-links --html-extension --page-requisites --user-agent="Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.2b5) Gecko/20091204 Firefox/3.6b5" -e robots=off http://legos.sourceforge.net/HOWTO
However I just get a single file and not the other pages and that file too is not html.
 
Old 01-19-2010, 07:11 AM   #2
Dave_Devnull
Member
 
Registered: May 2009
Posts: 142

Rep: Reputation: 24
I've run your command and I get a full directory:

legos.sourceforge.net/
total 20K
drwxr-xr-x 5 root root 4.0K 2010-01-19 13:09 .
drwxr-xr-x 11 root root 4.0K 2010-01-19 13:09 ..
drwxr-xr-x 2 root root 4.0K 2010-01-19 13:09 API
drwxr-xr-x 5 root root 4.0K 2010-01-19 13:09 files
drwxr-xr-x 2 root root 4.0K 2010-01-19 13:09 HOWTO
...
 
Old 01-19-2010, 07:20 AM   #3
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
Moved: This thread is more suitable in Linux Software and has been moved accordingly to help your thread/question get the exposure it deserves.
 
Old 01-19-2010, 07:36 AM   #4
craigevil
Senior Member
 
Registered: Apr 2005
Location: OZ
Distribution: Debian Sid
Posts: 4,732
Blog Entries: 12

Rep: Reputation: 455Reputation: 455Reputation: 455Reputation: 455Reputation: 455
wget --random-wait -r -p -e robots=off -U mozilla http://www.example.com

http://www.unixmen.com/linux-tutoria...ine-cheatsheet
 
Old 01-19-2010, 11:42 PM   #5
ravi.xolve
Member
 
Registered: Jul 2006
Posts: 89

Original Poster
Rep: Reputation: 15
I already know how to use wget.


The question I want to ask I just want to replicate the HOWTO directory and I am not getting what is expected and I can't find the reason for it.

Quote:
Originally Posted by Dave_Devnull View Post
I've run your command and I get a full directory:

legos.sourceforge.net/
total 20K
drwxr-xr-x 5 root root 4.0K 2010-01-19 13:09 .
drwxr-xr-x 11 root root 4.0K 2010-01-19 13:09 ..
drwxr-xr-x 2 root root 4.0K 2010-01-19 13:09 API
drwxr-xr-x 5 root root 4.0K 2010-01-19 13:09 files
drwxr-xr-x 2 root root 4.0K 2010-01-19 13:09 HOWTO
...
 
  


Reply

Tags
mirroring, wget


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Curiousity: FTP via Kget = "Username+pw req'd for this site" but FTP via Wget doesn't GrapefruiTgirl Linux - Software 6 09-18-2009 01:23 PM
How DNS works? - how does it replicate? neocontrol Linux - Networking 1 08-23-2007 07:56 AM
site download with wget ygloo Linux - General 7 09-27-2006 05:49 AM
download/save a site- wget kpachopoulos Linux - Software 1 07-26-2005 12:22 PM
I want to download ftp-site files via wget and socks5 proxy server. jiawj Red Hat 2 10-28-2004 03:32 PM


All times are GMT -5. The time now is 10:36 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration