LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-06-2012, 02:12 AM   #1
tawandasm
LQ Newbie
 
Registered: Mar 2012
Location: zimbabwe, mutare
Distribution: Fedora 16
Posts: 3

Rep: Reputation: Disabled
Question Tricky WGET


hie

I have this site im trying to mirror / download onto my machine for offline viewing but am failing to put the right switches / wget commands. I need the whole "Message section" & "Bible + Lexicon section" as well...

http://nt.scbbs.com/cgi-bin/om_isapi...wse_Frame_Pg42

http://nt.scbbs.com/cgi-bin/om_isapi...wse_Frame_Pg42

Pliz help {a bit new to WGET ;-/}

Thanx

[tawanda@eagle ~]$ wget -m -r -k -p -L --convert-links -P ./MsgArch http://nt.scbbs.com/cgi-bin/om_isapi...wse_Frame_Pg42
[1] 2564
[2] 2565
[tawanda@eagle ~]$ --2012-03-06 08:16:49-- http://nt.scbbs.com/cgi-bin/om_isapi...ntID=547754057
Resolving nt.scbbs.com... 66.6.218.75
Connecting to nt.scbbs.com|66.6.218.75|:80... connected.
HTTP request sent, awaiting response... 302 Object Moved
Location: http://nt.scbbs.com/cgi-bin/om_isapi...ntID=291922769 [following]
--2012-03-06 08:16:50-- http://nt.scbbs.com/cgi-bin/om_isapi...ntID=291922769
Reusing existing connection to nt.scbbs.com:80.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “./MsgArch/nt.scbbs.com/cgi-bin/om_isapi.dll?clientID=291922769”

[ <=> ] 4,245 --.-K/s in 0.004s

Last-modified header missing -- time-stamps turned off.
2012-03-06 08:16:50 (1.01 MB/s) - “./MsgArch/nt.scbbs.com/cgi-bin/om_isapi.dll?clientID=291922769” saved [4245]

FINISHED --2012-03-06 08:16:50--
Downloaded: 1 files, 4.1K in 0.004s (1.01 MB/s)
Converting ./MsgArch/nt.scbbs.com/cgi-bin/om_isapi.dll?clientID=291922769... nothing to do.
Converted 1 files in 0 seconds.

[1]- Done wget -m -r -k -p -L --convert-links -P ./MsgArch http://nt.scbbs.com/cgi-bin/om_isapi...ntID=547754057
[2]+ Done infobase=message2010.nfo
 
Old 03-07-2012, 03:41 PM   #2
T3RM1NVT0R
Senior Member
 
Registered: Dec 2010
Location: Internet
Distribution: Linux Mint, SLES, CentOS, Red Hat
Posts: 2,385

Rep: Reputation: 476Reputation: 476Reputation: 476Reputation: 476Reputation: 476
@ Reply

Hi tawandasm,

Welcome to LQ!!!

I have checked this site and it does not appear that it allow you to follow the link on the page via automated process. So basically you cannot download the pages which are linked on the site that you have mentioned.

For your information you can use the following command to create a mirror of a website on your system:

Code:
wget -mk http://www.sitename.com
But this command will not work in your case or in a case where site requires an authentication.
 
Old 03-09-2012, 05:04 AM   #3
tawandasm
LQ Newbie
 
Registered: Mar 2012
Location: zimbabwe, mutare
Distribution: Fedora 16
Posts: 3

Original Poster
Rep: Reputation: Disabled
hmmn...!!! thanx for the post & advice.... smway smhow will find a way to crack this, just nids a lil bit o patience & smthing should crop up.... havent given up yet.... Thanx
 
Old 03-09-2012, 08:30 AM   #4
T3RM1NVT0R
Senior Member
 
Registered: Dec 2010
Location: Internet
Distribution: Linux Mint, SLES, CentOS, Red Hat
Posts: 2,385

Rep: Reputation: 476Reputation: 476Reputation: 476Reputation: 476Reputation: 476
@ Reply

You're welcome.

I would like to mention that we at LQ do not promote the use of sms language as that creates confusion. Spell out your words correctly and you will get much better response for your post / queries.
 
1 members found this post helpful.
Old 03-14-2012, 02:55 AM   #5
tawandasm
LQ Newbie
 
Registered: Mar 2012
Location: zimbabwe, mutare
Distribution: Fedora 16
Posts: 3

Original Poster
Rep: Reputation: Disabled
thanks & my bad... sorry about the full language grammar here is a switch i found and am about to try try it out

Mask User Agent and Display wget like Browser Using wget –user-agent

Some websites can disallow you to download its page by identifying that the user
agent is not a browser. So you can mask the user agent by using –user-agent
options and show wget like a browser as shown below.

$ wget --user-agent="Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3)
Gecko/2008092416 Firefox/3.0.3" URL-TO-DOWNLOAD
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to resume an interrupted wget using wget.log? misterJ Linux - Software 2 06-19-2011 02:21 PM
tricky filename x/y ; ravi.ravix Linux - Newbie 1 06-26-2006 12:19 PM
tricky XF86Config Raphael M LinuxQuestions.org Member Success Stories 1 06-21-2004 06:07 PM
depencies what for :) tricky nakkaya Linux - General 3 02-19-2003 10:20 PM
Tricky!!!! isajera General 39 08-25-2002 06:52 PM


All times are GMT -5. The time now is 12:44 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration