LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-03-2007, 07:58 AM   #16
timmeke
Senior Member
 
Registered: Nov 2005
Location: Belgium
Distribution: Red Hat, Fedora
Posts: 1,515

Rep: Reputation: 61

Without looking into detail at your command, I immediately spotted something wrong:
Quote:
--follow-tags='*.wml'
--follow-tags is to specify the HTML/XML/WML tags that wget should follow, such as the anchor tag (<a href="..."> for hyperlinks to other files). It's not to be used to specify a filename pattern, like *.wml for "all .wml files". This is done with -A option instead:
-A '*.wml'
Note the single quotes, to prevent the shell's filename expansion.

To use --follow-tags, maybe you can start by looking up some examples via Google?
 
Old 04-03-2007, 11:11 PM   #17
brave heart
Member
 
Registered: Mar 2007
Distribution: Fedora-14
Posts: 155
Blog Entries: 8

Original Poster
Rep: Reputation: 15
Thank you, timmeke.

I looked "--follow-tags" from google&google linux, but no any result in this aspect.

Could you please provide any? Or could you just give out a command which can download this wml page and also give an example?

I feel I will almost give up this several days of trying to use wget! It's a horrible experience! :-(

Thank you once again.
 
Old 04-04-2007, 02:39 AM   #18
timmeke
Senior Member
 
Registered: Nov 2005
Location: Belgium
Distribution: Red Hat, Fedora
Posts: 1,515

Rep: Reputation: 61
Judging from the example you posted earlier, only anchor tags (of the form <a href="someURL">) are used in your case. These are supported by default, so the --follow-tags option isn't strictly needed.

If you do add it, it should probably be something like:
Code:
--follow-tags=a
--follow-tags supports a list of comma separated tagnames too, like --follow-tags=a,area for all <a> and <area> tags.

Don't feel discouraged yet. You're on the right path, so just be patient.

Searching for "wget --follow-tags example" on Google turned up:
http://www.editcorp.com/Personal/Lar...v1/wget_7.html and
http://en.wikipedia.org/wiki/Wget
both containing examples.

Maybe you could try a command like this?
Code:
wget -p -m -k --follow-tags=a -A '*.wml' http://195.235.160.80/index.wml
Leave out -A '*.wml' if you want to download also other files besides .wml ones.
 
Old 04-04-2007, 05:23 AM   #19
brave heart
Member
 
Registered: Mar 2007
Distribution: Fedora-14
Posts: 155
Blog Entries: 8

Original Poster
Rep: Reputation: 15
Thanks, buddy. All of you, acid_kewpie and XavierP are generous people.

This time I tried
# wget -p -m -k --follow-tags=a http://195.235.160.80/index.wml
got only "index.wml"

I tried
# wget -p -m -k --follow-tags=a http://195.235.160.80
got "index.html" and it's related file

Anyway, thank you all once again.
 
Old 04-04-2007, 06:15 AM   #20
timmeke
Senior Member
 
Registered: Nov 2005
Location: Belgium
Distribution: Red Hat, Fedora
Posts: 1,515

Rep: Reputation: 61
Well, I tried it too and it seems that wget has a problem with the fact that the downloaded file is not HTML nor XML (see content-type header).

To overcome this, I tried to first download just the index.wml file and then issuing
Code:
wget -H -i index.wml --force-html --base=http://195.235.160.80
-H for spanning hosts,
-i index.wml to parse the local index.wml file for URLs (using this option, the file doesn't have to be a HTML or XML file),
--force-html forces parsiing of the wml file as if it was a valid html file (for the <a href=...> tags);
--base=http://195.235.160.80 is used to interprete the relative paths in the <a href=...> tags.

This downloads a few files, but hangs on a specific file, which may be due to a slow internet connection, the remote server being temporarily unavailable, ...

In theory, you could re-iterate this for each .wml file downloaded. HTML and XHTML files that are downloaded can be parsed recursively using "-r -l inf" or "-m".
 
Old 04-05-2007, 02:47 AM   #21
brave heart
Member
 
Registered: Mar 2007
Distribution: Fedora-14
Posts: 155
Blog Entries: 8

Original Poster
Rep: Reputation: 15
Hi, timmeke. Thank you for your reply!

This time I tried
# wget -H -r -c -m -t2 -i index.wml --force-html --base=http://195.235.160.80

and I got index.wml and all related folders. However, only index*.*wml file in each folder and no other related files downloaded.
 
Old 04-09-2007, 06:01 AM   #22
brave heart
Member
 
Registered: Mar 2007
Distribution: Fedora-14
Posts: 155
Blog Entries: 8

Original Poster
Rep: Reputation: 15
Hi, all

What I mean is that I just want to use wget to download all the contents of "http://195.235.160.80/index.wml".

To use any option or need any script?

Pls help!

Thanks a lot!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Script to download file using wget linuxnewbie82 Programming 6 02-14-2007 05:48 AM
how to specify the name of the wget output file datasink Linux - Networking 2 01-23-2007 08:39 PM
wGet is there a command for file size? SonoranFun Linux - Software 2 01-02-2006 04:56 PM
where does konqueror and wget downloaded file go eatmars Linux - Software 2 08-28-2005 11:11 PM
wget any version of a file? shishimo Linux - General 1 07-08-2004 02:37 PM


All times are GMT -5. The time now is 02:35 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration