Web Page Retrieval & Processing with PERL
 

I want to retrieve a web page (which I can't list due to posting rules) and retrieve the stock information for storage in a file.

I've tried wget and it's doesn't seem to work properly. I believe wget doesn't work properly because to wed page is .jsp file with multiple options.

I'm trying to do this in PERL as I'm a bit more familar with processing the data in PERL line by line. I just need to be able to retrieve the data in an automated way.

Does anybody have any tips?

You could use the curl.

Or read the 'man perlipc' to find out how you can connect by yourself. The http-protocol
is easy to speak. Something like:[code]
$netcat www.hut.fi 80
GET http://www.hut.fi/index.html http/1.0

[code] gives me a web-page.

You can use the LWP module - lots of info available if google.

http://search.cpan.org/~gaas/libwww-....74/lib/LWP.pm

all you probably need is lwp-mirror - try http://search.cpan.org for mirror :)

Sil