script to download a webpage

pankaj99 · 09-23-2006, 04:07 AM

somebody please give me a perl/python script to download a url.

ghostdog74 · 09-23-2006, 04:12 AM

Quote:

Originally Posted by pankaj99

somebody please give me a perl/python script to download a url.

Instead of giving it to you, you can do a search on the web and try to program it yourself after reading an example like this:
http://docs.python.org/lib/module-urllib2.html
http://www.voidspace.org.uk/python/a.../urllib2.shtml

or this
http://www.perl.com/pub/a/2002/08/20/perlandlwp.html

pankaj99 · 09-23-2006, 04:22 AM

thanks ghostdog74.
Actually, I don't know perl/python just trying out something ,and for that
I need to download a webpage and parse it.

Anyway,
now how do i tunnel my request through a proxy server?
does perl provides any mechanism to use a proxy ?

ghostdog74 · 09-23-2006, 05:56 AM

Quote:

Originally Posted by pankaj99

thanks ghostdog74.
Actually, I don't know perl/python just trying out something ,and for that
I need to download a webpage and parse it.

Anyway,
now how do i tunnel my request through a proxy server?
does perl provides any mechanism to use a proxy ?

Perl:
http://www.oreilly.com/openbook/webc...h05.html#34288
look for "Using LWP::UserAgent"

pankaj99 · 09-23-2006, 06:28 AM

thanks that was really helpful.