-   Programming (
-   -   Interacting with a website using Perl and LWP (

saldon 05-05-2011 01:55 PM

Interacting with a website using Perl and LWP
I'm trying to write a Perl script that will go out to a website, log in, pull up a query page, fill-in and submit the query, extract a few specific lines from the query results, and finally place that output into a file.

I'm having difficulty with the log in portion. It appears that rather than using the built-in Apache authentication, the developer decided to write their own authentication scheme that uses cookies.

I setup a cookie jar but I'm not certain I'm actually getting and keeping the cookie. When I try to go to a page after the log in page I get an error saying I'm not logged in or my session has expired.

Here's the code as I have it thus far.


use strict;
use LWP;
use HTTP::Cookies;

my $url = '';
my $proxy = 'http://proxy:80';

# Create a useragent
my $ua = LWP::UserAgent->new;

# Set the proxy
$ua->proxy(['https'], $proxy);

# Setup my cookie jar
#$ua->cookie_jar( {} );
my $cookie_jar = HTTP::Cookies->new(
        file => "/Perl/cookies.lwp",
        autosave => 1,
$ua->cookie_jar( $cookie_jar );

# Post the username and password
my $res = $ua->post( $url ,
          'username' => 'XXXXXXXX',
          'password' => 'XXXXXXXX',

# Print out the response
print $res->content;
print "\n ***** \n";

$res = $ua->post(

print $res->content;
print "\n ***** \n";

Thanks for any help.


P.S. I had to sanitize a few things from my code. i.e. username and password

Sergei Steshenko 05-05-2011 03:33 PM

I haven't read the documentation, but does


path make sense ?

saldon 05-09-2011 09:27 AM

The path is not really relevant and I'm certain that is not the problem. I'm building the script in my home directory and the cookie file is actually in the same directory as the script itself. I just changed the path in my posting from the actual path I used for security reasons. I don't think sharing my username with the world is a good idea.

All times are GMT -5. The time now is 04:54 AM.