using wget to authenticate a SSL web page

AKviking · 07-04-2012, 12:38 PM

I have a web page that I had a script run at midnight and log on to and extract information to put into a database.

The maintainers of the page have upgraded and (good for them) made it more secure with SSL. However, now I'm trying to rewrite my script, and am unable to get past the SSL. The closest I get is a message that it cannot verify the certificate, so I know it's trying.

Code:

wget --secure-protocol=SSLv2 --http-user=USERNAME --http-password=PASSWORD https://usageinfo.website.com --no-check-certificate

This will return me to the logon screen, instead of logging on. If I remove the "--no-check-certificate" option, then I get:

Code:

https://usageinfo.website.com/
Resolving usageinfo.website.com... xxx.xxx.xxx.xxx
Connecting to usageinfo.website.com|xxx.xxx.xxx.xxx|:443... connected.
ERROR: cannot verify usageinfo.website.com's certificate, issued by â/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=Terms of use at https://www.verisign.com/rpa (c)10/CN=VeriSign Class 3 Secure Server CA - G3â
                Unable to locally verify the issuer's authority.
To connect to usageinfo.website.com insecurely, use "--no-check-certificate".

MisterBark · 07-04-2012, 10:04 PM

First I would put --no-check-certificate before, but I don't think it's the problem.

- Are you sure they didn't change anything else on the server?
- What is the http code returned ?
- Maybe you need to use the Basic authentication (encode a "login

assword" in base64)

AKviking · 07-05-2012, 10:29 AM

Quote:

Originally Posted by MisterBark

First I would put --no-check-certificate before, but I don't think it's the problem.

Tried that, but no change.

Quote:

- Are you sure they didn't change anything else on the server?

I have no idea, or know how to find out

Quote:

- What is the http code returned ?

302. When it fails on login, it just returns the main page, so it'll always be 302.

Quote:

- Maybe you need to use the Basic authentication (encode a "login : password" in base64)

That did not work.

It appears as though I'm unable to verify the Verisign CA certificate. There's options in wget to specify a file or directory to check certificates, but searching online, I've not found how to obtain such files that I could reference.

MisterBark · 07-05-2012, 10:48 AM

For the certificate try curl

AKviking · 07-06-2012, 10:27 AM

Quote:

Originally Posted by MisterBark

For the certificate try curl

I've gone down that road as well. I suspect I may be missing something, but I get similar results as with wget.

Code:

curl -uUSERNAME:PASSWORD https://web.domain.com -1 --cert-type DER --cacert /etc/ssl/certs/website.cer

curl: (77) error setting certificate verify locations:
  CAfile: /etc/ssl/certs/website.cer
  CApath: /etc/ssl/certs

I've saved the cert from my web browser, but am unsure if its format is correct. Still playing with that for a bit.

Thanks for your suggestions, it's keeping me motivated.

AKviking · 07-08-2012, 01:50 AM

Update: I believe I've got the proper certificate.

I used openssl to obtain the cert, and then ran through openssl again to verify, recieving an OK status.

So, now it appears as though my --post-data "UserName=user&Password=password" does not work, and so I'm thinking it's because now it's using requiring encryption, so perhaps a different approach?

I'm still trying with both wget & curl to see which one wins.