What I want: connect to my Rapidshare Premium page from my own application, written in Python.

Problem: how to pass username + password when requesting the page (https://ssl.rapidshare.de/cgi-bin/premiumzone.cgi)

I use the password manager in Firefox, so when I open the Rapidshare page, I get taken to the Premium section straightaway because FF fills in my credentials.
I want to write a screenscraper in Python (and perhaps my own uploadmanager, later) to read my own Premium page.
I can connect to RS using sgmllib.SGMLParser and urllib.urlopen, it let's me scrape my screen beautifully, except it's the login page, not the Premium section.

Can anyone explain to me how to send my credentials/authentication when requesting the url?



(note: I have a (paid for) Premium account, I don't intend to do any hacking beyond reading my own uploads.)

If Rapidshare has an XML RPC programming interface, then you can probably use that to pass remote procedure calls to the server from your client.

Unless the server-side app has "exposed" its functionality though an interface like XML-RPC, I don't see any easy way to achieve this.

Thx for your response.
I think the RS server has this exposed, they offer a tool called Rapidget which has functions like a down- and uploadqueue and a linkchecker, not sure though.

Firefox however seems to have a generic way to pass username/password to a login screen before requesting the page.

I'm new to the subject, the idea I've got is:
- request page
- walk through page looking for 'input'
- add 'value="<user>"' and 'value="<password>"' to code in the appropriate inputs
- send the page to the server

I doubt whether that's the way though, and then I don't know how to do it.
A better way seems to me (except I can't get it to work in code, it _does_ in a browser)
It might be a limitation of the library used (urllib or urllib2).

I'm open for suggestions and will continue experimenting in the meantime.
Might ask this question on a Mozilla forum too.