wget and links2 can't access web page.

fakie_flip · 12-12-2007, 11:55 PM

Why do none of these work?

wget http://www.blackcats-games.net/signup.php

links2 -source http://www.blackcats-games.net/signup.php

links2 -dump http://www.blackcats-games.net/signup.php

The only way I've been able to access that page is by clicking a link. I need to be able to access it from a script, so I can know when registration opens. Is there any possible way to do this? Here's what I have.

wget -P/tmp http://www.blackcats-games.net/signup.php > /tmp/logfile 2>&1; if (grep -q 'current user account limit' /tmp/signup.php); then DISPLAY=:0 xmessage 'account registration opened at blackcat'; cat /tmp/logfile; fi; rm -f /tmp/signup.php /tmp/logfile > /dev/null 2>&1

matthewg42 · 12-13-2007, 05:37 PM

The link doesn't do anything in firefox either, if you go directly to it. Probably there is something with javascript on the page where you found the link, which adds some parameters or something like that.

fakie_flip · 12-13-2007, 06:40 PM

Quote:

Originally Posted by matthewg42

The link doesn't do anything in firefox either, if you go directly to it. Probably there is something with javascript on the page where you found the link, which adds some parameters or something like that.

I think the web server knows if I clicked on the link or not by the http referrer sent to it, so I want to spoof it. How can I know what http referrer is sent to the server when I click on the register link from the homepage in Firefox?

gilead · 12-13-2007, 07:15 PM

Quote:

Originally Posted by fakie_flip

How can I know what http referrer is sent to the server when I click on the register link from the homepage in Firefox?

I'd just run wireshark (used to be called ethereal) and sniff the packets - you'll get all the gorey details that way.

matthewg42 · 12-13-2007, 07:50 PM

I think the referrer string is usually just the URL of the page where you found the link, no?

gilead · 12-13-2007, 08:02 PM

That's right, but I don't know enough about it to know if for example, the page is changing the referer (http://en.wikipedia.org/wiki/Referer#Referer_hiding) or if it's using the correct URL.

archtoad6 · 01-11-2008, 04:34 PM

http://www.blackcats-games.net/signup.php:

Quote:

NO, the link is NOT broken, we are closed to new signups. The page is missing intentionally!

That's today, any chance the same thing was happening a month ago?