Apache? Php? non-existent pages getting served...

bulliver · 08-14-2006, 06:34 PM

Sorry for the cryptic title but I am not sure exactly what is going on here, as it may be apache or php or....

I was looking at my apache access logs and noticed entries such as:
/sitemap.bot/writing/music/unix/code/photos/july_august_2005/
As far as I can tell all these were created by Yahoo! Slurp indexing my site.

Now: /sitemap.bot is a static page that has a link to every page on my site.
"writing", "music", "unix" and all other elements of this 'path' are top level directories of my webroot. So I figure Slurp is a little confused, but no big deal, right? But then I notice, these requests are returning code 200 - ok

Whaaa?

This got me curious so I tried this url:
http://badcomputer.org/index.bot/foo/bar/baz/this/doesn't/exist/

Sure enough, it returns 200 and shows me my homepage (badcomputer.org/index.bot) but without a stylesheet images etc (presumably because the path is wrong).

So my question is, what is causing this, and how to I get requests such as these to return 404 as they should?

My apache is 2.0.58
My PHP is 5.1.4

If I have forgotten something you need to diagnose this please ask.
Thanks for consideration.

w3bd3vil · 08-14-2006, 07:34 PM

this is probably a security patch. Cause its good that any page you open returns a 200 request, since automatic scanners wont work against your domain to find any vulnerabilities. Suppose the scanner is searching for some php flaws like remote inclusion it will get all false positives. So the hacker will have a hard time figuring out which are the true vulns on your site.

Anyway, search for something you installed to secure apache. That should be the cause to this.

bulliver · 08-14-2006, 07:48 PM

Quote:

Cause its good that any page you open returns a 200 request, since automatic scanners wont work against your domain to find any vulnerabilities.

Not sure that any good would outweigh the fact that Yahoo! is indexing pages on my site that don't exist. I don't think that is good at all. Besides, it seems to fly in the face of the entire HTTP protocol.

Quote:

Anyway, search for something you installed to secure apache.

I've done nothing of the sort, in fact, I do not even have mod_security installed. I do however have hardened php. Could this be it then? I still want it disabled.

Actually, after a test I don't think you are right at all...as when I try:
http://badcomputer.org/somepagethatdoesntexist.html
I get a 404 as I should. It just seems to do this when a directory path is grafted onto the en of the URL...