If you go take a look at
http://wiki.phpgedview.net/en/index....ict_bot_access there is a good explanation of how to restrict bot access (including blocking). The article is too long to reproduce here.
Here is an example (perhaps not relevant) that block access to Bugzilla and PhpGedViews along with other important things on my severs; read the article to see what's what and why's why:
Code:
User-agent: URL_Spider_Pro
Disallow: /
User-agent: CherryPicker
Disallow: /
User-agent: *
Disallow: /bin/
Disallow: /bugzilla/
Disallow: /cgi-bin/
Disallow: /dev/
Disallow: /downloads/
Disallow: /manual/
Disallow: /index.php?
Disallow: /index.php/Help
Disallow: /index.php/MediaWiki
Disallow: /index.php/Special:
Disallow: /index.php/Template
Disallow: /skins/
Disallow: /phpgedview/reportengine.php
Disallow: /phpgedview/fanchart.php
Disallow: /phpgedview/search.php
Disallow: /phpgedview/login.php
Disallow: /phpgedview/clippings.php
Disallow: /phpgedview/sosabook.php
Disallow: /phpgedview/timeline.php
Disallow: /phpgedview/calendar.php
Disallow: /phpgedview/hourglass.php
Disallow: /phpgedview/ancestry.php
Disallow: /phpgedview/descendancy.php
Disallow: /phpgedview/pedigree.php
Disallow: /phpgedview/family.php
Disallow: /phpgedview/relationship.php
Disallow: /phpgedview/famlist.php
Disallow: /phpgedview/patriarchlist.php
Disallow: /phpgedview/repolist.php
Disallow: /phpgedview/aliveinyear.php
Hope this helps some.
