Review your favorite Linux distribution.
Go Back > Forums > Linux Forums > Linux - Security
User Name
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.


  Search this Thread
Old 04-09-2005, 10:30 AM   #1
jc materi
LQ Newbie
Registered: Aug 2004
Location: Saskatoon, SK, Canada
Distribution: fedora 4
Posts: 25

Rep: Reputation: 15
configuring robots.txt

Apache has a default file in the document root called robots.txt restricting the scan of search bots. I have kept all the defaults but the line "/all_our_e-mail_addresses" must be changed and I'm not sure what should go there. Any examples would be appreciated (without using actual email addresses, of course).
Old 04-09-2005, 11:37 AM   #2
Senior Member
Registered: Jan 2005
Location: Manalapan, NJ
Distribution: Fedora x86 and x86_64, Debian PPC and ARM, Android
Posts: 4,593
Blog Entries: 2

Rep: Reputation: 332Reputation: 332Reputation: 332Reputation: 332
That's a sample directory or file that you would want to exclude (one that contained all your email addresses - a confusing choice for an example). Some better examples:

User-agent: *
Disallow: /some/secret/directory
Disallow: /tmp/
Disallow: /my_important_file

There's more information on the layout of this file here


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
robots.txt paleogryph Linux - Software 1 11-11-2005 03:32 PM
Linux Powered Robots. rvijay Linux - News 8 07-31-2005 10:01 AM
txt >> mysql LLS Programming 2 06-21-2005 12:15 PM
Stop the Google robots lothario Linux - Software 2 04-18-2005 06:47 AM
The Robots movie ahz General 4 03-24-2005 05:31 PM > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 10:50 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration