LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices

Reply
 
Search this Thread
Old 04-09-2005, 09:30 AM   #1
jc materi
LQ Newbie
 
Registered: Aug 2004
Location: Saskatoon, SK, Canada
Distribution: fedora 4
Posts: 25

Rep: Reputation: 15
configuring robots.txt


Apache has a default file in the document root called robots.txt restricting the scan of search bots. I have kept all the defaults but the line "/all_our_e-mail_addresses" must be changed and I'm not sure what should go there. Any examples would be appreciated (without using actual email addresses, of course).
 
Old 04-09-2005, 10:37 AM   #2
macemoneta
Senior Member
 
Registered: Jan 2005
Location: Manalapan, NJ
Distribution: Fedora x86 and x86_64, Debian PPC and ARM, Android
Posts: 4,593
Blog Entries: 2

Rep: Reputation: 327Reputation: 327Reputation: 327Reputation: 327
That's a sample directory or file that you would want to exclude (one that contained all your email addresses - a confusing choice for an example). Some better examples:

User-agent: *
Disallow: /some/secret/directory
Disallow: /tmp/
Disallow: /my_important_file

There's more information on the layout of this file here
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
robots.txt paleogryph Linux - Software 1 11-11-2005 02:32 PM
Linux Powered Robots. rvijay Linux - News 8 07-31-2005 09:01 AM
txt >> mysql LLS Programming 2 06-21-2005 11:15 AM
Stop the Google robots lothario Linux - Software 2 04-18-2005 05:47 AM
The Robots movie ahz General 4 03-24-2005 04:31 PM


All times are GMT -5. The time now is 09:50 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration