LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 06-19-2012, 06:30 AM   #1
arun5002
Member
 
Registered: Aug 2011
Location: Chennai,India
Distribution: Redhat,Centos,Ubuntu,Dedian
Posts: 549
Blog Entries: 5

Rep: Reputation: Disabled
Apache close wait error :Apache stop responding,Server get hangs


Hi
Today one of my apache server stop responding .it deliver the error

Quote:
#losf -i :80

httpd 4024 apache 4u IPv6 14114 0t0 TCP *:http (LISTEN)
httpd 4024 apache 13u IPv6 21471 0t0 TCP xxx:http->xxx.googlebot.com:45068 (ESTABLISHED)
httpd 4046 apache 13u IPv6 24518 0t0 TCP xxx:http->crawl-66-249-71-218.googlebot.com:59740 (CLOSE_WAIT)
Quote:
I read of completed google about these thread .I could int find out the complete solution for my error.As you can see on the output of the following lsof command, these two processes are "dead" Apache processes, they don't listen for new connections anymore. But the actual status of CLOSE_WAIT between Apache and the Googlebot doesn't allow the Apache Master process to kill his child:
I read oout the following link but they haveint given the proper cause of these issue ,Can anyone help me to find out the complete cause of issue


whether Googlebot which causes troubles.

Quote:
Any idea here? HOW CAN I BLOCK GOOGLEBOTS

http://www.gossamer-threads.com/lists/apache/dev/414558

Last edited by arun5002; 06-19-2012 at 07:21 AM.
 
Old 06-19-2012, 11:49 AM   #2
tronayne
Senior Member
 
Registered: Oct 2003
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 2,995

Rep: Reputation: 737Reputation: 737Reputation: 737Reputation: 737Reputation: 737Reputation: 737Reputation: 737
If you go take a look at http://wiki.phpgedview.net/en/index....ict_bot_access there is a good explanation of how to restrict bot access (including blocking). The article is too long to reproduce here.

Here is an example (perhaps not relevant) that block access to Bugzilla and PhpGedViews along with other important things on my severs; read the article to see what's what and why's why:
Code:
User-agent: URL_Spider_Pro
Disallow: /

User-agent: CherryPicker
Disallow: /

User-agent: *
Disallow: /bin/
Disallow: /bugzilla/
Disallow: /cgi-bin/
Disallow: /dev/
Disallow: /downloads/
Disallow: /manual/
Disallow: /index.php?
Disallow: /index.php/Help
Disallow: /index.php/MediaWiki
Disallow: /index.php/Special:
Disallow: /index.php/Template
Disallow: /skins/
Disallow: /phpgedview/reportengine.php
Disallow: /phpgedview/fanchart.php
Disallow: /phpgedview/search.php
Disallow: /phpgedview/login.php
Disallow: /phpgedview/clippings.php
Disallow: /phpgedview/sosabook.php
Disallow: /phpgedview/timeline.php
Disallow: /phpgedview/calendar.php
Disallow: /phpgedview/hourglass.php
Disallow: /phpgedview/ancestry.php
Disallow: /phpgedview/descendancy.php
Disallow: /phpgedview/pedigree.php
Disallow: /phpgedview/family.php
Disallow: /phpgedview/relationship.php
Disallow: /phpgedview/famlist.php
Disallow: /phpgedview/patriarchlist.php
Disallow: /phpgedview/repolist.php
Disallow: /phpgedview/aliveinyear.php
Hope this helps some.

Last edited by tronayne; 06-19-2012 at 11:54 AM. Reason: Forgot the example.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Apache server hangs sanjay87 Linux - Server 1 04-17-2012 11:14 AM
Server not responding - apache level? ganz_friedrich Linux - Networking 0 08-07-2006 10:59 AM
Apache Server not responding to 127.0.0.1 khairkhoah Linux - Networking 2 05-11-2006 08:15 AM
Stop Apache Server from generating index pages slackwarefan Linux - General 0 05-16-2004 01:45 PM
Apache Not responding wgollie Linux - General 5 02-02-2004 01:07 AM


All times are GMT -5. The time now is 06:23 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration