LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 02-10-2013, 08:04 PM   #16
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,953
Blog Entries: 54

Rep: Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733

The fact your server hasn't crashed since you made changes is hopeful but it does make me wonder what other changes you made that could have adversely affected performance. Maybe it would be best if you start back at the beginning and show us the machine hardware specs (you seem to have 8GB RAM?) and an inventory of installed software. (For example your CentOS 6.2 is one update behind and how about your web stack, Drupal and other software?) Collecting system statistics is a good move but you should monitor other aspects as well like network traffic, your web server and your database. For example Atsar works like "regular" SAR but can also log Apache HTTP traffic (or see "Apache Top"). Like Atsar Dstat aims to replace vmstat, iostat and ifstat but it can also log MySQL statistics (else see "MysQL Top" or "mtop" for short). As for your system logs I suggest you run them through Logwatch at detail level 10 as that's the easiest, quickest way to generate leads. I know this all wasn't very detailed but you have to start somewhere, right?
 
Old 02-13-2013, 04:15 AM   #17
david59
LQ Newbie
 
Registered: Feb 2013
Posts: 18

Original Poster
Rep: Reputation: Disabled
Hello, am traveling right now... will be back tomorrow and respond. Apologies for the delay and thank you so much, david
 
Old 02-14-2013, 05:35 AM   #18
david59
LQ Newbie
 
Registered: Feb 2013
Posts: 18

Original Poster
Rep: Reputation: Disabled
This is a reply to chrism01, from another thread I started on this topic in Linux - Newbie (after realizing that perhaps I had started in the wrong forum). That thread is here:

Code:
http://www.linuxquestions.org/questions/linux-newbie-8/trying-to-understand-this-readout-from-top-b-4175449304/#post4887575
My sincere apologies for the confusion.



Hello Chris, thank you for writing and apologies for the delay in getting back to you. The crashes began quite suddenly, seemingly every 24 hours at more or less the same time. As for the /var/log files, I have looked, and there is nothing remarkable there.

Code:
[root@u1584224530 /]# cat /etc/*release*
CentOS release 6.2 (Final)
CentOS release 6.2 (Final)
CentOS release 6.2 (Final)
cpe:/o:centos:linux:6:GA
Code:
[root@u16271690 /]# uname -a
Linux u16271690.onlinehome-server.com 2.6.32-220.13.1.el6.x86_64 #1 SMP Tue Apr 17 23:56:34 BST 2012 x86_64 x86_64 x86_64 GNU/Linux

Following is the .htaccess file.

.htaccess

Code:
#
# Apache/PHP/Drupal settings:
#

# Protect files and directories from prying eyes.
<FilesMatch "\.(engine|inc|info|install|make|module|profile|test|po|sh|.*sql|theme|tpl(\.php)?|xtmpl)$|^(\..*|Entries.*|Repository|Root|Tag|Template)$">
  Order allow,deny
</FilesMatch>


# Don't show directory listings for URLs which map to a directory.
Options -Indexes

# Follow symbolic links in this directory.
Options +FollowSymLinks

# Make Drupal handle any 404 errors.
ErrorDocument 404 /index.php

# Set the default handler.
DirectoryIndex index.php index.html index.htm

# Override PHP settings that cannot be changed at runtime. See
# sites/default/default.settings.php and drupal_environment_initialize() in
# includes/bootstrap.inc for settings that can be changed at runtime.

# PHP 5, Apache 1 and 2.
<IfModule mod_php5.c>
  php_flag magic_quotes_gpc                 off
  php_flag magic_quotes_sybase              off
  php_flag register_globals                 off
  php_flag session.auto_start               off
  php_value mbstring.http_input             pass
  php_value mbstring.http_output            pass
  php_flag mbstring.encoding_translation    off
</IfModule>

# Requires mod_expires to be enabled.
<IfModule mod_expires.c>
  # Enable expirations.
  ExpiresActive On

  # Cache all files for 2 weeks after access (A).
  ExpiresDefault A1209600

  <FilesMatch \.php$>
    # Do not allow PHP scripts to be cached unless they explicitly send cache
    # headers themselves. Otherwise all scripts would have to overwrite the
    # headers set by mod_expires if they want another caching behavior. This may
    # fail if an error occurs early in the bootstrap process, and it may cause
    # problems if a non-Drupal PHP file is installed in a subdirectory.
    ExpiresActive Off
  </FilesMatch>
</IfModule>

# Various rewrite rules.
<IfModule mod_rewrite.c>
  RewriteEngine on

  # Block access to "hidden" directories whose names begin with a period. This
  # includes directories used by version control systems such as Subversion or
  # Git to store control files. Files whose names begin with a period, as well
  # as the control files used by CVS, are protected by the FilesMatch directive
  # above.
  #
  # NOTE: This only works when mod_rewrite is loaded. Without mod_rewrite, it is
  # not possible to block access to entire directories from .htaccess, because
  # <DirectoryMatch> is not allowed here.
  #
  # If you do not have mod_rewrite installed, you should remove these
  # directories from your webroot or otherwise protect them from being
  # downloaded.
  RewriteRule "(^|/)\." - [F]

  # If your site can be accessed both with and without the 'www.' prefix, you
  # can use one of the following settings to redirect users to your preferred
  # URL, either WITH or WITHOUT the 'www.' prefix. Choose ONLY one option:
  #
  # To redirect all users to access the site WITH the 'www.' prefix,
  # (http://example.com/... will be redirected to http://www.example.com/...)
  # uncomment the following:
  # RewriteCond %{HTTP_HOST} !^www\. [NC]
  # RewriteRule ^ http://www.%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
  #
  # To redirect all users to access the site WITHOUT the 'www.' prefix,
  # (http://www.example.com/... will be redirected to http://example.com/...)
  # uncomment the following:
  # RewriteCond %{HTTP_HOST} ^www\.(.+)$ [NC]
  # RewriteRule ^ http://%1%{REQUEST_URI} [L,R=301]

  # Modify the RewriteBase if you are using Drupal in a subdirectory or in a
  # VirtualDocumentRoot and the rewrite rules are not working properly.
  # For example if your site is at http://example.com/drupal uncomment and
  # modify the following line:
  # RewriteBase /drupal
  #
  # If your site is running in a VirtualDocumentRoot at http://example.com/,
  # uncomment the following line:
  # RewriteBase /

  # Pass all requests not referring directly to files in the filesystem to
  # index.php. Clean URLs are handled in drupal_environment_initialize().
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteCond %{REQUEST_URI} !=/favicon.ico
  RewriteRule ^ index.php [L]

  # Rules to correctly serve gzip compressed CSS and JS files.
  # Requires both mod_rewrite and mod_headers to be enabled.
  <IfModule mod_headers.c>
    # Serve gzip compressed CSS files if they exist and the client accepts gzip.
    RewriteCond %{HTTP:Accept-encoding} gzip
    RewriteCond %{REQUEST_FILENAME}\.gz -s
    RewriteRule ^(.*)\.css $1\.css\.gz [QSA]

    # Serve gzip compressed JS files if they exist and the client accepts gzip.
    RewriteCond %{HTTP:Accept-encoding} gzip
    RewriteCond %{REQUEST_FILENAME}\.gz -s
    RewriteRule ^(.*)\.js $1\.js\.gz [QSA]

    # Serve correct content types, and prevent mod_deflate double gzip.
    RewriteRule \.css\.gz$ - [T=text/css,E=no-gzip:1]
    RewriteRule \.js\.gz$ - [T=text/javascript,E=no-gzip:1]

    <FilesMatch "(\.js\.gz|\.css\.gz)$">
      # Serve correct encoding type.
      Header set Content-Encoding gzip
      # Force proxies to cache gzipped & non-gzipped css/js files separately.
      Header append Vary Accept-Encoding
    </FilesMatch>
  </IfModule>
</IfModule>

Last edited by david59; 02-14-2013 at 05:51 AM.
 
Old 02-14-2013, 05:40 AM   #19
david59
LQ Newbie
 
Registered: Feb 2013
Posts: 18

Original Poster
Rep: Reputation: Disabled
httpd.conf file, part 1 (I moved the directives for bots there, as you advised above)

Code:
#
# This is the main Apache server configuration file.  It contains the
# configuration directives that give the server its instructions.
# See <URL:http://httpd.apache.org/docs/2.2/> for detailed information.
# In particular, see
# <URL:http://httpd.apache.org/docs/2.2/mod/directives.html>
# for a discussion of each configuration directive.
#
#
# Do NOT simply read the instructions in here without understanding
# what they do.  They're here only as hints or reminders.  If you are unsure
# consult the online docs. You have been warned.  
#
# The configuration directives are grouped into three basic sections:
#  1. Directives that control the operation of the Apache server process as a
#     whole (the 'global environment').
#  2. Directives that define the parameters of the 'main' or 'default' server,
#     which responds to requests that aren't handled by a virtual host.
#     These directives also provide default values for the settings
#     of all virtual hosts.
#  3. Settings for virtual hosts, which allow Web requests to be sent to
#     different IP addresses or hostnames and have them handled by the
#     same Apache server process.
#
# Configuration and logfile names: If the filenames you specify for many
# of the server's control files begin with "/" (or "drive:/" for Win32), the
# server will use that explicit path.  If the filenames do *not* begin
# with "/", the value of ServerRoot is prepended -- so "logs/foo.log"
# with ServerRoot set to "/etc/httpd" will be interpreted by the
# server as "/etc/httpd/logs/foo.log".
#

### Section 1: Global Environment
#
# The directives in this section affect the overall operation of Apache,
# such as the number of concurrent requests it can handle or where it
# can find its configuration files.
#

#
# Don't give away too much information about all the subcomponents
# we are running.  Comment out this line if you don't mind remote sites
# finding out what major optional modules you are running
ServerTokens OS

#
# ServerRoot: The top of the directory tree under which the server's
# configuration, error, and log files are kept.
#
# NOTE!  If you intend to place this on an NFS (or otherwise network)
# mounted filesystem then please read the LockFile documentation
# (available at <URL:http://httpd.apache.org/docs/2.2/mod/mpm_common.html#lockfile>);
# you will save yourself a lot of trouble.
#
# Do NOT add a slash at the end of the directory path.
#
ServerRoot "/etc/httpd"

#
# PidFile: The file in which the server should record its process
# identification number when it starts.  Note the PIDFILE variable in
# /etc/sysconfig/httpd must be set appropriately if this location is
# changed.
#
PidFile run/httpd.pid

#
# Timeout: The number of seconds before receives and sends time out.
#
Timeout 60

#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive Off

#
# MaxKeepAliveRequests: The maximum number of requests to allow
# during a persistent connection. Set to 0 to allow an unlimited amount.
# We recommend you leave this number high, for maximum performance.
#
MaxKeepAliveRequests 100

#
# KeepAliveTimeout: Number of seconds to wait for the next request from the
# same client on the same connection.
#
KeepAliveTimeout 15

##
## Server-Pool Size Regulation (MPM specific)
## 

# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# ServerLimit: maximum value for MaxClients for the lifetime of the server
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  4000
</IfModule>

# worker MPM
# StartServers: initial number of server processes to start
# MaxClients: maximum number of simultaneous client connections
# MinSpareThreads: minimum number of worker threads which are kept spare
# MaxSpareThreads: maximum number of worker threads which are kept spare
# ThreadsPerChild: constant number of worker threads in each server process
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule worker.c>
StartServers         4
MaxClients         200
MinSpareThreads     25
MaxSpareThreads     75 
ThreadsPerChild     25
MaxRequestsPerChild  0
</IfModule>

#
# Listen: Allows you to bind Apache to specific IP addresses and/or
# ports, in addition to the default. See also the <VirtualHost>
# directive.
#
# Change this to Listen on specific IP addresses as shown below to 
# prevent Apache from glomming onto all bound IP addresses (0.0.0.0)
#
#Listen 12.34.56.78:80
Listen 80

#
# Dynamic Shared Object (DSO) Support
#
# To be able to use the functionality of a module which was built as a DSO you
# have to place corresponding `LoadModule' lines at this location so the
# directives contained in it are actually available _before_ they are used.
# Statically compiled modules (those listed by `httpd -l') do not need
# to be loaded here.
#
# Example:
# LoadModule foo_module modules/mod_foo.so
#
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule auth_digest_module modules/mod_auth_digest.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authn_alias_module modules/mod_authn_alias.so
LoadModule authn_anon_module modules/mod_authn_anon.so
LoadModule authn_dbm_module modules/mod_authn_dbm.so
LoadModule authn_default_module modules/mod_authn_default.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_owner_module modules/mod_authz_owner.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_dbm_module modules/mod_authz_dbm.so
LoadModule authz_default_module modules/mod_authz_default.so
LoadModule ldap_module modules/mod_ldap.so
LoadModule authnz_ldap_module modules/mod_authnz_ldap.so
LoadModule include_module modules/mod_include.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule logio_module modules/mod_logio.so
LoadModule env_module modules/mod_env.so
#LoadModule ext_filter_module modules/mod_ext_filter.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule expires_module modules/mod_expires.so
LoadModule deflate_module modules/mod_deflate.so
LoadModule headers_module modules/mod_headers.so
LoadModule usertrack_module modules/mod_usertrack.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule mime_module modules/mod_mime.so
LoadModule dav_module modules/mod_dav.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule info_module modules/mod_info.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule dir_module modules/mod_dir.so
LoadModule actions_module modules/mod_actions.so
LoadModule speling_module modules/mod_speling.so
LoadModule userdir_module modules/mod_userdir.so
LoadModule alias_module modules/mod_alias.so
LoadModule substitute_module modules/mod_substitute.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule cache_module modules/mod_cache.so
LoadModule suexec_module modules/mod_suexec.so
LoadModule disk_cache_module modules/mod_disk_cache.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule version_module modules/mod_version.so

#
# The following modules are not loaded by default:
#
#LoadModule asis_module modules/mod_asis.so
#LoadModule authn_dbd_module modules/mod_authn_dbd.so
#LoadModule cern_meta_module modules/mod_cern_meta.so
#LoadModule cgid_module modules/mod_cgid.so
#LoadModule dbd_module modules/mod_dbd.so
#LoadModule dumpio_module modules/mod_dumpio.so
LoadModule filter_module modules/mod_filter.so
#LoadModule ident_module modules/mod_ident.so
#LoadModule log_forensic_module modules/mod_log_forensic.so
#LoadModule unique_id_module modules/mod_unique_id.so
#

#
# Load config files from the config directory "/etc/httpd/conf.d".
#
Include conf.d/*.conf

#
# ExtendedStatus controls whether Apache will generate "full" status
# information (ExtendedStatus On) or just basic information (ExtendedStatus
# Off) when the "server-status" handler is called. The default is Off.
#
#ExtendedStatus On

#
# If you wish httpd to run as a different user or group, you must run
# httpd as root initially and it will switch.  
#
# User/Group: The name (or #number) of the user/group to run httpd as.
#  . On SCO (ODT 3) use "User nouser" and "Group nogroup".
#  . On HPUX you may not be able to use shared memory as nobody, and the
#    suggested workaround is to create a user www and use that user.
#  NOTE that some kernels refuse to setgid(Group) or semctl(IPC_SET)
#  when the value of (unsigned)Group is above 60000; 
#  don't use Group #-1 on these systems!
#
User apache
Group apache

### Section 2: 'Main' server configuration
#
# The directives in this section set up the values used by the 'main'
# server, which responds to any requests that aren't handled by a
# <VirtualHost> definition.  These values also provide defaults for
# any <VirtualHost> containers you may define later in the file.
#
# All of these directives may appear inside <VirtualHost> containers,
# in which case these default settings will be overridden for the
# virtual host being defined.
#

#
# ServerAdmin: Your address, where problems with the server should be
# e-mailed.  This address appears on some server-generated pages, such
# as error documents.  e.g. admin@your-domain.com
#
ServerAdmin root@localhost

#
# ServerName gives the name and port that the server uses to identify itself.
# This can often be determined automatically, but we recommend you specify
# it explicitly to prevent problems during startup.
#
# If this is not set to valid DNS name for your host, server-generated
# redirections will not work.  See also the UseCanonicalName directive.
#
# If your host doesn't have a registered DNS name, enter its IP address here.
# You will have to access it by its address anyway, and this will make 
# redirections work in a sensible way.
#
#ServerName www.example.com:80

#
# UseCanonicalName: Determines how Apache constructs self-referencing 
# URLs and the SERVER_NAME and SERVER_PORT variables.
# When set "Off", Apache will use the Hostname and Port supplied
# by the client.  When set "On", Apache will use the value of the
# ServerName directive.
#
UseCanonicalName Off

#
# DocumentRoot: The directory out of which you will serve your
# documents. By default, all requests are taken from this directory, but
# symbolic links and aliases may be used to point to other locations.
#
DocumentRoot "/var/www/html"

#
# Each directory to which Apache has access can be configured with respect
# to which services and features are allowed and/or disabled in that
# directory (and its subdirectories). 
#
# First, we configure the "default" to be a very restrictive set of 
# features.  
#
<Directory />
    Options FollowSymLinks
    AllowOverride None
</Directory>

#
# Note that from this point forward you must specifically allow
# particular features to be enabled - so if something's not working as
# you might expect, make sure that you have specifically enabled it
# below.
#

#
# This should be changed to whatever you set DocumentRoot to.
#
<Directory "/var/www/html">

#
# Possible values for the Options directive are "None", "All",
# or any combination of:
#   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
#
# Note that "MultiViews" must be named *explicitly* --- "Options All"
# doesn't give it to you.
#
# The Options directive is both complicated and important.  Please see
# http://httpd.apache.org/docs/2.2/mod/core.html#options
# for more information.
#
    Options Indexes FollowSymLinks

#
# AllowOverride controls what directives may be placed in .htaccess files.
# It can be "All", "None", or any combination of the keywords:
#   Options FileInfo AuthConfig Limit
#
    AllowOverride None

#
# Controls who can get stuff from this server.
#
    Order allow,deny
    Allow from all

</Directory>

#
# UserDir: The name of the directory that is appended onto a user's home
# directory if a ~user request is received.
#
# The path to the end user account 'public_html' directory must be
# accessible to the webserver userid.  This usually means that ~userid
# must have permissions of 711, ~userid/public_html must have permissions
# of 755, and documents contained therein must be world-readable.
# Otherwise, the client will only receive a "403 Forbidden" message.
#
# See also: http://httpd.apache.org/docs/misc/FAQ.html#forbidden
#
<IfModule mod_userdir.c>
    #
    # UserDir is disabled by default since it can confirm the presence
    # of a username on the system (depending on home directory
    # permissions).
    #
    UserDir disabled

    #
    # To enable requests to /~user/ to serve the user's public_html
    # directory, remove the "UserDir disabled" line above, and uncomment
    # the following line instead:
    # 
    #UserDir public_html

</IfModule>

#
# Control access to UserDir directories.  The following is an example
# for a site where these directories are restricted to read-only.
#
#<Directory /home/*/public_html>
#    AllowOverride FileInfo AuthConfig Limit
#    Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec
#    <Limit GET POST OPTIONS>
#        Order allow,deny
#        Allow from all
#    </Limit>
#    <LimitExcept GET POST OPTIONS>
#        Order deny,allow
#        Deny from all
#    </LimitExcept>
#</Directory>

#
# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
# The index.html.var file (a type-map) is used to deliver content-
# negotiated documents.  The MultiViews Option can be used for the 
# same purpose, but it is much slower.
#
DirectoryIndex index.html index.html.var index.shtml index.cfm index.php index.htm

#
# AccessFileName: The name of the file to look for in each directory
# for additional configuration directives.  See also the AllowOverride
# directive.
#
AccessFileName .htaccess

#
# The following lines prevent .htaccess and .htpasswd files from being 
# viewed by Web clients. 
#
<Files ~ "^\.ht">
    Order allow,deny
    Deny from all
    Satisfy All
</Files>

#
# TypesConfig describes where the mime.types file (or equivalent) is
# to be found.
#
TypesConfig /etc/mime.types

#
# DefaultType is the default MIME type the server will use for a document
# if it cannot otherwise determine one, such as from filename extensions.
# If your server contains mostly text or HTML documents, "text/plain" is
# a good value.  If most of your content is binary, such as applications
# or images, you may want to use "application/octet-stream" instead to
# keep browsers from trying to display binary files as though they are
# text.
#
DefaultType text/plain

#
# The mod_mime_magic module allows the server to use various hints from the
# contents of the file itself to determine its type.  The MIMEMagicFile
# directive tells the module where the hint definitions are located.
#
<IfModule mod_mime_magic.c>
#   MIMEMagicFile /usr/share/magic.mime
    MIMEMagicFile conf/magic
</IfModule>

#
# HostnameLookups: Log the names of clients or just their IP addresses
# e.g., www.apache.org (on) or 204.62.129.132 (off).
# The default is off because it'd be overall better for the net if people
# had to knowingly turn this feature on, since enabling it means that
# each client request will result in AT LEAST one lookup request to the
# nameserver.
#
HostnameLookups Off

#
# EnableMMAP: Control whether memory-mapping is used to deliver
# files (assuming that the underlying OS supports it).
# The default is on; turn this off if you serve from NFS-mounted 
# filesystems.  On some systems, turning it off (regardless of
# filesystem) can improve performance; for details, please see
# http://httpd.apache.org/docs/2.2/mod/core.html#enablemmap
#
#EnableMMAP off

#
# EnableSendfile: Control whether the sendfile kernel support is 
# used to deliver files (assuming that the OS supports it). 
# The default is on; turn this off if you serve from NFS-mounted 
# filesystems.  Please see
# http://httpd.apache.org/docs/2.2/mod/core.html#enablesendfile
#
#EnableSendfile off

#
# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
# container, error messages relating to that virtual host will be
# logged here.  If you *do* define an error logfile for a <VirtualHost>
# container, that host's errors will be logged there and not here.
#
ErrorLog logs/error_log

#
# LogLevel: Control the number of messages logged to the error_log.
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
#
LogLevel warn

#
# The following directives define some format nicknames for use with
# a CustomLog directive (see below).
#
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

# "combinedio" includes actual counts of actual bytes received (%I) and sent (%O); this
# requires the mod_logio module to be loaded.
#LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio

#
# The location and format of the access logfile (Common Logfile Format).
# If you do not define any access logfiles within a <VirtualHost>
# container, they will be logged here.  Contrariwise, if you *do*
# define per-<VirtualHost> access logfiles, transactions will be
# logged therein and *not* in this file.
#
#CustomLog logs/access_log common

#
# If you would like to have separate agent and referer logfiles, uncomment
# the following directives.
#
#CustomLog logs/referer_log referer
#CustomLog logs/agent_log agent

#
# For a single logfile with access, agent, and referer information
# (Combined Logfile Format), use the following directive:
#
CustomLog logs/access_log combined

#
# Optionally add a line containing the server version and virtual host
# name to server-generated pages (internal error documents, FTP directory
# listings, mod_status and mod_info output etc., but not CGI generated
# documents or custom error documents).
# Set to "EMail" to also include a mailto: link to the ServerAdmin.
# Set to one of:  On | Off | EMail
#
ServerSignature On

#
# Aliases: Add here as many aliases as you need (with no limit). The format is 
# Alias fakename realname
#
# Note that if you include a trailing / on fakename then the server will
# require it to be present in the URL.  So "/icons" isn't aliased in this
# example, only "/icons/".  If the fakename is slash-terminated, then the 
# realname must also be slash terminated, and if the fakename omits the 
# trailing slash, the realname must also omit it.
#
# We include the /icons/ alias for FancyIndexed directory listings.  If you
# do not use FancyIndexing, you may comment this out.
#
Alias /icons/ "/var/www/icons/"

<Directory "/var/www/icons">
    Options Indexes MultiViews FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

#
# WebDAV module configuration section.
# 
<IfModule mod_dav_fs.c>
    # Location of the WebDAV lock database.
    DAVLockDB /var/lib/dav/lockdb
</IfModule>

#
# ScriptAlias: This controls which directories contain server scripts.
# ScriptAliases are essentially the same as Aliases, except that
# documents in the realname directory are treated as applications and
# run by the server when requested rather than as documents sent to the client.
# The same rules about trailing "/" apply to ScriptAlias directives as to
# Alias.
#
ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"

#
# "/var/www/cgi-bin" should be changed to whatever your ScriptAliased
# CGI directory exists, if you have that configured.
#
<Directory "/var/www/cgi-bin">
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
</Directory>

#
# Redirect allows you to tell clients about documents which used to exist in
# your server's namespace, but do not anymore. This allows you to tell the
# clients where to look for the relocated document.
# Example:
# Redirect permanent /foo http://www.example.com/bar

#
# Directives controlling the display of server-generated directory listings.
#

#
# IndexOptions: Controls the appearance of server-generated directory
# listings.
#
IndexOptions FancyIndexing VersionSort NameWidth=* HTMLTable Charset=UTF-8

#
# AddIcon* directives tell the server which icon to show for different
# files or filename extensions.  These are only displayed for
# FancyIndexed directories.
#
AddIconByEncoding (CMP,/icons/compressed.gif) x-compress x-gzip

AddIconByType (TXT,/icons/text.gif) text/*
AddIconByType (IMG,/icons/image2.gif) image/*
AddIconByType (SND,/icons/sound2.gif) audio/*
AddIconByType (VID,/icons/movie.gif) video/*

AddIcon /icons/binary.gif .bin .exe
AddIcon /icons/binhex.gif .hqx
AddIcon /icons/tar.gif .tar
AddIcon /icons/world2.gif .wrl .wrl.gz .vrml .vrm .iv
AddIcon /icons/compressed.gif .Z .z .tgz .gz .zip
AddIcon /icons/a.gif .ps .ai .eps
AddIcon /icons/layout.gif .html .shtml .htm .pdf
AddIcon /icons/text.gif .txt
AddIcon /icons/c.gif .c
AddIcon /icons/p.gif .pl .py
AddIcon /icons/f.gif .for
AddIcon /icons/dvi.gif .dvi
AddIcon /icons/uuencoded.gif .uu
AddIcon /icons/script.gif .conf .sh .shar .csh .ksh .tcl
AddIcon /icons/tex.gif .tex
AddIcon /icons/bomb.gif core

AddIcon /icons/back.gif ..
AddIcon /icons/hand.right.gif README
AddIcon /icons/folder.gif ^^DIRECTORY^^
AddIcon /icons/blank.gif ^^BLANKICON^^

#
# DefaultIcon is which icon to show for files which do not have an icon
# explicitly set.
#
DefaultIcon /icons/unknown.gif

#
# AddDescription allows you to place a short description after a file in
# server-generated indexes.  These are only displayed for FancyIndexed
# directories.
# Format: AddDescription "description" filename
#
#AddDescription "GZIP compressed document" .gz
#AddDescription "tar archive" .tar
#AddDescription "GZIP compressed tar archive" .tgz

#
# ReadmeName is the name of the README file the server will look for by
# default, and append to directory listings.
#
# HeaderName is the name of a file which should be prepended to
# directory indexes. 
ReadmeName README.html
HeaderName HEADER.html

#
# IndexIgnore is a set of filenames which directory indexing should ignore
# and not include in the listing.  Shell-style wildcarding is permitted.
#
IndexIgnore .??* *~ *# HEADER* README* RCS CVS *,v *,t

#
# DefaultLanguage and AddLanguage allows you to specify the language of 
# a document. You can then use content negotiation to give a browser a 
# file in a language the user can understand.
#
# Specify a default language. This means that all data
# going out without a specific language tag (see below) will 
# be marked with this one. You probably do NOT want to set
# this unless you are sure it is correct for all cases.
#
# * It is generally better to not mark a page as 
# * being a certain language than marking it with the wrong
# * language!
#
# DefaultLanguage nl
#
# Note 1: The suffix does not have to be the same as the language
# keyword --- those with documents in Polish (whose net-standard
# language code is pl) may wish to use "AddLanguage pl .po" to
# avoid the ambiguity with the common suffix for perl scripts.
#
# Note 2: The example entries below illustrate that in some cases 
# the two character 'Language' abbreviation is not identical to 
# the two character 'Country' code for its country,
# E.g. 'Danmark/dk' versus 'Danish/da'.
#
# Note 3: In the case of 'ltz' we violate the RFC by using a three char
# specifier. There is 'work in progress' to fix this and get
# the reference data for rfc1766 cleaned up.
part 2 below
 
Old 02-14-2013, 05:40 AM   #20
david59
LQ Newbie
 
Registered: Feb 2013
Posts: 18

Original Poster
Rep: Reputation: Disabled
part 2 of httpd.conf file

Code:
#
# Catalan (ca) - Croatian (hr) - Czech (cs) - Danish (da) - Dutch (nl)
# English (en) - Esperanto (eo) - Estonian (et) - French (fr) - German (de)
# Greek-Modern (el) - Hebrew (he) - Italian (it) - Japanese (ja)
# Korean (ko) - Luxembourgeois* (ltz) - Norwegian Nynorsk (nn)
# Norwegian (no) - Polish (pl) - Portugese (pt)
# Brazilian Portuguese (pt-BR) - Russian (ru) - Swedish (sv)
# Simplified Chinese (zh-CN) - Spanish (es) - Traditional Chinese (zh-TW)
#
AddLanguage ca .ca
AddLanguage cs .cz .cs
AddLanguage da .dk
AddLanguage de .de
AddLanguage el .el
AddLanguage en .en
AddLanguage eo .eo
AddLanguage es .es
AddLanguage et .et
AddLanguage fr .fr
AddLanguage he .he
AddLanguage hr .hr
AddLanguage it .it
AddLanguage ja .ja
AddLanguage ko .ko
AddLanguage ltz .ltz
AddLanguage nl .nl
AddLanguage nn .nn
AddLanguage no .no
AddLanguage pl .po
AddLanguage pt .pt
AddLanguage pt-BR .pt-br
AddLanguage ru .ru
AddLanguage sv .sv
AddLanguage zh-CN .zh-cn
AddLanguage zh-TW .zh-tw

#
# LanguagePriority allows you to give precedence to some languages
# in case of a tie during content negotiation.
#
# Just list the languages in decreasing order of preference. We have
# more or less alphabetized them here. You probably want to change this.
#
LanguagePriority en ca cs da de el eo es et fr he hr it ja ko ltz nl nn no pl pt pt-BR ru sv zh-CN zh-TW

#
# ForceLanguagePriority allows you to serve a result page rather than
# MULTIPLE CHOICES (Prefer) [in case of a tie] or NOT ACCEPTABLE (Fallback)
# [in case no accepted languages matched the available variants]
#
ForceLanguagePriority Prefer Fallback

#
# Specify a default charset for all content served; this enables
# interpretation of all content as UTF-8 by default.  To use the 
# default browser choice (ISO-8859-1), or to allow the META tags
# in HTML content to override this choice, comment out this
# directive:
#
#AddDefaultCharset UTF-8

#
# AddType allows you to add to or override the MIME configuration
# file mime.types for specific file types.
#
#AddType application/x-tar .tgz

#
# AddEncoding allows you to have certain browsers uncompress
# information on the fly. Note: Not all browsers support this.
# Despite the name similarity, the following Add* directives have nothing
# to do with the FancyIndexing customization directives above.
#
#AddEncoding x-compress .Z
#AddEncoding x-gzip .gz .tgz

# If the AddEncoding directives above are commented-out, then you
# probably should define those extensions to indicate media types:
#
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz

#
#   MIME-types for downloading Certificates and CRLs
#
AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl    .crl

#
# AddHandler allows you to map certain file extensions to "handlers":
# actions unrelated to filetype. These can be either built into the server
# or added with the Action directive (see below)
#
# To use CGI scripts outside of ScriptAliased directories:
# (You will also need to add "ExecCGI" to the "Options" directive.)
#
#AddHandler cgi-script .cgi

#
# For files that include their own HTTP headers:
#
#AddHandler send-as-is asis

#
# For type maps (negotiated resources):
# (This is enabled by default to allow the Apache "It Worked" page
#  to be distributed in multiple languages.)
#
AddHandler type-map var

#
# Filters allow you to process content before it is sent to the client.
#
# To parse .shtml files for server-side includes (SSI):
# (You will also need to add "Includes" to the "Options" directive.)
#
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml

#
# Action lets you define media types that will execute a script whenever
# a matching file is called. This eliminates the need for repeated URL
# pathnames for oft-used CGI file processors.
# Format: Action media/type /cgi-script/location
# Format: Action handler-name /cgi-script/location
#

#
# Customizable error responses come in three flavors:
# 1) plain text 2) local redirects 3) external redirects
#
# Some examples:
#ErrorDocument 500 "The server made a boo boo."
#ErrorDocument 404 /missing.html
#ErrorDocument 404 "/cgi-bin/missing_handler.pl"
#ErrorDocument 402 http://www.example.com/subscription_info.html
#

#
# Putting this all together, we can internationalize error responses.
#
# We use Alias to redirect any /error/HTTP_<error>.html.var response to
# our collection of by-error message multi-language collections.  We use 
# includes to substitute the appropriate text.
#
# You can modify the messages' appearance without changing any of the
# default HTTP_<error>.html.var files by adding the line:
#
#   Alias /error/include/ "/your/include/path/"
#
# which allows you to create your own set of files by starting with the
# /var/www/error/include/ files and
# copying them to /your/include/path/, even on a per-VirtualHost basis.
#

#block spider
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^Baiduspider [NC]
RewriteRule .* - [F]
#
#Block bots.
SetEnvIfNoCase user-Agent ^$ bad_bot
SetEnvIfNoCase user-Agent "^AESOP_com_SpiderMan" bad_bot
SetEnvIfNoCase user-Agent "^Alexibot" bad_bot
SetEnvIfNoCase user-Agent "Anonymouse.org" bad_bot
SetEnvIfNoCase user-Agent "^asterias" bad_bot
SetEnvIfNoCase user-Agent "^attach" bad_bot
SetEnvIfNoCase user-Agent "^BackDoorBot" bad_bot
SetEnvIfNoCase user-Agent "^BackWeb" bad_bot
SetEnvIfNoCase user-Agent "bandit" bad_bot
SetEnvIfNoCase user-Agent "^Baiduspider" bad_bot
SetEnvIfNoCase user-Agent "^BatchFTP" bad_bot
SetEnvIfNoCase user-Agent "^Bigfoot" bad_bot
SetEnvIfNoCase user-Agent "^Black.Hole" bad_bot
SetEnvIfNoCase user-Agent "^BlackWidow" bad_bot
SetEnvIfNoCase user-Agent "^BlowFish" bad_bot
SetEnvIfNoCase user-Agent "^Bot\ mailto:craftbot@yahoo.com" bad_bot
SetEnvIfNoCase user-Agent "^BotALot" bad_bot
SetEnvIfNoCase user-Agent "Buddy" bad_bot
SetEnvIfNoCase user-Agent "^BuiltBotTough" bad_bot
SetEnvIfNoCase user-Agent "^Bullseye" bad_bot
SetEnvIfNoCase user-Agent "^BunnySlippers" bad_bot
SetEnvIfNoCase user-Agent "^Cegbfeieh" bad_bot
SetEnvIfNoCase user-Agent "^CheeseBot" bad_bot
SetEnvIfNoCase user-Agent "^CherryPicker" bad_bot
SetEnvIfNoCase user-Agent "^ChinaClaw" bad_bot
SetEnvIfNoCase user-Agent "Collector" bad_bot
SetEnvIfNoCase user-Agent "Copier" bad_bot
SetEnvIfNoCase user-Agent "^CopyRightCheck" bad_bot
SetEnvIfNoCase user-Agent "^cosmos" bad_bot
SetEnvIfNoCase user-Agent "^Crescent" bad_bot
SetEnvIfNoCase user-Agent "^Curl" bad_bot
SetEnvIfNoCase user-Agent "^Custo" bad_bot
SetEnvIfNoCase user-Agent "^DA" bad_bot
SetEnvIfNoCase user-Agent "^DISCo" bad_bot
SetEnvIfNoCase user-Agent "^DIIbot" bad_bot
SetEnvIfNoCase user-Agent "^DittoSpyder" bad_bot
SetEnvIfNoCase user-Agent "^Download" bad_bot
SetEnvIfNoCase user-Agent "^Download\ Demon" bad_bot
SetEnvIfNoCase user-Agent "^Download\ Devil" bad_bot
SetEnvIfNoCase user-Agent "^Download\ Wonder" bad_bot
SetEnvIfNoCase user-Agent "Downloader" bad_bot
SetEnvIfNoCase user-Agent "^dragonfly" bad_bot
SetEnvIfNoCase user-Agent "^Drip" bad_bot
SetEnvIfNoCase user-Agent "^eCatch" bad_bot
SetEnvIfNoCase user-Agent "^EasyDL" bad_bot
SetEnvIfNoCase user-Agent "^ebingbong" bad_bot
SetEnvIfNoCase user-Agent "^EirGrabber" bad_bot
SetEnvIfNoCase user-Agent "^EmailCollector" bad_bot
SetEnvIfNoCase user-Agent "^EmailSiphon" bad_bot
SetEnvIfNoCase user-Agent "^EmailWolf" bad_bot
SetEnvIfNoCase user-Agent "^EroCrawler" bad_bot
SetEnvIfNoCase user-Agent "^Exabot" bad_bot
SetEnvIfNoCase user-Agent "^Express\ WebPictures" bad_bot
SetEnvIfNoCase user-Agent "Extractor" bad_bot
SetEnvIfNoCase user-Agent "^EyeNetIE" bad_bot
SetEnvIfNoCase user-Agent "^FileHound" bad_bot
SetEnvIfNoCase user-Agent "^FlashGet" bad_bot
SetEnvIfNoCase user-Agent "^Foobot" bad_bot
SetEnvIfNoCase user-Agent "^flunky" bad_bot
SetEnvIfNoCase user-Agent "^FrontPage" bad_bot
SetEnvIfNoCase user-Agent "^GetRight" bad_bot
SetEnvIfNoCase user-Agent "^GetSmart" bad_bot
SetEnvIfNoCase user-Agent "^GetWeb!" bad_bot
SetEnvIfNoCase user-Agent "^Go!Zilla" bad_bot
SetEnvIfNoCase user-Agent "Google\ Wireless\ Transcoder" bad_bot
SetEnvIfNoCase user-Agent "^Go-Ahead-Got-It" bad_bot
SetEnvIfNoCase user-Agent "^gotit" bad_bot
SetEnvIfNoCase user-Agent "Grabber" bad_bot
SetEnvIfNoCase user-Agent "^GrabNet" bad_bot
SetEnvIfNoCase user-Agent "^Grafula" bad_bot
SetEnvIfNoCase user-Agent "^Harvest" bad_bot
SetEnvIfNoCase user-Agent "^hloader" bad_bot
SetEnvIfNoCase user-Agent "^HMView" bad_bot
SetEnvIfNoCase user-Agent "^httplib" bad_bot
SetEnvIfNoCase user-Agent "^HTTrack" bad_bot
SetEnvIfNoCase user-Agent "^humanlinks" bad_bot
SetEnvIfNoCase user-Agent "^ia_archiver" bad_bot
SetEnvIfNoCase user-Agent "^IlseBot" bad_bot
SetEnvIfNoCase user-Agent "^Image\ Stripper" bad_bot
SetEnvIfNoCase user-Agent "^Image\ Sucker" bad_bot
SetEnvIfNoCase user-Agent "Indy\ Library" bad_bot
SetEnvIfNoCase user-Agent "^InfoNaviRobot" bad_bot
SetEnvIfNoCase user-Agent "^InfoTekies" bad_bot
SetEnvIfNoCase user-Agent "^Intelliseek" bad_bot
SetEnvIfNoCase user-Agent "^InterGET" bad_bot
SetEnvIfNoCase user-Agent "^Internet\ Ninja" bad_bot
SetEnvIfNoCase user-Agent "^Iria" bad_bot
SetEnvIfNoCase user-Agent "^Jakarta" bad_bot
SetEnvIfNoCase user-Agent "^JennyBot" bad_bot
SetEnvIfNoCase user-Agent "^JetCar" bad_bot
SetEnvIfNoCase user-Agent "^JOC" bad_bot
SetEnvIfNoCase user-Agent "^JustView" bad_bot
SetEnvIfNoCase user-Agent "^Jyxobot" bad_bot
SetEnvIfNoCase user-Agent "^Kenjin.Spider" bad_bot
SetEnvIfNoCase user-Agent "^Keyword.Density" bad_bot
SetEnvIfNoCase user-Agent "^larbin" bad_bot
SetEnvIfNoCase user-Agent "^LeechFTP" bad_bot
SetEnvIfNoCase user-Agent "^LexiBot" bad_bot
SetEnvIfNoCase user-Agent "^lftp" bad_bot
SetEnvIfNoCase user-Agent "^libWeb/clsHTTP" bad_bot
SetEnvIfNoCase user-Agent "^likse" bad_bot
SetEnvIfNoCase user-Agent "^LinkextractorPro" bad_bot
SetEnvIfNoCase user-Agent "^LinkScan/8.1a.Unix" bad_bo
SetEnvIfNoCase user-Agent "^LNSpiderguy" bad_bott
SetEnvIfNoCase user-Agent "^LinkWalker" bad_bot
SetEnvIfNoCase user-Agent "^lwp-trivial" bad_bot
SetEnvIfNoCase user-Agent "^LWP::Simple" bad_bot
SetEnvIfNoCase user-Agent "^Magnet" bad_bot
SetEnvIfNoCase user-Agent "^Mag-Net" bad_bot
SetEnvIfNoCase user-Agent "^MarkWatch" bad_bot
SetEnvIfNoCase user-Agent "^Mass\ Downloader" bad_bot
SetEnvIfNoCase user-Agent "^Mata.Hari" bad_bot
SetEnvIfNoCase user-Agent "^Memo" bad_bot
SetEnvIfNoCase user-Agent "^Microsoft.URL" bad_bot
SetEnvIfNoCase user-Agent "^Microsoft\ URL\ Control" bad_bot
SetEnvIfNoCase user-Agent "^MIDown\ tool" bad_bot
SetEnvIfNoCase user-Agent "^MIIxpc" bad_bot
SetEnvIfNoCase user-Agent "^Mirror" bad_bot
SetEnvIfNoCase user-Agent "^Missigua\ Locator" bad_bot
SetEnvIfNoCase user-Agent "^Mister\ PiX" bad_bot
SetEnvIfNoCase user-Agent "^moget" bad_bot
SetEnvIfNoCase user-Agent "^Mozilla/3.Mozilla/2.01" bad_bot
SetEnvIfNoCase user-Agent "^Mozilla.*NEWT" bad_bot
SetEnvIfNoCase user-Agent "^NAMEPROTECT" bad_bot
SetEnvIfNoCase user-Agent "^Navroad" bad_bot
SetEnvIfNoCase user-Agent "^NearSite" bad_bot
SetEnvIfNoCase user-Agent "^NetAnts" bad_bot
SetEnvIfNoCase user-Agent "^Netcraft" bad_bot
SetEnvIfNoCase user-Agent "^NetMechanic" bad_bot
SetEnvIfNoCase user-Agent "^NetSpider" bad_bot
SetEnvIfNoCase user-Agent "^Net\ Vampire" bad_bot
SetEnvIfNoCase user-Agent "^NetZIP" bad_bot
SetEnvIfNoCase user-Agent "^NextGenSearchBot" bad_bot
SetEnvIfNoCase user-Agent "^NG" bad_bot
SetEnvIfNoCase user-Agent "^NICErsPRO" bad_bot
SetEnvIfNoCase user-Agent "^NimbleCrawler" bad_bot
SetEnvIfNoCase user-Agent "^Ninja" bad_bot
SetEnvIfNoCase user-Agent "^NPbot" bad_bot
SetEnvIfNoCase user-Agent "^Octopus" bad_bot
SetEnvIfNoCase user-Agent "^Offline\ Explorer" bad_bot
SetEnvIfNoCase user-Agent "^Offline\ Navigator" bad_bot
SetEnvIfNoCase user-Agent "^Openfind" bad_bot
SetEnvIfNoCase user-Agent "^OutfoxBot" bad_bot
SetEnvIfNoCase user-Agent "^PageGrabber" bad_bot
SetEnvIfNoCase user-Agent "^Papa\ Foto" bad_bot
SetEnvIfNoCase user-Agent "^pavuk" bad_bot
SetEnvIfNoCase user-Agent "^pcBrowser" bad_bot
SetEnvIfNoCase user-Agent "^PHP\ version\ tracker" bad_bot
SetEnvIfNoCase user-Agent "^Pockey" bad_bot
SetEnvIfNoCase user-Agent "^ProPowerBot/2.14" bad_bot
SetEnvIfNoCase user-Agent "^ProWebWalker" bad_bot
SetEnvIfNoCase user-Agent "^psbot" bad_bot
SetEnvIfNoCase user-Agent "^Pump" bad_bot
SetEnvIfNoCase user-Agent "^QueryN.Metasearch" bad_bot
SetEnvIfNoCase user-Agent "^RealDownload" bad_bot
SetEnvIfNoCase user-Agent "Reaper" bad_bot
SetEnvIfNoCase user-Agent "Recorder" bad_bot
SetEnvIfNoCase user-Agent "^ReGet" bad_bot
SetEnvIfNoCase user-Agent "^RepoMonkey" bad_bot
SetEnvIfNoCase user-Agent "^RMA" bad_bot
SetEnvIfNoCase user-Agent "Siphon" bad_bot
SetEnvIfNoCase user-Agent "sitecheck.internetseer.com" bad_bot
SetEnvIfNoCase user-Agent "^SiteSnagger" bad_bot
SetEnvIfNoCase user-Agent "^SlySearch" bad_bot
SetEnvIfNoCase user-Agent "^SmartDownload" bad_bot
SetEnvIfNoCase user-Agent "^Snake" bad_bot
SetEnvIfNoCase user-Agent "^Snapbot" bad_bot
SetEnvIfNoCase user-Agent "^Snoopy" bad_bot
SetEnvIfNoCase user-Agent "^sogou" bad_bot
SetEnvIfNoCase user-Agent "^SpaceBison" bad_bot
SetEnvIfNoCase user-Agent "^SpankBot" bad_bot
SetEnvIfNoCase user-Agent "^spanner" bad_bot
SetEnvIfNoCase user-Agent "^Sqworm" bad_bot
SetEnvIfNoCase user-Agent "Stripper" bad_bot
SetEnvIfNoCase user-Agent "Sucker" bad_bot
SetEnvIfNoCase user-Agent "^SuperBot" bad_bot
SetEnvIfNoCase user-Agent "^SuperHTTP" bad_bot
SetEnvIfNoCase user-Agent "^Surfbot" bad_bot
SetEnvIfNoCase user-Agent "^suzuran" bad_bot
SetEnvIfNoCase user-Agent "^Szukacz/1.4" bad_bot
SetEnvIfNoCase user-Agent "^tAkeOut" bad_bot
SetEnvIfNoCase user-Agent "^Teleport" bad_bot
SetEnvIfNoCase user-Agent "^Telesoft" bad_bot
SetEnvIfNoCase user-Agent "^TurnitinBot/1.5" bad_bot
SetEnvIfNoCase user-Agent "^The.Intraformant" bad_bot
SetEnvIfNoCase user-Agent "^TheNomad" bad_bot
SetEnvIfNoCase user-Agent "^TightTwatBot" bad_bot
SetEnvIfNoCase user-Agent "^Titan" bad_bot
SetEnvIfNoCase user-Agent "^toCrawl/UrlDispatcher" bad_bot
SetEnvIfNoCase user-Agent "^True_Robot" bad_bot
SetEnvIfNoCase user-Agent "^turingos" bad_bot
SetEnvIfNoCase user-Agent "^TurnitinBot" bad_bot
SetEnvIfNoCase user-Agent "^URLy.Warning" bad_bot
SetEnvIfNoCase user-Agent "^Vacuum" bad_bot
SetEnvIfNoCase user-Agent "^VCI" bad_bot
SetEnvIfNoCase user-Agent "^VoidEYE" bad_bot
SetEnvIfNoCase user-Agent "^Web\ Image\ Collector" bad_bot
SetEnvIfNoCase user-Agent "^Web\ Sucker" bad_bot
SetEnvIfNoCase user-Agent "^WebAuto" bad_bot
SetEnvIfNoCase user-Agent "^Webbandit" bad_bot
SetEnvIfNoCase user-Agent "^Webclipping.com" bad_bot
SetEnvIfNoCase user-Agent "^WebCopier" bad_bot
SetEnvIfNoCase user-Agent "^WebEMailExtrac.*" bad_bot
SetEnvIfNoCase user-Agent "^WebEnhancer" bad_bot
SetEnvIfNoCase user-Agent "^WebFetch" bad_bot
SetEnvIfNoCase user-Agent "^WebGo\ IS" bad_bot
SetEnvIfNoCase user-Agent "^Web.Image.Collector" bad_bot
SetEnvIfNoCase user-Agent "^WebLeacher" bad_bot
SetEnvIfNoCase user-Agent "^WebmasterWorldForumBot" bad_bot
SetEnvIfNoCase user-Agent "^WebReaper" bad_bot
SetEnvIfNoCase user-Agent "^WebSauger" bad_bot
SetEnvIfNoCase user-Agent "^WebSite" bad_bot
SetEnvIfNoCase user-Agent "^Website\ eXtractor" bad_bot
SetEnvIfNoCase user-Agent "^Website\ Quester" bad_bot
SetEnvIfNoCase user-Agent "^Webster" bad_bot
SetEnvIfNoCase user-Agent "^WebStripper" bad_bot
SetEnvIfNoCase user-Agent "^WebWhacker" bad_bot
SetEnvIfNoCase user-Agent "^WebZIP" bad_bot
SetEnvIfNoCase user-Agent "^Wget" bad_bot
SetEnvIfNoCase user-Agent "Whacker" bad_bot
SetEnvIfNoCase user-Agent "^Widow" bad_bot
SetEnvIfNoCase user-Agent "^WISENutbot" bad_bot
SetEnvIfNoCase user-Agent "^WWWOFFLE" bad_bot
SetEnvIfNoCase user-Agent "^WWW-Collector-E" bad_bot
SetEnvIfNoCase user-Agent "^Xaldon" bad_bot
SetEnvIfNoCase user-Agent "^Xenu" bad_bot
SetEnvIfNoCase user-Agent "^Zeus" bad_bot
SetEnvIfNoCase user-Agent "^Zyborg" bad_bot
Order Allow,Deny
Allow from all
Deny from env=bad_bot

# Modification for dealing with botnet DDoS via high CPU utilization
# by overwhelming PHP with POST data
#
# Referer is empty
RewriteCond %{HTTP_REFERER} ^$
# User agent is empty
RewriteCond %{HTTP_USER_AGENT} ^$
# The request is for the home page
RewriteCond %{REQUEST_URI} ^/$
# It is a POST request
RewriteCond %{REQUEST_METHOD} POST
# Forbid the request
RewriteRule ^(.*)$ - [F,L]


Alias /error/ "/var/www/error/"

<IfModule mod_negotiation.c>
<IfModule mod_include.c>
    <Directory "/var/www/error">
        AllowOverride None
        Options IncludesNoExec
        AddOutputFilter Includes html
        AddHandler type-map var
        Order allow,deny
        Allow from all
        LanguagePriority en es de fr
        ForceLanguagePriority Prefer Fallback
    </Directory>

#    ErrorDocument 400 /error/HTTP_BAD_REQUEST.html.var
#    ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
#    ErrorDocument 403 /error/HTTP_FORBIDDEN.html.var
#    ErrorDocument 404 /error/HTTP_NOT_FOUND.html.var
#    ErrorDocument 405 /error/HTTP_METHOD_NOT_ALLOWED.html.var
#    ErrorDocument 408 /error/HTTP_REQUEST_TIME_OUT.html.var
#    ErrorDocument 410 /error/HTTP_GONE.html.var
#    ErrorDocument 411 /error/HTTP_LENGTH_REQUIRED.html.var
#    ErrorDocument 412 /error/HTTP_PRECONDITION_FAILED.html.var
#    ErrorDocument 413 /error/HTTP_REQUEST_ENTITY_TOO_LARGE.html.var
#    ErrorDocument 414 /error/HTTP_REQUEST_URI_TOO_LARGE.html.var
#    ErrorDocument 415 /error/HTTP_UNSUPPORTED_MEDIA_TYPE.html.var
#    ErrorDocument 500 /error/HTTP_INTERNAL_SERVER_ERROR.html.var
#    ErrorDocument 501 /error/HTTP_NOT_IMPLEMENTED.html.var
#    ErrorDocument 502 /error/HTTP_BAD_GATEWAY.html.var
#    ErrorDocument 503 /error/HTTP_SERVICE_UNAVAILABLE.html.var
#    ErrorDocument 506 /error/HTTP_VARIANT_ALSO_VARIES.html.var

</IfModule>
</IfModule>

#
# The following directives modify normal HTTP response behavior to
# handle known problems with browser implementations.
#
BrowserMatch "Mozilla/2" nokeepalive
BrowserMatch "MSIE 4\.0b2;" nokeepalive downgrade-1.0 force-response-1.0
BrowserMatch "RealPlayer 4\.0" force-response-1.0
BrowserMatch "Java/1\.0" force-response-1.0
BrowserMatch "JDK/1\.0" force-response-1.0

#
# The following directive disables redirects on non-GET requests for
# a directory that does not include the trailing slash.  This fixes a 
# problem with Microsoft WebFolders which does not appropriately handle 
# redirects for folders with DAV methods.
# Same deal with Apple's DAV filesystem and Gnome VFS support for DAV.
#
BrowserMatch "Microsoft Data Access Internet Publishing Provider" redirect-carefully
BrowserMatch "MS FrontPage" redirect-carefully
BrowserMatch "^WebDrive" redirect-carefully
BrowserMatch "^WebDAVFS/1.[0123]" redirect-carefully
BrowserMatch "^gnome-vfs/1.0" redirect-carefully
BrowserMatch "^XML Spy" redirect-carefully
BrowserMatch "^Dreamweaver-WebDAV-SCM1" redirect-carefully

#
# Allow server status reports generated by mod_status,
# with the URL of http://servername/server-status
# Change the ".example.com" to match your domain to enable.
#
#<Location /server-status>
#    SetHandler server-status
#    Order deny,allow
#    Deny from all
#    Allow from .example.com
#</Location>

#
# Allow remote server configuration reports, with the URL of
#  http://servername/server-info (requires that mod_info.c be loaded).
# Change the ".example.com" to match your domain to enable.
#
#<Location /server-info>
#    SetHandler server-info
#    Order deny,allow
#    Deny from all
#    Allow from .example.com
#</Location>

#
# Proxy Server directives. Uncomment the following lines to
# enable the proxy server:
#
#<IfModule mod_proxy.c>
#ProxyRequests On
#
#<Proxy *>
#    Order deny,allow
#    Deny from all
#    Allow from .example.com
#</Proxy>

#
# Enable/disable the handling of HTTP/1.1 "Via:" headers.
# ("Full" adds the server version; "Block" removes all outgoing Via: headers)
# Set to one of: Off | On | Full | Block
#
#ProxyVia On

#
# To enable a cache of proxied content, uncomment the following lines.
# See http://httpd.apache.org/docs/2.2/mod/mod_cache.html for more details.
#
#<IfModule mod_disk_cache.c>
#   CacheEnable disk /
#   CacheRoot "/var/cache/mod_proxy"
#</IfModule>
#

#</IfModule>
# End of proxy directives.

### Section 3: Virtual Hosts
#
# VirtualHost: If you want to maintain multiple domains/hostnames on your
# machine you can setup VirtualHost containers for them. Most configurations
# use only name-based virtual hosts so the server doesn't need to worry about
# IP addresses. This is indicated by the asterisks in the directives below.
#
# Please see the documentation at 
# <URL:http://httpd.apache.org/docs/2.2/vhosts/>
# for further details before you try to setup virtual hosts.
#
# You may use the command line option '-S' to verify your virtual host
# configuration.

#
# Use name-based virtual hosting.
#
#NameVirtualHost *:80
#
# NOTE: NameVirtualHost cannot be used without a port specifier 
# (e.g. :80) if mod_ssl is being used, due to the nature of the
# SSL protocol.
#

#
# VirtualHost example:
# Almost any Apache directive may go into a VirtualHost container.
# The first VirtualHost section is used for requests without a known
# server name.
#
#<VirtualHost *:80>
#    ServerAdmin webmaster@dummy-host.example.com
#    DocumentRoot /www/docs/dummy-host.example.com
#    ServerName dummy-host.example.com
#    ErrorLog logs/dummy-host.example.com-error_log
#    CustomLog logs/dummy-host.example.com-access_log common
#</VirtualHost>
GracefulShutDownTimeout 3
AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps

Last edited by david59; 02-14-2013 at 11:38 AM.
 
Old 02-14-2013, 05:57 AM   #21
david59
LQ Newbie
 
Registered: Feb 2013
Posts: 18

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by unSpawn View Post
The fact your server hasn't crashed since you made changes is hopeful but it does make me wonder what other changes you made that could have adversely affected performance. Maybe it would be best if you start back at the beginning and show us the machine hardware specs (you seem to have 8GB RAM?) and an inventory of installed software. (For example your CentOS 6.2 is one update behind and how about your web stack, Drupal and other software?) Collecting system statistics is a good move but you should monitor other aspects as well like network traffic, your web server and your database. For example Atsar works like "regular" SAR but can also log Apache HTTP traffic (or see "Apache Top"). Like Atsar Dstat aims to replace vmstat, iostat and ifstat but it can also log MySQL statistics (else see "MysQL Top" or "mtop" for short). As for your system logs I suggest you run them through Logwatch at detail level 10 as that's the easiest, quickest way to generate leads. I know this all wasn't very detailed but you have to start somewhere, right?
Thank you very much for your help, its greatly appreciated. Here is some information from the Plesk panel:
Code:
CPU 	AuthenticAMD, Quad-Core AMD Opteron(tm) Processor 1356
Version 	Parallels Plesk Panel v11.0.9_build110120608.16 os_CentOS 6
OS 	CentOS 6.2 (Final)
Key number 	PLSK.------08
System Uptime: 	1 day(s) 02:36
Please see screenshot for additional hardware information.

I will look into Atsar today.
Attached Images
File Type: jpg 112.jpg (38.5 KB, 10 views)
 
Old 02-14-2013, 09:10 AM   #22
david59
LQ Newbie
 
Registered: Feb 2013
Posts: 18

Original Poster
Rep: Reputation: Disabled
Here is some data from sar February 10:

Code:
[root@u1---~]# sar -r -f /var/log/sa/sa10
Linux 2.6.32-220.13.1.el6.x86_64 (u----0.onlinehome-server.com)      02/10/2013      _x86_64_        (4 CPU)

12:00:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
12:10:01 AM   1355352   6671460     83.11    275256   4224112   1719772     17.22
12:20:01 AM   1347736   6679076     83.21    275572   4228152   1711808     17.14
12:30:01 AM   1341460   6685352     83.29    275848   4232332   1710256     17.13
12:40:01 AM   1331908   6694904     83.41    276132   4235856   1711264     17.14
12:50:01 AM   1311024   6715788     83.67    276444   4239544   1726340     17.29
01:00:01 AM   1319112   6707700     83.57    276752   4242968   1712136     17.14
01:10:02 AM    404016   7622796     94.97    267872   5071832   1859188     18.62
01:20:02 AM    157812   7869000     98.03     70812   6142068   1797364     18.00
01:30:01 AM   3843188   4183624     52.12     78208   2541604   1736676     17.39
01:40:01 AM   3799716   4227096     52.66     89472   2582732   1727676     17.30
01:50:01 AM   3777992   4248820     52.93     93720   2602416   1723836     17.26
02:00:01 AM   3770644   4256168     53.02     96628   2614208   1712204     17.14
02:10:01 AM   3739440   4287372     53.41    100468   2629368   1721892     17.24
02:20:01 AM   3728912   4297900     53.54    103408   2645892   1712244     17.15
02:30:01 AM   3711172   4315640     53.77    106672   2657860   1717508     17.20
02:40:01 AM   3692440   4334372     54.00    110528   2670808   1711708     17.14
02:50:01 AM   3653576   4373236     54.48    115876   2696428   1714036     17.16
03:00:01 AM   3630904   4395908     54.77    118956   2709512   1714192     17.16
03:10:01 AM   3562868   4463944     55.61    126248   2748860   1736020     17.38
03:20:01 AM   3560440   4466372     55.64    128756   2759064   1719104     17.21
03:30:01 AM   3264592   4762220     59.33    141432   2958644   1806476     18.09
03:40:01 AM   3675164   4351648     54.21    149324   2927480   1364612     13.66
03:50:01 AM   3589136   4437676     55.29    151316   2934020   1439068     14.41
04:00:01 AM   3560944   4465868     55.64    153400   2940336   1462156     14.64
04:10:01 AM   3551776   4475036     55.75    156332   2950064   1454236     14.56
04:20:01 AM   3530152   4496660     56.02    158316   2954984   1457280     14.59
04:30:01 AM   3521112   4505700     56.13    160168   2962020   1460100     14.62
04:40:01 AM   3500772   4526040     56.39    164784   2968140   1459692     14.62
04:50:01 AM   3427584   4599228     57.30    168308   2978500   1540716     15.43
05:00:01 AM   3392552   4634260     57.73    170328   2984684   1565084     15.67
05:10:01 AM   3360012   4666800     58.14    172180   2990216   1582184     15.84
05:20:01 AM   3304364   4722448     58.83    174572   3006868   1620124     16.22
05:30:01 AM   3290316   4736496     59.01    176312   3012164   1629360     16.32
05:40:01 AM   3220564   4806248     59.88    178136   3051616   1661504     16.64
05:50:01 AM   3184756   4842056     60.32    179844   3056732   1682020     16.84
06:00:01 AM   3156744   4870068     60.67    181204   3064084   1696792     16.99
06:10:01 AM   3128436   4898376     61.03    183944   3075556   1702904     17.05
06:20:01 AM   3119308   4907504     61.14    185652   3083372   1701632     17.04
06:30:01 AM   3107744   4919068     61.28    187728   3085156   1694128     16.96
06:40:01 AM   3090916   4935896     61.49    189328   3096928   1697192     16.99
06:50:01 AM   3080460   4946352     61.62    191344   3106652   1692056     16.94
07:00:01 AM   3042240   4984572     62.10    194320   3129888   1694044     16.96
07:10:01 AM   3027148   4999664     62.29    195992   3137500   1706144     17.08
07:20:01 AM   3016716   5010096     62.42    198344   3143596   1697572     17.00
07:30:01 AM   2999120   5027692     62.64    199680   3152252   1712080     17.14
07:40:01 AM   3001892   5024920     62.60    202036   3146604   1705244     17.08
07:50:01 AM   2990264   5036548     62.75    203392   3152332   1704264     17.07
08:00:01 AM   2963760   5063052     63.08    205508   3170972   1702976     17.05
08:10:01 AM   2944092   5082720     63.32    206924   3177500   1706508     17.09
08:20:01 AM   2871024   5155788     64.23    208444   3202492   1798712     18.01
08:30:01 AM   2923464   5103348     63.58    209640   3189632   1702928     17.05
08:40:01 AM   2922968   5103844     63.58    211056   3189380   1706804     17.09
08:50:01 AM   2895176   5131636     63.93    212908   3205000   1707892     17.10
09:00:01 AM   2718220   5308592     66.14    215476   3331544   1769788     17.72

09:00:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
09:10:01 AM   2703536   5323276     66.32    217188   3337680   1762408     17.65
09:20:01 AM   2687320   5339492     66.52    218372   3345516   1768020     17.70
09:30:01 AM   2656192   5370620     66.91    220000   3361068   1774852     17.77
09:40:01 AM   2645080   5381732     67.05    221308   3365868   1778960     17.81
09:50:01 AM   2632692   5394120     67.20    222236   3371552   1775268     17.78
10:00:01 AM   2621316   5405496     67.34    224096   3380220   1768512     17.71
10:10:01 AM   2612988   5413824     67.45    224612   3383848   1770452     17.73
10:20:01 AM   2609348   5417464     67.49    225248   3389648   1764540     17.67
10:30:01 AM   2590028   5436784     67.73    226444   3395036   1770260     17.73
10:40:01 AM   2587984   5438828     67.76    227268   3399672   1764616     17.67
10:50:01 AM   2570140   5456672     67.98    228540   3406116   1765792     17.68
11:00:01 AM   2564992   5461820     68.04    229468   3411000   1762976     17.65
11:10:01 AM   2557824   5468988     68.13    230272   3413452   1768084     17.70
11:20:01 AM   2548216   5478596     68.25    231020   3419136   1769060     17.71
11:30:01 AM   2536932   5489880     68.39    231868   3424408   1767416     17.70
11:40:01 AM   2525760   5501052     68.53    232544   3429520   1763992     17.66
11:50:01 AM   2521576   5505236     68.59    233176   3436768   1761152     17.63
12:00:01 PM   2501488   5525324     68.84    233884   3442604   1768340     17.71
12:10:01 PM   2494156   5532656     68.93    234904   3447436   1764524     17.67
12:20:01 PM   2480624   5546188     69.10    235452   3451656   1769124     17.71
12:30:01 PM   2472196   5554616     69.20    236004   3456060   1769872     17.72
12:40:01 PM   2471804   5555008     69.21    236852   3456048   1764764     17.67
12:50:01 PM   2466120   5560692     69.28    237428   3460652   1761680     17.64
01:00:01 PM   2450580   5576232     69.47    238048   3466396   1768612     17.71
01:10:01 PM   2446028   5580784     69.53    238620   3470496   1762696     17.65
01:20:01 PM   2434016   5592796     69.68    239112   3474836   1764768     17.67
01:30:01 PM   2427672   5599140     69.76    239852   3479176   1761856     17.64
01:40:01 PM   2417816   5608996     69.88    240608   3483760   1768596     17.71
01:50:02 PM   2412276   5614536     69.95    241420   3487372   1763716     17.66
02:00:01 PM   2397192   5629620     70.14    242356   3492316   1772396     17.75
02:10:01 PM   2387056   5639756     70.26    243260   3497020   1773776     17.76
02:20:01 PM   2381680   5645132     70.33    243960   3501776   1770424     17.73
02:30:01 PM   2366860   5659952     70.51    244724   3506576   1775684     17.78
02:40:01 PM   2362772   5664040     70.56    245216   3511376   1764500     17.67
02:50:01 PM   2341988   5684824     70.82    245728   3516120   1781236     17.84
03:00:01 PM   2347132   5679680     70.76    246508   3520716   1759672     17.62
03:10:01 PM   2292188   5734624     71.44    247332   3550832   1775412     17.78
03:20:01 PM   2281888   5744924     71.57    247948   3558844   1768952     17.71
03:30:01 PM   2265076   5761736     71.78    248624   3566580   1775800     17.78
03:40:01 PM   2249956   5776856     71.97    249148   3573008   1783488     17.86
03:50:01 PM   2248472   5778340     71.99    249432   3580124   1777600     17.80
04:00:01 PM   2237216   5789596     72.13    249936   3585092   1775428     17.78
04:10:01 PM   2233164   5793648     72.18    250412   3581576   1776808     17.79
04:20:01 PM   2221880   5804932     72.32    250776   3583920   1786356     17.89
04:30:01 PM   2165256   5861556     73.02    251212   3588744   1842136     18.45
04:40:01 PM   2163656   5863156     73.04    251256   3592916   1835720     18.38
04:50:01 PM   2154944   5871868     73.15    251420   3595868   1841024     18.43
05:00:01 PM   2132256   5894556     73.44    251684   3600836   1850128     18.53
05:10:01 PM   2136352   5890460     73.38    251652   3604784   1840676     18.43
05:20:01 PM   2122292   5904520     73.56    251172   3610748   1841752     18.44
05:30:01 PM   2110556   5916256     73.71    251776   3620240   1849768     18.52
05:40:01 PM   2101516   5925296     73.82    251928   3629396   1842596     18.45
05:50:01 PM   2090316   5936496     73.96    251644   3636564   1848136     18.51
06:00:01 PM   2082312   5944500     74.06    251020   3643380   1836352     18.39

06:00:01 PM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
06:10:01 PM   2073308   5953504     74.17    251272   3650736   1840564     18.43
06:20:01 PM   2062928   5963884     74.30    250968   3654212   1850704     18.53
06:30:01 PM   2081324   5945488     74.07    250600   3638536   1848952     18.51
06:40:01 PM   2074808   5952004     74.15    250864   3642408   1844880     18.47
06:50:01 PM   2076008   5950804     74.14    250860   3646716   1834824     18.37
07:00:01 PM   2072724   5954088     74.18    251140   3649280   1833124     18.36
07:10:01 PM   2061124   5965688     74.32    251432   3652968   1838912     18.41
07:20:01 PM   2054356   5972456     74.41    252032   3657716   1837228     18.40
07:30:01 PM   2332972   5693840     70.94    253212   3735924   1386448     13.88
07:40:01 PM   2412664   5614148     69.94    253668   3768904   1320880     13.23
07:50:01 PM   2254920   5771892     71.91    254252   3750792   1443728     14.46
08:00:01 PM   2249132   5777680     71.98    254688   3754384   1442460     14.44
08:10:01 PM   2238192   5788620     72.12    255060   3758456   1444344     14.46
08:20:01 PM   2227924   5798888     72.24    255324   3761072   1446840     14.49
08:30:01 PM   2214836   5811976     72.41    255756   3765396   1450280     14.52
08:40:01 PM   2172272   5854540     72.94    256064   3767708   1497164     14.99
08:50:01 PM   2152788   5874024     73.18    256456   3771520   1503912     15.06
09:00:01 PM   2143084   5883728     73.30    256800   3776716   1509960     15.12
09:10:01 PM   2126156   5900656     73.51    257260   3781652   1511972     15.14
09:20:01 PM   2123084   5903728     73.55    257640   3787212   1504120     15.06
09:30:01 PM   2113648   5913164     73.67    257712   3792372   1506464     15.08
09:40:01 PM   2096256   5930556     73.88    258100   3796992   1514600     15.17
09:50:01 PM   2035768   5991044     74.64    258324   3799560   1587936     15.90
10:00:01 PM   1987748   6039064     75.24    258600   3802084   1630436     16.33
10:10:02 PM   1985500   6041312     75.26    258928   3804376   1628204     16.30
10:20:01 PM   1979544   6047268     75.34    258564   3806912   1625192     16.27
10:30:01 PM   1968372   6058440     75.48    258656   3810292   1624444     16.27
10:40:01 PM   1965624   6061188     75.51    258564   3813140   1621616     16.24
10:50:01 PM   1957800   6069012     75.61    258536   3816576   1625916     16.28
11:00:01 PM   1943560   6083252     75.79    258748   3820976   1631844     16.34
11:10:01 PM   1655944   6370868     79.37    258684   4116996   1629572     16.32
11:20:01 PM   1637616   6389196     79.60    258372   4119348   1638280     16.40
11:30:01 PM   1651516   6375296     79.43    258472   4122088   1613292     16.15
11:40:01 PM   1646488   6380324     79.49    258496   4124164   1612164     16.14
11:50:01 PM   1630488   6396324     79.69    258620   4126660   1622980     16.25
Average:      2534226   5492586     68.43    219371   3458278   1698423     17.01

And from today, the 14th:

Code:
[root@u1------- ~]# sar -r -f /var/log/sa/sa14
Linux 2.6.32-220.13.1.el6.x86_64 (u-----0.onlinehome-server.com)      02/14/2013      _x86_64_        (4 CPU)

12:00:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
12:10:01 AM   1594672   6432140     80.13    277968   4145112   1556908     15.59
12:20:01 AM   1584708   6442104     80.26    278356   4150880   1555892     15.58
12:30:01 AM   1570940   6455872     80.43    278748   4155996   1557404     15.59
12:40:01 AM   1554844   6471968     80.63    279112   4160532   1565224     15.67
12:50:01 AM   1562848   6463964     80.53    279372   4146252   1568824     15.71
01:00:01 AM   1557728   6469084     80.59    279612   4151184   1563592     15.66
01:10:01 AM    403232   7623580     94.98    280784   5208152   1687732     16.90
01:20:02 AM    164332   7862480     97.95     90432   6211500   1670400     16.73
01:30:01 AM   3660020   4366792     54.40     95652   2780508   1618220     16.20
01:40:01 AM   3581848   4444964     55.38    110616   2850616   1609084     16.11
01:50:01 AM   3543580   4483232     55.85    116252   2888660   1604168     16.06
02:00:01 AM   3524248   4502564     56.09    119452   2904868   1600156     16.02
02:10:01 AM   3472640   4554172     56.74    123620   2936368   1613460     16.16
02:20:01 AM   3450224   4576588     57.02    126520   2950824   1610060     16.12
02:30:01 AM   3410036   4616776     57.52    131580   2973408   1617100     16.19
02:40:01 AM   3388092   4638720     57.79    134972   2987932   1618908     16.21
02:50:01 AM   3359628   4667184     58.14    137756   2999156   1625912     16.28
03:00:01 AM   3364560   4662252     58.08    139560   3007840   1606996     16.09
03:10:01 AM   3340080   4686732     58.39    141260   3016648   1615180     16.17
03:20:01 AM   3345592   4681220     58.32    142896   3013348   1603952     16.06
03:30:01 AM   3065380   4961432     61.81    159012   3138520   1718940     17.21
03:40:01 AM   3216224   4810588     59.93    164572   3079468   1607972     16.10
03:50:01 AM   3188628   4838184     60.28    166080   3101652   1609524     16.12
04:00:01 AM   3157756   4869056     60.66    167476   3107312   1626484     16.29
04:10:01 AM   3169492   4857320     60.51    169172   3113968   1597416     16.00
04:20:01 AM   3130592   4896220     61.00    170880   3120256   1624576     16.27
04:30:01 AM   3144708   4882104     60.82    172832   3113652   1607912     16.10
04:40:01 AM   3134328   4892484     60.95    174520   3119744   1611920     16.14
04:50:01 AM   3117476   4909336     61.16    180956   3127768   1603884     16.06
05:00:01 AM   3106036   4920776     61.30    182108   3134232   1603052     16.05
05:10:01 AM   3093600   4933212     61.46    184132   3140092   1601756     16.04
05:20:01 AM   3080180   4946632     61.63    185840   3146848   1596844     15.99
05:30:01 AM   3023392   5003420     62.33    187896   3152980   1657348     16.60
05:40:01 AM   3018708   5008104     62.39    189284   3158528   1649656     16.52
05:50:01 AM   3008228   5018584     62.52    190796   3164352   1648952     16.51
06:00:01 AM   2992060   5034752     62.72    192516   3171620   1651828     16.54
06:10:01 AM   2907268   5119544     63.78    197328   3230216   1730732     17.33
06:20:01 AM   2720644   5306168     66.11    200852   3261320   1886436     18.89
06:30:01 AM   2780516   5246296     65.36    202184   3267704   1797136     18.00
06:40:01 AM   2866036   5160776     64.29    203892   3242436   1680224     16.82
06:50:01 AM   2839328   5187484     64.63    205432   3249724   1696008     16.98
07:00:01 AM   2841548   5185264     64.60    206672   3255416   1677136     16.79
07:10:01 AM   2827284   5199528     64.78    207788   3260888   1683712     16.86
07:20:01 AM   2820428   5206384     64.86    208968   3266712   1674184     16.76
07:30:01 AM   2786656   5240156     65.28    210212   3275360   1698120     17.00
07:40:01 AM   2782924   5243888     65.33    211520   3284872   1688504     16.91
07:50:01 AM   2777596   5249216     65.40    212816   3291456   1676960     16.79
08:00:01 AM   2758964   5267848     65.63    213872   3297668   1677248     16.79
08:10:01 AM   2741436   5285376     65.85    215060   3304056   1683088     16.85
08:20:01 AM   2739440   5287372     65.87    215952   3310120   1680016     16.82
08:30:01 AM   2726984   5299828     66.03    216948   3318444   1672452     16.75
08:40:01 AM   2690168   5336644     66.49    218332   3325784   1692940     16.95
08:50:01 AM   2699796   5327016     66.37    219484   3333796   1667504     16.70
09:00:01 AM   2682808   5344004     66.58    220640   3340428   1667380     16.70

09:00:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
09:10:01 AM   2671976   5354836     66.71    221308   3345988   1667512     16.70
09:20:01 AM   2660992   5365820     66.85    221912   3351500   1666020     16.68
09:30:01 AM   2649912   5376900     66.99    222564   3357816   1665384     16.68
09:40:01 AM   2634464   5392348     67.18    223044   3363040   1670984     16.73
09:50:01 AM   2627580   5399232     67.26    223676   3370496   1662568     16.65
10:00:01 AM   2614684   5412128     67.43    224356   3376472   1663108     16.65
10:10:01 AM   2594936   5431876     67.67    224928   3382304   1678372     16.81
Average:      2779098   5247714     65.38    192661   3350734   1645754     16.48

Last edited by colucix; 02-14-2013 at 10:04 AM. Reason: [CODE] tags fixed.
 
Old 02-14-2013, 10:50 AM   #23
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,953
Blog Entries: 54

Rep: Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733
Quote:
Originally Posted by david59 View Post
OS CentOS 6.2 (Final)
It's one release behind as CentOS has been at 6.3 for some time now.


Quote:
Originally Posted by david59 View Post
I will look into Atsar today.
Maybe I wasn't clear enough when I posted my reply but the general idea is to monitor as much aspects as possible and monitor them continuously.
The reason I explained those applications is so you have an idea which app can monitor what. So wrt choice think "and and" and not "or or".
The reason for continuously monitoring system, network, process and database stats is that over time it becomes easier to spot for example recurring events (say cron jobs), spikes you can trace back (say manually importing a humongous table), steadily increasing resource allocation you can correlate in other ways (advertising, new site launch, complete and utter misconfiguration, etc, etc.). So if you have baseline data, the machine being in rest without any activity, then over time it becomes easier to determine what the bottlenecks are and what can be attributed to usage or abuse.
So while posting SAR data (missing 3 days worth of data in between BTW) is neat, a single spike lasting approximately 40 minutes itself doesn't reveal much unless you correlate it with process and network resource usage at that time, amount and type of requests, etc, etc.

*BTW reading your httpd.conf my first questions would be 0) what changes exactly you made and 1) why would you be running Apache with about all modules including proxy_.* enabled?..

*BTW[1] If it is correct you host about 22 web sites and mail servers then I wonder at what stage you began experiencing problems. Right from the start or after you had x amount of sites running?
 
Old 02-14-2013, 11:44 AM   #24
david59
LQ Newbie
 
Registered: Feb 2013
Posts: 18

Original Poster
Rep: Reputation: Disabled
OK, sorry, I misunderstood. I understand fully what you are talking about. I appreciate the advice.

Re: specific changes to the httpd.conf, I highlighted them in blue bold italics in post #20 above. I have since removed those items and the server is running fine. If, as I expect, bots return to my site, I will have to restore some sort of script to block them.


You ask above: why would you be running Apache with about all modules including proxy_.* enabled?..

The short answer is because I am an absolute beginner and if someone with expertise told me which modules I did not need to be activated, I would gladly comment them out.

Let me think about the answer to your second question about 22 sites. I will get back to you. many thanks.
 
Old 02-14-2013, 12:12 PM   #25
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,953
Blog Entries: 54

Rep: Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733
Do you have a machine at home or work you could use? Doesn't have to be a dedicated one: you could install virtualization software like VMware, VirtualBox, QEmu or whatever you like. Point is once you have that you can install Linux. Then you have a machine you can test use for testing purposes w/o the fear of fscking up your production machine.
 
Old 02-15-2013, 09:14 AM   #26
david59
LQ Newbie
 
Registered: Feb 2013
Posts: 18

Original Poster
Rep: Reputation: Disabled
Thanks, I have set up VirtualBox and am experimenting with it.

I am not running 22 sites per se, I have three drupal installations (will be removing one of these) and each installation runs a few sites off the same codebase.

Regarding the proxy-related Apache modules, I looked at what I have enabled:

Code:
proxy 
proxy_ajp 
proxy_balancer 
proxy_connect 
proxy_ftp 
proxy_http
And one that is disabled:

Code:
proxy_scgi
My online help tells me that the following modules are always enabled for CentOS:

Code:
    env
    auth_digest
    authn_file
    authz_host
    authz_user
    actions
    alias
    autoindex
    dav
    dir
    log_config
    mime
    negotiation
    setenvif
    ssl
I am not at all sure that I need the proxy modules enabled. If I want to disable them, is there a particular order in which I must do so?

Thanks for your advice.
 
Old 02-15-2013, 10:17 AM   #27
unSpawn
Moderator
 
Registered: May 2001
Posts: 26,953
Blog Entries: 54

Rep: Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733Reputation: 2733
Quote:
Originally Posted by david59 View Post
Thanks, I have set up VirtualBox and am experimenting with it.
The idea is that you load your production configuration (rsync?) so you can test modification before you apply them for real. Processes like that aren't meant to convolute your work flow but to ensure your production environment doesn't crap out over misconfiguration or botched upgrades. Don't see it as a hassle but as you protecting your investment.


Quote:
Originally Posted by david59 View Post
I am not running 22 sites per se, I have three drupal installations (will be removing one of these) and each installation runs a few sites off the same codebase.
Yah, I should've said you've got x-something domains registered.
Are these Drupal installations all up to date?


Quote:
Originally Posted by david59 View Post
I am not at all sure that I need the proxy modules enabled. If I want to disable them, is there a particular order in which I must do so?
No, not really.


Quote:
Originally Posted by david59 View Post
Thanks for your advice.
That's OK, it's what we're here for. Now come on and supply the nfo that helps us help you!
 
  


Reply

Tags
crash, server


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Strange Log Readout showing lots of queries using named , cant understand tonechild Linux - Server 3 02-21-2012 01:44 PM
LXer: Top 10 Things Linux Users Don't Understand LXer Syndicated Linux News 0 08-12-2011 03:50 AM
trying to understand top command -H option connecticket Linux - Newbie 1 06-12-2009 12:46 AM
Help me understand my top output neocontrol Linux - Server 5 02-21-2008 05:46 PM
color ls readout? orange400 Linux - General 2 06-18-2004 04:58 AM


All times are GMT -5. The time now is 03:28 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration