[SOLVED] Need to exclude certain messages from Apache access_log
Linux - ServerThis forum is for the discussion of Linux Software used in a server related context.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I've tried various combinations including using "Referer" as the attribute and with or without the leading "/" in the regex. Nothing seems to work.
Does this version of Apache support SetEnvIf? The docs seem to indicate it does. Do I need to load some module? Perhaps I have the SetEnvIf directive in the wrong place in httpd.conf? Any ideas?
SetEnvIf Request_URI "^/calendars" dontlog=true
CustomLog "/var/log/httpd/access_log" common env=!dontlog
but it still doesn't work. The following link indicates that this does not work if redirecting to SSL: http://www.apachelounge.com/viewtopic.php?p=22266, which I am doing. The poster in that link says his SetEnvIf did not work until he turned off the SSL redirection. Otherwise, no solution was posted. I am not in a position to turn this off and for production use don't want to. Not really sure why this wouldn't work redirecting to SSL. Any ideas?
I have the following in my DOCUMENT_ROOT/.htaccess:
RewriteEngine On
RewriteCond {HTTPS} !^on
RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [L,R]
More info: I turned off the http rewrite. If I reference the /calendars directory as http://hostname.com/calendars, then the nolog function works. However, even with the rewrite off, if I use https://hostname.com/calendars the request gets logged to access_log. So, using https at all causes the SetEnvIf to fail.
More info: I turned off the http rewrite. If I reference the /calendars directory as http://hostname.com/calendars, then the nolog function works. However, even with the rewrite off, if I use https://hostname.com/calendars the request gets logged to access_log. So, using https at all causes the SetEnvIf to fail.
Why? How to fix?
Sorry don't know how to preserve the env. variable when using mod_rewrire
If it's acceptable, you can use the same technique inside the https vhost stanza and just use a different file to log ssl access, e.g:
Code:
SetEnvIf Request_URI "^/calendars" dontlog
CustomLog /var/log/httpd/ssl_access_log common env=!dontlog
The SetEnvIf I put into the httpd-ssl.conf worked. No "/calendars" URIs are logged to ssl_request_log. However, these URIs are still logged to access_log. Still at a loss as to how to not log things in access_log. Seems like an Apache bug!
How about if I log NOTHING to access_log in the main httpd.conf and direct all ssl logging to the access_log instead of ssl_request_log?
How do I turn off non-ssl logging? I've tried commenting out the CustomLog line in httpd.conf, but it's still logging!
This thing's like a zombie cockroach! I can't kill it! I've commented out the original CustomLog directives in httpd.conf and replaced with your suggested "CustomLog /dev/null common", yet I'm *still* getting log messages in access_log!!!! I've checked all the files included by httpd.conf and none other than the httpd-ssl.conf have a CustomLog directive. I've even turned off (commented out) the CustomLog in httpd-ssl.conf -- but they keep coming! I've shutdown httpd for several minutes, verified no httpd task are running, then started. Really, how do I stop logging?
Just to prove I'm not insane (in this instance) here's my httpd.conf, excluding the many LogModules:
Code:
ServerRoot "/usr"
Listen 80
LoadModule authn_file_module lib64/httpd/modules/mod_authn_file.so
:
many more
:
:
RewriteEngine On
RewriteRule ^/DVR$ DVR/ [R]
ProxyPass /DVR/ http://192.168.0.24/
<IfModule unixd_module>
User apache
Group apache
</IfModule>
ServerAdmin webadmin@ohprs.org
ServerName mail.ohprs.org:80
<Directory />
AllowOverride none
Require all denied
</Directory>
DocumentRoot "/srv/httpd/htdocs"
<Directory "/srv/httpd/htdocs">
Options FollowSymLinks
AllowOverride All
Require all granted
</Directory>
<IfModule dir_module>
DirectoryIndex index.html index.php
</IfModule>
<Files ".ht*">
Require all denied
</Files>
ErrorLog "/var/log/httpd/error_log"
LogLevel warn
<IfModule log_config_module>
LogFormat "h l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
<IfModule logio_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>
CustomLog /dev/null common
</IfModule>
<IfModule alias_module>
ScriptAlias /cgi-bin/ "/srv/httpd/cgi-bin/"
</IfModule>
<IfModule cgid_module>
</IfModule>
<Directory "/srv/httpd/cgi-bin">
AllowOverride None
Options None
Require all granted
</Directory>
<IfModule mime_module>
TypesConfig /etc/httpd/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
</IfModule>
Include /etc/httpd/extra/httpd-dav.conf
<IfModule proxy_html_module>
Include /etc/httpd/extra/proxy-html.conf
</IfModule>
Include /etc/httpd/extra/httpd-ssl.conf
<IfModule ssl_module>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>
Include /etc/httpd/mod_php.conf
This thing's like a zombie cockroach! I can't kill it! I've commented out the original CustomLog directives in httpd.conf and replaced with your suggested "CustomLog /dev/null common", yet I'm *still* getting log messages in access_log!!!! I've checked all the files included by httpd.conf and none other than the httpd-ssl.conf have a CustomLog directive. I've even turned off (commented out) the CustomLog in httpd-ssl.conf -- but they keep coming! I've shutdown httpd for several minutes, verified no httpd task are running, then started. Really, how do I stop logging?
It should work.
Could be that you're using "TransferLog" instead of "CustomLog" somewhere in your configs.
Or CustomLog is defined in an included file for vhosts setup. You can run a recursive find under the apache config directory, to see where it's used:
Yep, you nailed it! There was a TransferLog directive in httpd-ssl.conf. When I commented that out all logging stopped. So, I had the following in the httpd-ssl.conf <Vitural Host> section:
TransferLog "/var/log/httpd/access_log"
CustomLog "/var/log/httpd/ssl_request_log" "t h {SSL_PROTOCOL}x %{SSL_CIPHER}x "%r" %b"
and the following in httpd.conf in the <IfModule log_config_module> section:
LogFormat "%h %l %u %t "%r" %>s %b" common
CustomLog "/var/log/httpd/access_log" common
Seems like an awful lot of logging going on. With respect to logging, this was the default setup. I would have expected with the TransferLog (httpd-ssl.conf) and the CustomLog (httpd.conf) both specified to have gotten 2 messages per access in access_log, but I didn't. Any idea why?
I've commented out the "CustomLog /dev/null common" in httpd.conf and the TransferLog in httpd-ssl.conf and now have only the following in httpd-ssl.conf:
I'll let this run a bit, then test re-enabling the CustomLog in the main httpd.conf. I'm thinking that when I did the SetEnvIf there the TransferLog "took over" and logged the message anyway.
I would have expected with the TransferLog (httpd-ssl.conf) and the CustomLog (httpd.conf) both specified to have gotten 2 messages per access in access_log, but I didn't. Any idea why?
You didn't because TransferLog specifies the logging for the https server, while CustomLog (httpd.conf) does the same for the http.
If you leave both TransferLog and CustomLog enabled in the https section (the default), you'll get 2 logs for the same request. The usual from TransferLog (written in access_log) and one containing also the ssl protocol from CustomLog (written in ssl_request_log).
Quote:
I'm thinking that when I did the SetEnvIf there the TransferLog "took over" and logged the message anyway.
Nah. TransferLog was logging the request when using https regardless the SetEnvIf in the http section
bathory: "If you leave both TransferLog and CustomLog enabled in the https section (the default), you'll get 2 logs for the same request."
Well, when I did have both enabled I didn't see 2 messages. But no matter, I believe I've solved my problem by comments out ALL logging in httpd-ssl.conf and having only the following in httpd.conf:
LogLevel warn
LogFormat "h %l %u %t "%r" %>s %b" common
SetEnvIf Request_URI "^/calendars" dontlog=true
CustomLog "/var/log/httpd/access_log" common env=!dontlog
That kills off requests for the calendars folder.
I believe this issue is resolved, though confusing. Thanks for your help.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
