Apache Log

yackluminoso · 07-05-2017, 06:28 AM

Hello,

Can you define two identical LogFormat and then use them in the same vhost in two different CustomLogs?
Thank you

Yack

szboardstretcher · 07-05-2017, 06:49 AM

Would be simple enough to test it. Have you tested it? What was the result?

Maybe something like this if you haven't tried yet,..

Code:

LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog "logs/access_log_1" common
CustomLog "logs/access_log_2" common

Related Docs: http://httpd.apache.org/docs/current...html#customlog

yackluminoso · 07-05-2017, 07:32 AM

Certainly, I did some tests:

In httpd.conf:
...
LogFormat "%{X-Forwarded-For}i %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"" test
LogFormat "%{X-Forwarded-For}i %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"" combined
…
In virtual host config:
<VirtualHost *:80>
…
CustomLog /var/log/httpd/www.efaci.it.access_log combined
CustomLog "|/usr/bin/logger -t httpd -p local7.info" test
…

but it does not work

Only first CustomLog work (combined)

Logs are sent to the configured destination but do not come with the correct format.

If you change the configuration in this way:

<VirtualHost *:80>
…
CustomLog /var/log/httpd/www.efaci.it.access_log combined
CustomLog "|/usr/bin/logger -t httpd -p local7.info" combined
…

all work fine, but I need two different LogFormat

Thanks

Yack

szboardstretcher · 07-05-2017, 09:03 AM

You are only showing one log format with two different aliases.

Code:

LogFormat "%{X-Forwarded-For}i %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i""

What are the two different LogFormats you are trying to use?

yackluminoso · 07-05-2017, 09:20 AM

The two logformat are identical, they change the name only, but have two different purposes.
The first sends the logs into one file the second one in a siem.
Their content may, in time, differ.
I would like to use them both in the vhost configuration.
Logs are sent to the file in correct format but the log send to siem not have the correct format (LogFormat 'test')

szboardstretcher · 07-05-2017, 09:22 AM

What about specifying them without alias reference?

Code:

Virtualhost 80....

CustomLog "/var/log/httpd/www.efaci.it.access_log" "%{X-Forwarded-For}i %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i""
CustomLog "|/usr/bin/logger -t httpd -p local7.info" "%{X-Forwarded-For}i %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i""

...

yackluminoso · 07-05-2017, 09:40 AM

error!!!

CustomLog takes two or three arguments, a file name, a custom log format string or format name, and an optional "env=" or "expr=" clause

szboardstretcher · 07-05-2017, 10:46 AM

Lets start all over and break apart this format string that is giving the error:

Would it be better written like this with escapes?

Code:

"%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""

YES! Httpd loads up correctly now. If I add my local IP to /etc/hosts as example.com --- and if I write out a silly /etc/httpd/conf.d/vhosts.conf file for testing like this:

Code:

<VirtualHost *:80>
    ServerName example.com
    ServerAlias example.com
    DocumentRoot /var/www/html
    ErrorLog /var/www/html/error.log
    CustomLog /var/www/html/requests.log1 test1
    CustomLog /var/www/html/requests.log2 test2
    CustomLog /var/www/html/requests.log3 test1
    CustomLog /var/www/html/requests.log4 test2
</VirtualHost>

And add the formats to the conf/httpd.conf file, along with the test 'MYTEST' so we know its coming from these lines, in the "IfModule log_config_module" section like so:

Code:

<IfModule log_config_module>
    LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" MYTEST1" test1
    LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" MYTEST2" test2

Restart and look at our files that were created and their format:

Code:

[root@linux01 httpd]# cat /var/www/html/requests.log*
- - - [05/Jul/2017:15:45:00 +0000] "GET / HTTP/1.1" 200 12 "-" "curl/7.29.0" MYTEST1
- - - [05/Jul/2017:15:45:00 +0000] "GET / HTTP/1.1" 200 12 "-" "curl/7.29.0" MYTEST2
- - - [05/Jul/2017:15:45:00 +0000] "GET / HTTP/1.1" 200 12 "-" "curl/7.29.0" MYTEST1
- - - [05/Jul/2017:15:45:00 +0000] "GET / HTTP/1.1" 200 12 "-" "curl/7.29.0" MYTEST2

You can see that the aliases did their magic. The logformat string has to be formatted correctly and it has to be in the right place in the configuration file.

yackluminoso · 07-05-2017, 11:01 AM

Thank you, I will try this evening.

Yack

yackluminoso · 07-06-2017, 02:48 AM

It works but why does not it give me the remote ip that makes the web request?

<190>httpd: - - - [06/Jul/2017:09:41:39 +0200] "GET /costruction.gif HTTP/1.1" 304 - "http://www.xxxx.it/" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0" TEST

the option %{X-Forwarded-For}i not work!

Thanks

Yack

scasey · 07-06-2017, 10:58 AM

Quote:

Originally Posted by yackluminoso

It works but why does not it give me the remote ip that makes the web request?

<190>httpd: - - - [06/Jul/2017:09:41:39 +0200] "GET /costruction.gif HTTP/1.1" 304 - "http://www.xxxx.it/" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0" TEST

the option %{X-Forwarded-For}i not work!

Thanks

Yack

How is the variable X-Forwarded-For populated?

The format you've proposed doesn't include recording of the remote IP. Add %h to the format to see the remote IP.
My 'combined' format is defined as

Code:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\"" combined

which yields

Code:

73.186.197.82 - - [06/Jul/2017:07:09:43 -0700] "GET / HTTP/1.1" 200 39638 "-"

the hyphens indicate that the requested data is not available, so in my case, the remote log name and the remote user are not there.

Maybe also review the documentation Take note of the section on Piped Logs. And this for details on log formatting