LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices

Reply
 
Search this Thread
Old 04-06-2003, 09:48 AM   #1
Kayaker
LQ Newbie
 
Registered: Mar 2003
Posts: 21

Rep: Reputation: 15
How to read the access_log of Apache?


Folks,

This is really an apache question - not slackware question. But I have always received excellent advice from Slackware folks than anyone else. I am taking advantage of knowledge of slack floks once again.

A standard apache installation yields an access log file called
/usr/local/apache2/logs/access_log
and after the server is up running for a while, it is usually loaded with contents like:

65.33.94.190 - - [05/Apr/2003:17:26:27 -0500] "GET /scripts/root.exe?/c+dir HTTP/1.0" 404 276

65.33.94.190 - - [05/Apr/2003:17:26:28 -0500] "GET /MSADC/root.exe?/c+dir HTTP/1.0" 404 274

65.33.94.190 - - [05/Apr/2003:17:26:29 -0500] "GET /c/winnt/system32/cmd.exe?/c+dir HTTP/1.0" 404 284

65.33.94.190 - - [05/Apr/2003:17:26:41 -0500] "GET /scripts/..%%35c../winnt/system32/cmd.exe?/c+dir HTTP/1.0" 400 288

12.246.123.60 - - [04/Apr/2003:13:01:15 -0500] "GET /scripts/root.exe?/c+dir HTTP/1.0" 404 276

80.133.155.17 - - [03/Apr/2003:10:35:54 -0500] "PROPFIND /admin%24 HTTP/1.1" 405 305

What are these stuff?

How do I read the access log.

I would appreciate the answers to my question come in the following formats:

1) a simplified answer such as
the_meaning_of_the_1st_field
the_meaning_of_the_2nd_field
the_meaning_of_the_3rd_field
the_meaning_of_the_4th_field
the_meaning_of_the_5th_field
the_meaning_of_the_6th_field
....

or

2) a pointer (a hyperlink) to the documenation where a detailed explanation can be found.

By the way, does access_log provide any indication of security breach of the web server? If so, how does one tell by reading the log?

Many Thanks.

Kayaker
 
Old 04-06-2003, 10:30 AM   #2
david_ross
Moderator
 
Registered: Mar 2003
Location: Scotland
Distribution: Slackware, RedHat, Debian
Posts: 12,047

Rep: Reputation: 64
Take a look at
http://www.serverwatch.com/tutorials...le.php/1127521

I once wrote a script to show the error log (I use it a lot when debugging programs) You could probably adapt it to show the access log in a mor ereadable format too:
Code:
#!/usr/bin/perl
if(!$ENV{'QUERY_STRING'}){$do="15"}
else{$do=$ENV{'QUERY_STRING'}}
print "Content-type: text/html\n\n<HTML><TITLE>View Log</TITLE><BODY><H2>LOGS</H2><TABLE border=1>";
open(LOG, "/var/log/httpd/error_log") || die("Could not open log");
@line = <LOG>;
close(LOG);
print $line[1];
$num = @line;
for($done=0;$done<$do;$done++){
$num--;
if($line[$num] =~ /^\[.*:..:.*\]/){
$line[$num] =~ tr/\[/ /;
($time, $typ, $cli, $info) = split (/\]/, $line[$num]);
}
else{$info=$line[$num]}
if($line[$num] =~ /\(2\)/){
$cli = "";
$info=$cli
}
if($tm ne $time){print "<TR><TD colspan=4><HR></TD></TR>"}
print "<TR><TD width=100>$time</TD><TD>$typ</TD><TD width=50>$cli</TD><TD>$info</TD></TR>";
$tm=$time;
}
print "</TABLE></BODY></HTML>";
exit;
 
Old 04-06-2003, 12:07 PM   #3
Kayaker
LQ Newbie
 
Registered: Mar 2003
Posts: 21

Original Poster
Rep: Reputation: 15
Thank you David.

I found this page after posting the question.

http://httpd.apache.org/docs/logs.html#accesslog

perhaps it will be of interest to others as well.

Kayaker
 
Old 04-06-2003, 12:44 PM   #4
MasterC
Guru
 
Registered: Mar 2002
Location: Salt Lake City, UT - USA
Distribution: Gentoo ; LFS ; Kubuntu
Posts: 12,612

Rep: Reputation: 64
Quote:
Originally posted by david_ross
Take a look at
http://www.serverwatch.com/tutorials...le.php/1127521

I once wrote a script to show the error log (I use it a lot when debugging programs) You could probably adapt it to show the access log in a mor ereadable format too:
Code:
#!/usr/bin/perl
if(!$ENV{'QUERY_STRING'}){$do="15"}
else{$do=$ENV{'QUERY_STRING'}}
print "Content-type: text/html\n\n<HTML><TITLE>View Log</TITLE><BODY><H2>LOGS</H2><TABLE border=1>";
open(LOG, "/var/log/httpd/error_log") || die("Could not open log");
@line = <LOG>;
close(LOG);
print $line[1];
$num = @line;
for($done=0;$done<$do;$done++){
$num--;
if($line[$num] =~ /^\[.*:..:.*\]/){
$line[$num] =~ tr/\[/ /;
($time, $typ, $cli, $info) = split (/\]/, $line[$num]);
}
else{$info=$line[$num]}
if($line[$num] =~ /\(2\)/){
$cli = "";
$info=$cli
}
if($tm ne $time){print "<TR><TD colspan=4><HR></TD></TR>"}
print "<TR><TD width=100>$time</TD><TD>$typ</TD><TD width=50>$cli</TD><TD>$info</TD></TR>";
$tm=$time;
}
print "</TABLE></BODY></HTML>";
exit;
Where would I put this, and how would I use it (and what does it do )?

Looks cool though

Cool
 
Old 04-06-2003, 01:03 PM   #5
david_ross
Moderator
 
Registered: Mar 2003
Location: Scotland
Distribution: Slackware, RedHat, Debian
Posts: 12,047

Rep: Reputation: 64
I assume you are talking about the script.
1) Put it in a file called log.cgi within your cgi-bin
2) Give it executeable permissions
3) Make sure the error log file is pointing to the right place
4) Set the permissions on the logs/logs directory so the web server can read it.
5) Access http://yourhost/cgi-bin/log.cgi

That's it! It automatically displays the last 15 lines of the error log in an easy to read table. It tries to group errors related to each individual access and puts an <HR> (horizontal rule) between them.

If you want to see more than the last 15 lines then append the number of lines to view to the filename as a query string. To disply the last 100 - http://yourhost/cgi-bin/log.cgi?100

Let me know if you have any problems.
 
Old 04-06-2003, 01:33 PM   #6
MasterC
Guru
 
Registered: Mar 2002
Location: Salt Lake City, UT - USA
Distribution: Gentoo ; LFS ; Kubuntu
Posts: 12,612

Rep: Reputation: 64
Woohoo! Worked great, thanks! I tried to make an access log the same way, but it didn't quite work out the same, the date/time wasn't displayed, and all entries were grouped into one. But I will work on that...

Thanks!

Cool
 
Old 04-06-2003, 01:36 PM   #7
MasterC
Guru
 
Registered: Mar 2002
Location: Salt Lake City, UT - USA
Distribution: Gentoo ; LFS ; Kubuntu
Posts: 12,612

Rep: Reputation: 64
Unlrelated:

You should sign up for affero. I was going to "affero" you, but it looks like you aren't signed up. Here's a thread with some info on it:
http://www.linuxquestions.org/questi...threadid=25730

And here's my search string:
http://www.linuxquestions.org/questi...der=descending

For even more info

Cool
 
Old 04-06-2003, 01:40 PM   #8
david_ross
Moderator
 
Registered: Mar 2003
Location: Scotland
Distribution: Slackware, RedHat, Debian
Posts: 12,047

Rep: Reputation: 64
I keep meaning to - I just never get round to it. No time like the present I suppose!

I'm glad you like it. I was just getting annoyed at reading horrible raw logs. There are plenty of analysers for access logs but nothing for us poor developers. There are a few simple enhancements that I may make at some point but consider it as GPL
 
Old 04-06-2003, 03:15 PM   #9
MasterC
Guru
 
Registered: Mar 2002
Location: Salt Lake City, UT - USA
Distribution: Gentoo ; LFS ; Kubuntu
Posts: 12,612

Rep: Reputation: 64
Cool, thanks again, and can you provide me any tips on how to go about setting this up for other logs, such as access_log?

 
Old 04-06-2003, 03:17 PM   #10
david_ross
Moderator
 
Registered: Mar 2003
Location: Scotland
Distribution: Slackware, RedHat, Debian
Posts: 12,047

Rep: Reputation: 64
Since you just clicked the Affreo - give me 10 mins!
 
Old 04-06-2003, 05:17 PM   #11
david_ross
Moderator
 
Registered: Mar 2003
Location: Scotland
Distribution: Slackware, RedHat, Debian
Posts: 12,047

Rep: Reputation: 64
The longest 10 minutes of my life!
Code:
#!/usr/bin/perl

# Start Config

$access_log = "/var/log/httpd/access_log";
$error_log = "/var/log/httpd/error_log";
$default_log = "error"; # access or error
$default_lines = "15";

# End Config

# Print the top of the page
print <<"EOF";
Content-type: text/html

<HTML>
<HEAD>
<TITLE>Viewing $in{'log'} log</TITLE>
</HEAD>
<BODY>
<H1>Log viewing options</H1>
<FORM method="get">
<TABLE>
<TR><TH>Log Type:</TH><TD>Error<INPUT type="radio" value="error" name="log">
&nbsp;&nbsp;Access<INPUT type="radio" value="access" name="log"></TD></TR>
<TR><TH>Lines to show:</TH><TD><INPUT type="text" size="5" name="lines" value="$default_lines"></TD></TR>
<TR><TH>&nbsp</TH><TD><INPUT type="submit" value="Show Log"></TD></TR>
</TABLE>
</FORM>
EOF

# Get values from the query string (not many so we'll not use post)
foreach $pair (split(/&/, $ENV{'QUERY_STRING'})){
$pair =~ tr/+/ /;
($name, $value) = split(/=/, $pair);
$name =~ s/%(..)/pack("C", hex($1))/eg;
$value =~ s/%(..)/pack("C", hex($1))/eg;
$in{$name} = $value;
}
# If specific options were not given then use the defaults
if(!$in{'log'} && ($in{'log'} ne "access" || $in{log} ne "error")){$in{'log'} = $default_log}
if(!$in{'lines'}){$in{'lines'} = $default_lines}
&showlog;

print "</BODY></HTML>";

sub showlog{
print "<H1>Viewing $in{'log'} log</H1><TABLE border=1 width=90%>";
if($in{'log'} eq "access"){
open(LOG, "$access_log");
@line = <LOG>;
close(LOG);
$num = @line;
for($done=0;$done<$in{'lines'};$done++){
$num--;
($start, $request, $response, $ref, $other, $browser, $end) = split (/\"/, $line[$num]);
($browser, $end) = split (/ \(/, $browser);
($client, $other) = split (/\s/, $line[$num]);
($start, $end) = split (/\- /, $line[$num]);
($user, $bad) = split (/ \-/, $end);
($start, $end) = split (/\[/, $line[$num]);
($time, $line[$num]) = split (/\]/, $end);
if($response =~ /^ 200/){$resp = "OK!"}
if($response =~ /^ 500/){$resp = "Server error"}
if($response =~ /^ 404/){$resp = "Page not found"}
if($response =~ /^ 403/){$resp = "Authorisation required"}
if($response =~ /^ 401/){$resp = "Forbidden"}
if($response =~ /^ 400/){$resp = "Bad request"}
if($tm ne $time){print "<TR><TD colspan=4><HR></TD></TR>"}
print "<TR><TD width=35%><B>At:</B> $time<BR><B>User:</B> $client $user<BR><B>Browser:</B> $browser</TD>";
print "<TD width=65%><B>From page:</B> $ref<BR><B>Request:</B> $request<BR><B>Response:</B> $resp ($response)</TD></TR>";
$tm=$time;
}
}
elsif($in{'log'} eq "error"){
open(LOG, "$error_log");
@line = <LOG>;
close(LOG);
$num = @line;
for($done=0;$done<$in{'lines'};$done++){
$num--;
if($line[$num] =~ /^\[.*:..:.*\]/){
$line[$num] =~ tr/\[/ /;
($time, $typ, $cli, $info) = split (/\]/, $line[$num]);
}
else{$info=$line[$num]}
if($line[$num] =~ /\(2\)/){
$cli = "";
$info=$cli;
}
if($tm ne $time){print "<TR><TD colspan=4><HR></TD></TR>"}
print "<TR><TD width=100>$time</TD><TD>$typ</TD><TD width=50>$cli</TD><TD>$info</TD></TR>";
$tm=$time;
}
}
print "</TABLE>";
}

exit;
During which I almost completly rewrote a script then deleted half of it to find that for some reason my editor wouldn't let me undo. The I rewrote that stuff again, added bits - decided how to display the access log and tested it.

Oh I almost forgot the part where I tried to be clever and sent my server running 2 permenant loops (don't worry I fixed that :D)

Anyway I think I'll stop for the night before I delete anything else or accidentally advise soemone to run "rm -rf /"

Let me know how you get on with it - there are now some config options at the top that you may want to set first. if you have any problems or think I made a boo boo then I'll see what I can do.
 
Old 04-06-2003, 06:40 PM   #12
MasterC
Guru
 
Registered: Mar 2002
Location: Salt Lake City, UT - USA
Distribution: Gentoo ; LFS ; Kubuntu
Posts: 12,612

Rep: Reputation: 64
Wow! That works great! Thanks for all that work, really appreciate it

Cool
 
Old 06-02-2009, 01:22 PM   #13
ersimpson75
LQ Newbie
 
Registered: Jan 2009
Posts: 2

Rep: Reputation: 0
Variable for changing log name

Hi David,
I tried the first script and it works nicely!
What I would like to know is how to change it so that it will look for a log file that changes the name from day to day.
I tried several wild card substitutions but with no success.
Thanks, Ed Simpson
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
apache access_log questions shilo Linux - Software 9 06-07-2004 02:40 PM
apache access_log woes thew00t Linux - Software 1 02-23-2004 07:26 AM
apache access_log permissions mirage_3d Linux - Networking 2 12-06-2003 08:54 PM
Apache access_log question WorldBuilder Linux - Networking 7 11-01-2003 06:05 PM
apache access_log to printer plisken Linux - Software 4 02-11-2003 05:54 PM


All times are GMT -5. The time now is 06:20 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration