quick question with awstats and my log file of 270MB

bangsters · 03-09-2010, 12:23 PM

Hi.

I was wondering how I can get this done. I reinstalled my awstats and deleted my awstats data. How do I pull the data from my original access_log which is around 270MB in size.

Any suggestion? I tried updating awstats but it's timing out and I'm getting an internal server error which I assume is from the 270MB access_log size.

Thanks

alunduil · 03-09-2010, 12:36 PM

Have you tried running the update script from the command line and just letting it run as long as necessary?

Regards,

Alunduil

bangsters · 03-09-2010, 12:38 PM

that is what i'm doing now.

it's taking a while though...and i opened another console window and checked at the stats folder where awstats stores the data, still empty.

Also, how can I get awstats to work with logformat=1? Right now I am using logformat=4 because I get an error when I set it as logformat 1.

Thanks

alunduil · 03-09-2010, 12:48 PM

I haven't played with different log formats but the reason the data isn't showing up yet is that it does all of the processing in memory and will write it out when it has finished.

Regards,

Alunduil

bangsters · 03-09-2010, 12:52 PM

ahh.... i'll wait it and update you when process is done.

if anyone can chime it with getting logformat=1 working instead of logformat=4 would be great. from my existing data with my other sites (all logformat=4), and from what i've read online, logformat-4 does not show Browsers, OS, keywords, etc.

thanks again

alunduil · 03-09-2010, 12:56 PM

Do you mean getting a customlog format to get all of that information? I may have posted what you need here: http://www.alunduil.com/2010/03/02/r...ats-on-gentoo/ but if not good luck.

Regards,

Alunduil

bangsters · 03-09-2010, 01:10 PM

This is what I already have i my httpd.conf

ErrorLog logs/error_log
LogLevel crit
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combin$
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
CustomLog logs/access_log combined
ServerSignature off

alunduil · 03-09-2010, 01:36 PM

Just to cover all of our bases did you restart apache after configuring that log style? Also, if the log file was created in the old format I don't think it's possible to convert it to a new style, but all new log entries will have the information you want.

Regards,

Alunduil

bangsters · 03-09-2010, 02:23 PM

actually that is my existing logformat settings in my httpd.conf. i have not changed it. and from the looks of that, i should be able to use logformat=1 right?

thanks

alunduil · 03-09-2010, 02:56 PM

I'm not familiar with the logformat directive but I do know that specifying a log file with the logformat works: CustomLog /var/www/localhost/log/apache/production.log combined

Regards,

Alunduil

bangsters · 03-09-2010, 03:09 PM

hmmm....so i should have the same logformat which is 1 (combined) in awstats based on my post above. that's weird.

also, if I change my logformat, will it break awstats or will it break anything? how should I change my formatting to suit awstats logformat=1 without breaking existing data or beter yet without breaking the box

thanks again

alunduil · 03-09-2010, 03:32 PM

Those are excellent questions. Is the logformat coming from awstats or apache that you're having difficulty with (I've been probably wrongly assuming it's an apache issue)?

Regards,

Alunduil

bangsters · 03-09-2010, 05:41 PM

Actually I'm not so sure to be honest. I assumed it's apache as well, since AWSTATS only have 1 to 4 general options that I have there, but apache logformat has a bunch of syntax. Also, AWSTATS just pulls the apache logs and need to match with apache's format. so for awstats to work on 4 means apache is somehow configured that way despite in httpd saying it's combined (logformat=1).

alunduil · 03-10-2010, 10:54 AM

Code:

# Examples for Apache combined logs (following two examples are equivalent):
# LogFormat = 1
# LogFormat = "%host %other %logname %time1 %methodurl %code %bytesd %refererquot 
quot"
#
# Example for IIS:
# LogFormat = 2
#
LogFormat=1

I stumbled across this in the awstats config I'm using.

Regards,

Alunduil

bangsters · 03-11-2010, 07:04 AM

yes that was what i was concerned with. if i set awstats to logformat=1 it does not work. i have to set it in awstats as logformat=4. however, as from my previous post my apache is configured for combined logging, which i believe is equivalent to logformat=1.