"Real time" Apache log filtering with Perl
 

Wondering if anyone else here has ever messed around with access log filtering on Apache. I borrowed some perl code from the URL:
scrapped most of the contents and wrote my own body section with logic pertaining to the type of filtering that I need.

The problem that I'm having is that the filter doesn't launch once and then remain resident. It keeps launching, processing several lines of logging as they arrive from apache, and then terminating. It terminates with an exit code of 0 or 1, cause of the code unknown, and the code itself appears in the log file. I know that it's respawning because I added a print() statement at the top of the script that shows when it has been freshly instantiated, and sure enough, like clockwork the filter restarts every 10 seconds.

My understanding from the rather slim documentation I've found on filters is that the script should be instantiated once and then stay alive for the entire time that the server is running. Is this false? Is there any way to make it stop? The way the filter is invoked is by setting up:

CustomLog "| /myfilter.pl | /path/rotatelogs... yadda yadda"

It just gets inserted into the pipe there, receives log messages on stdin and pops filtered messages back on stdout which are received by the normal logging system.

Well I'd appreciate any tips on direction. My script is required to operate for days at a time in order to build internal records against which to filter. The 10 second intervals won't work be sufficient. Unfortunately I can't post my own script as it is considered intellectual property for the company, but the gist is the same as the code linked above. Mine appends each log line with a new field that I can use a log post-processor to retrieve and display new statistics on.

Thanks,
- SK

PS: if someone could edit the URL above and turn into a real link, that would be helpful. The site's anti-spam policy prevents me from doing it myself. I posted this in the Linux - Software forum originally which was apparently a mistake.

Nevermind, I figured it out: the filter is started and stopped for each Apache virtual server that is spawns and dies which on this server happens relatively frequently. I've switched to an alternative input model so that this is no longer an issue.