Logging via stdin

bubblenut · 02-12-2007, 06:02 AM

Is there a logging / log rotation tool which allows me to pipe output into it and it will log it in a particular way? Basically what I'm looging for would have an interface something like the following

Code:

$ ./run_my_tool | stdin2log --path=/path/to/logs --rotate=10 --compress

or maybe

Code:

$ ./run_my_tool | stdin2log --file=/path/to/logs/number%dfile --rotate=10 --compress=bz2

I have looked at logrotate and it doesn't really handle logging output from standard out if the programs you want to log can be run more than once a day. I have also thought about using the append operator (>>) but I want to keep the logs for each individual execution seperate. I have had a good look on Sourceforge and Freshmeat but cannot find anything like it. I can't believe no one has thought of this before, I can only think there is a reason why it is a bad idea. If no one knows of a tool like this and people think it's a good idea I'll happily implement it myself, I just don't want to go re-inventing the wheel.

unSpawn · 02-12-2007, 09:12 AM

Code:

#!/bin/bash
# Purpose: Catch stdin and archive when done.
# Deps: Bash, GNU utils
# set -xe
umask 027; progn=${0//*\//}; progn=${progn%.sh}
log_name="${progn}"; log_base="/var/log"
log_stamp=`/bin/date '+%Y%m%d_%H%M%S%N' 2>/dev/null`
log_file="${log_base}/${log_name}.${log_stamp}"

help() { # Echo some help lines here.
 echo "${progn}: -c command and arguments -l logfile"; exit 1; }

loglines() { # Formatting and logging
 while read line; do line="$(/bin/date "+%b %d %H:%M:%S:%N" 2>/dev/null) \
 $(hostname -s 2>/dev/null) ${progn} [$$]: ${line:="EMPTY LINE"}"
 echo "${line}" 2>&1>> "${log_file}"; done
 [ $PIPESTATUS -eq 1 ] && gzip "${log_file}" 2>/dev/null; }

[ $# -eq 0 ] && help; [ $# -gt 64 ] && help
[ "${1:0:1}" != "-" -o "${1:0:2}" = "--" ] && help
while getopts c:l:h OPT; do case "${OPT}" in
  c) [ -n "${OPTARG}" ] && cmdline="${OPTARG}";;
  l) log_file="${OPTARG}";;
  *|h) help;; esac
done; exec $cmdline 2>&1 | loglines
exit 0

Execute as

Code:

stdin2log -c "seq 1 100" -l /var/log/name.log

YMMV(VM), as usual ;-p

bubblenut · 02-12-2007, 09:50 AM

Very cool script! It's not quite what I'm looking for but it certainly shown me that implementing what I need will be a hell of a lot simpler than I'd imagined. Thanks.

unSpawn · 02-12-2007, 10:22 AM

Post your script when you're done?

bubblenut · 02-12-2007, 12:00 PM

This is what I've put together, much easier that what I had first envisioned. Thanks for your help unSpawn.

Code:

#!/bin/bash

umask 027;
rotate=10; path='';

# Fail and print the usage
help() {
    if [ $# -gt 0 ]
    then
        echo "error [$1]";
    fi
    echo "stdin2log -r [number of times to rotate the log]"\
      " -p [path to log directory or filename template]";
    exit 1;
}

# Build the filename
getFilename() {
    if [ $1 = 'file' ]
    then
        echo -n $(printf $2 $3);
    else
        echo -n "${2}log${3}";
    fi;
}

while getopts r:p:h: OPT;
do
    case "${OPT}" in
        r) [ ${OPTARG} -gt 0 ] && rotate=${OPTARG};;
        p) path="${OPTARG}";;
        *|h) help;;
    esac
done;

# Figure out whether it's a directory or a filename template
[ -z $path ] && help "Empty path";
if $(echo $path | grep -q '\/$')
then
    pathtype='dir';
elif $(echo $path | grep -q '%d' )
then
    pathtype='file';
else
    help "Invalid path ${path}";
fi;
# Run through the logs backwards moving them back
for i in $(seq $(($rotate-1)) 1)
do
    log_file=$(getFilename $pathtype $path $i)
    if [ -e $log_file ]
    then
        mv -f $log_file $(getFilename $pathtype $path $(($i+1)))
    fi
done;

# Write all lines from stdin to the first file
log_file=$(getFilename $pathtype $path 1);
touch $log_file;
while read line;
do
    echo "${line}" >> "${log_file}" 2>&1
done;