LinuxQuestions.org - Archiving Logs on a Central Syslog Server

- Linux - Server (https://www.linuxquestions.org/questions/linux-server-73/)

- - Archiving Logs on a Central Syslog Server (https://www.linuxquestions.org/questions/linux-server-73/archiving-logs-on-a-central-syslog-server-4175457349/)

Archiving Logs on a Central Syslog Server

Hey guys, I have a question regarding log management on my central syslog server.

My server is running syslog-ng and collects syslog messages from a few firewalls. The logs themselves are stored in /var/syslog with the following format:

/var/syslog/YYYY-MM-DD/device_name/YYYY-MM-DD-device_name-HH.log

A new log is created for every hour (the HH in the filename). This makes it easy to find historical logs by date, time, and device. Due to the large nature of these files (a single file for an hour for one device may be close to 1 GB) I need to make sure they are compressed until their retention period is reached.

The ultimate goal is to compress the previous day's logs into a single archive and leave them at their original location. So for Device_One on April 1, 2013, it would be this:

/var/syslog/2013-04-01/device_one/2013-04-01/2013-04-01-device_one-archive.zip

After the retention period is reached, the logs and their corresponding directory structure need to be removed.

I was looking at logrotate but I'm not sure if it will meet this exact scenario.

How would you suggest accomplishing this?

Thanks!

I would expect that logrotate should be able to handle that.

Thanks for the reply. I ended up just running the following script via cron:

Code:

#!/bin/bash

# Compress old logs

# Version 1.0 2013-04-08



# Define variables

yesterday=`date --date='1 day ago' +%Y-%m-%d`

#lastmonth=`date --date='1 month ago' +%Y-%m`

lastthreemonth=`date --date='3 months ago' +%Y-%m-%d`

logdir="/var/syslog/"



# Switch to previous day logs and compress

cd $logdir$yesterday

gzip -r ./*



# Remove directory older than 90 days from now

cd $logdir

rm -rf $lastthreemonth



exit