LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 04-08-2013, 09:12 AM   #1
bkendall
LQ Newbie
 
Registered: Jul 2012
Posts: 4

Rep: Reputation: Disabled
Archiving Logs on a Central Syslog Server


Hey guys, I have a question regarding log management on my central syslog server.

My server is running syslog-ng and collects syslog messages from a few firewalls. The logs themselves are stored in /var/syslog with the following format:

/var/syslog/YYYY-MM-DD/device_name/YYYY-MM-DD-device_name-HH.log

A new log is created for every hour (the HH in the filename). This makes it easy to find historical logs by date, time, and device. Due to the large nature of these files (a single file for an hour for one device may be close to 1 GB) I need to make sure they are compressed until their retention period is reached.

The ultimate goal is to compress the previous day's logs into a single archive and leave them at their original location. So for Device_One on April 1, 2013, it would be this:

/var/syslog/2013-04-01/device_one/2013-04-01/2013-04-01-device_one-archive.zip

After the retention period is reached, the logs and their corresponding directory structure need to be removed.

I was looking at logrotate but I'm not sure if it will meet this exact scenario.

How would you suggest accomplishing this?

Thanks!
 
Old 04-08-2013, 08:42 PM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,348

Rep: Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749
I would expect that logrotate should be able to handle that.
 
Old 04-09-2013, 09:00 AM   #3
bkendall
LQ Newbie
 
Registered: Jul 2012
Posts: 4

Original Poster
Rep: Reputation: Disabled
Thanks for the reply. I ended up just running the following script via cron:

Code:
#!/bin/bash
# Compress old logs
# Version 1.0 2013-04-08

# Define variables
yesterday=`date --date='1 day ago' +%Y-%m-%d`
#lastmonth=`date --date='1 month ago' +%Y-%m`
lastthreemonth=`date --date='3 months ago' +%Y-%m-%d`
logdir="/var/syslog/"

# Switch to previous day logs and compress
cd $logdir$yesterday
gzip -r ./*

# Remove directory older than 90 days from now
cd $logdir
rm -rf $lastthreemonth

exit
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
syslog to central server and store logs in separate host directories steve.goldner Linux - Enterprise 11 04-07-2012 02:26 PM
Syslog-ng, Central log server, trying to split logs out using filters helptonewbie Linux - Newbie 2 01-23-2009 08:55 AM
Central Log Server using syslog-ng sakthi.s Linux - Security 3 12-26-2007 05:04 PM
Syslog-ng central logserver is dropping logs humbletech99 Linux - Networking 2 06-22-2006 03:09 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 11:48 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration