LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 06-04-2008, 04:14 AM   #1
Twelveone
LQ Newbie
 
Registered: May 2006
Location: Dunfermline, Scotland
Distribution: Ubuntu 8.04 LTS
Posts: 10

Rep: Reputation: 0
logRotate on a directory


Hi all,

I am working on an application that generates its own log files. These logs are stored under /var/log/<application_name> and under that in unique directories named using the current date, Using a custom application running under cron, all log files more than 24 hours old are zip'd into one file and the originals deleted daily. This leaves /var/log/<application_name> full of compressed files that will continue to grow. After a while this will fill the /var/log/ partition, so I am looking for a tool that can monitor the size of a directory and delete the oldest files if the directory passes a specified size or that can monitor the directory and delete files older than a specified age (e.g. 7 days). Can anyone help?

Many thanks
 
Old 06-04-2008, 04:49 AM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,017
Blog Entries: 54

Rep: Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764Reputation: 2764
Quote:
Originally Posted by Twelveone View Post
I am looking for a tool that can monitor the size of a directory and delete (..) files older than a specified age (e.g. 7 days).
Size is more like 'logrotate', for aging see 'tmpwatch'.
 
Old 06-04-2008, 05:56 AM   #3
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
You could use the find command to locate older files and the delete them.
find /var/log/backups -mtime +2 -daystart -execdir rm '{}' \; 2>/dev/null

However, instead of what you are doing, you could use logrotate to compress the rotated files and use "rotate <n>" or "maxage <age>" to limit the number of backup logs used. Check if you have files in /etc/logrotate.d/. There is even an option to run a post rotate script, which could move the rotated file to /var/logs/backups/ and delete old log backups.

Just wondering, why you would use zip. bzip or gzip seem better choices and you can use tar with the -z or -j options to produce compressed archives.
 
Old 06-04-2008, 07:38 AM   #4
Twelveone
LQ Newbie
 
Registered: May 2006
Location: Dunfermline, Scotland
Distribution: Ubuntu 8.04 LTS
Posts: 10

Original Poster
Rep: Reputation: 0
Smile

Hi,

Thanks for the replies.

unSpawn:
I will need to have a look at tmpwatch and see if it will do what I'm after.

jschiwal:
This is a long story, but I've been brought in to a company test their system for a third party (customer) and I will need to ask the developers why they are not using logRotate to acheive what they want, because they use it to manage a single system log but not for these data logs where they have written a routine themselves to archive the data logs every day. As for the zip :-) I should probably have said compress or archive. The archive routine does as you suggest use tar -z.

Thanks Again
 
Old 06-04-2008, 09:56 AM   #5
Twelveone
LQ Newbie
 
Registered: May 2006
Location: Dunfermline, Scotland
Distribution: Ubuntu 8.04 LTS
Posts: 10

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by jschiwal View Post
You could use the find command to locate older files and the delete them.
find /var/log/backups -mtime +2 -daystart -execdir rm '{}' \; 2>/dev/null
Can you explain what the \ does after the rm '{}' and what the 2>/dev/null does?

I think I understand the rest of the find command but I cannot make it work. I have created a number of files and used touch to change the modification date then run the find command as above but it did nothing. Am I doing something wrong?

Thanks
 
Old 06-04-2008, 07:03 PM   #6
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
The "\;" delineates the end of the -exec argument of the find command. The 2>/dev/null will redirect stderr to /dev/null. You want to do this in a cron job because it doesn't have access to a console.

I haven't really studied logrotate that much but using a post rotate script might be useful for your situation. If you copy the compressed rotated files to a different location (maybe even on a remote share) you can do that in such a scrit and even delete aged files. If the files are copied to a remote computer, it may be more "proper" to let that remote host take care of deleting the aged logs.
 
  


Reply

Tags
find, logrotate, remove


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Logrotate: compress logs in a temp directory gbratig AIX 3 04-12-2008 10:42 PM
logrotate Ronin_tekorei Linux - Newbie 2 02-07-2008 05:44 AM
logrotate sidra Red Hat 10 08-01-2006 01:05 AM
Logrotate - did I get it right? Swakoo Linux - General 1 06-20-2006 05:14 AM
logrotate Rig24 Linux - Newbie 2 07-11-2003 05:08 AM


All times are GMT -5. The time now is 11:38 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration