LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 09-27-2006, 02:05 PM   #1
Kevad
LQ Newbie
 
Registered: Sep 2006
Posts: 3

Rep: Reputation: 0
Limit log file size


Hello,

I'm looking for a way to limit the file size of some of our log files. For instance, /var/log/messages might grow to over a few hundred megs on some of our boxes used for testing. What I'd like to do would be to limit the size of the /var/log/messages file to something like 50MB, and when additional data comes in, older data in the file is deleted.

Anyone done something similar?

Thanks,
Dave
 
Old 09-27-2006, 02:09 PM   #2
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,414

Rep: Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966
you wouldn't delete the data, you'd rotate the logs. you should have logrotate already runnign and if you have too much going into messages, you can change the rotation logic to rotate on file size instead of age easily enough.
 
Old 09-27-2006, 02:17 PM   #3
Kevad
LQ Newbie
 
Registered: Sep 2006
Posts: 3

Original Poster
Rep: Reputation: 0
Ahh..that makes sense. Didn't know you could change logrotate to use filesize instead of just at a predetermined size.

So, let me explain a little bit further...just to see if you guys have any better ideas.

We have a large number of servers that iSCSI boot to a NetApp filer (~250). So, all of these client's "local disks" are actually located on the filer. If all the log rotations kick off at the same time each night, it can cause the filer to become very busy with disk I/O for all 250 servers. I'm looking to eliminate this by either staggering the log rotation times for each of the servers, or possibly rotating based on the size.
 
Old 09-27-2006, 03:16 PM   #4
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,414

Rep: Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966
well yes, that's doable, there's a simple example on the logrotate manpage
Code:
       "/var/log/httpd/access.log" /var/log/httpd/error.log {
           rotate 5
           mail www@my.org
           size=100k
           sharedscripts
           postrotate
                                     /sbin/killall -HUP httpd
           endscript
       }
but be careful about avoiding problems you're going to crash into in the end. if you only have a toy SAN, then you've gotta do it for now though ehh?
 
Old 09-27-2006, 04:20 PM   #5
trickykid
Guru
 
Registered: Jan 2001
Posts: 24,133

Rep: Reputation: 199Reputation: 199
Quote:
Originally Posted by Kevad
If all the log rotations kick off at the same time each night, it can cause the filer to become very busy with disk I/O for all 250 servers. I'm looking to eliminate this by either staggering the log rotation times for each of the servers, or possibly rotating based on the size.
Either do this by size or alter the time logrotate runs, but if you have a SAN that is affected by rotating logs that don't exceed 50 megs in size each and only 250 of these machines, you seriously have something wrong with your SAN and it's I/O. I've dealt with SAN's with 20 to 30 servers attached to it while production continued to load data into our Databases using this SAN without problems and reaching/maintaining 100meg or so an hour for each database.

NetApp filers should have no problem handling log file rotations with 250 servers, if they do, I'd be calling NetApp to find out what the problem is.
 
Old 09-27-2006, 05:27 PM   #6
Kevad
LQ Newbie
 
Registered: Sep 2006
Posts: 3

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by trickykid
Either do this by size or alter the time logrotate runs, but if you have a SAN that is affected by rotating logs that don't exceed 50 megs in size each and only 250 of these machines, you seriously have something wrong with your SAN and it's I/O. I've dealt with SAN's with 20 to 30 servers attached to it while production continued to load data into our Databases using this SAN without problems and reaching/maintaining 100meg or so an hour for each database.

NetApp filers should have no problem handling log file rotations with 250 servers, if they do, I'd be calling NetApp to find out what the problem is.
Those were actually just some arbitrary sizes I pulled out of a hat . Our log files are actually *very* large (>1GB/day) due to the testing being done on these boxes. (Lots of output)

Thanks for the suggestions everyone.
 
Old 09-28-2006, 03:01 AM   #7
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,414

Rep: Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966Reputation: 1966
there is also the issue of where the logs go in the first place. again not sure exactly what the issues are but can you not work on the syslog itself and filter things more effectively if so much stuff is going into messages?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
any ideas to reduce log file size or make log file size managed? George2 Programming 2 08-13-2006 06:55 AM
File Size limit? compu73rg33k Linux - Software 1 02-27-2005 10:11 PM
limit file size ust Linux - General 2 02-24-2005 07:33 AM
Put a limit on the size of sys.log? Moloko Debian 2 08-02-2004 06:01 PM
file size limit jaysan Slackware 5 07-12-2002 03:43 AM


All times are GMT -5. The time now is 08:29 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration