I find that after running for a few hours (Red Hat 7.3), my system
runs incredibly slow. When I do a 'top', I see that there are between
one and three instances of 'logrotate' that are using up almost all available
CPU cycles.

Basically, I'm not that interested in the logs unless there is a system problem. So how do I keep these processes from using all my CPU without letting my logs get too big? I'm primarily interested in maximizing my system's overall performance.

Any ideas?

Hmmmm is logrotate a system service? You might be able to just disable it from starting up?

#Garry

See if you have a /etc/logrotate.conf file.If you run Redhat there should be one.It should contain comments on how to configure it.
It may be used for a cronjob too.
lynch

If you want to disable logrotate then move logrotate out of /etc/cron.daily - if you just want to change the process priority use nice, so for instance nice 19 /usr/sbin/logrotate /etc/logrotate.conf, so you can change your /etc/cron.daily/logrotate-script to match something like:

I wouldn't recommend on disabling logrotate. Better thing is to find out *why* it keeps hogging CPU. If you find it's running again, the visual way to see what's running for newbies would be to do a "ps ax -eo pid,args --forest" (or use "top") and watch the logrotate process and any of it's children. If you find out is's a process called "run-parts" or "awk" then report the *exact* full lines back here.

It is part of run-parts. Here's the output from
"ps ax -eo pid,args --forest"...

2451 crond
7082 \_ CROND
7084 \_ /bin/bash /usr/bin/run-parts /etc/cron.daily
7251 \_ /bin/sh /etc/cron.daily/logrotate
7253 | \_ /usr/sbin/logrotate /etc/logrotate.conf
7252 \_ awk -v progname=/etc/cron.daily/logrotate progname {????? print progname ":\n"????? progname="";????



Here's the upper portion of 'top'...
12:11pm up 18:15, 7 users, load average: 2.35, 2.57, 2.59
104 processes: 96 sleeping, 6 running, 2 zombie, 0 stopped
CPU states: 91.7% user, 7.6% system, 0.5% nice, 0.0% idle
Mem: 255884K av, 250260K used, 5624K free, 0K shrd, 30076K buff
Swap: 522104K av, 67276K used, 454828K free 39268K cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
7253 root 25 0 62652 60M 616 R 75.8 24.3 357:46 logrotate
2576 root 16 0 112M 20M 2300 S 21.7 8.1 217:11 X
9001 pdibona 16 0 1024 1024 800 R 1.3 0.4 0:00 top


Is this okay?

Excellent, actually output from ps forest was enough to see it's the damn run-parts' awk script keeps hanging. Maybe we should fire off a bugzilla request at RH, I've been having this starting at say, RH 5.2? :-]

What you can do is add a simple script to your crontab, since run-parts only get used during cron activation for a whole dir like /etc/cron.daily in the case of logrotate.

Save this script somewhere cron can reach/execute it but outside the /etc/cron.* directories, because we don't want it run inadvertedly. I use a dir called /etc/cron.scripts.

#!/bin/sh
# Hand message to syslog:
awklog() { /usr/bin/logger -i -t logrotate.post "${MSG}"; }
# Lame way to make sure we only have logrotate-awk pids:
awkpid=$(/bin/ps x | grep -v grep | grep "awk -v progname=/etc/cron.daily/logrotate" | awk '{print $1}')
# Pid of logrotate
lgrpid=$(/sbin/pidof /usr/sbin/logrotate)
# If logrotate ain't running (it's finished),
# but if the remaining awk script still works,
# then kill the awk script.
if [ "$lgrpid" = "" -a "$awkpid" != "" ]; then kill -s TERM "$awkpid"; MSG="Killed awk pid $awkpid"; awklog; fi
exit 0

Now add a line in your /etc/crontab, after /etc/cron.daily get's run (w/o quotes):
"05,10,15 20 * * * root /etc/cron.scripts/logrotate.post"

This will run the script at 5, 10 and 15 minutes past 8PM and should do. Logrotate may take between 1 and 15 minutes of time depending on how much processing (for me, a lot) you add to logrotates pre/post scripts.
When run-parts > logrotate > awk get's killed run-parts will start with the next job in queue.

If this doesn't work for you let me know, ok.

Great tip,unSpawn!
I saved that one to my RHFix dir for future use.
lynch

If you have the default "everything" installation of RedHat 7.3, odds are good that this problem is being caused by a program called mailman. With the default installation Mailman creates thousands of garbage log files in /var/log/mailman, which chokes logrotate.

Mailman is a mail usergroup program, which you probably have never used directly. If you don't use it, than the easiest way to fix your probably is to merely remove Mailman by typing:

rpm -e mailman

which will remove the rpm installation, and then:

rm -rf /var/log/mailman

which will remove the directory of mailman log files.

Once Mailman is gone, your logrotate problems will most likely vanish too.

Don't mess with the run-parts awk script. Your problem is most likely caused by the sheer number of mailman log files. run-parts isn't broken, it's actually doing its job (hence no error messages, and the high cpu usage); it's just taking a looooooooong time to do it.

You're not kidding. /var/log/mailman has over 12 Mb of logs. It took 10 minutes just to do a 'du' on that directory. I'll try your suggestion and post my results in a few days.

But in the mean time, what exactly does 'mailman' do?

While intalling sysklodg on my system, the INSTALL file had some warnings... maybe you could check..
apparentrly there are two ways of runnin logrotate.. through rc.* modules, and with the init scripts
it warns then that sometimes, if not configures correclt, and using the init method, logrotate may run more than once causin crashes

Mailman replaces the majordomo mailing list app in some newer distros,like SuSE 8.1.
lynch

Thanks, Azureash. Your suggestion about 'mailman' was right. Now logrotate still runs, and quits in a reasonable amount of time without using all of my CPU.

I appreciate your help.

Hi,

Logrotate is eating away my cpu aswell. But I don't have mailman. However, looking at my /etc/logrotate.conf, I noticed something I don't know why it is there :

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own lastlog or wtmp -- we'll rotate them here
/var/log/wtmp {
monthly
create 0664 root utmp
rotate 1
}

it is the wtmp I looked for. File size is about 35 Mb, and editing it results in a sort of encrypted text.

In logrotate.d is not that much:

-rw-r--r-- 1 root root 122 Feb 3 2000 cron
-rw-r--r-- 1 root root 78 Jun 23 2000 ftpd
-rw-r--r-- 1 root root 63 Dec 4 2000 hylafax
-rw-r--r-- 1 root root 262 Aug 27 11:29 linuxconf
-rw-r--r-- 1 root root 261 Aug 27 11:29 linuxconf~
-rw-r--r-- 1 root root 152 Nov 10 2000 named
-rw-r--r-- 1 root root 227 Feb 25 2000 samba
-rw-r--r-- 1 root root 410 Feb 3 2000 syslog
-rw-r--r-- 1 root root 544 Mar 7 2000 uucp
-rw-rw-r-- 1 root root 150 Nov 17 22:46 vhostlogs

Going through all of the files doesn't give me that much. Vhostlogs is for rotating all logs of several hundred vdomains, but I disabled them a while ago for testing purpose :

#/etc/httpd/logs/*log {
#weekly
#rotate 4
#}

#/etc/httpd/logs/wt/*access* {
#weekly
#rotate 4
#}

#/etc/httpd/logs/wt/*error* {
#weekly
#rotate 4
#}

So, it beats me :-( Allthough there is not much to rotate, this is how it looks like with top:

121 processes: 118 sleeping, 3 running, 0 zombie, 0 stopped
CPU states: 97.8% user, 1.9% system, 0.0% nice, 0.1% idle
Mem: 62080K av, 60496K used, 1584K free, 13456K shrd, 1288K buff
Swap: 265032K av, 96452K used, 168580K free 12168K cached

PID USER PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND
8197 root 12 0 26708 26M 160 R 0 97.3 42.9 452:58 logrotate
10522 root 4 0 900 900 668 R 0 1.7 1.4 0:00 top
611 root 0 0 524 320 232 S 0 0.3 0.5 4:38 dlist

Yes, 97,3% !!! Customers complain about beeing on a slow server. Grmpffff... Is there a way to find out actual what logrotate is doing at a specific time?

Regards,

Satman