Crond eating up 100% CPU constantly

gabriot · 06-20-2012, 04:15 PM

Can't seem to figure this one out, it's eating up 200% CPU (2 processors). here's top:

Code:

                                                                                                                                           
 7478 root      25   0 10212 2696 1748 R   50  0.1 321003:36 crond                                                                                                                                             
15488 root      25   0 10212 2692 1744 R   50  0.1  25888:51 crond                                                                                                                                             
20104 root      25   0 10212 2696 1748 R   50  0.1 351411:00 crond                                                                                                                                             
21474 root      25   0 10212 2696 1748 R   50  0.1  10132:38 crond

Here's: /usr/sbin/lsof -p `ps aux | grep crond | grep -v grep | awk '{print $2'}`

Code:

lsof: status error on 15488: No such file or directory
lsof: status error on 20104: No such file or directory
lsof: status error on 21474: No such file or directory
lsof: status error on 24189: No such file or directory
lsof 4.78
 latest revision: ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/
 latest FAQ: ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/FAQ
 latest man page: ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/lsof_man
 usage: [-?abhlnNoOPRstUvVX] [+|-c c] [+|-d s] [+D D] [+|-f]
 [-F [f]] [-g [s]] [-i [i]] [+|-L [l]] [+m [m]] [+|-M] [-o [o]]
 [-p s] [+|-r [t]] [-S [t]] [-T [t]] [-u s] [+|-w] [-x [fl]] [-Z [Z]] [--] [names]
Use the ``-h'' option to get more help information.

Here's tail -30 /var/log/cron

Code:

Jun 20 13:10:05 ecampweb01 crond[566]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 13:20:04 ecampweb01 crond[1195]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 13:30:02 ecampweb01 crond[1801]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 13:40:01 ecampweb01 crond[2429]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 13:50:04 ecampweb01 crond[3040]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 14:00:02 ecampweb01 crond[3756]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 14:01:04 ecampweb01 crond[3808]: (root) CMD (run-parts /etc/cron.hourly)
Jun 20 14:10:05 ecampweb01 crond[4354]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 14:20:03 ecampweb01 crond[4953]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 14:30:06 ecampweb01 crond[5647]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 14:40:04 ecampweb01 crond[6240]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 14:50:02 ecampweb01 crond[6875]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 15:00:03 ecampweb01 crond[7630]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 15:01:05 ecampweb01 crond[7680]: (root) CMD (run-parts /etc/cron.hourly)
Jun 20 15:10:02 ecampweb01 crond[8118]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 15:20:01 ecampweb01 crond[8869]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 15:30:05 ecampweb01 crond[9465]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 15:40:05 ecampweb01 crond[10073]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 15:50:05 ecampweb01 crond[10765]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 16:00:02 ecampweb01 crond[11370]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 16:01:02 ecampweb01 crond[11420]: (root) CMD (run-parts /etc/cron.hourly)
Jun 20 16:10:04 ecampweb01 crond[11978]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 16:20:01 ecampweb01 crond[12581]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 16:30:05 ecampweb01 crond[13179]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 16:40:04 ecampweb01 crond[13797]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 16:50:02 ecampweb01 crond[14486]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 17:00:05 ecampweb01 crond[15088]: (root) CMD (/usr/lib/sa/sa1 1 1)
Jun 20 17:01:04 ecampweb01 crond[15138]: (root) CMD (run-parts /etc/cron.hourly)
Jun 20 17:03:00 ecampweb01 crontab[15274]: (root) LIST (root)
Jun 20 17:10:03 ecampweb01 crond[15753]: (root) CMD (/usr/lib/sa/sa1 1 1)

Let me know if I can provide any further help, very frustrating trying to figure this one out

szboardstretcher · 06-20-2012, 04:25 PM

What is in your crontab for root?

gabriot · 06-20-2012, 04:29 PM

15 3 * * * /usr/bin/updatedb
12 19 * * * /etc/webmin/cron/tempdelete.pl
#*/2 * * * * /var/www/utilfo >/dev/null 2>&1

gabriot · 06-21-2012, 09:41 AM

maccas17 · 06-21-2012, 10:46 AM

Have you tried restarting cron:

Code:

/etc/init.d/crond stop
/etc/init.d/crond start

One of your 3 entries is commented out, the other is updatedb, which I doubt is the cause.
What is tempdelete.pl doing? (presumably its a Perl script)
I would look at what this does and possibly comment it out.

Hope this helps.

gabriot · 06-22-2012, 10:47 AM

Here's the cat of tempdelete.pl

Code:

#!/usr/bin/perl
open(CONF, "/etc/webmin/miniserv.conf");
while(<CONF>) {
        $root = $1 if (/^root=(.*)/);
        }
close(CONF);
$ENV{'WEBMIN_CONFIG'} = "/etc/webmin";
$ENV{'WEBMIN_VAR'} = "/var/webmin";
chdir("$root/cron");
exec("$root/cron/tempdelete.pl", @ARGV) || die "Failed to run $root/cron/tempdelete.pl : $!";

I don't really know perl unfortunately

gabriot · 06-22-2012, 11:02 AM

Also I restarted crond just now and the CPU is at 0% for the time being, I will let you know if it spikes back!

maccas17 · 06-22-2012, 11:10 AM

Looks like it sets up the webmin environment and then runs the script from a set location with the cmd-line arguments. I don't think this is causing your issue though. I would suggest you look in all your /etc/cron.hourly .daily etc. directories and see what scripts are in there - maybe one of these is your culprit?

gabriot · 06-22-2012, 11:15 AM

I didn't see anything in the hourly directly, here's the ls -l of daily:

Code:

-rwxr-xr-x 1 root root 379 Mar 28  2007 0anacron
lrwxrwxrwx 1 root root  39 Dec  6  2007 0logwatch -> /usr/share/logwatch/scripts/logwatch.pl
-rwxr-xr-x 1 root root 180 Dec  1  2007 logrotate
-rwxr-xr-x 1 root root 418 Jan  6  2007 makewhatis.cron
-rwxr-xr-x 1 root root 137 Mar 14  2007 mlocate.cron
-rwxr-xr-x 1 root root 296 Apr  3 12:52 rpm

Thanks a lot for your help so far, and my apologies for some of these questions I know they're probably pretty elementary.

Edit: Also as a side question that maybe you can answer for me - is there much danger in stopping and starting cron? I have 3 other webservers that have the same problem, and I don't want to potentially screw anything up - but if there's no harm in it I'm thinking of restarting on those other ones too considering how much this helped CPU to do so.