Hi all,

I am a sysadmin, but I mostly have experience in Windows.
Since Linux is much more attractive to businesses I'm now learning the ropes.

At my workplace we use Ubuntu 14.04 (we have around 7/8 linux servers), and we have a set of cron jobs that are executed everyday around midnight (01:00 or 01:30), but one machine in particular executes the cron jobs way too late (it begins around 07:30).

I already searched in dmesg, an in /var/log/syslog, but I'm not getting any wiser. I also Googled the problem, and various sites tell me that the timezone may be set-up wrong, but that's not the case.

Here I'll give an example of the output of the "date" command.
The first output is from the server in question with the cron problem:

root@caccini:/var/log# date
Fri Sep 18 14:46:54 CEST 2015

and a few moments later:

phouwen@encina:~$ date
Fri Sep 18 14:47:19 CEST 2015

Can anyone assist me or point me in the right direction?

(I also found that maybe if I ran "date -u", it would give a diffirent time, so here it goes:

root@caccini:/etc# date -u
Fri Sep 18 12:54:58 UTC 2015
phouwen@encina:~$ date -u
Fri Sep 18 12:55:03 UTC 2015
)


Thanks!

A couple of things offered for consideration:

1) are you editing the crontab file that is actually being used by cron, and did you restart cron after editing crontab to make it read the changes ?
2) are you aware that: if a user was not allowed to execute jobs when their crontab was last edited (ie. user was not listed in cron.allow),
then the user needs to be added to cron.allow AND THEN crontab needs to be re-edited to allow that user's jobs to execute, and CRON needs to be restarted to pickup the changes.
3) do you have a process running on the ssytem around that time, that is actually blocking the scrip from executing ?

I suggest that you write a quick script that dumps the date/time, and add that job to crontab, then restart cron (to pick up the job changes).
Use that approach to ensure that the job is starting at the correct system time. (Definitive evidence).
Even consider adding a dup of the active processes to that script.

If you have a mis-match between the crontab scheduled job time and the execution of the test job then you have your answer.

C.

If you use the approved way of editing your crontab i.e. by using the crontab command, there should be no need to restart anything.
crontab -l from the appropriate user will display your settings - perhaps you mistyped a setting?

date gives your local time, including DST.

date -u gives the UTC time.

JeremyBoden

Eeek!

(Smiles) Point taken. But I am a purist and prefer to edit crontab directly.

C.

Hi all, thanks for your quick reply.

Executing "crontab -l" gives me the following:

root@caccini:/etc# crontab -l
no crontab for root

While "nano /etc/crontab" gives me:

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user command
17 * * * * root cd / && run-parts --report /etc/cron.hourly
01 0 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 4 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 5 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
#

Any suggestions?

crontab -l lists the cron jobs of the active user.

the /etc/crontab files are system wide cron jobs. Mostly used by services. Also logrotate run through the crontab settings (should be /etc/cron.daily).
Check out the directories given and see which cronjobs are there. Guess /etc/cron.daily is where yours are hiding. Coming from the time you wrote in your first post.
As your settings for cron.daily is scheduled for the first hour of the day (01 0 * * *) you need to check why some jobs are run at 1:30 (also taken from your first post). Either the crontab was edited or there is a sleep 1800 at the start of the scripts to have it sleep for half a hour and then go to work. Maybe someone used this method but did the math wrong? grep "sleep" /etc/cron.* for a quick hint.

Can you post the time of the systems that are running correctly. Just to be sure that the time is good on all systems. Maybe use ntp on all machines?

What user did you create the crontab for?
Try (where username is the user who you specified the crontab for).

I've never tried to get root to run crontab jobs via the crontab mechanism - its a bit insecure isn't it?

However, root will run the scripts in /etc/cron.daily on a daily basis.
Similarly for hourly(?), weekly & monthly - it isn't usually time-critical as to when a root job is run.

Grep "sleep" /etc/cron.* did not give me any useful information:

root@caccini:/etc# grep "sleep" cron.*
grep: cron.d: Is a directory
grep: cron.daily: Is a directory
grep: cron.hourly: Is a directory
grep: cron.monthly: Is a directory
grep: cron.weekly: Is a directory
root@caccini:/etc#

I can't seem to figure out why. Also, I'm gonna edit the title of this post seeing as it's not only that one cron job that doesn't start at specified time, but all of them. (although there are some differences in how much too late they run.

And Jeremy, the specified command didn't give any valuable output:

root@caccini:~# crontab -u root -l
no crontab for root


---------- Post added 09-28-15 at 01:42 PM ----------

I can't seem to find an option which enables me to change the title of the thread. Perhaps I'm overlooking something?

Then it sounds like root doesn't have a crontab. How exactly did you add this command to the cron?

I don't see why it would be. Could you share why you think it would be a problem?

The crontab implementation either runs as a non-root user that is able to gain access to run as root
or crontab runs as a root daemon that can safely start non-root user applications.

Either way, running arbitrary commands/scripts by a process that switches its effective user name seems a little bit fraught?

BTW says, in part...

Actually I didn't. It was already setup when I started working at the company. How can root have no crontab while in /etc/crontab every cron job is run by root?

I re-ran the command a little different now, output is below:

root@caccini:/etc# grep "sleep" cron.*/*
cron.daily/apt:# sleep for a random interval of time (default 30min)
cron.daily/apt:random_sleep()
cron.daily/apt: debug_echo "sleeping for $TIME seconds"
cron.daily/apt: sleep $TIME
cron.daily/apt:# sleep random amount of time to avoid hitting the
cron.daily/apt:random_sleep

After looking at the code of apt a little more I see:

# sleep for a random interval of time (default 30min)
# (some code taken from cron-apt, thanks)
random_sleep()
{
RandomSleep=1800
eval $(apt-config shell RandomSleep APT::Periodic::RandomSleep)
if [ $RandomSleep -eq 0 ]; then
return
fi

Could this be the source of the problems?

No.

You have a cron job (called apt) that appears to be trying to run roughly every 30 minutes.
Many cron jobs can run simultaneously.