[SOLVED] Hardware clock being forced to UTC time

tux_dude · 10-01-2013, 06:14 PM

Quote:

Originally Posted by Richard Cranium

How about...

Code:

ps -ef | grep ntpd

Yes, I do have ntpd running. It keeps the system time (DATE: from my above output) in sync.

Code:

ps -ef | grep ntpd
root      5613     1  0 Sep30 ?        00:00:08 /usr/sbin/ntpd -g -p /var/run/ntpd.pid

Quote:

Originally Posted by Richard Cranium

...and...

Code:

grep ntp /etc/dhcpcd.conf

...? What do those two commands give?

Code:

grep ntp /etc/dhcpcd.conf
option ntp_servers

tux_dude · 10-01-2013, 06:35 PM

Quote:

Originally Posted by GazL

One of the recent kernels introduced these new options, which sound as if they may be relevant.

CONFIG_RTC_HCTOSYS:
If you say yes here, the system time (wall clock) will be set using the value read from a specified RTC device. This is useful to avoid unnecessary fsck runs at boot time, and to network better.

CONFIG_RTC_SYSTOHC:
If you say yes here, the system time (wall clock) will be stored in the RTC specified by RTC_HCTOSYS_DEVICE approximately every 11 minutes if userspace reports synchronized NTP status.

Also check out the help text on CONFIG_RTC_HCTOSYS_DEVICE:
This device should record time in UTC, since the kernel won't do timezone correction

If you're running your hw_clock on localtime then you probably want to avoid these new RTC options in your kernel config (especially the second one).

P.S. Both appear to be set to Y in config-generic-3.10.12.x64

This is interesting. If I read it correctly, it looks like I'm probably being affected by CONFIG_RTC_SYSTOHC, since this cause the RTC (BIOS) clock to be updated every 11 minutes.

I couldn't find any option to disable this feature without recompiling the kernel. I am not a fan of building my own kernel as it is too much maintenance work for me. What are other who are not affected doing?

volkerdi · 10-01-2013, 07:54 PM

Quote:

Originally Posted by tux_dude

This is interesting. If I read it correctly, it looks like I'm probably being affected by CONFIG_RTC_SYSTOHC, since this cause the RTC (BIOS) clock to be updated every 11 minutes.

I couldn't find any option to disable this feature without recompiling the kernel. I am not a fan of building my own kernel as it is too much maintenance work for me. What are other who are not affected doing?

Well, I'm assuming that GazL has nit the nail on the head as usual, and will remove those options from the 3.10.14 kernels. Test 'em when they go up... shouldn't be long.

wildwizard · 10-02-2013, 02:56 AM

I hope that does fix it Pat as I just got hit by this little kernel addition and if you also run your own DNS as I do dnssec will fall in a heap and it is pretty much a requirement these days.

GazL · 10-02-2013, 03:33 AM

I'd encourage folks to run their hw clock on UTC if at all possible and only use 'localtime' if absolutely necessary. It's clearly the way the system is intended to be used and avoids these sorts of issues.

If you dual-boot with Windows there's a registry entry you can change to make it also treat the hardware clock as UTC which is what I've done here.

allend · 10-02-2013, 05:53 AM

Quote:

CONFIG_RTC_SYSTOHC:
If you say yes here, the system time (wall clock) will be stored in the RTC specified by RTC_HCTOSYS_DEVICE approximately every 11 minutes if userspace reports synchronized NTP status.

I am going to point the finger at this setting. On my system that dual boots to Windows, I have the hardware clock set to local time, but I do not use NTP (I let Windows do the time synchronisation). I have not seen this issue.

chrisretusn · 10-02-2013, 11:58 AM

It does seem to be the issue. I switched to running all of my systems using UTC for the hardware clock a while ago using similar logic as Gazl.

tux_dude · 10-02-2013, 07:59 PM

I wouldn't call it necessary, however I would rather have my hardware clock set to localtime. Since the kernel does not do localtime, then this option should read:

Quote:

CONFIG_RTC_SYSTOHC:
If you say yes here, the system time (wall clock) will be stored in the RTC specified by RTC_HCTOSYS_DEVICE approximately every 11 minutes if the system time (wall clock) is set to UTC and the userspace reports synchronized NTP status.

wildwizard · 10-04-2013, 06:57 AM

Running the new kernel and the new options are right :-

Code:

root@indigo:/proc# zcat config.gz | grep RTC
CONFIG_HPET_EMULATE_RTC=y
CONFIG_RTC_LIB=y
CONFIG_RTC_CLASS=y
# CONFIG_RTC_HCTOSYS is not set
# CONFIG_RTC_SYSTOHC is not set

But it still happens :-

Code:

Fri Oct  4 21:46:34 EST 2013
Fri 04 Oct 2013 21:46:35 EST  -0.986628 seconds
Fri Oct  4 21:47:35 EST 2013
Fri 04 Oct 2013 21:47:36 EST  -1.001941 seconds
Fri Oct  4 21:48:36 EST 2013
Fri 04 Oct 2013 11:48:37 EST  -0.955072 seconds
Fri Oct  4 21:49:37 EST 2013
Fri 04 Oct 2013 11:49:38 EST  -1.001944 seconds

EDIT

Pulled the code from the kernel it it is :-

Code:

#if defined(CONFIG_GENERIC_CMOS_UPDATE) || defined(CONFIG_RTC_SYSTOHC)

So we have one more option to kill

EDIT2

Remove the offending config setting which is actually deprecated, the new option we've been talking about replaces this one

Code:

--- arch/x86/Kconfig.old        2013-10-04 22:47:31.925835050 +1000
+++ arch/x86/Kconfig    2013-10-04 22:47:40.047548184 +1000
@@ -101,7 +101,6 @@
        select ARCH_WANT_IPC_PARSE_VERSION if X86_32
        select HAVE_ARCH_SECCOMP_FILTER
        select BUILDTIME_EXTABLE_SORT
-       select GENERIC_CMOS_UPDATE
        select CLOCKSOURCE_WATCHDOG
        select GENERIC_CLOCKEVENTS
        select ARCH_CLOCKSOURCE_DATA if X86_64

bartgymnast · 10-04-2013, 07:49 AM

Hi All,

grabbed this part from arch linux

Recent kernels set the system time from the RTC directly on boot, assuming that the RTC is in UTC. This means that if the RTC is in local time, then the system time will first be set up wrongly and then corrected shortly afterwards on every boot. This is the root of certain weird bugs (time going backwards is rarely a good thing).

These will generate /etc/adjtime automatically; no further configuration is required.

During kernel startup, at the point when the RTC driver is loaded, the system clock may be set from the hardware clock. Whether this occurs or not depends on the hardware platform, the version of the kernel and kernel build options. If this does occur, at this point in the boot sequence, the hardware clock time is assumed to be UTC and the value of /sys/class/rtc/rtcN/hctosys (N=0,1,2,..) will be set to 1. Later, the system clock is set again from the hardware clock from systemd/udev, dependent on values in /etc/adjtime. Hence, having the hardware clock using localtime may cause some unexpected behavior during the boot sequence; e.g system time going backwards, which is always a bad idea.

so indeed the option of wildwizard is the other option that needs to be killed

wildwizard · 10-04-2013, 07:53 AM

Quote:

Originally Posted by bartgymnast

Recent kernels set the system time from the RTC directly on boot

We already have code in rc.S to set the system clock, having the kernel suddenly start butting in is not a good idea.

allend · 10-04-2013, 08:40 AM

From 'man hwclock' (Bold emphasis is mine)

Code:

Automatic Hardware Clock Synchronization By the Kernel
       You should be aware of another way that the Hardware Clock is kept synchronized in some systems.  The Linux kernel has a mode
       wherein it copies the System Time to the Hardware Clock every 11 minutes.  This is a good mode to  use  when  you  are  using
       something  sophisticated  like ntp to keep your System Time synchronized. (ntp is a way to keep your System Time synchronized
       either to a time server somewhere on the network or to a radio clock hooked up to your system.  See RFC 1305).

       This mode (we'll call it "11 minute mode") is off until something turns it on.  The ntp daemon xntpd is one thing that  turns
       it  on.   You  can  turn it off by running anything, including hwclock --hctosys, that sets the System Time the old fashioned
       way.

       To see if it is on or off, use the command adjtimex --print and look at the value of "status".  If the "64" bit of this  num-
       ber (expressed in binary) equal to 0, 11 minute mode is on.  Otherwise, it is off.

       If your system runs with 11 minute mode on, don't use hwclock --adjust or hwclock --hctosys.  You'll just make a mess.  It is
       acceptable to use a hwclock --hctosys at startup time to get a reasonable System Time until your system is able  to  set  the
       System Time from the external source and start 11 minute mode.

Is something going on that is turning "11 minute mode" off?

allend · 10-04-2013, 10:02 AM

I have been looking at /usr/src/linux/kernel/time/ntp.c
As I read it, if you turn off both CONFIG_GENERIC_CMOS_UPDATE and CONFIG_RTC_SYSTOHC, then the "11 minute mode" will be entirely turned off.

GazL · 10-04-2013, 11:49 AM

Yep it looks like the new RTC stuff is only part of the story and there's a second mechanism at play here.
Here's some background info I stumbled upon: https://lkml.org/lkml/2013/4/23/690

The interesting thing is that ntp.c contains this:

Code:

        getnstimeofday(&now);
        if (abs(now.tv_nsec - (NSEC_PER_SEC / 2)) <= tick_nsec / 2) {
                struct timespec adjust = now;

                fail = -ENODEV;
                if (persistent_clock_is_local)
                        adjust.tv_sec -= (sys_tz.tz_minuteswest * 60);
#ifdef CONFIG_GENERIC_CMOS_UPDATE
                fail = update_persistent_clock(adjust);
#endif
#ifdef CONFIG_RTC_SYSTOHC
                if (fail == -ENODEV)
                        fail = rtc_set_ntp_time(adjust);
#endif

... which looks like it should still deal with a clock in localtime assuming persistent_clock_is_local and sys_tz are being set correctly.

This whole area looks like a big mess. Maybe the latest util-linux might help?, but to be honest I'm inclined to go back to my previous suggestion which echoes one of the comments in kernel/time.c:

Code:

 
 *The best thing to do is to keep the CMOS clock in universal time (UTC)
 * as real UNIX machines always do it. This avoids all headaches about
 * daylight saving times and warping kernel clocks.
 */

astrogeek · 10-04-2013, 12:09 PM

I just encountered this on a -current machine after overnight extended power failure.

Nothing new to offer, but thanks for all who have posted - gives me an easier start!