[SOLVED] NTP issues and system time drift

timetraveler · 12-05-2011, 03:17 PM

This has worked no prob.
http://cr.yp.to/clockspeed/clockspeed-0.62.tar.gz

tronayne · 12-05-2011, 04:25 PM

NTP may not sync if your system clock is too far off the actual time, well, not just may, it just won't.

So, stop ntpd then (as root, sudo, su-, whatever)

Code:

ntpdate time.nist.gov
restart ntpd
wait a while
ntpq -p

It should sync.

By the way, when you shut down the system there's a good chance that one of the final steps during shut down is to save the system clock (which should be synchronized with NTP) to the hardware clock -- check your logs to see if that does happen.

Cyked · 12-05-2011, 05:31 PM

I don't have an issue with the time syncing. I have an issue with the time getting completely out of wack AFTER its synced.

Code:

ntpdate time.nist.gov
restart ntpd
wait a while
ntpq -p

I can do this all day long. My system time will still drift off.

And, if sudo hwclock,date shows me the currently HWclock time and the current system date/time I DO NOT have an issue with the hwclock. That is fine. It has been fine. It has never been an issue/
I don't reboot often, but even when I do both times are fine after logging back in.

I last restarted ntp around 4 hours ago and my time is still up to date.

michaelk · 12-05-2011, 08:09 PM

ntpdate forces the system clock to the current time but ntp never syncs up with a time server so it starts to drift. If system time runs to fast or slow ntp can not compensate. The suggested utility should update the kernel time variables and stabilise the system clock.

The purpose of the hardware clock is to provide a time source when the computer first boots up. The OS operates entirely from the system clock which is software. If ntp is running the hardware clock is automatically updated every 11 minutes.

Depending on the version of ntp running it will force a time update using the -g option so running ntpdate first is not required any more. However, if the time offset does become > 1000 secs ntp will quit assuming something is wrong.

BTW does this computer use CPU frequency scaling?

tronayne · 12-06-2011, 08:05 AM

In addition to michaelk's question about CPU frequency scaling, another question would be, what is your internet connection (DSL, broadband, satellite, wi-fi, something else)?

Asking because your system clock (nothing whatsoever to do with your hardware clock) drifts, apparently a lot. NTP will, every 11 minutes (as pointed out elsewhere) synchronize the system clock with a reference clock -- but, if the network goes away (say, because you use a cell-phone wi-fi connection that's not 24/7), NTP will simply stop. The daemon won't die, it'll simply stop doing anything and you would need to stop and restart the daemon (and, potentially, reset the system clock either with ntpdate or hwclock -s (which sets the system clock from the hardware clock)).

Keep in mind that if ntpq -p does not show an asterisk next to one of the external references, the system clock is not synchronized (the other references will show + signs indicating that they're candidates for synchronization) -- just because you see the display of external time references doesn't mean diddly-squat if there's no asterisk next to one of them. And, again, you need to wait a few minutes after the NTP daemon starts to see synchronization.

Hope this helps some.

Cyked · 12-06-2011, 10:10 AM

So why is there no * indicating that it is locked to a server for sync, or has synced to it? I can connect to any of the hosts listed in the conf file and update the time with ntpdate, So its not a connection issue.

I have comcast, no connection drops.

Code:

Dec  6 08:04:35 tux ntpd[1661]: ntpd 4.2.4p8@1.1612-o Tue Apr 19 07:08:29 UTC 2011 (1)
Dec  6 08:04:35 tux ntpd[1662]: precision = 1.000 usec
Dec  6 08:04:35 tux ntpd[1662]: ntp_io: estimated max descriptors: 1024, initial socket boundary: 16
Dec  6 08:04:35 tux ntpd[1662]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled
Dec  6 08:04:35 tux ntpd[1662]: Listening on interface #1 wildcard, ::#123 Disabled
Dec  6 08:04:35 tux ntpd[1662]: Listening on interface #2 lo, 127.0.0.1#123 Enabled
Dec  6 08:04:35 tux ntpd[1662]: Listening on interface #3 eth1, 192.168.2.102#123 Enabled
Dec  6 08:04:35 tux ntpd[1662]: kernel time sync status 2040
Dec  6 08:04:35 tux ntpd[1662]: frequency initialized 10.197 PPM from /var/lib/ntp/ntp.drift
Dec  6 08:04:45 tux ntpd[1662]: synchronized to 129.6.15.28, stratum 1
Dec  6 08:04:44 tux ntpd[1662]: time reset -0.618067 s
Dec  6 08:04:44 tux ntpd[1662]: kernel time sync status change 2001
Dec  6 08:05:01 tux CRON[1673]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)

This is in syslog after stopping NTP, ntpdate, and restarted NTP. When I got up this morning BOTH system time and hwclock had been syncing with each other, but were about 5 or so minutes ahead of my windows box. I haven't had any issues with the hw clock being off.

more output after I stopped NTP and synced and started it again. Note that right after my system time came back by about 7 min and hwclock was off

Code:

dpaul01@tux:/var/log$ ntpdate  server 0.pool.ntp.org
Error : Name or service not known
 6 Dec 08:08:51 ntpdate[1643]: can't find host server

 6 Dec 08:08:51 ntpdate[1643]: bind() fails: Permission denied
dpaul01@tux:/var/log$ date
Tue Dec  6 08:08:59 PST 2011
dpaul01@tux:/var/log$ sudo ntpdate  server 0.pool.ntp.org
Error : Name or service not known
 6 Dec 08:09:06 ntpdate[1654]: can't find host server

 6 Dec 08:02:43 ntpdate[1654]: step time server 169.229.70.64 offset -384.306965 sec
dpaul01@tux:/var/log$ date
Tue Dec  6 08:02:48 PST 2011
dpaul01@tux:/var/log$ sudo hwclock;date
Tue 06 Dec 2011 08:09:16 AM PST  -0.044874 seconds
Tue Dec  6 08:02:54 PST 2011
dpaul01@tux:/var/log$ ntpq -p
ntpq: read: Connection refused
dpaul01@tux:/var/log$ sudo ntpq -p
ntpq: read: Connection refused
dpaul01@tux:/var/log$ sudo ntpd
dpaul01@tux:/var/log$ sudo ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 time-a.nist.gov .ACTS.           1 u    -   64    1  103.168  -594.28   0.001
 tick.tadatv.com .INIT.          16 u    -   64    0    0.000    0.000   0.001
 pool-test.ntp.o .INIT.          16 u    -   64    0    0.000    0.000   0.001
 rigel.jeffkapla .INIT.          16 u    -   64    0    0.000    0.000   0.001
 conquest.kjsl.c .INIT.          16 u    -   64    0    0.000    0.000   0.001
 LOCAL(0)        .LOCL.          10 l    -   64    0    0.000    0.000   0.001
dpaul01@tux:/var/log$ sudo ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 time-a.nist.gov .ACTS.           1 u    1   64    1  112.351  -606.51  12.235
 tick.tadatv.com 10.0.22.51       2 u    3   64    1   20.519  -592.12   0.001
 pool-test.ntp.o 127.67.113.92    2 u    2   64    1   23.461  -598.17   0.001
 rigel.jeffkapla 216.128.67.248   3 u    -   64    1   98.565  -614.44   0.001
 conquest.kjsl.c .INIT.          16 u    -   64    0    0.000    0.000   0.001
 LOCAL(0)        .LOCL.          10 l    -   64    0    0.000    0.000   0.001
dpaul01@tux:/var/log$ sudo ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 time-a.nist.gov .ACTS.           1 u    -   64    1   92.328  -612.21  13.300
 tick.tadatv.com 10.0.22.51       2 u    4   64    1   20.519  -592.12   0.001
 pool-test.ntp.o 127.67.113.92    2 u    3   64    1   23.461  -598.17   0.001
 rigel.jeffkapla 216.128.67.248   3 u    1   64    1   98.565  -614.44   0.001
 conquest.kjsl.c 69.36.224.15     2 u    1   64    1   20.150  -609.32   0.001
 LOCAL(0)        .LOCL.          10 l    -   64    0    0.000    0.000   0.001
dpaul01@tux:/var/log$ sudo ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 time-a.nist.gov .ACTS.           1 u    -   64    1   92.328  -612.21  13.300
 tick.tadatv.com 10.0.22.51       2 u    4   64    1   20.519  -592.12   0.001
 pool-test.ntp.o 127.67.113.92    2 u    3   64    1   23.461  -598.17   0.001
 rigel.jeffkapla 216.128.67.248   3 u    2   64    1   98.565  -614.44   0.001
 conquest.kjsl.c 69.36.224.15     2 u    2   64    1   20.150  -609.32   0.001
 LOCAL(0)        .LOCL.          10 l    1   64    1    0.000    0.000   0.001
dpaul01@tux:/var/log$ sudo ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 time-a.nist.gov .ACTS.           1 u    1   64    1   92.328  -612.21  13.300
 tick.tadatv.com 10.0.22.51       2 u    5   64    1   20.519  -592.12   0.001
 pool-test.ntp.o 127.67.113.92    2 u    4   64    1   23.461  -598.17   0.001
 rigel.jeffkapla 216.128.67.248   3 u    2   64    1   98.565  -614.44   0.001
 conquest.kjsl.c 69.36.224.15     2 u    2   64    1   20.150  -609.32   0.001
 LOCAL(0)        .LOCL.          10 l    1   64    1    0.000    0.000   0.001

The a moment later both were back to being inline:

Code:

 sudo hwclock;date
Tue 06 Dec 2011 08:08:10 AM PST  -0.898316 seconds
Tue Dec  6 08:08:11 PST 2011

michaelk · 12-06-2011, 10:32 AM

Post the output from ntpq -p when reach displays 377.
We believe your system is not locking due to the system clock not being stable.
The offset has been decreasing so maybe it is just going to take awhile. Yesterday it was ~7.5 secs and now its down to 0.6. ntp applies small increments when it updates so large time jumps do not occur.

I would expect that ntp will sync to local(0) if nothing else.

Cyked · 12-06-2011, 10:41 AM

Code:

 sudo hwclock;date
Tue 06 Dec 2011 08:40:41 AM PST  -0.100172 seconds
Tue Dec  6 08:40:41 PST 2011
account@machine:~$ ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 time-a.nist.gov .ACTS.           1 u  149  512  377   92.547  -6058.1 2375.49
 tick.tadatv.com 10.0.22.51       2 u  162  512  377   15.728  -10616. 4084.71
 pool-test.ntp.o 69.25.96.13      2 u  100  512  377   21.152  -7635.2 1959.01
 rigel.jeffkapla 216.128.67.248   3 u  152  512  377   97.515  -9348.5 2919.97
 conquest.kjsl.c 69.36.224.15     2 u   93  512  377   16.234  -8305.4 2116.57
*LOCAL(0)        .LOCL.          10 l   25   64  377    0.000    0.000   0.001

NOW it does it lol

michaelk · 12-06-2011, 12:51 PM

High jitter and offset is keeping your your computer from syncing to a server. Have those values changed since you last posted?
I would try running clockspeed as suggested to see if you can stabilize the clock.

Cyked · 12-06-2011, 01:25 PM

No change, the values are still high.

tronayne · 12-06-2011, 02:31 PM

I don't see a list of your /etc/ntp.conf -- I'm wondering if you are using pool servers or have them hard-coded in your configuration file; e.g., time-a.nist.gov is a stratum 1 server (which we're not supposed to be using and would definitely not be a pool server).

As I understand the pool servers, you'll get servers that are electrically close to you, and, from the offset and jitter values you've posted, they ain't. Perhaps it would be useful to ping the servers you've defined (or that got defined for you) and look at the result. For example, pinging time.nist.gov on my server (which is in northern Michigan via HugesNet which adds delay for the satellite up-and-down, plus the connection in Texas, plus the up-and-down coming back) is

Code:

ping -c 5 time.nist.gov
PING time.nist.gov (192.43.244.18) 56(84) bytes of data.
64 bytes from time.nist.gov (192.43.244.18): icmp_req=1 ttl=50 time=821 ms
64 bytes from time.nist.gov (192.43.244.18): icmp_req=2 ttl=50 time=965 ms
64 bytes from time.nist.gov (192.43.244.18): icmp_req=3 ttl=50 time=1089 ms
64 bytes from time.nist.gov (192.43.244.18): icmp_req=4 ttl=50 time=1144 ms
64 bytes from time.nist.gov (192.43.244.18): icmp_req=5 ttl=50 time=866 ms

--- time.nist.gov ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 5263ms
rtt min/avg/max/mdev = 821.701/977.653/1144.324/124.230 ms, pipe 2

time-a.nist.gov does not respond to ping: trust me on this, if you can't ping it, it's not going to a usable time synchronization service.

You could have a terrible connection to your ISP -- try pinging a few sites and see what you get; e.g., news.google.com is going to be somewhere under 750 ms on average.

One other thing, just in case, check and see if you have more than one instance of NTP running; you should only see something pretty much like this:

Code:

ps -ef | grep ntpd
root      7641     1  0 Dec04 ?        00:00:05 /usr/sbin/ntpd -g -p /var/run/ntpd.pid
root     14765 14742  0 15:24 pts/1    00:00:00 grep ntpd

If you see more than one ntpd running, you'll need to stop the daemon then kill any corpses that may be left and restart the daemon.

Hope this helps some.

Cyked · 12-06-2011, 03:04 PM

Code:

# /etc/ntp.conf, configuration for ntpd; see ntp.conf(5) for help

driftfile /var/lib/ntp/ntp.drift


# Enable this if you want statistics to be logged.
statsdir /var/log/ntpstats/

statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable


# You do need to talk to an NTP server or two (or three).
server time-a.nist.gov iburst
server 1.pool.ntp.org
server 2.pool.ntp.org
server 0.pool.ntp.org
server 1.fedora.pool.ntp.org
server  127.127.1.0     # local clock
fudge   127.127.1.0 stratum 10

# Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for
# details.  The web page <http://support.ntp.org/bin/view/Support/AccessRestrictions>
# might also be helpful.
#
# Note that "restrict" applies to both servers and clients, so a configuration
# that might be intended to block requests from certain clients could also end
# up blocking replies from your own upstream servers.

# By default, exchange time with everybody, but don't allow configuration.
restrict -4 default kod notrap nomodify nopeer noquery
restrict -6 default kod notrap nomodify nopeer noquery

# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1

# Clients from this (example!) subnet have unlimited access, but only if
# cryptographically authenticated.
#restrict 192.168.123.0 mask 255.255.255.0 notrust


# If you want to provide time to your local subnet, change the next line.
# (Again, the address is an example only.)
#broadcast 192.168.123.255

# If you want to listen to time broadcasts on your local subnet, de-comment the
# next lines.  Please do this only if you trust everybody on the network!
#disable auth
#broadcastclient

NTPD is running only once and i have no issues pinging anything in my ntp.conf file.

Code:

 ping -c 5 1.pool.ntp.org
PING 1.pool.ntp.org (209.114.111.1) 56(84) bytes of data.
64 bytes from bindcat.fhsu.edu (209.114.111.1): icmp_seq=1 ttl=48 time=95.7 ms
64 bytes from bindcat.fhsu.edu (209.114.111.1): icmp_seq=2 ttl=48 time=91.5 ms
64 bytes from bindcat.fhsu.edu (209.114.111.1): icmp_seq=3 ttl=48 time=92.8 ms
64 bytes from bindcat.fhsu.edu (209.114.111.1): icmp_seq=4 ttl=48 time=91.3 ms
64 bytes from bindcat.fhsu.edu (209.114.111.1): icmp_seq=5 ttl=48 time=90.7 ms

--- 1.pool.ntp.org ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 90.798/92.460/95.710/1.802 ms

Cyked · 12-06-2011, 03:17 PM

This is as of around 1:15pm, just to check.

Code:

ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 time-a.nist.gov .ACTS.           1 u   87  128  377   95.111  -1813.6 1572.65
 mirror          204.9.54.119     2 u  100  128  377   84.130  -3419.4 1023.91
 name1.glorb.com 128.252.19.1     2 u  109  128  377   82.569  -1726.9 1556.14
*tick.tadatv.com 10.0.22.51       2 u  109  128  377   16.817  -3045.9 906.555
 bindcat.fhsu.ed 132.163.4.102    2 u   57  128  377   90.871  -3315.7 895.407
 LOCAL(0)        .LOCL.          10 l   44   64  377    0.000    0.000   0.001

Then it switched back to locking to local again.

michaelk · 12-08-2011, 04:50 PM

What is your frequency error?
http://www.eecis.udel.edu/~mills/ntp/html/debug.html

Cyked · 12-19-2011, 12:21 PM

So roundabout way I suppose. I fixed it by ripping out the NTP stuff and installing ntpd from source.