[SOLVED] NTP Client Error -- ntpstat shows unsynchronised

vikas027 · 08-23-2009, 12:51 AM

Hi All,

I am no expert in NTP But I have configured NTP Client many times, But this time something is going wrong.

This is my /etc/ntp.conf

Code:

[root@browser1 ~]# cat /etc/ntp.conf
restrict default ignore
restrict 127.0.0.1
server 10.2.29.26  prefer
driftfile /var/lib/ntp/drift
broadcastdelay  0.008
authenticate yes
keys            /etc/ntp/keys

Drift File

Code:

[root@browser1 ~]# cat /var/lib/ntp/drift
0.000

My NTP Server 10.2.29.26

Code:

[root@browser1 ~]# ntpq -p 10.2.29.26
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 SHM(0)          .MSF.            0 l    -   64    0    0.000    0.000 4000.00
*SHM(1)          .GPS.            0 l  139   16  377    0.000   -0.021   0.022
 255.255.255.255 .BCST.          16 -    -   64    0    0.000    0.000 4000.00

/var/log/messages on restarting NTP server, there is nothing else relating to NTP. I am not sure even if there should be anything relating to time sync etc.

Code:

Aug 23 11:13:11 browser1 ntpd: ntpd shutdown succeeded
Aug 23 11:13:11 browser1 ntpd: ntpd startup succeeded

Other commands

Code:

[root@browser1 ~]# /usr/sbin/ntpq -pn
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 10.2.29.26      .INIT.          16 u    -   64    0    0.000    0.000 4000.00

[root@browser1 ~]# /usr/sbin/ntpq -n -c peers
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 10.2.29.26      .INIT.          16 u    -   64    0    0.000    0.000 4000.00

[root@browser1 ~]# ntpstat
unsynchronised
  time server re-starting
   polling server every 64 s

[root@browser1 ~]# ntpdate -buv 10.2.29.26
23 Aug 11:17:19 ntpdate[17340]: ntpdate 4.2.0a@1.1190-r Thu Oct  5 04:11:32 EDT 2006 (1)
23 Aug 11:15:04 ntpdate[17340]: step time server 10.2.29.26 offset -135.099450 sec

Why is ntpstat still unsynchrnoized ?

Am I missing something ? Please Help .!!
If things does not work, I will have to put this in crontab.

Code:

60 * * * * /usr/sbin/ntpdate -u 10.2.29.26 &>/dev/null

kbp · 08-23-2009, 09:20 AM

Hi vikas027,

I'd have to guess a firewall issue, can you confirm 123/udp inbound to the client ?

thanks,

kbp

tronayne · 08-23-2009, 09:34 AM

Here's a known good ntp.conf that uses three pool servers (are you sure that 10.2.29.26 is serving time? That jitter value is way, way out of line) and does a fall-back to the local host when the network goes away for any reason. You can give this a try and see if it works for you; include 10.2.29.26 as your preferred source of time above the three pool servers and see if that helps. The ntp.conf below is well-commented to explain what's what in each section.

Your /etc/ntp directory should be owner and group root, mode 0755 and /etc/ntp/drift should be root.root and mode 0644; the ntp.keys file should be root.root and mode 600.

Another thing is that if your system clock is too far off, NTP won't synchronize it -- try setting the clock with ntpdate and one of the pool servers (just in case 10.2.29.26 isn't serving you, then start the daemon.

Hope this helps some.

Code:

#
# Undisciplined Local Clock. This is a fake driver intended for backup
# and when no outside source of synchronized time is available. The
# default stratum is usually 3, but in this case we elect to use stratum
# 0. Since the server line does not have the prefer keyword, this driver
# is never used for synchronization, unless no other other
# synchronization source is available. In case the local host is
# controlled by some external source, such as an external oscillator or
# another protocol, the prefer keyword would cause the local host to
# disregard all other synchronization sources, unless the kernel
# modifications are in use and declare an unsynchronized condition.
#
server  127.127.1.0     # local clock
fudge   127.127.1.0 stratum 10
#server  pool.ntp.org
server  0.us.pool.ntp.org
server  1.us.pool.ntp.org
server  2.us.pool.ntp.org

#
# Drift file.  Put this in a directory which the daemon can write to.
# No symbolic links allowed, either, since the daemon updates the file
# by creating a temporary in the same directory and then rename()'ing
# it to the file.
#
driftfile /etc/ntp/drift
multicastclient                 # listen on default 224.0.1.1
broadcastdelay  0.008

#
# Keys file.  If you want to diddle your server at run time, make a
# keys file (mode 600 for sure) and define the key number to be
# used for making requests.
# PLEASE DO NOT USE THE DEFAULT VALUES HERE. Pick your own, or remote
# systems might be able to reset your clock at will.
#
#keys           /etc/ntp/keys
#trustedkey     65535
#requestkey     65535
#controlkey     65535

# Don't serve time or stats to anyone else by default (more secure)
restrict default noquery nomodify
# Trust ourselves.  :-)
restrict 127.0.0.1

vikas027 · 08-25-2009, 04:04 AM

Quote:

Originally Posted by kbp

Hi vikas027,

I'd have to guess a firewall issue, can you confirm 123/udp inbound to the client ?

thanks,

kbp

Thanks for your time,
here it is

Code:

[root@browser1 ~]# netstat -an | grep 123
udp        0      0 10.133.23.239:123           0.0.0.0:*
udp        0      0 127.0.0.1:123               0.0.0.0:*
udp        0      0 0.0.0.0:123                 0.0.0.0:*
udp        0      0 :::123                      :::*

vikas027 · 08-25-2009, 04:13 AM

Quote:

Originally Posted by tronayne

Here's a known good ntp.conf that uses three pool servers (are you sure that 10.2.29.26 is serving time? That jitter value is way, way out of line) and does a fall-back to the local host when the network goes away for any reason.

I just have one ntp server with me at present. And outside network is not enabled.

Quote:

Originally Posted by tronayne

Your /etc/ntp directory should be owner and group root, mode 0755 and /etc/ntp/drift should be root.root and mode 0644; the ntp.keys file should be root.root and mode 600.

Code:

[root@browser1 ~]# ls -ld /etc/ntp
drwxr-xr-x  2 root root 4096 Aug 25 14:36 /etc/ntp
[root@browser1 ~]#
[root@browser1 ~]# ls -l /etc/ntp
total 24
-rw-r--r--  1 root root   0 Aug 25 14:36 drift
-rw-------  1 root root 266 Oct  5  2006 keys
-rw-r--r--  1 root root 186 Apr 20  2006 ntpservers
-rw-r--r--  1 root root   0 Oct  5  2006 step-tickers

Still, I am getting the same error. I am getting this unsynchronised but server time (NTP client) is fine at this moment.

Code:

[root@browser1 ~]# ntpstat
unsynchronised
  time server re-starting
   polling server every 64 s
[root@browser1 ~]#

tronayne · 08-25-2009, 07:25 AM

Have you tried starting ntpd with logging so you can see what's what?

Something like

Code:

>/tmp/ntp.log
/usr/sbin/ntpd -g -p /var/run/ntpd.pid -l /tmp/ntp.log

(or where and how your ntpd lives)

Again, that jitter time looks like your NTP server is not serving time to you; are you sure it's running, that it's configured to serve time and that you can get from "this" system to it (like, can you ping it?).

"Good" log entries will look something like (I use three pool servers, yours will be different)

Code:

23 Aug 08:50:33 ntpd[2919]: logging to file /tmp/ntp.log
23 Aug 08:50:33 ntpd[2919]: precision = 1.000 usec
23 Aug 08:50:33 ntpd[2919]: ntp_io: estimated max descriptors: 1024, initial socket boundary: 16
23 Aug 08:50:33 ntpd[2919]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled
23 Aug 08:50:33 ntpd[2919]: Listening on interface #1 wildcard, ::#123 Disabled
23 Aug 08:50:33 ntpd[2919]: Listening on interface #2 lo, ::1#123 Enabled
23 Aug 08:50:33 ntpd[2919]: Listening on interface #3 lo, 127.0.0.1#123 Enabled
23 Aug 08:50:33 ntpd[2919]: Listening on interface #4 eth0, 192.168.1.10#123 Enabled
23 Aug 08:50:33 ntpd[2919]: kernel time sync status 0040
23 Aug 08:50:35 ntpd[2919]: frequency initialized -50.347 PPM from /etc/ntp/drift
23 Aug 08:50:35 ntpd[2919]: Listening on interface #5 eth0, fe80::211:11ff:fe44:cb1f#123 Enabled
23 Aug 08:50:35 ntpd[2919]: new interface(s) found: waking up resolver
23 Aug 08:53:46 ntpd[2919]: synchronized to LOCAL(0), stratum 10
23 Aug 08:53:46 ntpd[2919]: kernel time sync status change 0001
23 Aug 08:53:50 ntpd[2919]: synchronized to 66.187.233.4, stratum 1
23 Aug 08:55:35 ntpd[2919]: Listening on interface #6 vmnet8, fe80::250:56ff:fec0:8#123 Enabled
23 Aug 08:55:35 ntpd[2919]: Listening on interface #7 vmnet1, fe80::250:56ff:fec0:1#123 Enabled
23 Aug 08:55:35 ntpd[2919]: Listening on interface #8 vmnet1, 172.16.114.1#123 Enabled

kbp · 08-25-2009, 09:47 AM

Hi vikas027,

Running a netstat only shows that it is listening, not that the ntp server can communicate with the ntp client

cheers,

kbp

abhijeetdutta · 08-25-2009, 10:16 AM

hi vikash

may be your server config is not correct.some day back i have also faced this issue.

Pl post your server config by viewing i will suggest some thing.

Either you can change with this conf.

server ur time server
server 127.127.1.0
fudge 127.127.1.0 stratum 10

before doing all this activity stop your ntpd service and after config client run ntpdate -u time server ip.for three times.

may be it works.....

bye

vikas027 · 09-05-2009, 10:19 PM

Quote:

Originally Posted by kbp

Running a netstat only shows that it is listening, not that the ntp server can communicate with the ntp client.

Sorry, for coming late.
Are you asking for this,

Code:

[root@browser1 ~]# ntpq -p 10.2.29.26
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 SHM(0)          .MSF.            0 l    -   64    0    0.000    0.000 4000.00
*SHM(1)          .GPS.            0 l  139   16  377    0.000   -0.021   0.022
 255.255.255.255 .BCST.          16 -    -   64    0    0.000    0.000 4000.00

vikas027 · 09-05-2009, 10:27 PM

Quote:

Originally Posted by tronayne

Have you tried starting ntpd with logging so you can see what's what?

Something like

Code:

>/tmp/ntp.log
/usr/sbin/ntpd -g -p /var/run/ntpd.pid -l /tmp/ntp.log

No, I had no knowledge about logging. Will keep a note of this in future.

I ran the above commands and found this in /tmp/ntp.log.

Code:

 6 Sep 08:54:25 ntpd[4650]: logging to file /tmp/ntp.log
 6 Sep 08:54:25 ntpd[4650]: ntpd 4.2.0a@1.1190-r Thu Oct  5 04:11:32 EDT 2006 (1)
 6 Sep 08:54:25 ntpd[4650]: precision = 1.000 usec
 6 Sep 08:54:25 ntpd[4650]: bind() fd 3, family 2, port 123, addr 0.0.0.0, in_classd=0 flags=8 fails: Address already in use
 6 Sep 08:54:25 ntpd[4650]: bind() fd 3, family 10, port 123, addr ::, in6_is_addr_multicast=0 flags=0 fails: Address already in use
 6 Sep 08:54:25 ntpd[4650]: bind() fd 3, family 2, port 123, addr 127.0.0.1, in_classd=0 flags=0 fails: Address already in use
 6 Sep 08:54:25 ntpd[4650]: bind() fd 3, family 2, port 123, addr 10.133.23.239, in_classd=0 flags=8 fails: Address already in use
 6 Sep 08:54:25 ntpd[4650]: kernel time sync status 0040
 6 Sep 08:54:25 ntpd[4650]: Frequency format error in /etc/ntp/drift
 6 Sep 08:54:25 ntpd[4650]: configure: keyword "authenticate" unknown, line ignored
 6 Sep 08:54:26 ntpd[4650]: sendto(10.2.29.26): Bad file descriptor
 6 Sep 08:55:31 ntpd[4650]: sendto(10.2.29.26): Bad file descriptor
 6 Sep 08:56:35 ntpd[4650]: sendto(10.2.29.26): Bad file descriptor
 6 Sep 08:57:38 ntpd[4650]: sendto(10.2.29.26): Bad file descriptor
 6 Sep 08:58:41 ntpd[4650]: sendto(10.2.29.26): Bad file descriptor
 6 Sep 08:59:45 ntpd[4650]: sendto(10.2.29.26): Bad file descriptor

Please help. Is there some issue with the NTP server itself ?

And yes, I am able to ping it.

Code:

[root@browser1 ~]# ping 10.2.29.26
PING 10.2.29.26 (10.2.29.26) 56(84) bytes of data.
64 bytes from 10.2.29.26: icmp_seq=0 ttl=57 time=42.5 ms
64 bytes from 10.2.29.26: icmp_seq=1 ttl=58 time=42.8 ms

--- 10.2.29.26 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 42.505/42.670/42.836/0.264 ms, pipe 2

Also I have found that my /etc/ntp/drift is empty. I do not think it should be empty as it should store error in frequency of the system clock. Am i right ?

tronayne · 09-07-2009, 07:37 AM

Back in the prehistory of the world with NTP I recall having to initialize /etc/ntp/drift with a real number; e.g.,

Code:

log in as root or su -
cat > /etc/ntp/drift
0.0
ctrl-d

or use an editor or some other method of your choosing, just get 0.0 in there and restart ntpd and see if that helps any. Once you get it going, that value will be updated periodically by the daemon so you only do this one time.

Looking up at your initial post, you have your driftfile defined as /var/lib/ntp/drift? The log is telling you that it's looking in /etc/ntp/drift (and it's maybe looking for other file in /etc/ntp too?

You may want to just try changing your server section in /etc/ntp.conf to something like this

Code:

server  127.127.1.0     # local clock
fudge   127.127.1.0 stratum 10
server 10.2.29.26
server  0.us.pool.ntp.org
server  1.us.pool.ntp.org
server  2.us.pool.ntp.org

(the prefer keyword is deleted above). Defining the pool servers won't hurt and might help -- if you can sync to a pool server but not to your intranet server...

Comment out the authenticate line; it's not doing anything.

You're looking for keys, does the file /etc/ntp/keys exist and is there anything in it? I don't use keys and the provided example file (/etc/ntp/ntp.keys) contains

Code:

65535   M       akey
1       M       pass

Note that those values are discussed in the example ntp.conf file:

Code:

#
# Keys file.  If you want to diddle your server at run time, make a
# keys file (mode 600 for sure) and define the key number to be
# used for making requests.
# PLEASE DO NOT USE THE DEFAULT VALUES HERE. Pick your own, or remote
# systems might be able to reset your clock at will.
#
#keys           /etc/ntp/keys
#trustedkey     65535
#requestkey     65535
#controlkey     65535

You can also just comment out that section entirely and see what happens (maybe you don't have legit keys?).

Then stop and restart the daemon and see what the log tells you. I have this in the daemon start up; you may want to add the logging to yours.

Code:

# Start/stop/restart ntpd.

# Start ntpd:
ntpd_start() {
  # Clear the log file
  >/tmp/ntp.log
  CMDLINE="/usr/sbin/ntpd -g"
  echo -n "Starting NTP daemon:  $CMDLINE"
  $CMDLINE -p /var/run/ntpd.pid -l /tmp/ntp.log
  echo
}

Hope all this helps some.

vikas027 · 09-07-2009, 03:11 PM

Thanks a tronayne for your time. I have done the changes as told by you.
My new ntp.conf is

Code:

[root@browser1 ~]# cat /etc/ntp.conf
server  127.127.1.0     # local clock
fudge   127.127.1.0 stratum 10
server 10.2.29.26
server  0.us.pool.ntp.org
server  1.us.pool.ntp.org
server  2.us.pool.ntp.org
driftfile /etc/ntp/drift
broadcastdelay  0.008
keys            /etc/ntp/keys

Drift File

Code:

[root@browser1 ~]# cat /etc/ntp/drift
0.0

All lines are commented in keys

Code:

[root@browser1 ~]# cat /etc/ntp/keys
#
# PLEASE DO NOT USE THE DEFAULT VALUES HERE. Pick your own, or remote
# systems might be able to reset your clock at will. Note also that
# ntpd is started with a -A flag, disabling authentication, that
# will have to be removed as well.
#
#65535  M       akey
#1      M       pass

New, log file

Code:

 8 Sep 01:35:30 ntpd[22438]: logging to file /tmp/ntp.log
 8 Sep 01:35:30 ntpd[22438]: ntpd 4.2.0a@1.1190-r Thu Oct  5 04:11:32 EDT 2006 (1)
 8 Sep 01:35:30 ntpd[22438]: precision = 1.000 usec
 8 Sep 01:35:30 ntpd[22438]: bind() fd 3, family 2, port 123, addr 0.0.0.0, in_classd=0 flags=8 fails: Address already in use
 8 Sep 01:35:30 ntpd[22438]: bind() fd 3, family 10, port 123, addr ::, in6_is_addr_multicast=0 flags=0 fails: Address already in use
 8 Sep 01:35:30 ntpd[22438]: bind() fd 3, family 2, port 123, addr 127.0.0.1, in_classd=0 flags=0 fails: Address already in use
 8 Sep 01:35:30 ntpd[22438]: bind() fd 3, family 2, port 123, addr 10.133.23.239, in_classd=0 flags=8 fails: Address already in use
 8 Sep 01:35:30 ntpd[22438]: kernel time sync status 0040
 8 Sep 01:35:30 ntpd[22438]: frequency initialized 0.000 PPM from /etc/ntp/drift
 8 Sep 01:35:30 ntpd[22439]: signal_no_reset: signal 17 had flags 4000000
 8 Sep 01:35:32 ntpd[22438]: sendto(10.2.29.26): Bad file descriptor
 8 Sep 01:35:32 ntpd[22439]: signal_no_reset: signal 14 had flags 4000000

Any idea, now ?

tronayne · 09-08-2009, 08:05 AM

You know, the messages about port 123 look like they might be an indication of your problem -- seems like it's telling you that port 123 is in use (by who knows what). If you take a look-see at /etc/services there may be something there; e.g.,

Code:

grep 123 /etc/services
ntp             123/tcp    #Network Time Protocol
ntp             123/udp    #Network Time Protocol

is what my system shows (yours may be different, but port 123 is the standard NTP port) -- there should only be those two entries in /etc/services for port 123.

It could also be that NTP is already running when you try to start it (make sure to shut it down first) or it's getting started twice or some other weird thing. Check around in your daemon start scripts to make sure that there aren't two daemons getting launched; I don't know what Red Hat uses for starting daemons (init.d, rcn.d or what) but try searching the tree in /etc for any instances of ntpd. Something like

Code:

cd /etc
find . -type f -print | xargs grep -l ntpd

There should be one file that does the start up and shut down and one file that contains the actual path and execution instructions (like /usr/sbin/ntpd -g -p /var/run/ntpd.pid -l /tmp/ntp.log).

For what it's worth, my system (yours may be different) turns up these files:

Code:

cd /etc
find . -type f -print | xargs grep -l ntpd
./rc.d/rc.M                           this contains the start up and shut down
./rc.d/rc.ntpd                        this executes the daemon
./ntp.conf                            this is the configuration file

If you don't have rc.d, you'll possibly have init.d instead, these things vary from system to system.

It just looks like there may be double entries somewhere in the start-up or something else has grabbed port 123 (and no other daemons should be accessing that port, it's meant for NTP). Too, take a look at the output of ps;

Code:

ps -ef | grep ntp
root      8188     1  0 Sep06 ?        00:00:02 /usr/sbin/ntpd -g -p /var/run/ntpd.pid -l /tmp/ntp.log

You should only see one process running; stop the daemon and make sure that it has stopped with ps (or you can kill -9 PID of the daemon(s) that may be running).

Hope this helps some.

vikas027 · 09-09-2009, 04:02 PM

You were absolutely right there were many ntp processes running. I killed them all.

Now, after running the below commands, I waited for around 5 mins.

Code:

>/tmp/ntp.log
/usr/sbin/ntpd -g  -p /var/run/ntpd.pid -l /tmp/ntp.log

Now, this is working. Many thanks Tronayne, you are a gem.

Code:

[root@browser1 ~]# cat /tmp/ntp.log
10 Sep 01:56:09 ntpd[14934]: logging to file /tmp/ntp.log
10 Sep 01:56:09 ntpd[14934]: ntpd 4.2.0a@1.1190-r Thu Oct  5 04:11:32 EDT 2006 (1)
10 Sep 01:56:09 ntpd[14934]: precision = 1.000 usec
10 Sep 01:56:09 ntpd[14934]: Listening on interface wildcard, 0.0.0.0#123
10 Sep 01:56:09 ntpd[14934]: Listening on interface wildcard, ::#123
10 Sep 01:56:09 ntpd[14934]: Listening on interface lo, 127.0.0.1#123
10 Sep 01:56:09 ntpd[14934]: Listening on interface bond0, 10.133.23.239#123
10 Sep 01:56:09 ntpd[14934]: kernel time sync status 0040
10 Sep 01:56:09 ntpd[14934]: frequency initialized 0.000 PPM from /etc/ntp/drift
10 Sep 02:00:28 ntpd[14934]: synchronized to 10.2.29.26, stratum 1
10 Sep 02:03:45 ntpd[14934]: kernel time sync disabled 0041
10 Sep 02:12:25 ntpd[14934]: kernel time sync enabled 0001

Code:

[root@browser1 ~]# cat /etc/ntp/drift
0.000

Code:

[root@browser1 ~]# ntpstat
synchronised to NTP server (10.2.29.26) at stratum 2
   time correct to within 83 ms
   polling server every 64 s

I have modified ntp.conf a little bit. Short & Simple

Code:

[root@browser1 ~]# cat /etc/ntp.conf
server 10.2.29.26 prefer
restrict 127.0.0.1
driftfile /etc/ntp/drift
broadcastdelay  0.008

Just one more small question, just for sake of knowledge:-
In /tmp/ntp.log it is showing as synchronized to stratum 1
while in ntpstat output, it is showing stratum 2.

What is this stratum and why it is showing differently (stratum 1 and stratum 2)

abhijeetdutta · 09-09-2009, 04:44 PM

A server operating at stratum 1 belongs to the class of best NTP servers available, because it has a reference clock attached to it. As accurate reference clocks are expensive, only rather few of these servers are publically available.

A stratum 1 server should not only have a precise and well-maintained and calibrated reference clock, but also should be highly available as other systems may rely on its time service. Maybe that's the reason why not every NTP server with a reference clock is publically available.