Seeing differences between ping and nping stats

Shaggy1 · 07-06-2020, 06:29 AM

system info:
cat /etc/centos-release
CentOS Linux release 7.5.1804 (Core)

Linux lb-cam-bca-centos7devel 3.10.0-862.11.6.el7.x86_64 #1 SMP Tue Aug 14 21:49:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
ping -V
ping utility, iputils-s20160308
[root@lb-cam-bca-centos7devel infping]# nping -V

Nping version 0.6.40 ( http://nmap.org/nping )
[root@lb-cam-bca-centos7devel infping]#

I've recently been using ping and nping to monitor some latency issues and have noticed that there seems to be a discrepancy in the output between the two, specifically with regards to min rtt.

The actual reason I use nping is so I can use the --tcp option to specifically monitor tcp trip times and initially thought the discrepancy was to do with using tcp, but I see it even when removing the use of that option.

When do a ping I see something like the following:
ping -c 10 remotehost
PING remotehost.net (10.110.48.94) 56(84) bytes of data.
64 bytes from remotehost.net (10.110.48.94): icmp_seq=1 ttl=54 time=110 ms
64 bytes from remotehost.net (10.110.48.94): icmp_seq=2 ttl=54 time=10.3 ms
64 bytes from remotehost.net (10.110.48.94): icmp_seq=3 ttl=54 time=10.3 ms
64 bytes from remotehost.net (10.110.48.94): icmp_seq=4 ttl=54 time=26.8 ms
64 bytes from remotehost.net (10.110.48.94): icmp_seq=5 ttl=54 time=59.8 ms
64 bytes from remotehost.net (10.110.48.94): icmp_seq=6 ttl=54 time=47.5 ms
64 bytes from remotehost.net (10.110.48.94): icmp_seq=7 ttl=54 time=64.7 ms
64 bytes from remotehost.net (10.110.48.94): icmp_seq=8 ttl=54 time=39.0 ms
64 bytes from remotehost.net (10.110.48.94): icmp_seq=9 ttl=54 time=57.0 ms
64 bytes from remotehost.net (10.110.48.94): icmp_seq=10 ttl=54 time=73.2 ms

--- remotehost.net ping statistics ---
10 packets transmitted, 10 received, 0% packet loss, time 9004ms
rtt min/avg/max/mdev = 10.323/49.928/110.192/28.873 ms

Because there are issues with this connection the reported values vary widely, but generally the min rtt is consistent with the individusl values and also with the output I see using mtr.

However when using nping I consistently get output looking something like this:
nping -c 10 remotehost

Starting Nping 0.6.40 ( http://nmap.org/nping ) at 2020-07-06 12:11 BST
SENT (0.0226s) ICMP [10.168.6.115 > 10.110.48.94 Echo request (type=8/code=0) id=62976 seq=1] IP [ttl=64 id=35732 iplen=28 ]
RCVD (0.0328s) ICMP [10.110.48.94 > 10.168.6.115 Echo reply (type=0/code=0) id=62976 seq=1] IP [ttl=54 id=55802 iplen=28 ]
SENT (1.0234s) ICMP [10.168.6.115 > 10.110.48.94 Echo request (type=8/code=0) id=62976 seq=2] IP [ttl=64 id=35732 iplen=28 ]
RCVD (1.0336s) ICMP [10.110.48.94 > 10.168.6.115 Echo reply (type=0/code=0) id=62976 seq=2] IP [ttl=54 id=55804 iplen=28 ]
SENT (2.0240s) ICMP [10.168.6.115 > 10.110.48.94 Echo request (type=8/code=0) id=62976 seq=3] IP [ttl=64 id=35732 iplen=28 ]
SENT (3.0244s) ICMP [10.168.6.115 > 10.110.48.94 Echo request (type=8/code=0) id=62976 seq=4] IP [ttl=64 id=35732 iplen=28 ]
SENT (4.0250s) ICMP [10.168.6.115 > 10.110.48.94 Echo request (type=8/code=0) id=62976 seq=5] IP [ttl=64 id=35732 iplen=28 ]
SENT (5.0258s) ICMP [10.168.6.115 > 10.110.48.94 Echo request (type=8/code=0) id=62976 seq=6] IP [ttl=64 id=35732 iplen=28 ]
RCVD (5.0259s) ICMP [10.110.48.94 > 10.168.6.115 Echo reply (type=0/code=0) id=62976 seq=3] IP [ttl=54 id=55806 iplen=28 ]
SENT (6.0268s) ICMP [10.168.6.115 > 10.110.48.94 Echo request (type=8/code=0) id=62976 seq=7] IP [ttl=64 id=35732 iplen=28 ]
SENT (7.0276s) ICMP [10.168.6.115 > 10.110.48.94 Echo request (type=8/code=0) id=62976 seq=8] IP [ttl=64 id=35732 iplen=28 ]
SENT (8.0287s) ICMP [10.168.6.115 > 10.110.48.94 Echo request (type=8/code=0) id=62976 seq=9] IP [ttl=64 id=35732 iplen=28 ]
RCVD (8.0288s) ICMP [10.110.48.94 > 10.168.6.115 Echo reply (type=0/code=0) id=62976 seq=4] IP [ttl=54 id=55808 iplen=28 ]
SENT (9.0336s) ICMP [10.168.6.115 > 10.110.48.94 Echo request (type=8/code=0) id=62976 seq=10] IP [ttl=64 id=35732 iplen=28 ]

Max rtt: 10.177ms | Min rtt: 0.042ms | Avg rtt: 5.104ms
Raw packets sent: 10 (280B) | Rcvd: 4 (184B) | Lost: 6 (60.00%)
Nping done: 1 IP address pinged in 9.03 seconds

This is indicating packet loss where (usually) ping does not and the reported min rtt is consistently very low (<0.1) which is clearly (to my mind) incorrect.

As I understood it both ping an nping use icmp by default and I should be getting similar output. Is that not the case ? Have I mis-understood what nping is doing ?

Does anyone have an explanation for the differences I am seeing here ?

Shaggy1 · 07-06-2020, 11:19 AM

Some additional info:
I did some quick tests pinging to a machine on the local sub-net, since we are having network latency issues that could be causing something strange, but there still seems to be a discrepancy between ping an nping.

ping -c 100 -i 0.1 10.168.6.126 # done ten times
...
100 packets transmitted, 100 received, 0% packet loss, time 9928ms
rtt min/avg/max/mdev = 0.142/0.256/3.982/0.378 ms

100 packets transmitted, 100 received, 0% packet loss, time 9926ms
rtt min/avg/max/mdev = 0.120/0.234/0.545/0.062 ms

100 packets transmitted, 100 received, 0% packet loss, time 9926ms
rtt min/avg/max/mdev = 0.139/0.225/0.572/0.063 ms

100 packets transmitted, 100 received, 0% packet loss, time 9925ms
rtt min/avg/max/mdev = 0.108/0.202/0.890/0.082 ms

100 packets transmitted, 100 received, 0% packet loss, time 9908ms
rtt min/avg/max/mdev = 0.100/0.213/0.623/0.071 ms

100 packets transmitted, 100 received, 0% packet loss, time 9928ms
rtt min/avg/max/mdev = 0.134/0.262/4.433/0.422 ms

100 packets transmitted, 100 received, 0% packet loss, time 9912ms
rtt min/avg/max/mdev = 0.098/0.201/0.516/0.048 ms

100 packets transmitted, 100 received, 0% packet loss, time 9952ms
rtt min/avg/max/mdev = 0.121/0.606/9.890/1.615 ms

100 packets transmitted, 100 received, 0% packet loss, time 9909ms
rtt min/avg/max/mdev = 0.136/0.217/0.394/0.046 ms

100 packets transmitted, 100 received, 0% packet loss, time 9920ms
rtt min/avg/max/mdev = 0.105/0.317/3.826/0.569 ms

nping -c 100 --rate 10 10.168.6.126 # done ten times
...
Max rtt: 14.412ms | Min rtt: 0.033ms | Avg rtt: 1.433ms
Raw packets sent: 100 (2.800KB) | Rcvd: 99 (4.554KB) | Lost: 1 (1.00%)
Nping done: 1 IP address pinged in 9.92 seconds

Max rtt: 10.952ms | Min rtt: 0.031ms | Avg rtt: 0.543ms
Raw packets sent: 100 (2.800KB) | Rcvd: 99 (4.554KB) | Lost: 1 (1.00%)
Nping done: 1 IP address pinged in 9.97 seconds

Max rtt: 11.317ms | Min rtt: 0.033ms | Avg rtt: 0.517ms
Raw packets sent: 100 (2.800KB) | Rcvd: 99 (4.554KB) | Lost: 1 (1.00%)
Nping done: 1 IP address pinged in 9.94 seconds

Max rtt: 11.162ms | Min rtt: 0.032ms | Avg rtt: 0.400ms
Raw packets sent: 100 (2.800KB) | Rcvd: 100 (4.600KB) | Lost: 0 (0.00%)
Nping done: 1 IP address pinged in 9.93 seconds

Max rtt: 10.970ms | Min rtt: 0.033ms | Avg rtt: 1.012ms
Raw packets sent: 100 (2.800KB) | Rcvd: 99 (4.554KB) | Lost: 1 (1.00%)
Nping done: 1 IP address pinged in 9.97 seconds

Max rtt: 10.897ms | Min rtt: 0.033ms | Avg rtt: 0.504ms
Raw packets sent: 100 (2.800KB) | Rcvd: 97 (4.462KB) | Lost: 3 (3.00%)
Nping done: 1 IP address pinged in 9.93 seconds

Max rtt: 0.660ms | Min rtt: 0.032ms | Avg rtt: 0.122ms
Raw packets sent: 100 (2.800KB) | Rcvd: 95 (4.370KB) | Lost: 5 (5.00%)
Nping done: 1 IP address pinged in 9.92 seconds

Max rtt: 10.278ms | Min rtt: 0.031ms | Avg rtt: 0.635ms
Raw packets sent: 100 (2.800KB) | Rcvd: 99 (4.554KB) | Lost: 1 (1.00%)
Nping done: 1 IP address pinged in 9.93 seconds

Max rtt: 10.914ms | Min rtt: 0.032ms | Avg rtt: 0.446ms
Raw packets sent: 100 (2.800KB) | Rcvd: 100 (4.600KB) | Lost: 0 (0.00%)
Nping done: 1 IP address pinged in 9.92 seconds

Max rtt: 11.039ms | Min rtt: 0.032ms | Avg rtt: 0.522ms
Raw packets sent: 100 (2.800KB) | Rcvd: 100 (4.600KB) | Lost: 0 (0.00%)
Nping done: 1 IP address pinged in 9.99 seconds

Shaggy1 · 07-07-2020, 06:24 AM

Another update:
nping looks to be working fine on another of my machines with centos 6:

$ cat /etc/centos-release
CentOS release 6.8 (Final)

$ nping -V
Nping version 0.5.51 ( http://nmap.org/nping )

I noticed an Nmap developers thread "nping appears to miss or drop a packet when run with newer kernel/libraries":
https://seclists.org/nmap-dev/2014/q1/57
which seems to indicate there have bee problems seen with nping 0.6.40 with certain kernels.

Does anyone know anything about that ?

Shaggy1 · 07-07-2020, 07:38 AM

Further update:
I tried downloading and building the latest version of nmap (7.80) and using the nping from that, but it's still showing packet loss and (probably as a result) incorrect stats.
Guess there must be some incompatability with the kernel.