Hi,
I've been plagued with the unstable connection to the server in a private network connected to our router, a H3C Quidway AR28-31, recently. When there is connection, I can ping the remote server 10.16.1.6 but not the gateway. It says "Time to live exceeded"
frank@yad:~$ traceroute -n 10.16.1.6
traceroute to 10.16.1.6 (10.16.1.6), 30 hops max, 40 byte packets
1 168.8.8.1 2.967 ms 2.932 ms 2.929 ms
2 192.168.1.1 2.096 ms 2.187 ms 2.335 ms
3 10.16.109.1 14.621 ms 23.025 ms 28.440 ms
4 10.16.255.29 30.425 ms 36.452 ms 40.361 ms
5 10.16.1.6 46.371 ms 50.395 ms 54.468 ms

frank@yad:~$ ping 10.16.1.6
PING 10.16.1.6 (10.16.1.6) 56(84) bytes of data.
64 bytes from 10.16.1.6: icmp_seq=1 ttl=124 time=14.8 ms
64 bytes from 10.16.1.6: icmp_seq=2 ttl=124 time=12.4 ms

frank@yad:~$ ping 10.16.109.1
PING 10.16.109.1 (10.16.109.1) 56(84) bytes of data.
From 10.16.255.29 icmp_seq=1 Time to live exceeded
From 10.16.255.29 icmp_seq=2 Time to live exceeded

I can't explain why the return package is from the next hop 10.16.255.29, and the ttl exceeded stuff. A little Google search says it might be route loop, but traceroute seems ok. Interestingly, if I set the ping ttl to an odd number above 3, then it works, but any even number will raise the same ttl exceeded response.
frank@yad:~$ ping -t 3 10.16.109.1
PING 10.16.109.1 (10.16.109.1) 56(84) bytes of data.
64 bytes from 10.16.109.1: icmp_seq=1 ttl=253 time=14.4 ms
64 bytes from 10.16.109.1: icmp_seq=2 ttl=253 time=14.1 ms

frank@yad:~$ ping -t 4 10.16.109.1
PING 10.16.109.1 (10.16.109.1) 56(84) bytes of data.
From 10.16.255.29 icmp_seq=1 Time to live exceeded
From 10.16.255.29 icmp_seq=2 Time to live exceeded

frank@yad:~$ ping -t 5 10.16.109.1
PING 10.16.109.1 (10.16.109.1) 56(84) bytes of data.
64 bytes from 10.16.109.1: icmp_seq=1 ttl=253 time=15.5 ms
64 bytes from 10.16.109.1: icmp_seq=2 ttl=253 time=14.9 ms

The router maker ask us to connect only a PC to the interface, and overnight ping to the PC works flawlessly without a single lost. The private network technician ask us to connect only a PC too to access their network and the ping to the server 10.16.1.6 works most of the time with only a few lost, which considered normal. If we put them together, the unstable problem repeats. We are stuck in the middle.
Another clue is that if the network drops, a traceroute to the server can always make the connection come back.
Can someone shed a light on this?
Thanks in advance.

Regards,
Frank Wang

I happen to find if the static arp bind of the remote gateway 10.16.109.1 is deleted from the router, the network seems normal again, at least for the past eight hours. I set the bind last year during several arp virus attacks to our network. It had worked fine until this problem emerged. I don't understand why the dynamic arp works, because the learned MAC of the remote gateway is still the same as the static one. Any way, I still need more time to see if it's the real cure.

It happened again tonight with/out the static bind. I just don't get it.

This is due to time to live (TTL) is reaches a zero value from any of the gateways to the destination, try to ping to destination with custom TTL value (higher than default)
Refer this url for fixing
http://servercomputing.blogspot.com/...eded-ping.html

What me wonders in this output is the increase of time from hop 2 to hop 3. Or the hop is so damm buysy, or the link is bad eg speed , half/full duplex , or its makes a detour ,not likely