Our code runs on a ubuntu 16.04 VM (under HyperV of Windows Server 2016). We have periodic timer to print trace information every 30 seconds. Each trace is composed of a timestamp retrieved via clock_gettime(MONOTONIC), the process id, thread id, and the message. For some messages, we also print the wall clock using gettimeofday(). There was an occurrence that clock_gettime(MONOTONIC) apparently lagged for 13 second, while gettimeofday() was still getting the correct time.

We are pretty sure that the gettimeofday() is correctly representing the real world time: Since we are also tracing the network-receive from another machine who sends packets every 15 second; during the above abnormal period, the two timestamps of the message-receive traces showed an time interval of 2 seconds only.

Is there any theory explaining this, or what could possibly cause it, say, the host is suspending the VM or the host is overloaded? Our code heavily depends on the correctness of MONOTONIC, so needs to find the cause and work around.

I understand that in kernel, gettimeofday() is just another layer on top of MONOTONIC, and assume they should be always consistent.

When you say "periodic timer to print traceinfo", what clock is that based on?

We used timer_create(CLOCK_MONOTONIC..), and real-time signal for delivering timer expiration.