Every Week for the last half-year, we have had some kind of issue with ping/icmp-related zabbix checks. We couldn't nail it down.
Finally, in the end, I disabled ping checks via zabbix and (not-so-)silently declared icmp checks in zabbix as 'fluff'. Useless.
This has been ok up until this week. We hit our cap Hard and Fast on Wednesday morning.
It took out my zabbix and Kibana dashboards and caused another client's app portal to stop working.
We found...