In my experience when I see activity on a dm device I can usually see activity on a corresponding native disk. In this case I'm not and can't help but wonder if it's just a bug in the hypervisor this VM is running on? dm-4 is continuously 99% utilized.

collectl -sD -c1
waiting for 1 second sample...

# DISK STATISTICS (/sec)
# <---------reads---------------><---------writes--------------><--------averages--------> Pct
#Name KBytes Merged IOs Size Wait KBytes Merged IOs Size Wait RWSize QLen Wait SvcTim Util
vda 0 0 0 0 0 12 1 2 6 0 6 0 0 0 0
vdb 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
vdc 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
vdd 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
vde 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
vdf 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
vdg 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
dm-0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
dm-1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
dm-2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
dm-3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
dm-4 0 0 0 0 0 0 0 0 0 0 0 3 0 0 99
dm-5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

There are also 4 CPU and 2 are pegged at 100% iowait. I've never seen iowait high when there was no block i/o.

collectl -sC -c1
waiting for 1 second sample...

# SINGLE CPU STATISTICS
# Cpu User Nice Sys Wait IRQ Soft Steal Guest NiceG Idle
0 0 0 0 100 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0 100
2 0 0 0 100 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0 0 100

I think I found the problem. Thanks to collectl -sm --verbose I found 3 process were blocked waiting on I/O. More digging found they were all trying to access openstack cinder and were in uninterruptable sleeps. More digging found LVM hung!