Try it out with something easier to track first, maybe icmp?
Works fine for me on debian, and I see the drop counter go up and the pings drop.
num pkts bytes target prot opt in out source destination
38 18 2476 DROP icmp -- eth1 * x.x.x.x 0.0.0.0/0 statistic mode random probability 0.200000
For a moment there I thought it was buggy... To test with ssh i connected to the dropping machine and then forgot about it and proceeded to test pings again. Ofcourse it didn't drop anything as the packets didn't match the rule and were sourced locally