I think that "DenyGroups
denyusergroup@!yourdomain.com" might do it. You will need to test this.
The "denyusergroup" name is an arbitrary example. I'm not certain is group@host is allowed. It might be better to instead use "AllowUsers" instead if the number of users isn't too large.
"AllowUsers john@10.0.0.0/8 sally@10.0.0.0/8 mike tim sally"
This will allow john and sally on the local 10/8 LAN, and mike, tim, & sally from anywhere. AllowUsers will also deny system user logins.
The username@host pattern refers to a local user (on the server) connecting from a remote host. It doesn't refer to a user at that host, unless that users username is the same by coincidence.