If you're passing through a NAT device (i.e. your home router?) along the way, then that's likely the cause.
For an OpenSSH client, I solve that problem with the following directive in ~/.ssh/config:
Check your Mac ssh client for some sort of "keep alive" functionality and enable it.