First of all please make sure you can login to the box from an unprivileged user account using passphrase and then explicitly set the following options (regardless of your question and regardless of where the box resides):
Protocol 2
PermitRootLogin no
PasswordAuthentication no
If you want an explanation please lookup any threads about SSH in the Linux Security forum.
after about 10 mins of inactivity the remote ssh session will freeze
I have experienced similar. I tried using
Login_keepalived but that didn't help (didn't pursue finding out why), backgrounded pingbacks (no help either). Now I use AutoSSH to keep tunnels alive and added ClientAliveInterval and ClientAliveCountMax to sshd_config which works (check your man sshd_config for ClientAliveInterval and ClientAliveCountMax).