In your sshd_config file:
1) PermitRootLogin no
2) AllowUsers list_of_users_who_can_ssh
3)Use PublicKey authentication and disable pass auth.
PasswordAuthentication no
PermitEmptyPasswords no
4) StrictModes yes
5) Use Only protocol 2
6) Use iptables to restrict the IPs from where somebody can ssh to your server
7) Eventually run ssh on a port other than 22
5)You can also use port knocking (
www.portknocking.org )