Hope this gets you started, there are many different ways of making it more secure.
One way is to limit access, for gentoo I do this;
Limit SSHD Access
Deny access to everyone, including root, except user1 and user2.
AllowUsers user1 user2