OK, recently I had to open my firewall a bit to allow more SSH connectivity. SO along with that I beefed up my SSH security. Mandatory PKI being one of them. I was wondering if any of you had some more ideas to make my SSH server more secure, hopefully without adding frustration. But thats more of my
decision. Here's my sshd_config file:

Port 22
Protocol 2
ListenAddress 192.168.0.1
ListenAddress xx.xx.38.237
AllowUsers <me only>
HostKey /etc/ssh/ssh_host_key
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
ServerKeyBits 1024
LoginGraceTime 30
KeyRegenerationInterval 900
PermitRootLogin no
#Compression yes

IgnoreRhosts yes

StrictModes yes
X11Forwarding yes
X11DisplayOffset 0
KeepAlive yes

SyslogFacility AUTH
LogLevel INFO

RhostsAuthentication no
RhostsRSAAuthentication no
#HostbasedAuthentication yes
RSAAuthentication yes
#PubkeyAuthentication yes

PasswordAuthentication no
PermitEmptyPasswords no

#ChallengeResponseAuthentication no


#MaxStartups 10:30:60
Banner /etc/issue.net
VerifyReverseMapping yes

Subsystem sftp /usr/libexec/openssh/sftp-server



tarballedtux

Looking at the sshd_config man page I'd recommend:
As a side note I'd recommend using latest OpenSSH which is at this time 3.5

i increased ssh security by only allowing connections from certain (trusted) hosts. i used iptables for this on my firewall...

I think this is the whole point. Allowing a wider range of hosts to connect, securing SSHD is neccessary more than with a very limited range of hosts.

Basically SSHD should be secured anyways so