I am having trouble figuring out where my delays are SSH'n into my computer.
Is instant password prompt
Takes about 20 seconds for a password prompt. The process hangs here (ssh -vvv user@IP):
debug2: key: /root/.ssh/id_dsa ((nil))
Ha, and as I was writing this I solved my problem
I added this to /etc/ssh/sshd_config
Then restarted sshd.