@jonaskellens: Just be careful with this. From the hosts_access(5) manpages:
The access control language supports explicit wildcards:
Matches any host whose name does not match its address. When
tcpd is built with -DPARANOID (default mode), it drops requests
from such clients even before looking at the access control
tables. Build without -DPARANOID when you want more control
over such requests.
Thus, if your installation has this default enabled, your scheme is not going to work. Be sure to test this out before relying on certain behaviour (and potentially locking yourself out).
The suggestion to lock things down to an ISP-issued IP range is fine (if you can determine that range correctly). You could take other steps to harden sshd -- e.g. using pubkey authentication.