It's also possible that you aren't listed as a known user in Server #1. This depends on how you authenticate with the server. ( E.g. passwordless logins with your public key being listed in known_users)
Also check the /etc/ssh/sshd_config file for options that may deny you access. Perhaps it uses "AllowUsers" and you aren't listed.
The entries can have the form user@host, where "user" the local user on the server, and host is the remote host you are connecting from. If the host is different, or the hostname can't be resolved via DNS or /etc/hosts, you can be denied that way as well.
If you can log in directly to server 1, or indirectly via server 2, also read the logs (/var/logs/messages or /var/logs/security) for information on the login attempt.
Also, are you trying to login under your username or as root. It is common (and a good practice) to deny root login attempts.