[SOLVED] SSH key being ignored

Hello,
I need to be able to connect to another computer with SSH keys so that a password is not needed. This is because the BackupPC software does not allow me to enter a password. Both are on CentOS 6.
I followed this guide exactly:
http://wiki.centos.org/HowTos/Network/SecuringSSH

The backuppc user on the backup server needs to connect to the root user on the test server. I copied the public key from the backup server to the test server, then added it to the authorized_users file as root. Opening the public key file does show that it was generated by the right user: backuppc.

I've had issues with this before but managed to get around it one way or another. I'm not sure what I should do this time. Thank you all for your help.

Here is the verbose output:

-bash-4.1$ ssh -v root@(IP Address) -p(Port)
OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to (IP Address) [(IP Address)] port (Port).
debug1: Connection established.
debug1: identity file /var/lib/BackupPC/.ssh/identity type -1
debug1: identity file /var/lib/BackupPC/.ssh/identity-cert type -1
debug1: identity file /var/lib/BackupPC/.ssh/id_rsa type 1
debug1: identity file /var/lib/BackupPC/.ssh/id_rsa-cert type -1
debug1: identity file /var/lib/BackupPC/.ssh/id_dsa type -1
debug1: identity file /var/lib/BackupPC/.ssh/id_dsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '[(IP Address)]:(Port)' is known and matches the RSA host key.
debug1: Found key in /var/lib/BackupPC/.ssh/known_hosts:3
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mi
c,password
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_498' not found

debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_498' not found

debug1: Unspecified GSS failure.  Minor code may provide more information


debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_498' not found

debug1: Next authentication method: publickey
debug1: Trying private key: /var/lib/BackupPC/.ssh/identity
debug1: Offering public key: /var/lib/BackupPC/.ssh/id_rsa
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mi
c,password
debug1: Trying private key: /var/lib/BackupPC/.ssh/id_dsa
debug1: Next authentication method: password
root@(IP Address)'s password:

-bash-4.1$ ssh -vv root@(IP Address) -p(Port)
OpenSSH_5.3p1, OpenSSL 1.0.1e-fips 11 Feb 2013
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to (IP Address) [(IP Address)] port (Port).
debug1: Connection established.
debug1: identity file /var/lib/BackupPC/.ssh/identity type -1
debug1: identity file /var/lib/BackupPC/.ssh/identity-cert type -1
debug2: key_type_from_name: unknown key type '-----BEGIN'
debug2: key_type_from_name: unknown key type '-----END'
debug1: identity file /var/lib/BackupPC/.ssh/id_rsa type 1
debug1: identity file /var/lib/BackupPC/.ssh/id_rsa-cert type -1
debug1: identity file /var/lib/BackupPC/.ssh/id_dsa type -1
debug1: identity file /var/lib/BackupPC/.ssh/id_dsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug2: fd 3 setting O_NONBLOCK
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-g
roup-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh
.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour12
8,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rij
ndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour12
8,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rij
ndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,
hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,
hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-g
roup-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour12
8,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rij
ndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour12
8,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rij
ndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,
hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,
hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_setup: found hmac-md5
debug1: kex: server->client aes128-ctr hmac-md5 none
debug2: mac_setup: found hmac-md5
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug2: dh_gen_key: priv key bits set: 147/256
debug2: bits set: 511/1024
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host '[(IP Address)]:(Port)' is known and matches the RSA host key.
debug1: Found key in /var/lib/BackupPC/.ssh/known_hosts:3
debug2: bits set: 509/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /var/lib/BackupPC/.ssh/identity ((nil))
debug2: key: /var/lib/BackupPC/.ssh/id_rsa (0x7fc43bebe6e0)
debug2: key: /var/lib/BackupPC/.ssh/id_dsa ((nil))
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mi
c,password
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug2: we did not send a packet, disable method
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_498' not found

debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_498' not found

debug1: Unspecified GSS failure.  Minor code may provide more information


debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_498' not found

debug2: we did not send a packet, disable method
debug1: Next authentication method: publickey
debug1: Trying private key: /var/lib/BackupPC/.ssh/identity
debug1: Offering public key: /var/lib/BackupPC/.ssh/id_rsa
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mi
c,password
debug1: Trying private key: /var/lib/BackupPC/.ssh/id_dsa
debug2: we did not send a packet, disable method
debug1: Next authentication method: password
root@(IP Address)'s password:

Just to clarify:

Home directory for BackupPC user is /var/lib/BackupPC/.ssh?

That home directory contains id_rsa.pub (and id_rsa) and/or id_dsa.pub (and id_dsa)?

What is the home directory for root on the target server? (Usually /root).
Does $HOME/.ssh exist for root on the target server?

You copied the contents of either id_rsa.pub or id_dsa.pub into root's $HOME/.ssh/authorized_keys file on the target server?

Do the permissions on /var/lib/BackupPC on source server and root's $HOME on target server only allow the respective user on each to modify the directory? (That is no write permission for group or other.)

Do the permissions on /var/lib/BackupPC/.ssh on source server and root's $HOME/.ssh on the target server only allow read/write by the respective users on each? (That is no permissions at all for group or other.)

The most common reason for ssh trusts to fail is because the source user or the target user is not actually viewed as "secure" by ssh so the first thing I usually check are the last 2 things I mentioned.

You mentioned you can't login with a password. Did you mean you "shouln't" or did you really mean you "can't" but you know the password. If the latter then it may be whatever prevents password logins also prevents trusts. (e.g. if you had root logins restricted to the console.)

Originally Posted by MensaWater Do the permissions on /var/lib/BackupPC on source server and root's $HOME on target server only allow the respective user on each to modify the directory? (That is no write permission for group or other.) Do the permissions on /var/lib/BackupPC/.ssh on source server and root's $HOME/.ssh on the target server only allow read/write by the respective users on each? (That is no permissions at all for group or other.) The most common reason for ssh trusts to fail is because the source user or the target user is not actually viewed as "secure" by ssh so the first thing I usually check are the last 2 things I mentioned.

Agreed

The following commands (run on the destination machine) will set the permissions correctly for ssh to use the authorized_keys file:

chmod 750 ~
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

I usually don't have a problem with the permissions on the originating machine.

Originally Posted by MensaWater Just to clarify: Home directory for BackupPC user is /var/lib/BackupPC/.ssh? That home directory contains id_rsa.pub (and id_rsa) and/or id_dsa.pub (and id_dsa)?

Yes

What is the home directory for root on the target server? (Usually /root).

Yes, that's it.

Does $HOME/.ssh exist for root on the target server?

It didn't at first but I created it.

You copied the contents of either id_rsa.pub or id_dsa.pub into root's $HOME/.ssh/authorized_keys file on the target server?

Yes

Do the permissions on /var/lib/BackupPC on source server and root's $HOME on target server only allow the respective user on each to modify the directory? (That is no write permission for group or other.) Do the permissions on /var/lib/BackupPC/.ssh on source server and root's $HOME/.ssh on the target server only allow read/write by the respective users on each? (That is no permissions at all for group or other.)

Permissions are at 700 for the .ssh folder as a whole. 600 for the authorized_keys and id_rsa files.

You mentioned you can't login with a password. Did you mean you "shouln't" or did you really mean you "can't" but you know the password. If the latter then it may be whatever prevents password logins also prevents trusts. (e.g. if you had root logins restricted to the console.)

I do know the password but the BackupPC software does not have an option for entering a password.

Does $HOME/.ssh exist for root on the target server?

It didn't at first but I created it.

You copied the contents of either id_rsa.pub or id_dsa.pub into root's $HOME/.ssh/authorized_keys file on the target server?

Yes

Those two responses seem contradictory. If the /root/.ssh didn't exist until you created it then you couldn't have updated authorized_keys there until after you created it. Are you saying you did the creation and copy after I asked or before?

Permissions are at 700 for the .ssh folder as a whole. 600 for the authorized_keys and id_rsa files.

You didn't mention the permissions on the parent directory that I'd asked about (the one above .ssh). Having a wide open parent directory breaks trust as anyone could then modify the permissions of the .ssh subdirectory.

Also you have to be sure of permissions for /root and /root.ssh on the target server.

I ended up just having to regenerate the keys and re-do the pairing. It works now. I think I used the wrong public key file before.