ssh remote host identification has changed

Arzach · 10-14-2016, 08:25 AM

This is the situation: in my working place I have a small LAN. Here I have the firewall (pfsense), a file-server (debian 8) a workstation (debian 8), some other win PC. All the machine has a static IP. From my laptop (debian 8), I use static IP too, I need to connect to the file-server and the workstation via ssh. So I generated the keys (ssh-keygen -t dsa) and I copied the public key into both machines (ssh-copy-id -i .ssh/id_dsa.pub). When I try to connect to the file-server (ssh server) all goes right. When I try to connect to the workstation (ssh workstation) I get this:

Code:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX.
Please contact your system administrator.
Add correct host key in /home/r/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /home/r/.ssh/known_hosts:22
remove with: ssh-keygen -f "/home/r/.ssh/known_hosts" -R 10.1.1.20
ECDSA host key for 10.1.1.20 has changed and you have requested strict checking.
Host key verification failed.

I tried to remove the key as suggested (ssh-keygen -f "/home/r/.ssh/known_hosts" -R 10.1.1.20) but when I try to connect to the workstation again the shell asks for the password but even if I type the right password the login is rejected. So I have to login the workstation machine locally, remove the content of the .ssh/ folder, and restart the ssh service (systemctl restart ssh.service) to have the possibility to connect using ssh. Each time I disconnect/reconnect I get the same issue.

Do you have any suggestion?

Turbocapitalist · 10-14-2016, 01:29 PM

Quote:

Originally Posted by Arzach

This is the situation: in my working place I have a small LAN. Here I have the firewall (pfsense), a file-server (debian 8) a workstation (debian 8), some other win PC. All the machine has a static IP. From my laptop (debian 8), I use static IP too, I need to connect to the file-server and the workstation via ssh. So I generated the keys (ssh-keygen -t dsa)

DSA keys are deprecated. RSA or Ed25119 would be appropriate. You can dig around for references if you like.

Quote:

Originally Posted by Arzach

The fingerprint for the ECDSA key sent by the remote host is
XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX.

Was your remote machine upgraded recently? If so, maybe the keys were replaced accidentally. Either way, it is important to get to the bottom of it.

Over on the remote host, does the key fingerprint there match what you are getting from your client? Someone has to run the following from the remote machine's console:

Code:

ssh-keygen -lf /etc/ssh/ssh_host_ecdsa_key.pub

They can read it to you over the phone or send it as SMS or in e-mail or something else out of band.

Then compare that fingerprint with the one you were getting above when you try to connect. If they match, you are ok. If they do not match, contact your network administrator ASAP because that means you are trying to log in to a machine other than the one you want to.

Elizine · 10-15-2016, 06:12 AM

use

Code:

ssh-keygen -R hostname

example ssh-keygen -R 168.9.9.2

This will update the offending of your host from the known_hosts

wpeckham · 10-15-2016, 09:02 AM

Quote:

Originally Posted by Elizine

use

Code:

ssh-keygen -R hostname

example ssh-keygen -R 168.9.9.2

This will update the offending of your host from the known_hosts

To pick an important nit, no it will not. It will remove the OLD key.
You will have to accept the NEW key on your next manual connect to update known_hosts with the new key.

Turbocapitalist · 10-15-2016, 09:16 AM

Quote:

Originally Posted by Elizine

example ssh-keygen -R 168.9.9.2

This will update the offending of your host from the known_hosts

Yes, and Arzach wrote that was tried already in the original post. But given the other information, specifically that the server's key's fingerprint does not match, it is possibly exceedingly unsafe to override the warnings to try connecting anyway. If the fingerprints don't match, it is very important^Wessential to find out why before proceeding. The network administrator can probably help with that and would need to be notified anyway if a man-in-the-middle attack is being carried out.

A further indication of such an attack is happening is not being able to log in with valid credentials, like what is described above in the original post. However, that's not something you want to try with password authentication because it gives the password to the attacking machine, if it is set up to harvest login attempts.

Key-based authentication won't work either obviously but that is safe because no secret data is leaving the client. The key is just used to exchange messages with the server in such a way as to prove to the server that the private key is in possession of the client.

Arzach · 10-20-2016, 09:34 AM

Quote:

The network administrator can probably help with that and would need to be notified anyway if a man-in-the-middle attack is being carried out.

I am the sysadmin and I'm pretty sure no man-in-the-middle attack has been done.

Quote:

Over on the remote host, does the key fingerprint there match what you are getting from your client? Someone has to run the following from the remote machine's console:

Code:

ssh-keygen -lf /etc/ssh/ssh_host_ecdsa_key.pub

No they are different: from the host I get 49:0e....., while from ssh worning I get dd:44...

Yesterday I removed the .ssh folder in the home directory of the workstation and rebooted the system. I send the pub key again and than I was able to log-in. This morning the WARNING again...

I found a strange thing: the problem occur each time I close the lid (hybernate) my laptop. When I try to login to the remote machine I get the warning message

Turbocapitalist · 10-20-2016, 09:40 AM

Quote:

Originally Posted by Arzach

No they are different: from the host I get 49:0e....., while from ssh worning I get dd:44...

Since you are sysadmin, when you do log in (from the console) to the remote machine, do any of the public keys in /etc/ssh/ have a fingerprint that matches?

If not, then the main remaining option is that you are not connecting to the machine you intend to connect to...

What does the network administrator have to say about things?

Arzach · 10-20-2016, 10:31 AM

Quote:

do any of the public keys in /etc/ssh/ have a fingerprint that matches?

None.

I controlled all the address in the network and they seems ok. I verified the key in /etc/ssh on both the remote machine I'm able to connect and the remote machine I'm not able to connect.

From the remote machine that gives to me the error I get:

Code:

The authenticity of host '10.1.1.20 (10.1.1.20)' can't be established.
ECDSA key fingerprint is dd:44:fa...
Are you sure you want to continue connecting (yes/no)?

I controled all the key in /etc/ssh and there is not a key with the value above.

From the machine which accept remote connection I get:

Code:

ssh-keygen -lf /etc/ssh/ssh_host_ecdsa_key.pub 
256 dd:44:fa... /etc/ssh/ssh_host_ecdsa_key.pub (ECDSA)

The same key. I'm confused...

Turbocapitalist · 10-20-2016, 10:41 AM

The machine that accepts the remote connection is where to check. You've done that and the key matches and on the occasions that it does that would be a sign that you are connecting to the right machine. On the occasions that it does not, that would be a sign that you are not connecting to the right machine but instead connecting elsewhere. I say when that happens you need the help of the network administrator to track your connection and see where it is really going.

It's possible for someone who has gotten inside your network to redirect your connection to a machine of their choice. (ARP-spoofing is one way to do that.) But that's only one possibility, so when the client machine says that the keys are mismatched, you need to check the network itself not either of the two end machines.