I am running several servers with multiple LAN ports.

Two of the machines can communicate via two separate subnets to control bandwidth issues.

Machine 1: 10.0.0.1 (w/ additional alias 12.0.0.1)
192.168.1.1
Machine 2: 10.0.0.2
192.168.1.2

I see three separate intermittent symptoms on the 10.0.0.0 network.
1) Both machines will cut off in the middle of the data transfer (rsync) if it takes too long.
2) Both machines will claim that the RSA key host has changed on the other...this will happen every few minutes.
3) Both machines will, at times, disallow login from the other...ssh prompts for password but will not accept the password.

Performing any operation between the same machines via the 192.168.1.0 subnet has no issue (as of yet).

From what I can see, the routing tables are set correctly. Machines were exact clones of each other. Machine 2 is a recently rebuilt machine with a fresh suse11 distro install. Symptoms on Machine 2 appeared immediately. I had this exact setup but reversed ip addresses before Machine 2 burned out without issue. All networks but the 12.0.0.1 are internal and there hasn't been any indication of attack.

At times, running ssh-keygen -R <ipaddress> will fix issues 1-3, other times only 1-2.

Thanks for any help,
Paul

Are you confident about the security of your network? It almost sounds like what you might see during a poorly implemented MITM attack. It could be a stretch, but you might want to check your ARP tables to make sure nothing odd is going on there.

I don't see anything abnormal in the arp list...every ip address/hostname is accounted for.

If this is a MITM attack, what would I need to do? Would a MITM attack account for my inability to login via the ethernet device?

At least a temp fix, I switched the relevant ip address onto a different subnet. So far so good.

Thanks.