Originally Posted by Ser Olmy
For a host to add an entry to its ARP table, it must receive an ARP Reply packet from another host. And for that to happen, there has to be a Layer 2 connection between the hosts. Perhaps the router has a built-in switch and has two IP addresses assigned to the same Layer 3 interface, what is commonly known as a "router on a stick" setup?
Somehow it seems the hosts in question are sharing the same physical network infrastructure. It is entirely possible to have two logical (layer 3) networks (for instance, 192.168.1.0/24 and 192.168.2.0/24) share the same Layer 2 (typically Ethernet) network infrastructure. This will not cause routing issues in and by itself, but hosts on either network are bound to pick up ARP replies from the other network.
If you run tcpdump -i eth0 on either host, you should be able to see whether the host is picking up traffic (broadcasts) from the opposite network. traceroute will tell you whether packets to hosts on the other network are sent via the gateway router or not.
All three machines are standard Fedora/Centos setups, the router has two interfaces, one on eth1 facing the
outside world, and eth0 facing the inside world.
eth0 has 192.168.1.1/24 and 192.168.2.1/24 assigned to it.
machines A and B are connected via a standard dell switch to each other and the router. There are no vlan's on
the switch except the default vlan 1.
Machine A is 192.168.1.3/24
Machine B is 192.168.2.3/24
Both A and B hear each other's arp requests as they are broadcasts. However they should not be hearing
each other's replies as they are unicast through the switch. Each machine maybe offering gratuitous arps as
broadcasts and thus learning about each other's mac addresses.
However A has no business putting mac addresses in its arp table for B's subnet 192.168.2.x/24
traceroute shows that A talks to B via the router.
However tcpdump shows that pings sometimes go through the router both going out and coming back,
and sometimes one goes through the router and the other path goes directly, and sometimes both paths
are direct between A and B.
Is this a 'feature' or a bug?
Homer W. Smith
CEO Lightlink Internet