I have two hosts that are on different subnets but on the same physical network. Host 1 broadcasts a UDP message that includes it's IP address. Host 2 gets this message and attempts to establish a TCP connection the Host 1. The correct port is known to both hosts. I see the traffic with a sniffer. Host 2 does send a request to host 1's address, with its address as the source. Host 1 never responds to the initial connection request from host 2.

This all works as long as both hosts are on the same subnet. For instance if both are on 192.168.0.0/24 all works. If one is on 192.168.5.0/24 and the other on 192.168.0.0/24 it fails.

If it makes any difference, both hosts are Linux on a Raspberry Pi.

Is it possible to make this work?

One might want to read up on the meanings and functions of the words subnet, subnet mask, router, and perhaps bridge just as a start. You might also want to consider how easy it is to give an interface addresses in two different subnets on the same host, and why one might WANT to do that.

Giving host2 and/or host1 an address in the other respective subnet seems to be the easiest solution.

Well, I was hoping to find a method to avoid pre-assigning a static address. Most of the places that this will be installed there will be no DHCP server. I'm not hearing that it is possible to establish a TCP connection across subnet lines, so assignment it will have to be. Someone will be managing all the addresses on the subnet, so you can't just arbitrarily pick an unused address.

Not the answer I was looking for, but a good one nevertheless.

If you don't want to give Host A an IP address that is in the same subnet as Host B, you need a router between the subnets.

Either "super" net or fix subnet.

Or use static arp entry. Be careful with static arp.