Where you mention 'Network Segments' - you don't mean subnets do you? That setup isn't going to work without proxying the arp packets on each segment, assuming your linux box is the only piece of equipment between A and B.
Otherwise, if you actually do mean subnets, then you need to put each interface into *actual* subnets - 192.168.35.20 and 192.168.35.21 are in the same network - no matter what netmask you use
Try splitting it out into 192.168.35.0/27 and 192.168.35.32/27 for eth0 and eth1 respectively, then just forward between the subnets (with routes as required pointing to the linux box)
What's the topology - is everything connected into the same hub or switch (or a cluster thereof)?