Server with 2 NIC's on different Networks

plisken · 08-09-2023, 02:12 AM

I've had this issue before and couldn't really resolve it but it's come back to bite me, so...

I have an old Debian Server (also same issue on Slackware as it happens) which has two network cards, configured as below;

Code:

 # The loopback network interface 
 auto lo 
 iface lo inet loopback 
  
 # The primary network interface 
 allow-hotplug eth6 eth7 
 iface eth7 inet static 
              address  23.128.28.20 
              netmask  255.255.240.0 
              network  23.128.16.0 
 #           gateway  23.128.16.1 
   
 iface eth6 inet static 
     address 10.28.10.21 
     netmask 255.255.255.0 
     network 10.28.10.0 
 #          gateway 10.28.10.1

this result in a routing (route -n) table similar to;

Code:

Kernel IP routing table 
 Destination     Gateway         Genmask         Flags Metric Ref    Use Iface 
 10.28.10.0      0.0.0.0         255.255.255.0   U     0      0        0 eth6 
  23.128.16.0     0.0.0.0         255.255.240.0   U     0      0        0 eth7

The idea is that this server can reach either of the 23. or 10. networks by means of eth6 and eth7 respectively and in the above state, traceroutes to IP addresses in those ranges seem to indicate that traffic is indeed going through the correct interfaces. However, I also need to receive traffic from both of those networks, each coming specifically through those interfaces and this I'm not so sure is reliable.
Additionally, there is a specific IP of 10.29.42.103 which resides on the network to which eth6 connects and I am specifically having trouble sending and more so, receiving connectivity from it. I've tried adding a static route with;

Code:

route add 10.29.42.103 dev eth6

which then changes the routing table to something like;

Code:

 Kernel IP routing table
 Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
 10.29.42.103    0.0.0.0         255.255.255.255 UH    0      0        0 eth6
 10.28.10.0      0.0.0.0         255.255.255.0   U     0      0        0 eth6
 23.128.16.0     0.0.0.0         255.255.240.0   U     0      0        0 eth7

This does seem to help but I cant be 100%

Additionally, I have tried adding in the gateways for each of the interfaces (currently commented out above) and this changes the routing table to something like;

Code:

Kernel IP routing table 
 Destination     Gateway         Genmask         Flags Metric Ref    Use Iface 
 10.28.10.0      0.0.0.0         255.255.255.0   U     0      0        0 eth6 
 23.128.16.0     0.0.0.0         255.255.240.0   U     0      0        0 eth7 
 0.0.0.0         23.128.16.1     0.0.0.0         UG    0      0        0 eth7 
  0.0.0.0         10.28.10.1      0.0.0.0         UG    0      0        0 eth6

This also has limited success but I still have possible issues and questions generally related to networking in this context. I feel as though I'm getting confused around the mask terms, as used when setting up an interface and when used in routing;

How exactly should I be setting up my server in order to be able to send and receive traffic from 2 networks as defined above?
I think understand that with networking, if an IP address falls within the network mask then it will not need the gateway to route the traffic, such that with 10.28.10.0 and 255.255.255.0, anything within that mask, will be sent directly, anything outwith that mask, will be sent to the gateway 10.28.16.1 to be routed, is this correct?
Following on from this then in the routing table, the Genmask, is that kind of the opposite from my understanding in point 2 above? Such that in order for traffic to to be routed through a specific interface, then the IP must fall within the Genmask?

Any help would be greatly appreciated.

elgrandeperro · 08-09-2023, 09:29 PM

Hold on. 10.29.42.103 isn't on the same network as 10.28.10.0/24, so no way throwing it out there on the wire is going to work. If there is a gateway, you can forward the /32 to it, or the network (/24) to it. Defining it as default gateway isn't going to work 1/2 the time because you have the same metric on the interfaces. So it is going to round robin, try one interface then the next time the other.

So if you have a gateway to the 10.29.42.0/24 network, you have to define it as such. And that network needs a route back.

wpeckham · 08-09-2023, 10:46 PM

#1 your initial config should work so that each of your subnets can be reached via the interface that lives on that subnet. I am confused about what you want that might require more than that, other than access to the internet. Since you have interfaces ON those subnets you would require a special filter or device to BLOCK you FROM receiving the traffic that lives on those subnets.

#2 To reach the internet you only need add ONE default gateway, and it can be on EITHER of the subnets (you should not need TWO, unless you are configuring a router or edge device). If you DO set two, you either need some special rules to channel traffic for a special purpose, or a trigger for exception conditions or routes. There are tricks one can play with complex configurations, but keeping it simple makes it more reliable.

jefro · 08-10-2023, 07:14 PM

Two thoughts.
One is supernetting.
Two is adding secondary ip to the nic in the desired subnet.