Routing Question with eth0 private (192.x.x.x) and eth1 public (234.x.x.x)

SteveJenkins · 11-27-2009, 01:24 PM

After googling like a madman and not ever finding the right key words to uncover the correct answer, I'm making this my first post on LQ. I'm not technically a Linux n00b, but routing is like voodoo to me, so I'm appealing to the knowledge of this group.

I've got two CentOS 5.4 servers that share a cabinet. Both have public/external IPs, but they aren't on the same subnet. That means that if I want to transfer data between them, the bandwidth counts against my allotment at my ISP.

So I want to set up their secondary NICs with private IPs in the 192.168.1.x range, connect them to a separate switch, and be able to transfer data between them through the switch, and without going through the public IPs.

So here's what I'm trying to do:

Server A: eth0=192.168.1.101 eth1=222.222.222.101
Server B: eth0=333.333.333.102 eth1=192.168.1.102

I want traffic intended for the 192.x.x.x network to go across the appropriate NICs, and the rest to go out the public IPs.

I've got all the NICs enabled on boot, traffic is flowing nicely out the public IPs. If I'm on Server A and ping its 192 address, it replies. But if I try to ping Server B's 192 address, I get an undeliverable.

What's the right way to make this work? Should I be messing with the ifcfg-eth0 and eth1 files? Or should I be messing with the route?

Thanks in advance for any help!

ddaemonunics · 11-27-2009, 03:27 PM

please post output from ifconfig on both servers.
please post output from route -n on both servers.

it has nothing to do with routing...the network 192.168.1.0 is directly connected so traffic destined to any host on 192.168.1.0 network will be sent by default out to the interface configured with the address from 192.168.1.0 network.
make sure no firewall blocks traffic
make sure the switch that connects the servers is ok

SteveJenkins · 11-27-2009, 09:00 PM

Server A is CentOS 5.4. Server B is Fedora 10.

Server A ifconfig:

Code:

eth0      Link encap:Ethernet  HWaddr 00:06:5B:F3:BE:81  
          inet addr:192.168.1.12  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::206:5bff:fef3:be81/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:38 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:7669 (7.4 KiB)
          Interrupt:193 Memory:fcf10000-fcf20000 

eth1      Link encap:Ethernet  HWaddr 00:06:5B:F3:BE:82  
          inet addr:333.xxx.xx.12  Bcast:333.xxx.xx.255  Mask:255.255.255.0
          inet6 addr: 333::xxx:xx:be82/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:468047 errors:0 dropped:0 overruns:0 frame:0
          TX packets:208802 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:49677714 (47.3 MiB)  TX bytes:18809945 (17.9 MiB)
          Interrupt:201 Memory:fcf00000-fcf10000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:522 errors:0 dropped:0 overruns:0 frame:0
          TX packets:522 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:376870 (368.0 KiB)  TX bytes:376870 (368.0 KiB)

ppp0      Link encap:Point-to-Point Protocol  
          inet addr:192.168.234.235  P-t-P:192.168.234.236  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:5 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3 
          RX bytes:62 (62.0 b)  TX bytes:62 (62.0 b)

Server A route -n:

Code:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.234.236 0.0.0.0         255.255.255.255 UH    0      0        0 ppp0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
333.xxx.xx.0    0.0.0.0         255.255.255.0   U     0      0        0 eth1
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth1
0.0.0.0         333.xxx.xx.1    0.0.0.0         UG    0      0        0 eth1

Server B ifconfig:

Code:

eth0      Link encap:Ethernet  HWaddr 00:22:19:83:42:47  
          inet addr:222.x.x.139  Bcast:222.xxx.xx.255  Mask:255.255.255.0
          inet6 addr: 222::xxx:xx:x:4247/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2823532 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3870972 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:421557728 (402.0 MiB)  TX bytes:278132392 (265.2 MiB)
          Interrupt:16 Memory:da000000-da012700 

eth1      Link encap:Ethernet  HWaddr 00:15:17:91:A5:7C  
          inet addr:192.168.1.139  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Memory:d5ee0000-d5f00000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:843305 errors:0 dropped:0 overruns:0 frame:0
          TX packets:843305 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:282861357 (269.7 MiB)  TX bytes:282861357 (269.7 MiB)

Server B route -n:

Code:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
222.xxx.xx.0    0.0.0.0         255.255.255.0   U     1      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1004   0        0 eth1
0.0.0.0         222.xxx.xx.1    0.0.0.0         UG    0      0        0 eth0

SteveJenkins · 11-28-2009, 09:11 AM

Also, switch works fine, and iptables was turned off during testing (but even so, I can't see anything in the firewall that would block the connections).

Any suggestions?