Bridged network, devices using bridge fail until pinged.

exodist · 02-15-2010, 02:21 AM

I am pretty sure the problem here is networking side, not virtualization side. If I am wrong than an op should move this.

My network is as follows
eth0 and eth1, both are nvidia network cards
both interfaces are up, but have no ip address

bond0 has eth0 and eth1 as slaves, no ip address set

br0 is created using bond0, it has 2 networks, my static ip network, as well as my internal network.

Here is what all this looks like:

Code:

bond0     Link encap:Ethernet  HWaddr 00:04:4b:15:5e:8d  
          inet6 addr: fe80::204:4bff:fe15:5e8d/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:40823 errors:0 dropped:0 overruns:0 frame:0
          TX packets:35122 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:2980353 (2.8 MiB)  TX bytes:2648064 (2.5 MiB)

br0       Link encap:Ethernet  HWaddr 00:04:4b:15:5e:8d  
          inet addr:192.168.0.3  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::204:4bff:fe15:5e8d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:38047 errors:0 dropped:0 overruns:0 frame:0
          TX packets:31908 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:2258097 (2.1 MiB)  TX bytes:2290816 (2.1 MiB)

br0:1     Link encap:Ethernet  HWaddr 00:04:4b:15:5e:8d  
          inet addr:75.148.94.148  Bcast:75.148.94.151  Mask:255.255.255.248
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth0      Link encap:Ethernet  HWaddr 00:04:4b:15:5e:8d  
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:20583 errors:0 dropped:0 overruns:0 frame:0
          TX packets:17562 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1501770 (1.4 MiB)  TX bytes:1324897 (1.2 MiB)
          Interrupt:26 Base address:0xc000 

eth1      Link encap:Ethernet  HWaddr 00:04:4b:15:5e:8d  
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:20240 errors:0 dropped:0 overruns:0 frame:0
          TX packets:17560 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1478583 (1.4 MiB)  TX bytes:1323167 (1.2 MiB)
          Interrupt:27 

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
75-148-94-144-O *               255.255.255.248 U     0      0        0 br0
192.168.0.0     *               255.255.255.0   U     0      0        0 br0
loopback        *               255.0.0.0       U     0      0        0 lo
default         75-148-94-150-O 0.0.0.0         UG    1000   0        0 br0

cat /etc/resolv.conf
nameserver 75.148.94.150

Now, my virtual machine simply uses the bridge (I admit to not fully understanding how it does this)

It uses dhcpcd to obtain a lease, this works fine, it obtains an ip address on my internal network (192.168.0.x), the proper route is added (192.168.0.1) and the nameserver (192.168.0.1) is set in the resolv.conf file

However, I cannot ping any ip address on any network, nor can I ping internet addresses or resolve their dns.

If I use another box on my network to ping or ssh to the vm suddenly the vm networking works 100% and it can ping and connect to any external or internal address.

This happens on every vhost, gentoo, ubuntu, freebsd, weather it is a livecd or an already installed image. Since all vm's have the same problem, and there is no configuration for network aside from pointing at a bridge, I am assuming the problem is with my non-vm networking config

exodist · 02-15-2010, 03:19 AM

Google to the rescue:

https://bugzilla.redhat.com/show_bug.cgi?id=487763#c11

This explains the problem, it seems to be a kernel bug. The comment linked to includes a perl script workaround. I have yet to test it.

exodist · 02-15-2010, 04:05 AM

didn't help.

exodist · 02-15-2010, 02:26 PM

For now I have given up. One network interface has my static and internal ip, the other has no ip but is bridged, vm's use that one.

jschiwal · 02-16-2010, 07:08 PM

I haven't used bonding, but for bridging, you want to add interfaces to the bridge and then configure the bridge device with an IP address. The individual NICs don't get their own IP addresses. They are just brought up after bonding.

I think it is the same for bonding. The bond interface (e.g. bond0) should have an IP and routes defined, but the interfaces enslaved should not.
Also, make sure that if you create a VPN tunnel that the bonding is done first. The order is important.
Make sure you don't have configurations for the interfaces making up the bond device that could cause them to be configured before the bond. Looking at some howto's on Google, an interface that has a route defined can cause problems.

exodist · 02-16-2010, 07:16 PM

No vpn.

Take a look at the ifconfig I posted, I did not give the devices addresses, I added them to the bond, I did not give the bond an address, I used it to create the bridge, I then gave my bridge the addresses.

jschiwal · 02-17-2010, 05:19 PM

Your other post said "One network interface has my static and internal ip" which seemed to indicate otherwise, and didn't match the original ifconfig results you posted.

Does the bonded interface work on the master without first being pinged?

exodist · 02-18-2010, 02:50 PM

What I meant was the bond had 2 ip addresses, sorry.

The bonded interface worked fine on the master at all times. The problem only occured within VM's.

At this point I have a working config that just doesn't use bonding. I am no longer able to simply go back to the bonded form for testing to try out any solutions this thread may offer.