| krkishore |
05-29-2013 03:34 PM |
Problems getting guest to connect to network on second bridge
I am having trouble enabling accessing the network from the guest VMs on one of the two NICs. Here is my setup
PHP Code:
Server1 Server2
+-----+ +-----+ +-----+ +-----+
vm1--| br0 |<--->| p4p1|<=====10G======>| p4p1|<--->| br0 |--vm1
vm2--| | | | | | | |--vm2
+-----+ +-----+ +-----+ +-----+
+-----+ +-----+ +-----+ +-----+
vm1--| br1 |<--->| em1 |<== Internet ==>| em1 |<--->| br1 |--vm1
vm2--| | | | | | | |--vm2
+-----+ +-----+ +-----+ +-----+
[S1]br0: 192.168.10.1 [S2]br0: 192.168.20.2
[S1]br1: 10.18.64.27 [S2]br1: 10.18.64.28
[S1]vm1: 10.18.66.12 [S2]vm1: 10.18.66.13
[S1]vm2: 10.18.67.145 [S2]vm2: 10.18.66.25
- Server 1&2 - Running RHEL6.4/x86_64 KVM; All guest vm's running CentOS 6.4
- I have two NICs on each server;
- 1G intel NIC to connect to internet [interface: em1]
- 10G Broadcom 57810 NIC (dual port), [interface: p4p1, p4p2]
- 10G ports connected back-to-back on Server1 & Server2
- VMs were created with bridged networking enabled and used br1 at VM creation time [eth0 was created]. I "added hardware" and enabled the second NIC (p4p1/br0) later [eth1 created].
- VMs can connect to the internet when I enable wired networking (via the icon on title bar) using eth0.
- When I try to enable networking using eth1, the network is unavailable.
- I can ping S2 from S1 using br0, I can even run netperf using these interfaces and get the higher network throughput.
- Why can't the VMs see br0 and use the 10G link?
- I have attached the screen log of ifconfig, netstat -r and brctl below
PHP Code:
--------------------------------------------------------------------
Server 1
[root@lab-s1-140 kishore]# ifconfig
br0 Link encap:Ethernet HWaddr AC:16:2D:81:A5:C8
inet addr:192.168.10.1 Bcast:192.168.31.255 Mask:255.255.224.0
inet6 addr: fe80::ae16:2dff:fe81:a5c8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:228913 errors:0 dropped:0 overruns:0 frame:0
TX packets:398997 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:29837772 (28.4 MiB) TX bytes:25934881358 (24.1 GiB)
br1 Link encap:Ethernet HWaddr 90:B1:1C:91:70:B8
inet addr:10.18.64.27 Bcast:10.18.95.255 Mask:255.255.224.0
inet6 addr: fe80::92b1:1cff:fe91:70b8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8215543 errors:0 dropped:0 overruns:0 frame:0
TX packets:1385980 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1910293267 (1.7 GiB) TX bytes:15369696166 (14.3 GiB)
em1 Link encap:Ethernet HWaddr 90:B1:1C:91:70:B8
inet6 addr: fe80::92b1:1cff:fe91:70b8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:9120138 errors:0 dropped:0 overruns:0 frame:0
TX packets:11582243 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2143679613 (1.9 GiB) TX bytes:16083465219 (14.9 GiB)
Interrupt:20 Memory:fb400000-fb420000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:9276072 errors:0 dropped:0 overruns:0 frame:0
packets:9276072 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4875534021 (4.5 GiB) TX bytes:4875534021 (4.5 GiB)
p4p1 Link encap:Ethernet HWaddr AC:16:2D:81:A5:C8
inet6 addr: fe80::ae16:2dff:fe81:a5c8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:213681 errors:0 dropped:0 overruns:0 frame:0
TX packets:17946202 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:23883754 (22.7 MiB) TX bytes:27173686290 (25.3 GiB)
Interrupt:40 Memory:f4000000-f47fffff
virbr0 Link encap:Ethernet HWaddr 52:54:00:72:42:7F
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1506 errors:0 dropped:0 overruns:0 frame:0
TX packets:2104 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:82111 (80.1 KiB) TX bytes:2646696 (2.5 MiB)
vnet0 Link encap:Ethernet HWaddr FE:54:00:E8:47:91
inet6 addr: fe80::fc54:ff:fee8:4791/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:11577 errors:0 dropped:0 overruns:0 frame:0
TX packets:1522538 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:830532 (811.0 KiB) TX bytes:132650796 (126.5 MiB)
vnet1 Link encap:Ethernet HWaddr FE:54:00:D7:16:E4
inet6 addr: fe80::fc54:ff:fed7:16e4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:17864 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:468 (468.0 b) TX bytes:1484333 (1.4 MiB)
vnet2 Link encap:Ethernet HWaddr FE:54:00:4A:D9:48
inet6 addr: fe80::fc54:ff:fe4a:d948/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:16216 errors:0 dropped:0 overruns:0 frame:0
TX packets:58548 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:5544288 (5.2 MiB) TX bytes:17236290 (16.4 MiB)
vnet3 Link encap:Ethernet HWaddr FE:54:00:49:98:7B
inet6 addr: fe80::fc54:ff:fe49:987b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:188 errors:0 dropped:0 overruns:0 frame:0
TX packets:636 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:62712 (61.2 KiB) TX bytes:184800 (180.4 KiB)
[root@lab-s1-140 kishore]# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.ac162d81a5c8 no p4p1
vnet2
vnet3
br1 8000.90b11c9170b8 no em1
vnet0
vnet1
virbr0 8000.52540072427f yes virbr0-nic
[root@lab-s1-140 kishore]# netstat -r
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.122.0 * 255.255.255.0 U 0 0 0 virbr0
10.18.64.0 * 255.255.224.0 U 0 0 0 br1
192.168.0.0 * 255.255.224.0 U 0 0 0 br0
link-local * 255.255.0.0 U 0 0 0 br1
link-local * 255.255.0.0 U 0 0 0 br0
192.0.0.0 192.168.122.100 255.0.0.0 UG 0 0 0 virbr0
default lab-dvt-gateway 0.0.0.0 UG 0 0 0 br1
[root@lab-s1-140 kishore]# exit
--------------------------------------------------------------------
VM1
root@pc1-vm1 kishore]#
[root@pc1-vm1 kishore]# ifconfig
eth0 Link encap:Ethernet HWaddr 52:54:00:E8:47:91
inet addr:10.18.66.12 Bcast:10.18.95.255 Mask:255.255.224.0
inet6 addr: fe80::5054:ff:fee8:4791/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1522913 errors:0 dropped:0 overruns:0 frame:0
TX packets:11576 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:132683323 (126.5 MiB) TX bytes:830490 (811.0 KiB)
eth1 Link encap:Ethernet HWaddr 52:54:00:4A:D9:48
inet6 addr: fe80::5054:ff:fe4a:d948/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:58556 errors:0 dropped:0 overruns:0 frame:0
TX packets:16219 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:17238744 (16.4 MiB) TX bytes:5545314 (5.2 MiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1058996 errors:0 dropped:0 overruns:0 frame:0
TX packets:1058996 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:12543923920 (11.6 GiB) TX bytes:12543923920 (11.6 GiB)
[root@pc1-vm1 kishore]# brctl show
bridge name bridge id STP enabled interfaces
[root@pc1-vm1 kishore]# netstat -r
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
10.18.64.0 * 255.255.224.0 U 0 0 0 eth0
default lab-dvt-gateway 0.0.0.0 UG 0 0 0 eth0
[root@pc1-vm1 kishore]#
--------------------------------------------------------------------
|
