I have a host machine with 2 ethernet cards on two different subnets. I've set up bridges for both of them:
host> brctl show
bridge name bridge id STP enabled interfaces
br0 8000.003048c655e0 no vnet7
br1 8000.003048c655e1 no vnet6
virbr0 8000.000000000000 yes
In my virtual machines I've connected one virtual ethernet card to br0 and the other to br1. The problem is that only eth1 (in the virtual machine) that is connected to br0 on the host works fully. eth0 that is connected to br1 does not work. I can't connect to machine on the same subnet. I can ping them, but I can't open a tcp connection:
vm> telnet d2 25
telnet: connect to address 22.214.171.124: Connection refused
telnet: Unable to connect to remote host: Connection refused
I can tell from tcpdump that the target machine does not see anything from the vm besides an arp request.
The problem is the same in the opposite direction. I can ping the vm from an entirely different network, but if I try to telnet to it tcpdump shows me that the vm never sees anything.
So the question is why does one virtual ethernet card work and not the other? On the host I compared the files in /proc/sys/net/ipv4/conf and they are exactly the same for eth0/eth1 br0/br1 tap0/tap1
The command line for my vm (started by libvirtd) is :
/usr/bin/qemu-kvm -S -M pc -m 2048 -smp 3 -name dmail -uuid <snip> -monitor pty -pidfile /var/run/libvirt/qemu//dmail.pid -boot c -drive file=/var/lib/libvirt/images/dmail.img,if=ide,index=0,boot=on -drive file=/var/lib/libvirt/images/dmail-b.img,if=ide,index=1 -drive file=/local/download/CentOS-5.3-x86_64-bin-DVD/CentOS-5.3-x86_64-bin-DVD.iso,if=ide,media=cdrom,index=2 -net nic,macaddr=54:52:00:11:cb:ae,vlan=0 -net tap,fd=22,script=,vlan=0,ifname=vnet4 -net nic,macaddr=54:52:00:11:cb:af,vlan=1 -net tap,fd=23,script=,vlan=1,ifname=vnet5 -serial pty -parallel none -usb -vnc 127.0.0.1:2 -k en-us