[SOLVED] Using KVM public bridge manually

sfxpt · 08-14-2010, 11:30 PM

EDIT:

problem solved, please check out
http://www.linuxquestions.org/questi...c-bridge-3107/
or
http://www.linuxquestions.org/questi...ml#post4070277

Here is the OP:

Hi,

Following up on
http://www.linuxquestions.org/questi...6/#post3950901

which setup KVM public bridge manually, and referencing another helpful article,

KVM network scripts
http://blog.bodhizazen.net/linux/kvm_network_scripts/

I am still not able to get my KVM bridge networking works properly.

This is what I've been doing on the host side:

Code:

$ ifconfig 
eth0      Link encap:Ethernet  HWaddr 00:e0:4d:b8:98:ef  
          inet addr:192.168.0.106  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::2e0:4dff:feb8:98ef/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:45338 errors:0 dropped:0 overruns:0 frame:0
          TX packets:25537 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:24727443 (23.5 MiB)  TX bytes:5100629 (4.8 MiB)
          Interrupt:27 Base address:0xa000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:18569 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18569 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:6515922 (6.2 MiB)  TX bytes:6515922 (6.2 MiB)


sudo brctl addbr br0
USERID=`whoami`
iface=`sudo tunctl -b -u $USERID`
sudo brctl addif br0 $iface
sudo ifconfig $iface up
sudo ifconfig br0 192.168.1.1 up

$ ifconfig 
br0       Link encap:Ethernet  HWaddr c6:2d:ab:4f:01:e7  
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::c42d:abff:fe4f:1e7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:29 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:5577 (5.4 KiB)

eth0      Link encap:Ethernet  HWaddr 00:e0:4d:b8:98:ef  
          inet addr:192.168.0.106  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::2e0:4dff:feb8:98ef/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:45393 errors:0 dropped:0 overruns:0 frame:0
          TX packets:25556 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:24743382 (23.5 MiB)  TX bytes:5102232 (4.8 MiB)
          Interrupt:27 Base address:0xa000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:18608 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18608 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:6519650 (6.2 MiB)  TX bytes:6519650 (6.2 MiB)

tap0      Link encap:Ethernet  HWaddr c6:2d:ab:4f:01:e7  
          inet6 addr: fe80::c42d:abff:fe4f:1e7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:17 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

model=virtio

kvm -m 1024 -smp 2 -net nic,macaddr=52:54:00:39:81:49,model=$model -net tap,ifname=$iface,script=no,downscript=no -hda . . .

The KVM guest network doesn't work. ifconfig shows that eth0 doesn't have any ip address, whereas if I don't use any -net parameters, guest has the network just fine.

Please help.
Thanks

Chuck56 · 08-15-2010, 07:16 AM

The variable $iface is possibly the issue:

Change this:

-net tap,ifname=$iface,script=no,downscript=no

to this:

-net tap,ifname=tap0,script=no

sfxpt · 08-15-2010, 10:25 AM

Quote:

Originally Posted by Chuck56

The variable $iface is possibly the issue:

Change this:

-net tap,ifname=$iface,script=no,downscript=no

to this:

-net tap,ifname=tap0,script=no

No, $iface is exactly the same as tap0:

$ echo $iface
tap0

Yet, I did give your suggestion a try:

Code:

$ echo $model
virtio

kvm -m 1024 -smp 2 -net nic,macaddr=52:54:00:39:81:49,model=$model -net tap,ifname=tap0,script=no,downscript=no . . .

and the result is the same.

Chuck56 · 08-16-2010, 08:26 AM

Are the virtio modules loaded?

Code:

# lsmod | grep virtio
virtio_net             12607  0 
virtio_blk              4565  0 
virtio                  3741  2 virtio_net,virtio_blk
#

The virtio-net module is used for networking and the virtio-blk module is used for block devices.

HasC · 08-16-2010, 08:50 AM

I believe you should set promiscuous mode on your tap device

Code:

ifconfig $iface 0.0.0.0 promisc up

sfxpt · 08-16-2010, 11:45 AM

Quote:

Originally Posted by HasC

I believe you should set promiscuous mode on your tap device

Thanks everyone for the reply.

I've remove the virtio model from the command line and set promiscuous mode on for the tap device, but still no luck.

Code:

$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:e0:4d:b8:98:ef  
          inet addr:192.168.0.106  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::2e0:4dff:feb8:98ef/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:297618 errors:0 dropped:0 overruns:0 frame:0
          TX packets:246743 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:92873039 (88.5 MiB)  TX bytes:123962649 (118.2 MiB)
          Interrupt:27 Base address:0xa000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:81188 errors:0 dropped:0 overruns:0 frame:0
          TX packets:81188 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:106905891 (101.9 MiB)  TX bytes:106905891 (101.9 MiB)

sudo brctl addbr br0
iface=`sudo tunctl -b -g kvm`

$ echo $iface
tap0

sudo brctl addif br0 $iface
sudo ifconfig $iface up
sudo ifconfig br0 192.168.1.1 up
sudo ifconfig $iface 0.0.0.0 promisc up

$ ifconfig
br0       Link encap:Ethernet  HWaddr 1a:8f:d2:be:bf:03  
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::188f:d2ff:febe:bf03/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:24 errors:0 dropped:0 overruns:0 frame:0
          TX packets:70 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:7872 (7.6 KiB)  TX bytes:12788 (12.4 KiB)

eth0      Link encap:Ethernet  HWaddr 00:e0:4d:b8:98:ef  
          inet addr:192.168.0.106  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::2e0:4dff:feb8:98ef/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:297672 errors:0 dropped:0 overruns:0 frame:0
          TX packets:246771 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:92887234 (88.5 MiB)  TX bytes:123972678 (118.2 MiB)
          Interrupt:27 Base address:0xa000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:81224 errors:0 dropped:0 overruns:0 frame:0
          TX packets:81224 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:106907917 (101.9 MiB)  TX bytes:106907917 (101.9 MiB)

tap0      Link encap:Ethernet  HWaddr 1a:8f:d2:be:bf:03  
          inet6 addr: fe80::188f:d2ff:febe:bf03/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:16 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

$ ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_req=1 ttl=64 time=0.164 ms
64 bytes from 192.168.1.1: icmp_req=2 ttl=64 time=0.057 ms
64 bytes from 192.168.1.1: icmp_req=3 ttl=64 time=0.055 ms
^C

kvm -m 1024 -smp 2 -net nic,macaddr=52:54:00:39:81:49 -net tap,ifname=tap0,script=no,downscript=no . . .

still dhcp discover failed for the guest, and pinging 192.168.1.1 from guest failed as well.

Any way that I can troubleshoot this?

Thanks

sfxpt · 08-16-2010, 11:49 AM

Quote:

Originally Posted by sfxpt

Thanks everyone for the reply.

I've remove the virtio model from the command line and set promiscuous mode on for the tap device, but still no luck.

BTW, I don't think set promiscuous mode on for the tap device would make any difference, because I saw that Cotun has already tested it:

From
http://www.linuxquestions.org/questi...6/#post3950901

Quote:

I removed the promisc parameter from the ifconfig line as I found it wasn't needed and made no difference to performance. I also can't be sure of this, but I think using the promisc flag may introduce a security implication as the guest virtual machine may receive transmissions meant for the real host, i.e allowing packet sniffing on the virtual machine.

but i tried it anyway.

please help.

HasC · 08-16-2010, 12:08 PM

Did you tried to set a static IP on your VM? what results did you get?

also, the promisc mode *is* needed, to make your virtual switch behave like a real physical switch (doing retransmission of broadcasts and such), AFAIK. if your virtual switch has just one port, then it may work with or without "promisc", but YMMV. For me, it didn't work.

Chuck56 · 08-16-2010, 12:11 PM

I noticed you're running 2 subnets? I usually run hosts and guests on the same subnet. I used dhcp in the past but now use fixed ip addresses. I also bind both eth0 and tap0 to br0.

Code:

      ip addr flush eth0
      ifconfig eth0 0.0.0.0

      brctl addbr br0
      brctl addif br0 eth0

      tunctl -t tap0 -u your_user_name
      ifconfig tap0 0.0.0.0
      brctl addif br0 tap0

      ifconfig br0 192.168.0.x netmask 255.255.255.0
      route add default gw 192.168.0.1 metric 1

YMMV!

sfxpt · 08-16-2010, 05:37 PM

Quote:

Originally Posted by HasC

Did you tried to set a static IP on your VM? what results did you get?

I totally don' have any idea how the guest VM network would normally be, so last time when I tried, pinging 192.168.1.1 from guest failed with destination unreachable.

Today, I just shoot into the dark and tried again. Hooray, now at least I can ping my host IP now using a static IP. However,

- I can't ping outside my local network
- I tried to setup /etc/resolv.conf but dns look up still fails
- DHCP request still fails

So, great, at least I'm made a great progress. I've seen the light at the end of the tunnel.

Any idea why I can't make it totally work?

Thanks

sfxpt · 08-16-2010, 05:51 PM

Quote:

Originally Posted by Chuck56

I noticed you're running 2 subnets? I usually run hosts and guests on the same subnet.

I'm running 2 subnets because I want to use manual KVM public bridge as defined in
http://www.linux-kvm.org/page/Networking#public_bridge

The reason was that,

I want to setup my KVM bridge so that,

- assign an IP address to the virtual machines and make them accessible
from the local network
- leave my physical interface eth0 as is, and only bring up the KVM
bridge device if I am using KVM.

The actual reason behind why I chose the above approach is that I still want my eth0 to be eth0, instead of need to use br0 for all my network applications.

Quote:

Originally Posted by Chuck56

I also bind both eth0 and tap0 to br0.

On seeing this, I now realized that although eth0 doesn't have an IP any more with such setting, it is still *bind* to the bridge device, and I can still use eth0 the same way as before. From the application using eth0 prospective, nothing is changed, right?

Thanks

Chuck56 · 08-16-2010, 06:02 PM

Quote:

Originally Posted by sfxpt

... I now realized that although eth0 doesn't have an IP any more with such setting, it is still *bind* to the bridge device, and I can still use eth0 the same way as before. From the application using eth0 prospective, nothing is changed, right?

Correct. You can ifconfig eth0 [up|down] just like before.

sfxpt · 08-16-2010, 08:37 PM

> From the application using eth0 prospective, nothing is changed, right?

Quote:

Originally Posted by Chuck56

Correct. You can ifconfig eth0 [up|down] just like before.

Oh, no. Found that it is no longer the same:

from
http://ubuntuforums.org/archive/inde...t-1147139.html

Quote:

For anyone also using Firestarter for Internet Connection Sharing, please note that you will need to change the Firestarter preferences to use br0 as the local network connected device, instead of eth0 (which Firestarter reports as not ready).

this is what I most worry about -- I need to check back my applications one by one. . .

Chuck56 · 08-16-2010, 09:13 PM

Well if you use Firestarter's Internet Connection Sharing feature then you have the workaround. What other apps do you use that would manipulate the eth0 interface like that?

sfxpt · 08-16-2010, 09:14 PM

Quote:

Originally Posted by Chuck56

I noticed you're running 2 subnets?

Did you mean that I should bring up br0 within the same subnet? I just gave that option a try, and the result is that I can't even ping my host from the guest VM.

Here is the log:

Code:

. . . everything same as before. . . 
sudo ifconfig br0 192.168.0.10 up

$ ifconfig
br0       Link encap:Ethernet  HWaddr 76:0c:3d:fd:74:7d  
          inet addr:192.168.0.10  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::740c:3dff:fefd:747d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:117 errors:0 dropped:0 overruns:0 frame:0
          TX packets:195 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:28408 (27.7 KiB)  TX bytes:33537 (32.7 KiB)

eth0      Link encap:Ethernet  HWaddr 00:e0:4d:b8:98:ef  
          inet addr:192.168.0.106  Bcast:192.168.0.255  Mask:255.255.255.0
. . . everything same as before. . . 

$ ping 192.168.0.106
PING 192.168.0.106 (192.168.0.106) 56(84) bytes of data.
64 bytes from 192.168.0.106: icmp_req=1 ttl=64 time=0.093 ms
64 bytes from 192.168.0.106: icmp_req=2 ttl=64 time=0.050 ms
64 bytes from 192.168.0.106: icmp_req=3 ttl=64 time=0.053 ms
^C

$ ping 192.168.0.10
PING 192.168.0.10 (192.168.0.10) 56(84) bytes of data.
64 bytes from 192.168.0.10: icmp_req=1 ttl=64 time=0.035 ms
64 bytes from 192.168.0.10: icmp_req=2 ttl=64 time=0.049 ms
^C

kvm -m 1024 -smp 2 -net nic,macaddr=52:54:00:39:81:49 -net tap,ifname=tap0,script=no,downscript=no . . .