LinuxQuestions.org
LinuxAnswers - the LQ Linux tutorial section.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices

Reply
 
Search this Thread
Old 04-20-2010, 10:54 AM   #1
HasC
Member
 
Registered: Oct 2009
Location: South America - Paraguay
Distribution: Debian 5 - Slackware 13.1 - Arch - Some others linuxes/*BSDs through KVM and Xen
Posts: 329

Rep: Reputation: 55
Virtual bridged network with too high load - without IP configuration


Not sure if this issue belongs in "Virtualization" or "Networking". A little of both, I assume. And this will be a little bit long, so thanks for your patience :-)

I'm trying to setup a little virtual network, using 2 bridges (using bridge-utils), and 2 virtual machines (using QEMU/KVM), on Debian Lenny (no unstable-testing-backports-whatever which is not stable). With the "virtual-bridged-network.png" included graphic you can see my idea.

I created the virtual network with this script (as root):
Code:
#!/bin/bash

brctl addbr switch0
brctl addbr switch1

for interface in $(seq 100 103); do
	tunctl -t tap"$interface" -u 1000
	brctl addif switch0 tap"$interface"
	ifconfig tap"$interface" promisc up
done
for interface in $(seq 104 107); do
	tunctl -t tap"$interface" -u 1000
	brctl addif switch1 tap"$interfaz"
	ifconfig tap"$interface" promisc up
done

ifconfig switch0 up
ifconfig switch1 up
and created the virtual machines with this script (as my normal user):
Code:
#!/bin/bash

kvm -cdrom systemrescuecd.iso -m 256 \
	-net nic,macaddr=aa:bb:cc:00:11:01 \
	-net tap,ifname=tap100,script=no,downscript=no \
	-net nic,macaddr=aa:bb:cc:00:11:02 \
	-net tap,ifname=tap104,script=no,downscript=no &

kvm -cdrom systemrescuecd.iso -m 256 \
	-net nic,macaddr=aa:bb:cc:00:11:03 \
	-net tap,ifname=tap103,script=no,downscript=no \
	-net nic,macaddr=aa:bb:cc:00:11:04 \
	-net tap,ifname=tap107,script=no,downscript=no &
As you can read, I started two instances of an "System Rescue CD" ISO (v1.5.1 BTW), but you can use any distro you want.

Now, let's see the symptoms:
When the VMs finish booting and I get the welcome screen, I try to configure eth0 or eth1, on any guest.
Code:
ifconfig eth0 up
(Note I don't assign any IP address - But I can, the result will be the same)
Then, I get a bunch of syslog output on my Debian host
Code:
[ 4000.536746] tap104: received packet with  own address as source address
[ 4000.536746] tap104: received packet with  own address as source address
[ 4005.573240] __ratelimit: 109501 messages suppressed
[ 4005.573240] tap103: received packet with  own address as source address
[ 4010.588751] __ratelimit: 108864 messages suppressed
[ 4010.588754] tap103: received packet with  own address as source address
[ 4015.627566] __ratelimit: 111212 messages suppressed
[ 4015.627566] tap107: received packet with  own address as source address
[ 4020.682822] __ratelimit: 110245 messages suppressed
[ 4020.682825] tap107: received packet with  own address as source address
and the bandwith usage goes from 0 to an average of 20MBps, max 40MBps and min 5MBps (checked with
Code:
# bwm-ng
).

On the guests, I got too syslog output:
Code:
 eth0: IPv6 duplicate address fe80::a8bb:ccff:fe00:1103 detected!
[   74.900262] eth0: IPv6 duplicate address fe80::a8bb:ccff:fe00:1103 detected!
[   74.900265] eth0: IPv6 duplicate address fe80::a8bb:ccff:fe00:1103 detected!
[   74.900268] eth0: IPv6 duplicate address fe80::a8bb:ccff:fe00:1103 detected!
[   74.900271] eth0: IPv6 duplicate address fe80::a8bb:ccff:fe00:1103 detected!
[   74.900274] eth0: IPv6 duplicate address fe80::a8bb:ccff:fe00:1103 detected!
[   74.900277] eth0: IPv6 duplicate address fe80::a8bb:ccff:fe00:1103 detected!
[   74.900280] eth0: IPv6 duplicate address fe80::a8bb:ccff:fe00:1103 detected!
[   74.900283] eth0: IPv6 duplicate address fe80::a8bb:ccff:fe00:1103 detected!
[   74.900287] eth0: IPv6 duplicate address fe80::a8bb:ccff:fe00:1103 detected!
[   79.923074] __ratelimit: 3754 callbacks suppressed
(IPv6? WTH? I didn't even configured IPv4 yet...)

Using some network-analyzers for debugging:
  • With Wireshark/Ethereal, I got output like this:
    Code:
    Frame 803 (90 bytes on wire, 90 bytes captured)
    Ethernet II, Src: aa:bb:cc:00:11:03 (aa:bb:cc:00:11:03), Dst: IPv6mcast_00:00:00:16 (33:33:00:00:00:16)
    Internet Protocol Version 6
    Internet Control Message Protocol v6
    
    No.     Time        Source                Destination           Protocol Info
        804 0.035921    ::                    ff02::16              ICMPv6   Multicast Listener Report Message v2
    
    Frame 804 (90 bytes on wire, 90 bytes captured)
    Ethernet II, Src: aa:bb:cc:00:11:03 (aa:bb:cc:00:11:03), Dst: IPv6mcast_00:00:00:16 (33:33:00:00:00:16)
    Internet Protocol Version 6
    Internet Control Message Protocol v6
    
    No.     Time        Source                Destination           Protocol Info
        805 0.035977    ::                    ff02::16              ICMPv6   Multicast Listener Report Message v2
    
    Frame 811 (90 bytes on wire, 90 bytes captured)
    Ethernet II, Src: aa:bb:cc:00:11:03 (aa:bb:cc:00:11:03), Dst: IPv6mcast_00:00:00:16 (33:33:00:00:00:16)
    Internet Protocol Version 6
    Internet Control Message Protocol v6
    
    No.     Time        Source                Destination           Protocol Info
        812 0.036157    ::                    ff02::16              ICMPv6   Multicast Listener Report Message v2
    
    Frame 812 (90 bytes on wire, 90 bytes captured)
    Ethernet II, Src: aa:bb:cc:00:11:03 (aa:bb:cc:00:11:03), Dst: IPv6mcast_00:00:00:16 (33:33:00:00:00:16)
    Internet Protocol Version 6
    Internet Control Message Protocol v6
    
    No.     Time        Source                Destination           Protocol Info
        813 0.036213    ::                    ff02::16              ICMPv6   Multicast Listener Report Message v2
  • With Tcpdump, I got output like this:
    Code:
    09:25:38.288125 IP6 :: > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
    09:25:38.288126 IP6 :: > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
    09:25:38.288158 IP6 :: > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28
(ICMP6? IPv6 again? Why? I didn't configure IPv6...)

For my guests, I used first System Rescue CD v1.5.1, then Wolvix 2.0.0, then Knoppix 6.2, and last Debian Lenny. Always the same results.

Well, here is the question: why I get the high load (and the reduced performance) with my virtual net?

I thought first on a "broadcast storm", but I can't understand why the guests will pass the broadcast packets on, like bridges/switches do (the guests aren't like switches, they are more like routers, right?)

Any pointer will be appreciated.
Attached Images
File Type: png virtual-bridged-network.png (40.7 KB, 9 views)
 
Old 04-20-2010, 12:54 PM   #2
Robhogg
Member
 
Registered: Sep 2004
Location: Old York, North Yorks.
Distribution: Debian 7 (mainly)
Posts: 653

Rep: Reputation: 85
I've no experience with QEMU/KVM, but one thing I notice (from my scant knowledge of IPv6) is that the fe80 prefix of the duplicated address (fe80::a8bb:ccff:fe00:1103) indicates that it is a link-local address.

These are the IPv6 equivalent of the IPv4 APIPA 169.254.0.0/16 addresses, that tend to get assigned automatically, in the absence of any other network configuration. In the case of IPv6, though, they are based on the network card's NIC number.

My bet is that, as the two machines were created from identical images, their "virtual" MAC addresses are the same. Often, also, IPv6 is configured as enabled by default. The result: clashing addresses.

Hope this helps,
Rob.
 
Old 04-20-2010, 01:25 PM   #3
HasC
Member
 
Registered: Oct 2009
Location: South America - Paraguay
Distribution: Debian 5 - Slackware 13.1 - Arch - Some others linuxes/*BSDs through KVM and Xen
Posts: 329

Original Poster
Rep: Reputation: 55
Quote:
Originally Posted by Robhogg View Post
My bet is that, as the two machines were created from identical images, their "virtual" MAC addresses are the same.
AFAIK, the virtual MACaddrs are not the same.
If you read the script which creates the VMs
Code:
-net nic,macaddr=aa:bb:cc:00:11:01
-net nic,macaddr=aa:bb:cc:00:11:02
-net nic,macaddr=aa:bb:cc:00:11:03
-net nic,macaddr=aa:bb:cc:00:11:04
they all have different macaddrs, at least the last byte is different on each mac address.
 
Old 04-20-2010, 04:32 PM   #4
Robhogg
Member
 
Registered: Sep 2004
Location: Old York, North Yorks.
Distribution: Debian 7 (mainly)
Posts: 653

Rep: Reputation: 85
Quote:
Originally Posted by HasC View Post
If you read the script which creates the VMs
Which I obviously failed to do

Well, I'm out of ideas. Sorry.
 
Old 04-20-2010, 09:02 PM   #5
HasC
Member
 
Registered: Oct 2009
Location: South America - Paraguay
Distribution: Debian 5 - Slackware 13.1 - Arch - Some others linuxes/*BSDs through KVM and Xen
Posts: 329

Original Poster
Rep: Reputation: 55
oh, don't worry. We'll solve this thing :-)

BTW, I've been doing some more tests, and the results seem to support the thesis of broadcast storm, though I can't understand why... I'll post more on that later.
 
Old 04-21-2010, 01:41 PM   #6
HasC
Member
 
Registered: Oct 2009
Location: South America - Paraguay
Distribution: Debian 5 - Slackware 13.1 - Arch - Some others linuxes/*BSDs through KVM and Xen
Posts: 329

Original Poster
Rep: Reputation: 55
Well, I solved it... Though I still can't understand how nor why

I modified the script which creates the VMs:
Code:
#!/bin/bash

kvm -cdrom systemrescuecd.iso -m 256 \
	-net nic,macaddr=aa:bb:cc:00:11:01 \
	-net tap,ifname=tap100,script=no,downscript=no \
	-net nic,macaddr=aa:bb:cc:00:11:02,vlan=1 \
	-net tap,ifname=tap104,script=no,downscript=no,vlan=1 &

kvm -cdrom systemrescuecd.iso -m 256 \
	-net nic,macaddr=aa:bb:cc:00:11:03 \
	-net tap,ifname=tap103,script=no,downscript=no \
	-net nic,macaddr=aa:bb:cc:00:11:04,vlan=1 \
	-net tap,ifname=tap107,script=no,downscript=no,vlan=1 &
Note the "vlan=X" pieces.

Apparently, the 4 virtual NICs (2 for each VM, you see) belonged to the same VLAN, vlan0. Because of that, the VMs worked as bridges too. And that's why I had so much network traffic: a broadcast storm.

Changing the VLAN value of one of the NIC pairs (to vlan1 in the example), I finally got separated subnets.

Man, I thought a PC couldn't work as a bridge without some fancy configuration... Here I needed the fancy configuration to achieve the opposite
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
qemu-kvm, virtual network stalling under load cosmonate Linux - Virtualization and Cloud 4 03-10-2010 05:37 AM
High load, high RAM usage and unresponsive VPS saeed22 Linux - Server 1 08-20-2009 11:58 AM
cannot set up network with suse vmware using bridged network config. lxvor Suse/Novell 1 02-08-2009 03:50 PM
Load Avg High/Phys Mem High teamh Debian 2 12-26-2006 05:03 PM
DL380 + Linux + High Load=Network Drop king0770 Linux - Hardware 0 05-25-2006 04:53 PM


All times are GMT -5. The time now is 05:51 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration