LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Virtualization and Cloud (http://www.linuxquestions.org/questions/linux-virtualization-and-cloud-90/)
-   -   How to get guest on same subnet as host (http://www.linuxquestions.org/questions/linux-virtualization-and-cloud-90/how-to-get-guest-on-same-subnet-as-host-4175458796/)

irreverentryan 04-19-2013 03:17 PM

How to get guest on same subnet as host
 
Hi all,

I'm building what I assume is a relatively complex setup. I have a physical host (VMHOST01) running RHEL 6.4 x64. This machine is hosting a virtualized RHEVM guest. I cannot seem to (successfully) get the guest to operate on the same subnet as the host. If this is not possible, how would I set it up so that hosts on the production network are able to access the guest in its virtual network?

My host setup is as follows:

em1
>bond0-->192.168.0.0/22 network (production)
em2


em3
>bond1-->192.168.10.0/24 network (storage / RHEV)
em4

These bonds seem to be working correctly. I have no packet loss with pings on either network from the host, I am able to go online, and add SAN storage from the RHEV network.

The problem is, when I select "Host device bond0 : macvtap" I receive DHCP (which I don't want, but am using for testing) and I am able to ping SOME addresses on my production LAN.

Working addresses:
192.168.1.244 (our prod network gateway)
192.168.1.39 (DNS1)
192.168.1.150 (DNS2)
8.8.8.8 (Google's Public DNS)

Non-Working addresses:
192.168.1.16 (the host bond0)
192.168.1.17 (the guest's only nic)
173.194.43.8 (Google's resolved IP address)


If I select "Virtual network 'default' : NAT" then I receive proper DHCP, and am able to ping across the board, as well as access the internet (which I couldn't do with the bond0 interface selected.) The only reason I don't stay with this configuration is that my guest receives an address in the 192.168.122.0/24 network (the virtual network between the host and guest) and I am (seemingly) unable to access that address from my production network.

I am sure I could probably make some static route entries, but what I'd really like to accomplish is to have my Guest on the same subnet as my host.

Thanks!

dyasny 04-20-2013 07:14 AM

1. what does RHEV have to do with this configuration? You are obviously talking about libvirt and virt-manager?
2. Use a normal bridge, not a macvtap

irreverentryan 04-22-2013 07:36 AM

RHEV
 
1 Attachment(s)
Hi, thanks for the reply. RHEV does not neccessarily have anything to do with this setup, orhter than the fact that it is the guest. I am indeed talking about libvirt and virt-manager. All of the options in the drop-down menu for virt-manager include "macvtap." Do I need to create a new confif file to create the bridge?

Thanks

irreverentryan 04-22-2013 11:21 AM

New Bridge
 
Alright, so I've created a bridge (br0) and set my virtual interface to Host Device bond0 (Bridge 'br0') and I was able to get DHCP (no change yet, I was able to do this before.)

I set the virtual nic to a static on my subnet (192.168.1.17)
I pinged google.com and got this result:
Code:

32 packets transmitted, 2 received (93% loss)
The traceroute was almost instant, though...

brctl show br0:
Code:

bridge name        bridge id                STP enabled        interfaces
br0                8000.00219ba7404c        no                bond0
                                                        vnet0
virbr0                8000.525400da074f        yes                virbr0-nic

I did some additional testing, and it seems to get only 2 responses, no matter how many requests it sends out, and now matter which sequence the first answer is (sometimes immediately, sometimes later.) but it is always 2 responses.

I am very confused as to why it would connect, get dhcp, allow a static address, and traceroute all while also haveing horible ping stats.

dyasny 04-22-2013 12:59 PM

what bonding mode do you use? modes 0 and 6 are not compatible with a bridge, I always recommend mode 4 if the switches support it

irreverentryan 04-22-2013 02:47 PM

Mode=1 solved it
 
I switched it to mode=1 and removed the bridge configs on the host machine. I then selected the "Bond0" option from the virt-manager drop down menu, and all seems great now. 100% ping success from host and guest to google.com

nicolasdiogo 04-22-2013 04:50 PM

had similar problem.
http://www.linuxquestions.org/questi...tu-4175458587/

i will try your mode=1

however, i am sure that RedHat has a bug posted about problems of bonding and bridge with KVM.but i will try it..

thanks,

dyasny 04-23-2013 01:33 AM

There is a bugreport, somewhere, but the fact remains - bridging and bond modes 0 and 6 don't work together, because of the way these bonding modes operate. You can read up on what exactly they do, and think about that in conjunction with using a bridge and virtual machine NICs on top - will make sense after a few passes

irreverentryan 04-24-2013 08:46 AM

Still having some issues
 
Although my previous solution worked to get my VM on the host network, I am unable to ping the host from the guest, and vice versa. My guest VM can ping my physical gateway, google's DNS servers, and other hosts on my physical LAN, however it cannot ping the host address. I am able to connect to RHEVM, which is running on my VM, but I am unable to run the spice console for the RHEV vms. I am pretty sure this is a relative issue with the odd ping results I'm experiencing.

irreverentryan 04-24-2013 02:35 PM

I actually cannot confirm that my host is properly forwarding traffic... I have enables ip forwarding in sysctl.conf, and I have disabled iptables. What could be stopping it at this point?

dyasny 04-24-2013 11:49 PM

when you use a bridge, there is no need to forward anything. can you post your ifcfg-* files?

irreverentryan 04-25-2013 08:08 AM

ifcfg-bondx and ifcfg-emx
 
DEVICE=bond0
IPADDR=192.168.1.16
PREFIX=22
DNS1=192.168.1.39
DNS2=192.168.1.150
DEFROUTE=yes
GATEWAY=192.168.1.244
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
//////////////////////////////////////////////////////
DEVICE=bond1
IPADDR=192.168.10.100
PREFIX=24
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
DEFROUTE=no
//////////////////////////////////////////////////////
DEVICE=em4
BOOTPROTO=none
ONBOOT=yes
MASTER=bond1
SLAVE=yes
USERCTL=no
//////////////////////////////////////////////////////
DEVICE=em3
BOOTPROTO=none
ONBOOT=yes
MASTER=bond1
SLAVE=yes
USERCTL=no
//////////////////////////////////////////////////////
DEVICE=em2
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
/////////////////////////////////////////////////////
DEVICE=em1
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no
////////////////////////////////////////////////////

---------- Post added 04-25-13 at 09:09 AM ----------

I also swapped the bridge for the forwarding. I'll have to undo the forwarding and reconfigure the bridge.

irreverentryan 04-25-2013 08:12 AM

Also, I'd be bridging bond0 and bond1 for passthrough (I think) but would there be an issue with the two bonds being on different subnets, or will it route for me?

Or am I confused? I was also thinking I might have to create a bridge with only bond0 and point the VM at the bridge as the host device? and the same for bond1?

irreverentryan 04-25-2013 01:01 PM

Alright, so I've completely redone my bonds and bridges. I can confirm that, on the host, they are working properly. Here are the VMHOST config files:


////////////////////
DEVICE=bond0
BOOTPROTO=none
ONBOOT=yes
USERCTL=NO

### BRIDGING PARAMS ###
BRIDGE=br0


//////////////////////
DEVICE=bond1
BOOTPROTO=none
ONBOOT=yes
USERCTL=NO

### BRIDGING PARAMS ###
BRIDGE=br1


/////////////////////
TYPE=Bridge
ONBOOT=yes
DEVICE=br0
IPADDR=192.168.1.16
GATEWAY=192.168.1.244
DNS1=192.168.1.39
DNS2=192.168.1.150
BOOTPROTO=static
USERCTL=no
IPV6INIT=no
STP=no


////////////////////
TYPE=Bridge
ONBOOT=yes
DEVICE=br1
IPADDR=192.168.10.100
BOOTPROTO=static
USERCTL=no
IPV6INIT=no
STP=no


////////////////////
DEVICE=em1
BOOTPROTO=none
ONBOOT=yes

### Bonding ###
MASTER=bond0
SLAVE=yes


////////////////////
DEVICE=em2
BOOTPROTO=none
ONBOOT=yes

### Bonding ###
MASTER=bond0
SLAVE=yes


////////////////////
DEVICE=em3
BOOTPROTO=none
ONBOOT=yes

### BONDING PARAMS ###
MASTER=bond1
SLAVE=yes


//////////////////////
DEVICE=em4
BOOTPROTO=none
ONBOOT=yes

###BONDING PARAMS ###
MASTER=bond1
SLAVE=yes
////////////////////////////////////////////////////////////////////////////////////
I am able to ping the following from my VMHOST:
192.168.1.244 (default gateway on production LAN)
192.168.1.150 (dns server on production LAN)
8.8.8.8 (public Google DNS server)
173.194.43.4 (Google's public address)
192.168.10.110 (RHEVH01)
192.168.10.120 (RHEVH02)
192.168.10.50 (SAN VIP)
////////////////////////////////////////////////////////////////////////////////////
Where I am getting lost is how to properly configure the virtual machine to be able to do the same. I am using virt-manager and I was able to set up one connection (Host device bond0 {bridge 'br0'}. With this setup, I made the following config file on my VMGUEST:

###ifcfg-Auto_eth0###
HWADDR=52:54:00:B1:F9:2F
TYPE=Ethernet
BOOTPROTO=none
IPADDR=192.168.1.17
PREFIX=22
GATEWAY=192.168.1.16
DNS1=192.168.1.39
DNS2=192.168.1.150
DEFROUTE=yes
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME="Auto eth0"
UUUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
ONBOOT=yes
LAST_CONNECT=1366907507
///////////////////////////////////////////////////////
With this config, I am able to ping the following:
192.168.1.244 (GW)
192.168.1.16 (VMHOST IP)
192.168.1.150 (DNS)
8.8.8.8 (Google)
173.194.43.4 (Google)
192.168.10.100 VMHOST IP on br1

I am UNABLE to ping anything else on the 192.168.10.0 network from my VM. I tried adding a second virtual nic to br1, in the same fashion I did for br0, but it did not seem to fix anything.


All times are GMT -5. The time now is 05:20 AM.