How Can I Create Two Bond Interfaces in CentOS 7

rdill · 06-06-2017, 12:05 PM

Hi,

I have a machine with four NIC cards. I have successfully bonded two NIC's together. Now, I want to bond two more and assign them a different IP address. Eventually, I want to hook a bridge up to the new bond interface and use it for a VM or multiple VM's. I have been reading for hours, and I can't find a solution to get both bond interfaces working simultaneously. Below are my configuration files. I have changed the IP and MAC addresses for security purposes. Any help would be appreciated.

/etc/modprobe.d/bonding.conf

Code:

alias bond0 bonding
alias bond1 bonding

/etc/sysconfic/network-scripts/ifcfg-eth0

Code:

DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
HWADDR=00:00:00:00:00:01

/etc/sysconfic/network-scripts/ifcfg-eth1

Code:

DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
HWADDR=00:00:00:00:00:02

/etc/sysconfic/network-scripts/ifcfg-eth2

Code:

DEVICE=eth2
USERCTL=no
ONBOOT=yes
MASTER=bond1
SLAVE=yes
BOOTPROTO=none
HWADDR=00:00:00:00:00:03

/etc/sysconfic/network-scripts/ifcfg-eth3

Code:

DEVICE=eth3
USERCTL=no
ONBOOT=yes
MASTER=bond1
SLAVE=yes
BOOTPROTO=none
HWADDR=00:00:00:00:00:04

/etc/sysconfic/network-scripts/ifcfg-bond0

Code:

TYPE=Bond
DEVICE=bond0
NAME=bond0
BOOTPROTO=static
USERCTL=no
ONBOOT=yes
IPADDR=192.168.1.2
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
BONDING_OPTS="max_bonds=2 mode=balance-alb miimon=100"

/etc/sysconfic/network-scripts/ifcfg-bond1

Code:

TYPE=Bond
DEVICE=bond1
NAME=bond1
BOOTPROTO=static
USERCTL=no
ONBOOT=yes
IPADDR=192.168.1.3
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
BONDING_OPTS="max_bonds=2 mode=balance-alb miimon=100"

Bond0 always works, but bond1 never works.

Thanks in advance.

jefro · 06-08-2017, 02:22 PM

Hello and welcome to LQ.

Might look here for changes to how 6 and 7 do it. https://www.unixmen.com/linux-basics...n-centos-76-5/

For some reason I get the feeling that not all nic's allow bonding but I don't know why I think that right now. Might end up checking on that.

voleg · 06-09-2017, 03:04 AM

The bond's IP addresses belongs to same subnet. Is this by design ? What exactly you want to achieve ? If you want aggregate all four ports, just add all of them to same bond.
You can force to use second IP on same subnet using "ip route add" to specific destination, but this is very pervert solution.

rdill · 06-16-2017, 10:43 AM

Hi All,

First, thanks to all those who read this post and tried to help. I finally resolved the problem. My goal was to assign two bonded NIC's to the host OS and two bonded NIC's to the guest OS. I apologize if I was a little unclear about the end goal.

When I posted this question, I thought that the bond driver was limiting me to one working bonded interface. This was not the case. I reproduced the same behavior by removing the bond interfaces and simply configuring eth0 and eth2 with different IP addresses. When you have a machine with two or more interfaces with two or more IP addresses, you have to be very careful. The machine will pick one interface act as the default route. That interface will work as expected, the other one will not. By that I mean that you can ping from outside the machine to either interface, and you will get an answer, but if you try to ping from the machine to anywhere else, one interface will ping successfully and the other wont. This is really odd behavior until you understand how routing within your machine works. To make a long story short, unless you are an expert at networking, don't assign two IP's to the same machine because you will get strange behavior. Anyways, the problems I was having with the two bonded interfaces was a red herring.

For anyone else attempting to assign two bonded NIC's to a host OS and two bonded NIC's to a guest OS, here's how to set that up. In my setup eth0 and eth1 are bonded for the host, and eth2 and eth3 are bonded for the guest.

On the host OS, use the following settings.

/etc/modprobe.d/bonding.conf (create this file if it doesn't exist)

Code:

alias bond0 bonding
alias bond1 bonding

/etc/sysconfig/network-scripts/ifcfg-eth0

Code:

DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
HWADDR=00:00:00:00:00:01

/etc/sysconfig/network-scripts/ifcfg-eth1

Code:

DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
HWADDR=00:00:00:00:00:02

/etc/sysconfig/network-scripts/ifcfg-eth2

Code:

DEVICE=eth2
USERCTL=no
ONBOOT=yes
MASTER=bond1
SLAVE=yes
BOOTPROTO=none
HWADDR=00:00:00:00:00:03

/etc/sysconfig/network-scripts/ifcfg-eth3

Code:

DEVICE=eth3
USERCTL=no
ONBOOT=yes
MASTER=bond1
SLAVE=yes
BOOTPROTO=none
HWADDR=00:00:00:00:00:04

/etc/sysconfig/network-scripts/ifcfg-bond0 (host machines bonded network interface)

Code:

TYPE=Bond
DEVICE=bond0
NAME=bond0
BOOTPROTO=static
USERCTL=no
ONBOOT=yes

# This is where you configure you bond device 
# NOT as some examples suggest inside bonding.conf
BONDING_OPTS="max_bonds=4 mode=balance-alb miimon=100"

# The numbers
IPADDR=192.168.1.2
NETMASK=255.255.255.0
GATEWAY=192.168.1.1

# Additional options
DOMAIN=domain.edu
IPV4_FAILURE_FATAL=no
IPV6INIT=no
DEFROUTE=yes

The following bond interface is for the VM. Do not assign an IP address to it!
/etc/sysconfig/network-scripts/ifcfg-bond1

Code:

TYPE=Bond
DEVICE=bond1
NAME=bond1
BOOTPROTO=none
USERCTL=no
ONBOOT=yes
BONDING_OPTS="max_bonds=4 mode=balance-alb miimon=100"
BRIDGE=br0

You need a bridge device for the VM to connect to. This is like a virtual network switch. You can control it using brctl. It has commands like brctl showmacs br0 which lists the MAC addresses in the switching table.
/etc/sysconfig/network-scripts/ifcfg-br0

Code:

DEVICE=br0
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=none
USERCTL=no
DELAY=0

Notice that no IP address has been assigned to eth2, eth3, bond1, or br0. This is because we want the guest OS to set the IP addresses. If the host sets it, the VM will have no control over the devices. Now, I used KVM for my virtualization solution. You might use something different, but the concept will be the same. Your VM will create a tunneling device to the bridge interface and use that for communications. Below is how to configure the guest OS in KVM.

KVM uses an XML file to configure its machines. You can edit it using 'virsh edit <domain>.' In my case, we'll call the VM "destiny01". You will need to modify the interface tag as follows.

Code:

    <interface type='bridge'>
      <source bridge='br0'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

Finally, inside destiny01, make the following modification.

/etc/sysconfig/network-scripts

Code:

DEVICE=eth0
HWADDR=00:00:00:00:00:05
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
IPADDR=192.168.1.3
NETMASK=255.255.255.0
GATEWAY=192.168.1.1

When the guest OS starts, it creates a tunneling device called vnet0 that will connect to your bridge. Here's my attempt at drawing the diagram in ASCII artform

Code:

eth0 --- \
          |--- bond0
eth1 --- /

eth2 --- \                        / --- vnet0 --- VM1
          |--- bond1 --- br0 --- |
eth3 --- /                        \ --- vnet1 --- VM2

As you can see, you can attach multiple VM's to the bridge device and share the bonded interface if you want. Although, I haven't tried that yet.

I hope this helps someone in the future. Thanks again!

voleg · 06-16-2017, 12:46 PM

Sorry to dissapoint you, the classic solution is to build bridge that include bonding interface and VM. An IP (that use by server), assigned to bridge. VM will do with it what it want. You can add all four NICs to this bond if you want to get 4G. Use LACP mode for that, you have to talk with network guy.