LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Server (https://www.linuxquestions.org/questions/linux-server-73/)
-   -   Debian server 8 nic bonding help (https://www.linuxquestions.org/questions/linux-server-73/debian-server-8-nic-bonding-help-4175579300/)

mati-92 05-08-2016 06:32 PM

Debian server 8 nic bonding help
 
good evening

im still pretty new to the linux world and i have setup a debian 8 server in my home.
i have a cisco switch setup with 6 ports with LACP.
i have set up NIC bonding on the debian server using mode 4 to coincide with my cisco switch and for some weird reason when i perform this command

cat /proc/net/bonding/bond0

output

Bonding mode: Load Balancing (round-robin)
MII Status: up MII Polling Interval (ms):100

i know i set this up as mode 4 for lacp

just wondering if anyone else has come across this issue and how they resolved it as im not sure whether it actually using mode 4 or round robin

cheers

mati

lazydog 05-09-2016 11:35 AM

What does your bond interface config look like? Do you have "BONDING_OPTS=" set?

mati-92 05-09-2016 05:01 PM

my /etc/network/interfaces looks like this:

# This file describes the network interfaces available pn you system
# and how to activate them. For more information, see interfaces (5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# eth0 is manually configured, and slave to the "bond0" bonded nic
auto eth0
iface eth0 inet manual
bond-master bond0
bond #-primary = eth0

auto eth1
iface eth1 inet manual
bond-master bond 0

auto eth2
iface eth2 inet manual
bond-master bond 0

auto eth3
iface eth3 inet manual
bond-master bond 0

auto eth4
iface eth4 inet manual
bond-master bond 0

auto eth5
iface eth5 inet manual
bond-master bond 0

#The bonded inteface
auto bond 0
iface bond 0 inet static
address 192.168.0.25
netmask 255.255.255.0
gateway 192.168.0.1
dns-nameservers 8.8.8.8
bond-mode 4
bond-primary eth0
bond- miimon 100
bond-slaves none

mati-92 05-09-2016 05:02 PM

Quote:

Originally Posted by lazydog (Post 5542715)
What does your bond interface config look like? Do you have "BONDING_OPTS=" set?

my /etc/network/interfaces looks like this:

# This file describes the network interfaces available pn you system
# and how to activate them. For more information, see interfaces (5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# eth0 is manually configured, and slave to the "bond0" bonded nic
auto eth0
iface eth0 inet manual
bond-master bond0
bond-primary = eth0

auto eth1
iface eth1 inet manual
bond-master bond 0

auto eth2
iface eth2 inet manual
bond-master bond 0

auto eth3
iface eth3 inet manual
bond-master bond 0

auto eth4
iface eth4 inet manual
bond-master bond 0

auto eth5
iface eth5 inet manual
bond-master bond 0

#The bonded inteface
auto bond 0
iface bond 0 inet static
address 192.168.0.25
netmask 255.255.255.0
gateway 192.168.0.1
dns-nameservers 8.8.8.8
bond-mode 4
bond-primary eth0
bond-miimon 100
bond-slaves none

also seems to be a bit slow for some reason i am connected to the server via ssh and its lagging when copying large dumps of data over to the server

lazydog 05-10-2016 07:46 AM

Quote:

Originally Posted by mati-92 (Post 5542849)
my /etc/network/interfaces looks like this:

# eth0 is manually configured, and slave to the "bond0" bonded nic
auto eth0
iface eth0 inet manual
bond-master bond0
bond-primary = eth0

auto eth1
iface eth1 inet manual
bond-master bond 0

Change your setup from "bond 0" to "bond0"

Quote:

#The bonded inteface
auto bond 0
iface bond 0 inet static
address 192.168.0.25
netmask 255.255.255.0
gateway 192.168.0.1
dns-nameservers 8.8.8.8
bond-mode 4
bond-primary eth0
bond-miimon 100
bond-slaves none
Shouldn't bond-slave read as follows:
Code:

bond-slaves eth0 eth1 eth2 eth3 eth4 eth5
Quote:

also seems to be a bit slow for some reason i am connected to the server via ssh and its lagging when copying large dumps of data over to the server
Not sure if you are aware of the fact that any conversation will only use one (1) interface for communications. Doesn't matter how many interfaces you bond together.

mati-92 05-10-2016 04:07 PM

Quote:

Originally Posted by lazydog (Post 5543159)
Change your setup from "bond 0" to "bond0"



Shouldn't bond-slave read as follows:
Code:

bond-slaves eth0 eth1 eth2 eth3 eth4 eth5


Not sure if you are aware of the fact that any conversation will only use one (1) interface for communications. Doesn't matter how many interfaces you bond together.

thanks Robert i have made the changes that you have suggested. as for being aware of the information being sent and received from one card i didn't know that.

dgorack 05-11-2016 12:35 PM

Quote:

Originally Posted by lazydog (Post 5543159)
Change your setup from "bond 0" to "bond0"



Shouldn't bond-slave read as follows:
Code:

bond-slaves eth0 eth1 eth2 eth3 eth4 eth5


Not sure if you are aware of the fact that any conversation will only use one (1) interface for communications. Doesn't matter how many interfaces you bond together.

Not sure the above changes fixed OP's issue but just to add more info: "bond-slaves none" is fine because he has "bond-master bond 0" (which should be "bond-master bond0") under the slave interface definitions. I find the configuration more flexible when done like that.

On Debian (and debian based distros), for LACP, /proc/net/bonding/bond0 should display:
Code:

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
...

For reference and if it may help, I generally setup lacp as follows:

/etc/network/interfaces
Code:

auto eth0
iface eth0 inet manual
bond-master bond0

auto eth1
iface eth1 inet manual
bond-master bond0

auto bond0
iface bond0 inet static
address 192.168.0.25
netmask 255.255.255.0
broadcast 192.168.0.255
gateway 192.168.0.1
dns-nameservers 8.8.8.8 8.8.4.4
dns-search optional.com
bond-mode 802.3ad
bond-miimon 100
bond-lacp-rate 1
bond-slaves none

You can also set the bonding options like:

/etc/modprobe.d/bonding.conf
Code:

bonding mode=4 miimon=100 lacp_rate=1
On the switch end and for cisco switches in particular for LACP your "channel-group" config line under all your member interfaces should end in "mode active".

lazydog 05-11-2016 12:57 PM

Thnx for your feed back dgorack. There are many ways to do things in Linux and I don't mind learning a new way at all.

mati-92 05-11-2016 01:42 PM

Quote:

Originally Posted by dgorack (Post 5543919)
Not sure the above changes fixed OP's issue but just to add more info: "bond-slaves none" is fine because he has "bond-master bond 0" (which should be "bond-master bond0") under the slave interface definitions. I find the configuration more flexible when done like that.

On Debian (and debian based distros), for LACP, /proc/net/bonding/bond0 should display:
Code:

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
...

For reference and if it may help, I generally setup lacp as follows:

/etc/network/interfaces
Code:

auto eth0
iface eth0 inet manual
bond-master bond0

auto eth1
iface eth1 inet manual
bond-master bond0

auto bond0
iface bond0 inet static
address 192.168.0.25
netmask 255.255.255.0
broadcast 192.168.0.255
gateway 192.168.0.1
dns-nameservers 8.8.8.8 8.8.4.4
dns-search optional.com
bond-mode 802.3ad
bond-miimon 100
bond-lacp-rate 1
bond-slaves none

You can also set the bonding options like:

/etc/modprobe.d/bonding.conf
Code:

bonding mode=4 miimon=100 lacp_rate=1
On the switch end and for cisco switches in particular for LACP your "channel-group" config line under all your member interfaces should end in "mode active".

This is so weird i have made the changes that you suggested and at the present moment i am copying stuff over to the linux server.
i have logged into my cisco switch to check my configuration to make sure that my channel-group is set to active and it is. so i then did a show ip int br and it says that it is down but i have lights on flashing to indicate that it is communicating and my files are on the server. so strange
mati

dgorack 05-11-2016 02:12 PM

Quote:

Originally Posted by mati-92 (Post 5543954)
This is so weird i have made the changes that you suggested and at the present moment i am copying stuff over to the linux server.
i have logged into my cisco switch to check my configuration to make sure that my channel-group is set to active and it is. so i then did a show ip int br and it says that it is down but i have lights on flashing to indicate that it is communicating and my files are on the server. so strange
mati

Provide us a little more info and maybe we can help. Let's make sure LACP is properly configured. I assume you're restarting your networking/interfaces after you've made config changes.

On your server

Post your updated configuration (/etc/network/interfaces and anywhere else you set your network/lacp options)

Post the new output of /proc/net/bonding/bond0


On your switch

Post the your relevant interface and channel-group config or if you can link to a pastebin of your sanitized "sh run"

Post the output of "sh lacp neighbor" and "sh lacp internal"

Post the output of "sh int port-channel n" where n is your port-channel number

mati-92 05-11-2016 02:45 PM

4 Attachment(s)
Quote:

Originally Posted by dgorack (Post 5543977)
Provide us a little more info and maybe we can help. Let's make sure LACP is properly configured. I assume you're restarting your networking/interfaces after you've made config changes.

On your server

Post your updated configuration (/etc/network/interfaces and anywhere else you set your network/lacp options)

Post the new output of /proc/net/bonding/bond0


On your switch

Post the your relevant interface and channel-group config or if you can link to a pastebin of your sanitized "sh run"

Post the output of "sh lacp neighbor" and "sh lacp internal"

Post the output of "sh int port-channel n" where n is your port-channel number

please see the images attached i grabbed as much information as i could.

dgorack 05-11-2016 07:33 PM

Quote:

Originally Posted by mati-92 (Post 5543996)
please see the images attached i grabbed as much information as i could.

It would really be helpful if you'd post the output instead of attaching screen grabs. I think it's just good etiquette and it also my life easier in trying to help you (copy/paste).

Your output of /proc/net/bonding/bond0 shows link failures which is probably why it's falling back to round-robin. Like @lazydog said, there are a number of way to do this but I can show you the way I use. Your switch output also shows the each port in your channel-group in state "indep" which mean it failed to bond lacp.

First of all let's clean up your /etc/network/interfaces file.

Check for typos as I'm typing this manually, you know, because you weren't bothered to copy/paste here.
Code:

...

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
bond-master bond0

auto eth1
iface eth1 inet manual
bond-master bond0

auto eth2
iface eth2 inet manual
bond-master bond0

auto eth3
iface eth3 inet manual
bond-master bond0

auto eth4
iface eth4 inet manual
bond-master bond0

auto eth5
iface eth5 inet manual
bond-master bond0

auto bond0
iface bond0 inet static
address 192.168.0.25
netmask 255.255.255.0
broadcast 192.168.0.255
gateway 192.168.0.1
dns-nameservers 8.8.8.8 8.8.4.4
bond-mode 802.3ad
bond-miimon 100
bond-lacp-rate 1
bond-slaves none

Also make sure the bonding module is loaded
Code:

lsmod | grep bond
I can't help you with your switch because you didn't post your running-config, post that in a pastebin or in code blocks, no screen grabs please.

mati-92 05-12-2016 03:48 AM

Quote:

Originally Posted by dgorack (Post 5544117)
It would really be helpful if you'd post the output instead of attaching screen grabs. I think it's just good etiquette and it also my life easier in trying to help you (copy/paste).

Your output of /proc/net/bonding/bond0 shows link failures which is probably why it's falling back to round-robin. Like @lazydog said, there are a number of way to do this but I can show you the way I use. Your switch output also shows the each port in your channel-group in state "indep" which mean it failed to bond lacp.

First of all let's clean up your /etc/network/interfaces file.

Check for typos as I'm typing this manually, you know, because you weren't bothered to copy/paste here.
Code:

...

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet manual
bond-master bond0

auto eth1
iface eth1 inet manual
bond-master bond0

auto eth2
iface eth2 inet manual
bond-master bond0

auto eth3
iface eth3 inet manual
bond-master bond0

auto eth4
iface eth4 inet manual
bond-master bond0

auto eth5
iface eth5 inet manual
bond-master bond0

auto bond0
iface bond0 inet static
address 192.168.0.25
netmask 255.255.255.0
broadcast 192.168.0.255
gateway 192.168.0.1
dns-nameservers 8.8.8.8 8.8.4.4
bond-mode 802.3ad
bond-miimon 100
bond-lacp-rate 1
bond-slaves none

Also make sure the bonding module is loaded
Code:

lsmod | grep bond
I can't help you with your switch because you didn't post your running-config, post that in a pastebin or in code blocks, no screen grabs please.

Sorry i wasnt sure how to copy from putty. I hand typed the previous post i thought it would of been easier. Sorry again i shall copy and paste everything in now.

mati-92 05-12-2016 04:00 AM

Code:

***********************************************
UNAUTHORISED ACCESS
***********************************************

LoftR12950G>en
Password:
LoftR12950G#show run
Building configuration...

Current configuration : 3265 bytes
!
version 12.1
no service pad
service timestamps debug uptime
service timestamps log uptime
service password-encryption
!
hostname LoftR12950G
!
enable secret 5
enable password 7
!
username mati secret 5 ip subnet-zero
!
no ip domain-lookup
ip domain-name bennetthome.com
ip ssh time-out 120
ip ssh authentication-retries 3
ip ssh version 2
!
spanning-tree mode pvst
no spanning-tree optimize bpdu transmission
spanning-tree extend system-id
!
!
!
!
interface Port-channel1
 switchport mode access
 flowcontrol send off
!
interface FastEthernet0/1
 switchport mode access
!
interface FastEthernet0/2
 switchport mode access
!
interface FastEthernet0/3
 switchport mode access
 shutdown
!
interface FastEthernet0/4
 switchport mode access
!
interface FastEthernet0/5
 switchport mode access
 shutdown
!
interface FastEthernet0/6
 switchport mode access
 shutdown
!
interface FastEthernet0/7
 switchport mode access
 shutdown
!
interface FastEthernet0/8
 switchport mode access
 shutdown
!
interface FastEthernet0/9
 switchport mode access
 shutdown
!
interface FastEthernet0/10
 switchport mode access
 shutdown
!
interface FastEthernet0/11
 switchport mode access
 shutdown
!
interface FastEthernet0/12
 switchport mode access
 shutdown
!
interface FastEthernet0/13
 switchport mode trunk
 shutdown
!
interface FastEthernet0/14
 description trunk port to second switch
 switchport trunk allowed vlan 1
 switchport mode trunk
 switchport nonegotiate
 shutdown
!
interface FastEthernet0/15
 switchport mode access
 shutdown
!
interface FastEthernet0/16
 switchport mode access
 shutdown
!
interface FastEthernet0/17
 description Server ports
 switchport mode access
 channel-group 1 mode active
!
interface FastEthernet0/18
 description Server ports
 switchport mode access
 channel-group 1 mode active
!
interface FastEthernet0/19
 description Server ports
 switchport mode access
 channel-group 1 mode active
!
interface FastEthernet0/20
 description Server ports
 switchport mode access
 channel-group 1 mode active
!
interface FastEthernet0/21
 description Server ports
 switchport mode access
 channel-group 1 mode active
!
interface FastEthernet0/22
 description Server ports
 switchport mode access
 channel-group 1 mode active
!
interface FastEthernet0/23
 description Server ports
 switchport mode access
!
interface FastEthernet0/24
 switchport mode access
!
interface GigabitEthernet0/1
 shutdown
!
interface GigabitEthernet0/2
 shutdown
!
interface Vlan1
 ip address 192.168.0.150 255.255.255.0
 no ip route-cache
!
interface Vlan20
 no ip address
 no ip route-cache
 shutdown
!
interface Vlan50
 no ip address
 no ip route-cache
 shutdown
!
ip default-gateway 192.168.0.1
ip http server
banner motd ^C
***********************************************
UNAUTHORISED ACCESS
***********************************************
^C
!
line con 0
 exec-timeout 0 0
 password 7 11081D081E1C5C545D
 logging synchronous
 login
line vty 0 4
 exec-timeout 0 0
 password 7
 logging synchronous
 login local
 transport input ssh
line vty 5 15
 exec-timeout 0 0
 password 7
 logging synchronous
 login local
 transport input ssh
!
!
end

Code:

root@server:/home/mati# lsmod | grep bonding
bonding              124989  0


mati-92 05-12-2016 04:06 AM

Quote:

Originally Posted by mati-92 (Post 5544283)
Code:

***********************************************
UNAUTHORISED ACCESS
***********************************************

LoftR12950G>en
Password:
LoftR12950G#show run
Building configuration...

Current configuration : 3265 bytes
!
version 12.1
no service pad
service timestamps debug uptime
service timestamps log uptime
service password-encryption
!
hostname LoftR12950G
!
enable secret 5 $1$g693$XM3jlK2NgRbghIDtakST20
enable password 7 022C0B5A05080A7018175B
!
username mati secret 5 $1$EIjG$vnrL4Lx39OL92vM8bCi8P/
ip subnet-zero
!
no ip domain-lookup
ip domain-name bennetthome.com
ip ssh time-out 120
ip ssh authentication-retries 3
ip ssh version 2
!
spanning-tree mode pvst
no spanning-tree optimize bpdu transmission
spanning-tree extend system-id
!
!
!
!
interface Port-channel1
 switchport mode access
 flowcontrol send off
!
interface FastEthernet0/1
 switchport mode access
!
interface FastEthernet0/2
 switchport mode access
!
interface FastEthernet0/3
 switchport mode access
 shutdown
!
interface FastEthernet0/4
 switchport mode access
!
interface FastEthernet0/5
 switchport mode access
 shutdown
!
interface FastEthernet0/6
 switchport mode access
 shutdown
!
interface FastEthernet0/7
 switchport mode access
 shutdown
!
interface FastEthernet0/8
 switchport mode access
 shutdown
!
interface FastEthernet0/9
 switchport mode access
 shutdown
!
interface FastEthernet0/10
 switchport mode access
 shutdown
!
interface FastEthernet0/11
 switchport mode access
 shutdown
!
interface FastEthernet0/12
 switchport mode access
 shutdown
!
interface FastEthernet0/13
 switchport mode trunk
 shutdown
!
interface FastEthernet0/14
 description trunk port to second switch
 switchport trunk allowed vlan 1
 switchport mode trunk
 switchport nonegotiate
 shutdown
!
interface FastEthernet0/15
 switchport mode access
 shutdown
!
interface FastEthernet0/16
 switchport mode access
 shutdown
!
interface FastEthernet0/17
 description Server ports
 switchport mode access
 channel-group 1 mode active
!
interface FastEthernet0/18
 description Server ports
 switchport mode access
 channel-group 1 mode active
!
interface FastEthernet0/19
 description Server ports
 switchport mode access
 channel-group 1 mode active
!
interface FastEthernet0/20
 description Server ports
 switchport mode access
 channel-group 1 mode active
!
interface FastEthernet0/21
 description Server ports
 switchport mode access
 channel-group 1 mode active
!
interface FastEthernet0/22
 description Server ports
 switchport mode access
 channel-group 1 mode active
!
interface FastEthernet0/23
 description Server ports
 switchport mode access
!
interface FastEthernet0/24
 switchport mode access
!
interface GigabitEthernet0/1
 shutdown
!
interface GigabitEthernet0/2
 shutdown
!
interface Vlan1
 ip address 192.168.0.150 255.255.255.0
 no ip route-cache
!
interface Vlan20
 no ip address
 no ip route-cache
 shutdown
!
interface Vlan50
 no ip address
 no ip route-cache
 shutdown
!
ip default-gateway 192.168.0.1
ip http server
banner motd ^C
***********************************************
UNAUTHORISED ACCESS
***********************************************
^C
!
line con 0
 exec-timeout 0 0
 password 7 11081D081E1C5C545D
 logging synchronous
 login
line vty 0 4
 exec-timeout 0 0
 password 7 07252E4D40071C54434B59
 logging synchronous
 login local
 transport input ssh
line vty 5 15
 exec-timeout 0 0
 password 7 07252E4D40071C54434B59
 logging synchronous
 login local
 transport input ssh
!
!
end

Code:

root@server:/home/mati# lsmod | grep bonding
bonding              124989  0


Just had a look at this as i noticed this on startup

Code:

root@server:/home/mati# systemctl status systemd-modules-load
● systemd-modules-load.service - Load Kernel Modules
  Loaded: loaded (/lib/systemd/system/systemd-modules-load.service; static)
  Active: failed (Result: exit-code) since Thu 2016-05-12 09:51:14 BST; 13min ago
    Docs: man:systemd-modules-load.service(8)
          man:modules-load.d(5)
  Process: 173 ExecStart=/lib/systemd/systemd-modules-load (code=exited, status=1/FAILURE)
 Main PID: 173 (code=exited, status=1/FAILURE)

May 12 09:51:14 server systemd-modules-load[173]: Inserted module 'ppdev'
May 12 09:51:14 server systemd-modules-load[173]: Inserted module 'parport_pc'
May 12 09:51:14 server systemd-modules-load[173]: Inserted module 'fuse'
May 12 09:51:14 server systemd-modules-load[173]: Inserted module 'loop'
May 12 09:51:14 server systemd-modules-load[173]: Failed to find module 'rtc'
May 12 09:51:14 server systemd-modules-load[173]: Inserted module 'bonding'
May 12 09:51:14 server systemd-modules-load[173]: Inserted module 'mii'
May 12 09:51:14 server systemd[1]: systemd-modules-load.service: main proce...RE
May 12 09:51:14 server systemd[1]: Failed to start Load Kernel Modules.
May 12 09:51:14 server systemd[1]: Unit systemd-modules-load.service entere...e.
Hint: Some lines were ellipsized, use -l to show in full.



All times are GMT -5. The time now is 10:40 PM.