bond always in 10Half-Duplex?

PhillipHuang · 01-03-2007, 09:35 PM

Hi all,

Today I run into a strange problem, which so far, I've seen reported in this forums before:
http://www.linuxquestions.org/questi...10+Half+Duplex
While read the long manual, still could I not fix this problem. So I had to paste my process here for your kind help.

The Operating System is CentOS4.4(2.6.9),and both two ethernet cards in mobo are e1000. Then I configure bond(fail-over), here's "/etc/modprobe.conf":

Code:

[root@Muyang ~]# less /etc/modprobe.conf
alias eth0 e1000
alias eth1 e1000
alias scsi_hostadapter aic79xx
alias scsi_hostadapter1 ata_piix
alias usb-controller ehci-hcd
alias usb-controller1 uhci-hcd
alias bond0 bonding
options bond0 miimon=100 mode=1

Check eth0 and eth1 by mii-tools:

Code:

[root@Muyang ~]# mii-tool -v eth0
eth0: negotiated 100baseTx-FD flow-control, link ok
  product info: vendor 00:50:43, model 12 rev 2
  basic mode:   autonegotiation enabled
  basic status: autonegotiation complete, link ok
  capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
  advertising:  100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
  link partner: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control
[root@Muyang ~]# mii-tool -v eth1
eth1: no link
  product info: vendor 00:50:43, model 12 rev 2
  basic mode:   autonegotiation enabled
  basic status: no link
  capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
  advertising:  100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD flow-control

and then, view the bond0:

Code:

[root@Muyang ~]# mii-tool -v bond0
bond0: 10 Mbit, half duplex, link ok
  product info: vendor 00:01:00, model 0 rev 4
  basic mode:   10 Mbit, half duplex
  basic status: link ok
  capabilities:
  advertising:

At this point, I connect the cable to eth1, and still find bond0 is 10M HD. It seems very strange. For more testing, I tune the status of eth0 and eth1 by "mii-tool -F media interface", and get the following result:

(100-100M 10-10M FD-Full Duplex HD-Half Duplex)
bond mode=1
eth0 100FD : eth1 100HD ----- bond0 10HD
eth0 100FD : eth1 100FD ----- bond0 10HD
eth0 10FD : eth1 100HD ----- bond0 10HD
eth0 10HD : eth1 10HD ----- bond0 10HD

bond mode=0(I tried load-balance mode later)
eth0 100FD : eth1 100HD ----- bond0 10HD
eth0 100FD : eth1 100FD ----- bond0 10HD
eth0 10FD : eth1 100HD ----- bond0 10HD
eth0 10HD : eth1 10HD ----- bond0 10HD

The "mii-tool -v bond0" always output 10HD. as the referred link in top, it sounds that mii-tool "guesses" the values, need I have to modify the interal database?
Is there another way I could choose?

Please give me some hints.
Thanks in advance.

--Phillip

acid_kewpie · 01-05-2007, 05:58 AM

i think it's just mii-tool being lazy - ethtool just says nothing about bond0 as i guess it knows it's not real. the bond itself can't have a speed as such, it'll just work based upon the underlying status of the real nics it binds. remember to look at /proc/net/bonding/bond0 for details of bond0 too...

MensaWater · 01-05-2007, 10:02 AM

I for one would be willing to accept the "mii-tool doesn't really report bond0 correctly" argument given ethtool doesn't except:

1) There are some REAL interfaces that ethtool will not address but mii-tool will.

2) Ethtool which DOES report my Intel PRO NICs shows speed/duplex/autonegoatiate values for eth2 and eth3 after they've been slaved to bond0 even though the entries for those NICs have ETHTOOLOPTS set for different values.

Unfortunately the bonding info under /proc doesn't tell you anything about speed, duplex or autonegoatiate.

So the real question is: How does one determine the speed/duplex settings for bond0 if mii-tool isn't it?

Just found the following link:
http://wiki.centos.org/TipsAndTricks/BondingInterfaces

It seems to suggest one could do mode 5 in modules.conf (we don't set the mode here and I can't test it since its in production already). It requires insuring the switch knows how to do agregation. It may be this is what one needs to do to solve what we're seeing with bonding.

PhillipHuang · 01-08-2007, 12:16 AM

Thank you all. I try to modify the mode in "/etc/modprobe.conf" and restart system.

Code:

alias bond0 bonding
options bond0 miimon=100 mode=5

As the result, the output of "mii-tool -v bond0" still keeps "10HD", and I'm confused by the following red text, where does it read from?

Code:

[root@Muyang ~]# mii-tool -v bond0
bond0: 10 Mbit, half duplex, link ok
  product info: vendor 00:01:00, model 0 rev 4
  basic mode:   10 Mbit, half duplex
  basic status: link ok
  capabilities:
  advertising:

Here's the /proc/net/bonding/bond0, "Bonding Mode: transmit load balancing" prove the mode has been set to "5".

Code:

[root@Muyang ~]# cat /proc/net/bonding/bond0 
Ethernet Channel Bonding Driver: v2.6.3 (June 8, 2005)

Bonding Mode: transmit load balancing
Primary Slave: None
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:30:48:87:d5:42

Slave Interface: eth1
MII Status: down
Link Failure Count: 0
Permanent HW addr: 00:30:48:87:d5:43

acid_kewpie · 01-08-2007, 02:24 AM

as above, mii-tool output for a bond interface is meaningless. don't use it. it's presumably just reading a default value in it's own code. i'd suggest you forget about it.

PhillipHuang · 01-08-2007, 03:09 AM

Thank you, Chris. I just did some experiences in lab, just try to idenfy bond duplex mode. First, I add "ETHTOOL_OPTS="speed 10 duplex half autoneg off"
to "/etc/sysconfig/network-script/ifcfg-eth0", run "service network restart" and find eth0 is 10HD as shown:

Code:

-bash-2.05b# ethtool eth0
Settings for eth0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Supports auto-negotiation: Yes
        Advertised link modes:  Not reported
        Advertised auto-negotiation: No
        Speed: 10Mb/s
        Duplex: Half
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: umbg
        Wake-on: g
        Current message level: 0x00000007 (7)
        Link detected: yes

Yes,it's way to preserve the duplex mode even reboot. Then, I think, if I add such line to "ifcfg-bond0", would it run on identied mode? Modify "ifcfg-bond0" and add "ETHTOOL_OPTS="speed 10 duplex half autoneg off", restart network service and issue "ethtool bond0", the result is:

Code:

[root@Muyang ~]# ethtool bond0
Settings for bond0:
No data available

And mii-tool still display 10HD. Chris, as your comment #5, I would have to forget mii-tool output about bond. I think these labs could be some helpful to guys who are finding a way to preserve the current duplex setting after reboot, which is another question perhaps many people concern. So I paste the process and result here.

Now, I'll keep researching on fixing this "bond" issue.

--Phillip

acid_kewpie · 01-08-2007, 03:18 AM

yeah i still think we're at odds here. you seem to think there's a problem, but there is not. there is no problem, no issue at all. nothing is wrong.

PhillipHuang · 01-08-2007, 04:08 AM

Quote:

So the real question is: How does one determine the speed/duplex settings for bond0 if mii-tool isn't it?

Chris, what I want to find is how to check and switch the duplex setting for bond. for example, if underlying nic is 10HD, and another is 100FD, what about bond?

Regards,
--Phillip

acid_kewpie · 01-08-2007, 04:41 AM

the bond does not have a duplex. it doesn't physically exist, so that makes no sense. how the links are used depends on the mode you use. by default, it'll do a round robin on it, to half the data will go over a 10/half and the other half on the 100/full.

http://www.linuxhorizon.ro/bonding.html

PhillipHuang · 01-08-2007, 05:54 AM

OK, I see.

Thank you, Chris.

Regards,
Phillip

MensaWater · 01-08-2007, 08:26 AM

Acid - I don't see where the link you provided gives information other than the one I'd provided.

My question still remains. Assuming one can never tell the "status" of the bond0 interface why is it the underlying REAL interfaces don't report the correct status based on the ETHTOOL options hard coded into their ifcfg files?

Below follows all the ifcfg files and the related ethtool status information for the two REAL interfaces. (MAC addresses obfuscated):

cat ifcfg-eth2
# Intel Corp.|82546EB Gigabit Ethernet Controller (Copper)
DEVICE=eth2
HWADDR=00:00:00:00:00:01
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
ETHTOOL_OPTS="autoneg off speed 100 duplex full"

ethtool eth2
Settings for eth2:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Half
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: umbg
Wake-on: d
Link detected: yes

cat ifcfg-eth3
# Intel Corp.|82546EB Gigabit Ethernet Controller (Copper)
DEVICE=eth3
HWADDR=00:00:00:00:00:02
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
ETHTOOL_OPTS="autoneg off speed 100 duplex full"

ethtool eth3
Settings for eth3:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Half
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: umbg
Wake-on: d
Link detected: yes

cat ifcfg-bond0
# Bonding Device for eth2 and eth3 for Oracle RAC
DEVICE=bond0
IPADDR=172.16.0.1
NETMASK=255.255.255.0
NETWORK=172.16.0.0
BROADCAST=172.16.0.255
ONBOOT=yes
BOOTPROTO=none
USRCTL=no

greerm · 07-31-2008, 09:32 PM

If you tail -f /var/log/messages your current status for your eth devices should be available regardless if there is a bug in the ethtool or mii-tool outputs.

Example:
# tail -f /var/log/messages
Jul 31 18:35:02 dp-n kernel: bonding: bond1: link status definitely up for interface eth0.
Jul 31 18:35:17 dp-n kernel: bonding: bond1: link status definitely down for interface eth1, disabling it
Jul 31 18:35:17 dp-n kernel: bonding: bond1: making interface eth0 the new active one.
Jul 31 18:35:19 dp-n kernel: e1000: eth1: e1000_watchdog: NIC Link is Up 100 Mbps Full Duplex
Jul 31 18:35:19 dp-n kernel: bonding: bond1: link status definitely up for interface eth1

Hope this helps!