We have two Dell R310s. Both have a 2xIntel Gigabit ET Dual Port Server Adapter card and also two Broadcom on-board ports. Both are configured in a similar fashion and exhibit the same behavior.
The Behavior:
The left port on the Intel card is associated with the Linux interfaces eth0 and eth1. The right port is associated with the Linux interfaces eth2 and eth3. eth2 and eth3 are recognized as being up and running, while eth0 and eth1 do not.
The `ip link show` command results in the following output when eth0, eth2, and eth3 are configured and "up":
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether eth0macaddr brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether eth0macaddr+1 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether eth2macaddr brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether eth2macaddr+1 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether eth4macaddr brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether eth5macaddr brd ff:ff:ff:ff:ff:ff
I don't know whether what I am looking for is correct, but in order for eth0 to work correctly (as an individual interface or as a member of a channel bonding interface), I believe that it needs to be considered an "up" link.
When eth0 is not "up", the `ip` output is:
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc mq state DOWN qlen 1000
link/ether eth0macaddr brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether eth0macaddr+1 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether eth2macaddr brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether eth2macaddr+1 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether eth4macaddr brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether eth5macaddr brd ff:ff:ff:ff:ff:ff
The Configuration:
The operating system is RHEL6.3 (2.6.32-279.2.1.el6.x86_64). Both ports on the Intel card are connected (2 different subnets - subnet A and subnet B), and the active and link lights are lit on both ports.
The dmesg output is:
Code:
# dmesg | grep eth
igb 0000:04:00.0: eth0: (PCIe:2.5Gb/s:Width x4) eth0macaddr
igb 0000:04:00.0: eth0: PBA No: G18758-003
igb 0000:04:00.1: eth1: (PCIe:2.5Gb/s:Width x4) eth0macaddr+1
igb 0000:04:00.1: eth1: PBA No: G18758-003
igb 0000:05:00.0: eth2: (PCIe:2.5Gb/s:Width x4) eth2macaddr
igb 0000:05:00.0: eth2: PBA No: G18758-003
igb 0000:05:00.1: eth3: (PCIe:2.5Gb/s:Width x4) eth2macaddr+1
igb 0000:05:00.1: eth3: PBA No: G18758-003
bnx2 0000:02:00.0: eth4: Broadcom NetXtreme II BCM5716 1000Base-T (C0) PCI Express found at mem d8000000, IRQ 16, node addr eth4macaddr
bnx2 0000:02:00.1: eth5: Broadcom NetXtreme II BCM5716 1000Base-T (C0) PCI Express found at mem da000000, IRQ 17, node addr eth5macaddr
ACPI Error (psparse-0537): Method parse/execution failed [\_SB_.PMI0._GHL] (Node ffff88013d6a2f60), AE_NOT_EXIST
ACPI Error (psparse-0537): Method parse/execution failed [\_SB_.PMI0._PMC] (Node ffff88013d6a2420), AE_NOT_EXIST
ADDRCONF(NETDEV_UP): eth2: link is not ready
igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
ADDRCONF(NETDEV_UP): eth3: link is not ready
The /var/log/messages file shows the following:
Code:
Aug 17 17:51:05 hostname kernel: igb 0000:04:00.0: eth0: (PCIe:2.5Gb/s:Width x4) eth0macaddr
Aug 17 17:51:05 hostname kernel: igb 0000:04:00.0: eth0: PBA No: G18758-003
Aug 17 17:51:05 hostname kernel: igb 0000:04:00.1: eth1: (PCIe:2.5Gb/s:Width x4) eth0macaddr+1
Aug 17 17:51:05 hostname kernel: igb 0000:04:00.1: eth1: PBA No: G18758-003
Aug 17 17:51:05 hostname kernel: igb 0000:05:00.0: eth2: (PCIe:2.5Gb/s:Width x4) eth2macaddr
Aug 17 17:51:05 hostname kernel: igb 0000:05:00.0: eth2: PBA No: G18758-003
Aug 17 17:51:05 hostname kernel: igb 0000:05:00.1: eth3: (PCIe:2.5Gb/s:Width x4) eth2macaddr+1
Aug 17 17:51:05 hostname kernel: igb 0000:05:00.1: eth3: PBA No: G18758-003
Aug 17 17:51:05 hostname kernel: bnx2 0000:02:00.0: eth4: Broadcom NetXtreme II BCM5716 1000Base-T (C0) PCI Express found at mem d8000000, IRQ 16, node addr eth4macaddr
Aug 17 17:51:05 hostname kernel: bnx2 0000:02:00.1: eth5: Broadcom NetXtreme II BCM5716 1000Base-T (C0) PCI Express found at mem da000000, IRQ 17, node addr eth5macaddr
Aug 17 17:51:05 hostname kernel: ACPI Error (psparse-0537): Method parse/execution failed [\_SB_.PMI0._GHL] (Node ffff88013d6a2f60), AE_NOT_EXIST
Aug 17 17:51:05 hostname kernel: ACPI Error (psparse-0537): Method parse/execution failed [\_SB_.PMI0._PMC] (Node ffff88013d6a2420), AE_NOT_EXIST
Aug 17 17:51:05 hostname kernel: ADDRCONF(NETDEV_UP): eth2: link is not ready
Aug 17 17:51:05 hostname kernel: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Aug 17 17:51:05 hostname kernel: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
Aug 17 17:51:05 hostname kernel: ADDRCONF(NETDEV_UP): eth3: link is not ready
Aug 17 17:51:05 hostname kernel: igb: eth3 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
Aug 17 17:51:05 hostname kernel: ADDRCONF(NETDEV_CHANGE): eth3: link becomes ready
Things that I have tried include:
- Updated the BIOS and firmware
- Installed the latest version of ethtool (ethtool version 3.5)
- Installed the latest version of the igb driver (version 3.4.8)
There has been no change in behavior.
Has anyone run into this before? Any suggestions as to what I should look at next would be greatly appreciated.
