Posting this in case it helps anyone else.
We had a Dell PE 1955 blade server that had eth0 and eth1 configured for two separate VLANs. For some reason communication on eth1 quit working.
ethtool indicated link was OK. Also looking at status lights on the port on back of the blade chassis I saw same status lights for this port as I did for all other working ports on this blade and others.
In our blade chassis we have 4 of the GbE Pass-Through I/O cards (as opposed to the the network switch cards). In my reading I've discovered that Pass-Throughs in I/O slot 1 and 2 are for the embedded networking (e.g. eth0 and eth1) on each blade connected. The other two Pass-Throughs in I/O slots 2 and 3 are only used by the blades (e.g. as eth2 and eth3) if there is an ethernet daughter card installed on the blade itself. In our environment we have 4 blades with these daughter cards and 6 without in this chassis. The one with the issue didn't have the daughter card. You can see whether the daughter card is installed by going to Dell OpenManage for the blade in question and looking at DC_CONN under slots. (Including this info as it was not all conveniently in one place as it is here.)
I had installed Dell OpenManage but on trying to modify iptables to open port 1311 for that I had other issues. Given both problems I had tried doing a reboot and had pushed the power button on the blade itself.
After the reboot my issues with iptables were resolved but I saw that my ifcfg-eth1 had been renamed ifcfg-eth1.bak and a new ifcfg-eth1 using DHCP defaults and HWAddre (MAC) EA:5E:00:0C:EA:5E had been created.
Typically Dell MACs start with 00:15:C5 so I wasn't sure what this other MAC was.
I tried modifying the new ifcfg-eth1 to use the new MAC but change from DHCP to static and provide the information for using the original IP. However on trying to start the interface I got:
Quote:
SIOCSIFFLAGS: Device or resource busy
Failed to bring up eth1.
|
A web search on that revealed some hits which talk about the interrupt.
On running cat /proc/interrupts it dislayed MSI interrupt for eth0 but nothing for eth1. On comparing this to the other blades they all have it for eth0 and eth1 and some also have for eth2 and eth3.
Therefore I tried changing MAC to what was in old file but it said this was not the expected address so was ignoring it.
I had about given up on this thinking it must be an issue with the blade's baseboard (since eth0 and eth1 are on the embedded controller) but on searching again for the odd MAC I found a link that talks about seeing same thing on Dell PE 1950 and 2950 that indicated it was also seen on some HPs and appears to be a problem with Broadcom NICs (which are what I have in the PE 1955). The link was:
https://stomp.colorado.edu/blog/blog...rily-changing/
Based on that poster's comments about simply needing to remove power from his 1950 and 2950 to resolve this I again shutdown the blade and this time physically removed it from the blade chassis for more than a minute. After re
inserting it and powering up the NIC restored the original MAC (00.15.c5…) expected for Dells.
For some reason it again moved my ifcfg-eth1 to ifcfg-eth1.bak on doing this (maybe during shutdown) so I had to manually move that back to ifcfg-eth1 and run ifup eth1 but on doing so the network came back up successfully.