I'm trying to get a nic failover type of situation on an IBM blade server running RHEL3. I've got two nics visible to the OS, both using the bcm5700 driver. They are statically assigned, and have all the correct info in their respective ifcfg files. But i am not getting what I expect out of the setup.

Normal boot, eth0 is set to come up and eth1 is set to stay off. It comes up without a problem and networking works great. If I change this by making eth0 stay off and eth1 come up on boot, things don't work. ifconfig shows eth1 assigned the right ip and everything, but the network just doesn't work. That is until I use ethtool to turn on auto-negotiation. As soon as I do that, it's fine. But that apparently isn't a static setting and rebooting puts us back at square one.

The odd thing about it is that eth0 doesn't seem to need auto negotation turned on for it to work properly. Just eth1. My modules.conf contains the following, in case this info is helpful:

alias eth0 bcm5700
alias eth1 bcm5700
options bcm5700 auto_speed=0,0 line_speed=100,100 full_duplex=1,1

ifcfg-eth0:
DEVICE=eth0
BOOTPROTO=static
HWADDR=00:00:00:00:00:00
IPADDR=x.x.x.x
NETMASK=x.x.x.x
ONBOOT=no
TYPE=Ethernet

ifcfg-eth1:
DEVICE=eth1
BOOTPROTO=static
HWADDR=00:00:00:00:00:00
IPADDR=x.x.x.x
NETMASK=x.x.x.x
ONBOOT=yes
TYPE=Ethernet

Obviously, the HWADDR has been changed, that's not what i've actually got the file. It is set correctly, as is the ipaddr and netmask. Can anyone else point to something i'm doing wrong? or some solution to this problem? i'm running out of ideas.

what do "route" and "ifconfig" say about the state of things? maybe the default gateway is trying to route through eth0 still?

everything looks ok in route and ifconfig. the only thing is that on boot when booting to eth1, the network doesn't come up and the route command shows the default route as the ip address. but when i turn autonegotiate on, and do route again, it shows the hostname of that ip address instead of just the ip address.

I don't get why simply turning autonegotiate on cures the problem right away. And why this doesn't seem to be neccessary on eth0.