Alright, I am starting to loose my mind trying to get this two node cluster setup. I'm currently stuck on getting the crossover connection between my two blades working. The network admin just got it plugged in for me. I've currently got 2 FCoE cards in each blade, trunked for front-end/back-end/back-up network access. Then I've got the copper port with a crossover cable between the blades. I've tried setting up the interface like such:

node1
# Broadcom Corporation NetXtreme II BCM5709S Gigabit Ethernet
DEVICE=eth4
BOOTPROTO=static
HWADDR=84:2B:2B:25:E4:FD
ONBOOT=yes
HOTPLUG=no
IPADDR=10.200.23.118
NETMASK=255.255.255.252
BROADCAST=10.200.23.119
GATEWAY=10.200.23.117

node2
# Broadcom Corporation NetXtreme II BCM5709S Gigabit Ethernet
DEVICE=eth4
BOOTPROTO=static
HWADDR=84:2B:2B:25:E4:F0
ONBOOT=yes
HOTPLUG=no
IPADDR=10.200.23.117
NETMASK=255.255.255.252
BROADCAST=10.200.23.119
GATEWAY=10.200.23.118

But when I bring the interface up it's not attaching an IP:
eth4 Link encap:Ethernet HWaddr 84:2B:2B:25:E4:F0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Interrupt:178 Memory:da000000-da012800

And of course I can't actually use the interface. Am I setting something up wrong here?

I'm confused, why would your gateway be the IP of the other machine and vice versa? Are both these blades on the same switch or vlan? How are you bringing them up? What if you manually add the ip info for each, what happens?

I did the gateway thing after reading around. It was mentioned that it wasn't necessary, but I thought I'd try it. Origionally the gateway wasn't specified. Both blades are on the same switch and in the same VLANs. This crossover connection is directly from one blade to the other, though, no switch involved. I am just doing an "ipconfig eth4 up" to bring up the interface. I am now able to get an IP assigned to the interface, but it's still not usable. When I try to force a route into the routing table for the crossover I cannot add it:

[root@usxsl227 network-scripts]# route add -net 10.200.23.0 gw 10.200.23.118 netmask 255.255.255.0 dev eth5
SIOCADDRT: Network is unreachable


Should I try something else as the gateway? I mean, logically I shouldn't need a gateway because there's no switch involved, the problem is the box won't use the interface, at all.

So you have eth4 on each connected by crossover? You know if you have open ports on the switch you don't need a crossover to achieve clustering. Just connect eth4 to the switch, don't use the crossover and configure accordingly for the cluster.

The switch is an FCoE swich (a cisco nexus) so there's only a handful of copper ports available. I mean, if I don't need to use a crossover that's excellent, but I can't get the multicast to work over the trunk at all. I'm guessing because something's not setup on the switch for it. When I sent the documentation to the network admin his was response was that he'd put in this crossover cable for me. I assumed it would be a pretty simple setup! (always a mistake, assuming!)

You better to ask your company to get you another cheap switch to do the complex Linux clustering configuration.............