Hi,

after some generic network advice actually, so i'll break my own rules and post in a linux forum anyway...

With regards to ethernet best practises and such, i've read various statements saying that two servers should never have a crossover cable connecting them, and instead should go via a switch, even if the connectivity is only for those two boxes. My situation here is a pair of Nokia Checkpoint IP1220's private ClusterXL HA network. Our firewall guys demanded that the netwrok there goes via a cisco switch, whereas my knowledge of 100/Full networks says that there shouldn't be any problem at all in using a single crossover cable between the boxes, indeed it'd be more resilient and theoretically faster.

Anyone able to comment on this?

I would be interested in hearing what your firewall guys have as reasons for this.

By connecting directly via a crossover cable you are doing two things:

1. Removing a latency factor of a switch, which has to take some amount of time to analyze the frames being sent and send them to the right place.
2. Removing a fault point. Cables don't really go bad that often, as long as they aren't beat around and rolled over by your chair. Switches on the other hand are subject to breakdowns since they are powered pieces of equipment, and if they are managed switches they are even more prone to failure, be it by software crashes of Cisco equipment (yes, I have seen it) or human mistakes when reconfiguring the router.

The disadvantage as I see it in this case is that you are going to lose the flexibility that would be created by having them plugged into a switch. Since this is an HA situation, and I assume that you have other network connections to the machines going out to the real world, this is pretty much moot unless you decide to go to an even large HA situation. Basically this cable is going to act as a heartbeat lifeline for you in the case that your primary machine goes down.

One other possible problem you may run into is there are cases where NICs just don't like playing with each other in a direct connect situation. That is just something that you will just have to test and see. You DO have a testing phase, right?

From what you have explained so far, I think you would be better off telling your firewall guys to go play somewhere else. I won't even go into the cost savings that using a simple crossover cable creates as compared to using up two ports on a potentially very expensive Cisco switch (hey... we got rid of a nearly fully loaded Cisco 6509 recently because we realized that we were never going to use it to its' full potential. Instead we went down to a 4948 and six other 48 port switches in a leaf model. Works fine for what we are using it for, which is a private network for a cluster of 200 machines.)

That all seems like what i wanted to be hearing... in another part of the business i deal with we have a pair of VMware ESX servers with a similar requirement (for vMotion if you know about vmware) and the network guys there all said to just stick in a cable and be done with it.

The thing about our firewall guys is that it's not about what does and doesn't work, it's about what they normally do, regardless of global best practises.

The reason this came about is actaully cos one of the two Cisco 2970's they were connected to popped a power supply. Their setup of choice was to go IP1220---2970---2970---IP1220. doesn't seem logical to me in anyway as it's only the two nodes involved, why have 5 points of failure when you can just have 1?

luckily he doesn't work in our head office, so will not see the single crossover cable anyway, so i'll probably just lie to him!

One draw-back of the cross-over method is that you cannot look at the interface statistics on the switch to diagnose possible problems. I suppose you could look at each box individually and check the netstat -s or what have you, but I've found in the past that switch port interface stats are more granular than what an OS typically provides.

Why are the firewalls clustered, is it for HA or higher performance? If it's for HA, how does the second box detect that it's supposed to take over? You want to be careful not to defeat the method it's using to detect failure. If it's counting on the first box being unreachable, and you just plug them together directly, there are some scenarios where the primary wouldn't be available via the switch, but would be available via the cross-over. In that case would the secondary not take over?

they run ClusterXL, which is basically Active/Active so it's lots of data like state tables more than the availability checking side. I'm not really that sure on how going via a switch is much use on the HA side anyway, i mean if the other box is unreachable, how do you know if it's your fault or theirs? same with a cable i guess.

So it doesn't really matter than. They don't need to detect reachability failures.

The idea is that if the other device is on another switch, the switch might have gone down. If your fail-over depended on the passive box detecting that the active had failed, but you were plugged right into it with cross-over, when their switch goes down you wouldn't detect it across your heart-beat link (because it would still be reachable). In your case it doesn't matter because they're active/active any way.

For sharing state data I would prefer cross-over. Removing switches from the picture removes a lot of potential complication and failures, and somewhat increases performance. Primary reason would be avoiding failures though, IMO.