This is beyond weird.

Before:

SwitchA <--> SwitchB <--> Swiches C, D, and E.

We removed switches b,c,d.

Replaced Switch B with Switch F and connected switch F to switch E:

Switcha <--> SwitchF <--> SwitchE

Get this:
Devices connected to switch F AND to Switch E that have HTTP/HTTPS management web servers on them ALL were pingable but SOME wouldn't let me connect to the http interface until the devices were rebooted.

Is there ANY plausible explanation for why?

Obviously the problem is gone now but I would LOVE to go down this rabbit hole and see why. However without it being a reproducible problem I suspect we're limited to speculation.

Most likely auto-negotiation of duplex:

http://en.wikipedia.org/wiki/Autoneg...ility_problems

A switch creates network tables and when you change the network topology the switch gets confused .

1 members found this post helpful.

Arp cache or other router management negotiation cache.

Moral is to reboot switches and test.

1 members found this post helpful.

That's what's very odd about this - we did reboot them. But we didn't lose the ability to communicate with these devices until some time T+6 to T+18 hours after the reboot operation.

8) The plot thickens. I too am suspicious of the arp table. That's the only thing it could be.

FWIW they were avaya 4500 and and 2500 series switches.

I've seen this too. Lots of times. It /could/ be this. But then why would pings have no isuses?

As it explains in the article, pings don't care whether its half or full duplex, since they are very simple one-packet-at-a-time protocol. Some appliances only negotiate duplex when they boot.

"Duplex mismatch problems are difficult to diagnose because the network is apparently working, and simple programs used for network tests such as ping report a valid connection; however, the network is much slower than expected."

1 members found this post helpful.

Now that is something I hadn't considered. That could easily explain it. Nmap showed http open, firefox said page not found for these devices. It probably was a duplex mismatch.

We still have 2 devices not responding even after they've been booted. I'm going to try changing the duplex settings on those ports this evening. Perhaps that's the magic bullet!