Im interested in working with 10GBit Ethernet, specifically with the idea of an iSCSI server.
However, 10GBit switches are extremely expensive.
I have found several PCIe cards which aren't going to break my budget though, and I thought perhaps I could build my own switch out of a linux box and a few 10GBit PCIE ports.

Does anyone know where to start with the OS layer to build the switch software?
Oh, I did think about just doing point-to-point directly from the iSCSI server, but I was hoping to have something more scalable, but still not break the bank like an actual switch would.
I have a HP 2U rackmount box which holds 25 of the 2.5 inch drives, which will be the iSCSI server. I can easily add 2x 10GBit ports for just a few hundred.

If you are hoping for 10g performance you may be disappointed. You can't do in software what the switch does in hardware. Running at line rate would be well over 100M interrupts per second per pair of active ports.

1 members found this post helpful.

What you will end up with is a router rather than a switch.

A switch only handles the IP header - and then uses electronic switches to connect the output to the input, thus avoiding having to handle the data. MUCH faster than the usual router.

A router has to receive the input packet, then output the packet to the destination interface - thus requiring two copy operations, one input, one output.

In the base throughput, a router is less than 1/2 the switch throughput for two ports. Added ports do degrade the same with independent messages - but merging two inputs to one output is even slower.

That said, a router is completely doable - and is MUCH cheaper than a switch.

1 members found this post helpful.

You're not going to get anywhere near 10Gb performance out of a computer acting as a switch, sorry. 10Gb switches use dedicated hardware for switching (ASICs) that are MUCH faster at that job than any general purpose CPU could ever hope for.

That said, I don't know why you think they're so expensive. You can pick up a Netgear XS708E 8 port 10GbE switch (just as an example) for just over $800. I have its big brother (712T) running in the server room at my office, and it hits 10Gb without issue. I can't really comment on the long term reliability since I've only had it for ~7 months, but that's been 7 months without a single hiccup.

2 members found this post helpful.

I did some searching about this last night. All I could find about a DIY switch, as opposed to a router, that seemed reasonable concerned building a switch using Arduino.

That is something that might be doable at a reasonable cost.

1 members found this post helpful.

Thanks for all the input. The Netgear switches are just barely affordable, and I may have to go that route.
The 8-port I can afford, but the 12-port doesn't fit my budget.
A pair of 8-port switches (multipathing) would work for the short term, but if this does grow in scale, I will likely need to sell the 8-port as used and get a 12 or 16 port.