Fault tolerant Raspberry Pi cluster, maybe?
 

Hi group,
Hope this is the right group?
On a lark, I am contemplating building a Fault Tolerate Linux Cluster using 4 Raspberry Pis.
There are a few components that I can not figure out how to make redundant.
The set up will be as follows:

1. 4 Raspberry Pis
2. two power supplies, diodes to separate them.
3. 2 USB hubs
4. 8 USB flash drives ( 2 RAID arrays ).
5. Ethernet hub

The main problem is all hubs are a single point of failure. Since there is only one Ethernet port on each Pi, it is not possible to connect more. The only way is to add another port would be a USB interface?

But the USB hubs are a real problem. The USB hubs will be connected to 2 different Pis. Is there a way to mirror the data from 1 Pi to another?

I hope my intentions are clear? I can figure out everything except the USB hubs. Any ideas?

Thanks for your time.

As I understand the USB design, it is strictly a master/slave connection. There can only be ONE master. Thus no "mirror the data".

There is a patent (https://www.google.com/patents/US8447890) for a USB hub to allow two masters to be connected - but I have seen no products.

You can make use of the network by using a distributed filesystem that provides the redundancy. Each system would then perform the I/O to its respective USB devices.

You might be able to use iscsi to share disk level access - but that again tends to be a "master-slave" connection in that the iscsi client controls the I/O done on the iscsi target.

But doing either of these may require a kernel rebuild to that includes the iscsi modules (or distributed filesystem) for both systems.

A more complex/expensive solution is to use 4 pi nodes, two are I/O nodes acting as controllers for the respective USB attachments, the other two are the HA nodes. At that point the two I/O nodes can communicate to provide distributed access transparent to the two HA nodes. This way the two I/O processors can provide redundant NFS services - and the HA nodes use NFS failover for data, as well as HA services for one node to failover to the other.

What this doesn't get you is failover of things like USB mice/keyboards - which have to attached to only ONE hub. Using a USB-over-ethernet you can get such "sharing" during the failover from one HA node to another. But if one of the I/O nodes fail, you would lose the keyboard/mouse if they are attached to that node.

With that many nodes, you would want a network switch rather than usb network adapters (it would also simplify the network configuration (otherwise each node has to have 3 network adapters, using up the built-in USB connections - the native network connection would be to an administrative network for logging purposes... leaving only one USB connection for all the devices). Of course this also makes the network switch the single point of failure...

Consider using a NAS device for your "large storage" and then access it over ethernet rather than USB.