Hi

I am trying to come up with the best solution for data clustering on RHEL/CentOS.
I currently have about 9TB of data on XFS RAID5 filesystem, and the server is at capacity. I can add a secondary server, and mount over NFS, but that will give me 2 partitions 10TB each. I would like to be able to use all available space between multiple servers. End goal is to have 20-30TB of raw data.

What the best way to do it? I have played with RedHat HA Cluster, but seems like it needs a single data location (like SAN), opposite of what I'm trying to do. It also has to be flexible and easy to add additional servers when needed.

Thanks!

I've not done anything like this myself, but have an idea of where I would start with something like this. Have you thought about setting up a SAN? I know it sounds complicated but it's really not that bad. You probably don't even need special hardware, just a decent network switch with one of the computers connected to both networks.

As to how to access the drives/arrays I'm not sure exactly how you'd go about it, but I'd start by looking for iSCSI server availability for linux.Google iSCSI

Also found a how-to configure iSCSI targets on Ubuntu here.

This sounds like some kind of software RAID 50 would do it. I.e. stripe the data across both RAID 5 instances. In both RAID 5 instances one disk might fail, before you lose any data.

Can you also mount the remote RAID 5 with iSCSI to use it like a local disk?

Thanks. I haven't had any real experience with iSCSI but I get the general idea behind it.
If I can have remote drive visible on local server, I should be able to use LVM to combine it, no?

I also google around a bit and found some info on GFS2 in RH HA Cluster, but I don't know if its a good way. Some say its limited to 25TB only and Cluster itself to 16 nodes.

I have just now tried iSCSI in vmware, seems like I can easily add remote drives.

Has anyone tried using LVM on top of it? Will it be stable or cause corruptions?

If a cluster is not the architecture you had in mind, would you care to describe what you are planning instead?

You can get an iSCSI adapter for HP's EVA product. As long as it's configured N+1 you can deal with equipment failure during regular business hours. Make sure you have the switch infrastructure for iSCSI before going down that road.

So that two server or more to access the same filesystem simultaneously is necessary to usage a cluster filesystem, you can to usage the GFS2 (Global File System) of the the "Red Hat Cluster Suite" package, or usage OCFS2 (Oracle Cluster File System).
In both situation will be necessary a disk in the storage SAN or iSCSI.

From my understanding an iSCSI disk will appear as a local hard drive (ex /dev/sdb) so you should be able to use them exactly like a local disk. (Just remember that I haven't used such a setup myself.)

From what I know about clustered filesystems in general you will still need an iSCSI or NAS storage for it to work with.

I'd really like to find out what kind of setup you eventually end up with!

You will need of a Cluster FileSystem for can to write simultaneously in same filesystem, still using the LVM will be necessary to format with GFS2 or other Cluster FileSystem, the EXT3/4, not permit write simultaneously.

You can use the OCF2, that has great performance, because it is used in Cluster Oracle Data Base.

This link "http://oss.oracle.com/projects/ocfs2/dist/documentation/v1.2/ocfs2_faq.html#LIMITS" has entire procedure for install and configure the OCFS2 among other questions.

1 members found this post helpful.

Thank you all for suggestions!

I have a few things Im going to try and test:
1) iSCSI + CLVM + GFS2
2) GlusterFS

I have found some info on GlusterFS that makes it quite interesting. I am not sure how well it would perform, but seems like an very simple solution.

1 members found this post helpful.