Okay here's the background: My company just bought an Intel Modular Server (IMS) system and now we're still tinkering about with it. http://www.intel.com/products/server...r-overview.htm

I am tasked with setting up 3 out of the 6 servers and dividing up 500GB of space in the most efficient manner amongst the 3 servers. The space is in a pool which can be assigned to virtual drives. Each virtual drive can be assigned as disk0 or disk1 and so on to one or more servers. They'll be running CentOS.

On the second try I came up with this scheme:
shared sda1 -- /boot (ext3)
shared sda2 -- /home (ext3)
shared sda3 -- /backup (ext3)
dedicated sdb1 -- swap
dedicated sdb2 -- / (ext3)

Well it worked better than the first try (where it was all shared in one big sda).

Aside from saving space, what we want is for a storage location where the multiple servers can share read/write access. What happened with the layout I tried above is that if I were to create a file in the shared location from server 1, it doesn't appear in the shared area of server 2 until I restart. But after restarting a few times it would say I need to fsck the filesystem. Doing that however causes everything to unravel until a reboot causes kernel panic. At least it lasted longer than the first scheme, which also ended with a panicking kernel.

So this scheme doesn't work either. I have a feeling it might be the filesystem. Maybe the scheme would work if I used vfat for /boot and ntfs for /home and /backup? Or maybe I'm approaching this from the wrong angle.

Help? I read somewhere that it might require setting up clustering. If that is so, can somebody point me in the right direction?

Sharing regular filesystems with multiple machines is not supported and is very bad, as yoi saw! You CAN share disks, but the filesystem used has to support multiple concurrent mounts. ext* will just not do. GFS is RedHat's cluster filesystem, for example. OCFS is Oracle's.

Ah, thanks for the tip. GFS eh? I did not know about that. So I should mount /home and /backup to a GFS partition then? Would you say its a good idea or a bad one to mount /boot as GFS (is that even possible?)?

I didn't think it was possible, but I found http://www.open-sharedroot.org/docum...ot-mini-howto/

I'm still not sure it's a good idea.

Ok, I've experimented with the shared root thingy, it's not a good idea.

I did more research and experimentation on GFS, now I've got a cluster of 2 servers running ricci on them with one of them running luci. I still can't get the GFS partition to be shared properly. Apparently I need to install and configure GNBD.

That's where I paused to reflect. It appears that things are getting more complex. The way it is, just to achieve a shared, parallel access storage area, I have to set up a cluster, set up a GNBD server and serve the drive/partition as a network storage. This is complicating what should be simple. Not to mention now the data will be sent over ethernet. I'm concerned that this will make I/O sucky.

Isn't there a way to format the filesystem and disallow the OS from locking it? I just need multiple servers to access the storage in parallel. It's a shared storage that is detected as a second hard disk in the server. It's not a SAN, it's not a network drive or NFS or anything else. It's just a damned extra disk. I don't think building a Rube Goldberg of a complicated setup will do here.

Quick update, I found out it is possible to achieve what I set out to do (shared /home and /backup) with luci, ricci and just the clustering and storage clustering suite. The trick is to use lvm and make sure cman, rgmanager and clvmd services are running on all nodes. I am in the process of adding a third node to the cluster, then I will document the process so others can try it out.

One thing I'm wondering is does the data travel through the NIC when a write is conducted on the shared partitions or is it just the locking protocols? Basically I'm wondering if performance will be dependent on network conditions?

Update: here is the documentation I promised: http://blog.ds.my/how-to/having-mult...s2-part-1.html