I'm cought in a dilema of which I have very limited knowledge. I have a rack of servers (15 Sun V20Zs-Red Hat Enterprise Linux WS release 3)that are running sungrid engine. In the past they have been doing jobs by writing to a nfs mounted (on each node) file system. Obviously this causes lots of traffic and is extremely slow, so the proposed solution is to install a cluster software and have the nodes write to a SAN via fiber. They have to write to the same files concurrently, so the cluster software is a requirement. The problem is that sw is damn expensive! I've noticed a few options out there and I was hoping you can guide me to the right choice. Here are a few I've seen (other good ones?, please mention!):

GFS - (Red Hat product) $$ - do they have a free version?problems?

QFS - (Sun product) $$$$ - I think this is out of the question, but feedback is welcome.

Polyserve - ?? I know nothing about this one but have heard bad feedback about it...

lustre - Been there, have it. What a pain in the @ss, I'd rather avoid it if I can. The only advantage is it's FREE.

Thanks in advance for your help all!

take a look at OpenAFS

I do not think this is the type of workload that AFS is intended for.

I think GFS may be the "best" option, if it does what you want.

from the opengfs homepage:
one of those links points at this.

hi,
why not ocfs?
its gpl and easy to configure, but i have no idea of benchmarks and stability between ocfs and gfs
Also,in fedora 6, we can use even yum to install gfs2
regards,
Nirmal Tom.

How about the cluster software offered by IBM?

hi,
to my knowledge,its GPFS ( General Parallel File System)
IBM has tools and i heard ,GPFS is implemented in many of top 500 super computers and for AIX ,linux grid computing.

regards,
Nirmal Tom.

hi,
any tutorial for gfs on fedora core 6? Is it necessary to set clusters?All i need is the shared disk to available on both systems.
i install gfs2-utils , system-config-cluster and its dependencies using yum
when i try ,
service gfs2 start
Mounting GFS2 filesystems: /sbin/mount.gfs2: waiting for gfs_controld to start
/sbin/mount.gfs2: waiting for gfs_controld to start
/sbin/mount.gfs2: waiting for gfs_controld to start
/sbin/mount.gfs2: waiting for gfs_controld to start
/sbin/mount.gfs2: waiting for gfs_controld to start
/sbin/mount.gfs2: waiting for gfs_controld to start
/sbin/mount.gfs2: waiting for gfs_controld to start
/sbin/mount.gfs2: waiting for gfs_controld to start
/sbin/mount.gfs2: waiting for gfs_controld to start
/sbin/mount.gfs2: waiting for gfs_controld to start
/sbin/mount.gfs2: gfs_controld not running
/sbin/mount.gfs2: error mounting lockproto lock_dlm
[FAILED]

regards,
Nirmal Tom.

Were you working from any documentation (howto, walkthrough, tutorial, man pages, etc..), or did you just install it and try to mount the filesystem?

hi,
i think i sorted out.The problem is installed the cman, gfs2-utils packages from updates repository and my kernel version is 2.6.19.But they are meant for 2.6.20.


I didnt follow any guides,just tried it out.bcaz i found only older ones.


my question is cluster package has many tools like cmirror, rgmanager, gnbd,cman,dlm etc in it.

dlm is needed to lock a shared partition.so, is dlm and gfs2 enough?(to give shared access to two systems.the partition is shared using iscsi)

thanks and regards,
Nirmal Tom

If you work with GFS regularily you will find out that it is really no rocket science.

But you must follow some guidelines to setup a working GFS. If you configure it wrong, you may cause data corruptiency when writing data from multiple nodes (for example if you are using the wrong logical volume manager).

GFS is free if you use it e. g. with CentOS. Only if you are running productive environments where you need vendor support from Red Hat this will cost you.

I do not recommend GFS 2 at the moment and would prefer GFS 1 for productive settings.

Here you may find appropriate information about using GFS:

[1] https://open.datacore.ch/DCwiki.open/Wiki.jsp?page=GFS
[2] http://www.redhat.com/software/rha/gfs/
[3] http://www.open-sharedroot.org/docum...shop-howto.pdf

Did you fix it? I have the same errors and don't understad how to win it... :/
Can you help me?

Most Sun software is already free and Open Source.
According to this press release, QFS too will be open sourced in the coming month.
http://www.sun.com/aboutsun/pr/2007-...20070410.1.xml

You should not use GFS2 for the first. It is still not ready for production.

Try to get a hand on any RHEL4/5 based distribution and use GFS1 (6.1).

Also think about the concurrent access on these files. Are these small files or bigfiles?
What kind of concurrent access do you have?

Normally if having concurrent write access on the same file from different nodes a cluster filesystem does not help much except from keeping the filesystem itself in a consistent state but not necessarily the contents of the file.

Think about Databases (Oracle RAC or the like) for instance or any other such applications.

I also read about mentioning OCFS or GPFS or Poliserve. You should be told that IMHO any of those cfs is not posix compliant (especially when flocks are used) and therefore cannot be used for any application that easily. GFS - right now - is the only CFS which is of general usage. Next when thinking about ocfs use ocfs2.

But a cluster filesystem is really cool stuff.

Last but not least analyse your application carefully and understand the differences between a distributed filesystem like (NFS/OpenAFS) and a Cluster Filesystem (like GFS and OCFS) and which fs scales better in what environment.

Have fun
MG

hi antras,
is it fedora or rhel , u r using?
sorry for the late reply,i was out of city,nowadays no time to spare in linux.
yum makes me painless and thats the truth

regards,
nirmal tom

Sorry and you for my late answer
I'he already done successful everything
I'm using CentOS (RHEL)