Hi,

I have some problem implementing redhat cluster using RHCS 6.

I have implemented the redhat cluster and it is showing the running nodes. I am using luci interface to look upon the status of the node in cluster.

My cluster configuration is as follows:-

<?xml version="1.0"?>
<cluster config_version="63" name="ExCluster">
<clusternodes>
<clusternode name="Node-1" nodeid="1">
<fence>
<method name="Method-1">
<device name="fence_dev" port="0"/>
</method>
</fence>
<unfence>
<device action="on" name="fence_dev" port="0"/>
</unfence>
</clusternode>
<clusternode name="Node-2" nodeid="2">
<fence>
<method name="Method-1">
<device name="fence_dev" port="1"/>
</method>
</fence>
<unfence>
<device action="on" name="fence_dev" port="1"/>
</unfence>
</clusternode>
</clusternodes>
<cman expected_votes="1" two_node="1"/>
<fencedevices>
<fencedevice agent="fence_brocade" ipaddr="192.168.2.166" login="admin" name="fence_dev" passwd="xyz"/>
</fencedevices>
<rm>
<failoverdomains>
<failoverdomain name="Domain-1" restricted="1">
<failoverdomainnode name="Node-1"/>
<failoverdomainnode name="Node-2"/>
</failoverdomain>
</failoverdomains>
<resources>
<fs device="/dev/mapper/mpath1p1" force_fsck="1" fsid="49353" fstype="ext3" mountpoint="/mnt/d000" name="fs0" quick_status="1"/>
<fs device="/dev/mapper/mpath2p1" force_fsck="1" fsid="61535" fstype="ext3" mountpoint="/mnt/d001" name="fs1" quick_status="1"/>
<fs device="/dev/mapper/mpath3p1" force_fsck="1" fsid="41445" fstype="ext3" mountpoint="/mnt/d002" name="fs2" quick_status="1"/>
<script file="/etc/init.d/script-1" name="Scr-1"/>
</resources>
<service domain="Domain-1" exclusive="1" name="Service-1" recovery="relocate">
<fs ref="fs0">
<script ref="Scr-1"/>
</fs>
</service>
</rm>
</cluster>

Following are the problems I am facing:

Problem-1
The main problem is that whenever I am starting the service by giving the command " service cman restart", I am getting the following error:

Stopping cluster:
Leaving fence domain... [ OK ]
Stopping gfs_controld... [ OK ]
Stopping dlm_controld... [ OK ]
Stopping fenced... [ OK ]
Stopping cman... [ OK ]
Unloading kernel modules... [ OK ]
Unmounting configfs... [ OK ]
Starting cluster:
Checking if cluster has been disabled at boot... [ OK ]
Checking Network Manager... [ OK ]
Global setup... [ OK ]
Loading kernel modules... [ OK ]
Mounting configfs... [ OK ]
Starting cman... [ OK ]
Waiting for quorum... [ OK ]
Starting fenced... [ OK ]
Starting dlm_controld... [ OK ]
Tuning DLM kernel config... [ OK ]
Starting gfs_controld... [ OK ]
Unfencing self... unfence rhcs1.systec.local failed
[FAILED]
Stopping cluster:
Leaving fence domain... [ OK ]
Stopping gfs_controld... [ OK ]
Stopping dlm_controld... [ OK ]
Stopping fenced... [ OK ]
Stopping cman... [ OK ]
Waiting for corosync to shutdown: [ OK ]
Unloading kernel modules... [ OK ]
Unmounting configfs... [ OK ]

Problem-2
Also Node-1 goes offline from the cluster by giving the message "Not a cluster member".

Problem-3
I am not able to add other instances to my fence device.

Could anyone please tell me the solution for the above issues.

Thanks in advance.

Since your problem is with fencing (jugding from when the first error message appears), try to disable it temporarily and see if the cluster starts normally without it. It's worth mentioning that you have only one fencing device, whereas in a cluster with N nodes you would normally have N fencing devices. In your case you would need two.

Hi Marios,

Thanks for your quick reply.

Yes, after disabling the fence device from the cluster the cluster is able to start normally without it.

But as per my requirement, I can use only one fence device(Brocade Switch). And the moment I add one of the port of this fence device as Instance to the Fence method, it does not allow me to add the other port of the Fence device.

I have attahced an architecture diagram of my setup for your reference.

Also the status of the Service Group is displayed as "disabled", and when ever I try to start I am not able to do so.


Kinldy look into the above issue and help me in resolving it.


Thanks in advance.

Attached Thumbnails

 

The the error " Unfencing self... unfence rhcs1.systec.local failed " shows that there is something wrong with your fence/unfence configuration. Is the name "rhcs1.systec.local" correct? Are the credentials correct? Is the fencing agent for Brocade sending the proper commands?

You need to check for errors that appear in files /var/log/cluster/*.log and /var/log/messages, and experiment with settings until you get it right. Keep in mind that setting up a cluster is not a simple task, you need to experiment a lot and break the system many times before you succeed.