LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices



Reply
 
Search this Thread
Old 03-15-2009, 12:22 PM   #1
aaron28
LQ Newbie
 
Registered: Mar 2009
Posts: 4

Rep: Reputation: 0
Redhat CLuster fence failed - problem


Hi I have a RED-HAT 5.3 and configured cluster RHCS on two nodes.
Everythings works ok,services are relocated when one of the active nodes shutdown or reboot.
But I have a problem when I unpluded power cord of the active node. Recources cannot relocate. In logs I see that node2 cannot fence node1 (whitch is power off) -fening failed.
How to resolve that problem, thanks in advance.
 
Old 03-16-2009, 05:45 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

A cluster needs a minimum of three nodes. If only 2 physical nodes are present a so called quorum disk is needed.

Why? With an even amount of nodes the cluster cannot determine which one is faulty and needs to be taken out of service (a so calles 'split-brain' situation can arise).

Take a look here: RedHat - 2.5. Considerations for Using Quorum Disk and here: RedHat - Enhancing cluster quorum with QDisk

Hope this helps.
 
Old 03-20-2009, 09:05 AM   #3
aaron28
LQ Newbie
 
Registered: Mar 2009
Posts: 4

Original Poster
Rep: Reputation: 0
fence failed

I Add the quorum disk but the result is the same ,when I unpluge teh power cord from the active node,the second one cannot fence
Icreated quorum disk and added:

<cman expected_votes="3" two_node="0"/>
<quorumd interval="3" tko="23" device="/dev/mapper/quorum_diskp1" votes="1"/>
<cman deadnode_timeout="135" expected_nodes="3"/>

the result is the same node cannot facing
 
Old 03-20-2009, 01:38 PM   #4
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

Are you using (external) power fencing or the dummy fencing (fence_manual)? The last one (fence_manual) doesn't fence by itself, it needs human intervention.

Take a look here: Red Hat GFS 6.0: Administrator's Guide - 10.2 Fencing Methods
 
Old 03-23-2009, 06:47 AM   #5
aaron28
LQ Newbie
 
Registered: Mar 2009
Posts: 4

Original Poster
Rep: Reputation: 0
fence failed

Hi there is any method to automate manual fencing?

After I added quorum disk I observed that situation.
When I for example shutdown node2, and reboot node1
The node1 gets up correctyly,but after that I power node2 ,and when It comes up the cman deamon on node1 is killed.And that node doesnt reboot.

Before adding quorum disk when I had in cluster.conf
<cman expected_votes="1" two_node="1"/> it worked.
The node reboots.
Where is the problem?
 
Old 03-23-2009, 07:37 AM   #6
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

Quote:
Hi there is any method to automate manual fencing?
Not that I know of, that is one of the down sides of manual fencing (which should not be used other then testing/evaluation or be used as a last resort fallback next to 'real' fencing). The human intervention is part of the mechanism (fence_ack_manual must be used to acknowledge the failed node).

Did you read the fence_manual and fence_ack_manual man page?

I'm not sure which docs you are using as a reference, but this one is a good start: Red Hat Cluster Suite for Red Hat Enterprise Linux (Red Hat 5.2, 5.3 version is not available yet).

Hope this gets you going again.
 
Old 03-23-2009, 09:56 AM   #7
aaron28
LQ Newbie
 
Registered: Mar 2009
Posts: 4

Original Poster
Rep: Reputation: 0
cluster fencing problem

Do I have to have fencing configured to quorum disk work properly?
 
Old 03-23-2009, 10:45 AM   #8
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374Reputation: 2374
Hi,

Quote:
Do I have to have fencing configured to quorum disk work properly?
That depends.

- If this is 'just' an exercise and meant to get some understanding in (Red Hat) clustering: No you don't need it, as long as you know/understand what manual fencing does (and doesn't!) and its place in the bigger picture. At certain points your cluster will not react as described in the docs/manuals (that is a given! and very annoying at times) because manual fencing isn't set up and/or isn't configured properly.

- If this cluster is for real: Yes you do. A quorum disk is needed to end up with an uneven amount of nodes, without an uneven amount of nodes fencing cannot properly decide which is the 'bad node'. You could decide, if you have the resources, to add a third physical node and drop the quorum disk. But that leaves you with manual fencing. As I stated before, manual fencing should not be used other then testing, evaluation or be used as a last resort fallback next to 'real' fencing.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Fence question with Red Hat Cluster Suite kenneho Linux - Server 2 10-02-2008 10:03 AM
Fence device ( redhat cluster) m3lyan Linux - Server 6 08-21-2008 04:24 AM
Cluster accounting failed at 135593 (0x211a9): missing cluster in $Bitmap fakie_flip Linux - Software 1 01-02-2008 04:08 AM
RH Cluster Fence Password Script quackerjack_98 Red Hat 0 08-15-2007 01:38 PM
Possible problem with RedHat Cluster andreseso Linux - Enterprise 3 09-22-2005 04:16 AM


All times are GMT -5. The time now is 10:18 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration