LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Enterprise Linux Forums > Linux - Enterprise
User Name
Password
Linux - Enterprise This forum is for all items relating to using Linux in the Enterprise.

Notices

Reply
 
Search this Thread
Old 07-30-2010, 06:39 PM   #1
lxf
LQ Newbie
 
Registered: Jul 2010
Posts: 22

Rep: Reputation: 2
Got stuck with CMAN - unable to establish quorum


Hi,
I was trying most of the day today to get a demo installation of Red Hats cman up und and running, as I rely on clvm afterwards. I did it on a 2 node cluster each site running a recent CentOS 5.5 and connected in a local network. Basically I followed the instructions on this link (plus surrounding chapters).

This is my /etc/cluster/cluster.conf
Code:
<?xml version="1.0" ?>
<cluster alias="wavecloud" config_version="5" name="wavecloud">
        <fence_daemon post_fail_delay="0" post_join_delay="3"/>
        <clusternodes>
                <clusternode name="centos1" nodeid="1" votes="1">
                        <fence>
                                <method name="1">
                                        <device name="hands_on" nodename="centos1"/>
                                </method>
                        </fence>
                </clusternode>
                <clusternode name="centos2" nodeid="2" votes="1">
                        <fence>
                                <method name="1">
                                        <device name="hands_on" nodename="centos2"/>
                                </method>
                        </fence>
                </clusternode>
        </clusternodes>
        <cman expected_votes="1" two_node="1"/>
        <fencedevices>
                <fencedevice agent="fence_manual" name="hands_on"/>
        </fencedevices>
        <rm>
                <failoverdomains/>
                <resources/>
        </rm>
</cluster>
Both sites are reachable from the other site via their hostname:

Code:
[root@centos2 ~]# ping centos1
PING centos1 (X.X.X.X) 56(84) bytes of data.
64 bytes from centos1 (X.X.X.X): icmp_seq=1 ttl=64 time=0.286 ms
64 bytes from centos1 (X.X.X.X): icmp_seq=2 ttl=64 time=0.500 ms

--- centos1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.286/0.393/0.500/0.107 ms


[root@centos1 ~]# ping centos2
PING centos2 (X.X.X.X) 56(84) bytes of data.
64 bytes from centos2 (X.X.X.X): icmp_seq=1 ttl=64 time=1.83 ms

--- centos2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.838/1.838/1.838/0.000 ms

I synchronized the cluster.conf on both sites, If I now try to start the cluster I run into the following problem:

Code:
[root@centos1 ssmping-0.9.1]# /etc/init.d/cman start
Starting cluster: 
   Loading modules... done
   Mounting configfs... done
   Starting ccsd... done
   Starting cman... failed
Timed-out waiting for cluster
                                                           [FAILED]
whichs turns out to be:


Interestingly I do not even join the pool (for both nodes):

Code:
[root@centos2 ~]# cman_tool nodes
cman_tool: cman_get_node_count failed: Node is not yet a cluster member
The log states:

Code:
Jul 31 01:34:37 centos1 ccsd[4432]: Starting ccsd 2.0.115: 
Jul 31 01:34:37 centos1 ccsd[4432]:  Built: Apr 26 2010 13:46:08 
Jul 31 01:34:37 centos1 ccsd[4432]:  Copyright (C) Red Hat, Inc.  2004  All rights reserved. 
Jul 31 01:34:37 centos1 ccsd[4432]: cluster.conf (cluster name = wavecloud, version = 6) found. 
Jul 31 01:34:40 centos1 openais[4438]: [MAIN ] AIS Executive Service RELEASE 'subrev 1887 version 0.80.6' 
Jul 31 01:34:40 centos1 openais[4438]: [MAIN ] Copyright (C) 2002-2006 MontaVista Software, Inc and contributors. 
Jul 31 01:34:40 centos1 openais[4438]: [MAIN ] Copyright (C) 2006 Red Hat, Inc. 
Jul 31 01:34:40 centos1 openais[4438]: [MAIN ] AIS Executive Service: started and ready to provide service. 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] Token Timeout (10000 ms) retransmit timeout (495 ms) 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] token hold (386 ms) retransmits before loss (20 retrans) 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] join (60 ms) send_join (0 ms) consensus (20000 ms) merge (200 ms) 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] downcheck (1000 ms) fail to recv const (50 msgs) 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] seqno unchanged const (30 rotations) Maximum network MTU 1402 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] window size per rotation (50 messages) maximum messages per rotation (17 messages) 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] send threads (0 threads) 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] RRP token expired timeout (495 ms) 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] RRP token problem counter (2000 ms) 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] RRP threshold (10 problem count) 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] RRP mode set to none. 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] heartbeat_failures_allowed (0) 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] max_network_delay (50 ms) 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] Receive multicast socket recv buffer size (262142 bytes). 
Jul 31 01:34:40 centos1 ccsd[4432]: Initial status:: Inquorate 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] Transmit multicast socket send buffer size (262142 bytes). 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] The network interface [X.X.X.X] is now up. 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] Created or loaded sequence id 0.X.X.X.X for this ring. 
Jul 31 01:34:40 centos1 openais[4438]: [TOTEM] entering GATHER state from 15. 
Jul 31 01:34:40 centos1 openais[4438]: [CMAN ] CMAN 2.0.115 (built Apr 26 2010 13:46:11) started 
Jul 31 01:34:40 centos1 openais[4438]: [MAIN ] Service initialized 'openais CMAN membership service 2.01' 
Jul 31 01:34:40 centos1 openais[4438]: [SERV ] Service initialized 'openais extended virtual synchrony service' 
Jul 31 01:34:40 centos1 openais[4438]: [SERV ] Service initialized 'openais cluster membership service B.01.01' 
Jul 31 01:34:40 centos1 openais[4438]: [SERV ] Service initialized 'openais availability management framework B.01.01' 
Jul 31 01:34:40 centos1 openais[4438]: [SERV ] Service initialized 'openais checkpoint service B.01.01' 
Jul 31 01:34:40 centos1 openais[4438]: [SERV ] Service initialized 'openais event service B.01.01' 
Jul 31 01:34:40 centos1 openais[4438]: [SERV ] Service initialized 'openais distributed locking service B.01.01' 
Jul 31 01:34:40 centos1 openais[4438]: [SERV ] Service initialized 'openais message service B.01.01' 
Jul 31 01:34:40 centos1 openais[4438]: [SERV ] Service initialized 'openais configuration service' 
Jul 31 01:34:40 centos1 openais[4438]: [SERV ] Service initialized 'openais cluster closed process group service v1.01' 
Jul 31 01:34:40 centos1 openais[4438]: [SERV ] Service initialized 'openais cluster config database access v1.01' 
Jul 31 01:34:40 centos1 openais[4438]: [SYNC ] Not using a virtual synchrony filter. 
Jul 31 01:35:10 centos1 openais[4438]: [TOTEM] The consensus timeout expired. 
Jul 31 01:35:10 centos1 openais[4438]: [TOTEM] entering GATHER state from 3. 
...
And additionally

Code:
[root@centos1 ssmping-0.9.1]# cman_tool join
cman_tool: Node is already active
[root@centos1 ssmping-0.9.1]# fence_tool join
fence_tool: waiting for cluster quorum
fence_tool: waiting for cluster quorum
I really don't see, what I'm doing wrong, as I can also verify that multicast is working:

Code:
[root@centos1 ssmping-0.9.1]# ping -t 1 -c 2 224.0.0.1
PING 224.0.0.1 (224.0.0.1) 56(84) bytes of data.
64 bytes from centos1: icmp_seq=1 ttl=64 time=0.257 ms
64 bytes from centos2: icmp_seq=1 ttl=64 time=1.16 ms (DUP!)
...
 
Old 08-02-2010, 03:38 AM   #2
lxf
LQ Newbie
 
Registered: Jul 2010
Posts: 22

Original Poster
Rep: Reputation: 2
Shit, I forgot that CentOS uses by default a firewall. Now everything is up and working.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
redhat cluster - cman lost contact with quorum device fft9qh Linux - Enterprise 2 04-15-2010 02:33 AM
Unable to establish NFS share No route to host teluguswan Linux - Security 5 11-28-2009 10:22 PM
Unable to establish GPRS link with Vodafone via GSM modem stuckfast Linux - Networking 1 11-13-2005 04:48 AM
unable to establish connection neerajchaudhari Linux - Software 2 09-14-2005 10:22 PM
Unable to establish a connection to my ISP? phobos Linux - Networking 2 10-22-2002 11:59 AM


All times are GMT -5. The time now is 11:02 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration