LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Enterprise Linux Forums > Linux - Enterprise
User Name
Password
Linux - Enterprise This forum is for all items relating to using Linux in the Enterprise.

Notices

Reply
 
Search this Thread
Old 02-12-2010, 01:01 PM   #1
hostmaster
Member
 
Registered: Feb 2007
Posts: 55

Rep: Reputation: 17
Cannot Add new services to Cluster


Hi I have installed RHEL 5.4 and Cluster Suite. While adding new services to cluster the new services are not recognized. Here is the response from clusvcadm

Local machine trying to enable service:Tertio_IP...Service does not exist

I am attaching the output of some commands and also i have checked proper version is loaded by cluster manager but still it cannot recognize the newly added service. This is a production server and restarting the server is not possible. All nodes are properly recognized by the cluster

[root@tertioapp01 ~]# cman_tool nodes
Node Sts Inc Joined Name
0 M 0 2010-02-01 03:10:39 /dev/emcpowerc
1 M 196 2010-02-01 03:09:56 tertioapp02.bss.telenor.com.pk
2 M 192 2010-02-01 03:09:51 tertioapp03.bss.telenor.com.pk
3 M 188 2010-02-01 03:09:47 tertioapp01.bss.telenor.com.pk
4 M 200 2010-02-01 03:13:54 tertioapp04.bss.telenor.com.pk

Restarting the cluster will hopefully recognize the added service but for that all services in Production
will need to be restarted which is not possible at the moment.

#################################################################################################### ##
[root@tertioapp01 ~]# cman_tool status
Version: 6.2.0
Config Version: 54
Cluster Name: tertio
Cluster Id: 7025
Cluster Member: Yes
Cluster Generation: 200
Membership state: Cluster-Member
Nodes: 4
Expected votes: 5
Quorum device votes: 1
Total votes: 5
Quorum: 3
Active subsystems: 10
Flags: Dirty
Ports Bound: 0 11 177
Node name: tertioapp02.mydomain.com
Node ID: 1
Multicast addresses: 239.192.27.140
Node addresses: x.x.x.190
#################################################################################################### ##
[root@tertioapp01 ~]# clustat
Cluster Status for tertio @ Fri Feb 12 21:54:09 2010
Member Status: Quorate

Member Name ID Status
------ ---- ---- ------
tertioapp02.mydomain.com 1 Online, rgmanager
tertioapp03.mydomain.com 2 Online, rgmanager
tertioapp01.mydomain.com 3 Online, Local, rgmanager
tertioapp04.mydomain.com 4 Online, rgmanager
/dev/emcpowerc 0 Online, Quorum Disk

Service Name Owner (Last) State
------- ---- ----- ------ -----
service:App_IP tertioapp01.mydomain.com started
serviceB tertioapp04.mydomain.com started
service:GFS_tertioapp01 tertioapp01.mydomain.com started
service:GFS_tertioapp02 tertioapp02.mydomain.com started
service:GFS_tertioapp03 tertioapp03.mydomain.com started
#################################################################################################### ##
CLUSTER.CONF
#################################################################################################### ###
<?xml version="1.0"?>
<cluster alias="tertio" config_version="54" name="tertio">
<fence_daemon clean_start="0" post_fail_delay="0" post_join_delay="3"/>
<clusternodes>
<clusternode name="tertioapp02.my.domain.com" nodeid="1" votes="1">
<fence>
<method name="1">
<device name="IPMI_tertioapp02"/>
</method>
</fence>
</clusternode>
<clusternode name="tertioapp03.my.domain.com" nodeid="2" votes="1">
<fence>
<method name="1">
<device name="IPMI_tertioapp03"/>
</method>
</fence>
</clusternode>
<clusternode name="tertioapp01.my.domain.com" nodeid="3" votes="1">
<fence>
<method name="1">
<device name="IPMI_tertioapp01"/>
</method>
</fence>
</clusternode>
<clusternode name="tertioapp04.my.domain.com" nodeid="4" votes="1">
<fence>
<method name="1">
<device name="IPMI_tertioapp04"/>
</method>
</fence>
</clusternode>
</clusternodes>
<cman expected_votes="5"/>
<fencedevices>
<fencedevice agent="fence_ipmilan" ipaddr="x.x.x.74" login="root" name="IPMI_tertioapp01" passwd="calvin"/>
<fencedevice agent="fence_ipmilan" ipaddr="x.x.x.75" login="root" name="IPMI_tertioapp02" passwd="calvin"/>
<fencedevice agent="fence_ipmilan" ipaddr="x.x.x.76" login="root" name="IPMI_tertioapp03" passwd="calvin"/>
<fencedevice agent="fence_ipmilan" ipaddr="x.x.x.77" login="root" name="IPMI_tertioapp04" passwd="calvin"/>
</fencedevices>
<rm>
<failoverdomains>
<failoverdomain name="FD_DB" nofailback="1" ordered="1" restricted="1">
<failoverdomainnode name="tertioapp03.my.domain.com" priority="2"/>
<failoverdomainnode name="tertioapp04.my.domain.com" priority="1"/>
</failoverdomain>
<failoverdomain name="FD_GFS_tertioapp01" nofailback="0" ordered="0" restricted="1">
<failoverdomainnode name="tertioapp01.my.domain.com" priority="1"/>
</failoverdomain>
<failoverdomain name="FD_GFS_tertioapp02" nofailback="0" ordered="0" restricted="1">
<failoverdomainnode name="tertioapp02.my.domain.com" priority="1"/>
</failoverdomain>
<failoverdomain name="FD_GFS_tertioapp03" nofailback="0" ordered="0" restricted="1">
<failoverdomainnode name="tertioapp03.my.domain.com" priority="1"/>
</failoverdomain>
<failoverdomain name="FDapp_ip" nofailback="1" ordered="1" restricted="1">
<failoverdomainnode name="tertioapp01.my.domain.com" priority="1"/>
</failoverdomain>
<failoverdomain name="FDtibco_ip" nofailback="1" ordered="1" restricted="1">
<failoverdomainnode name="tertioapp02.my.domain.com" priority="1"/>
<failoverdomainnode name="tertioapp03.my.domain.com" priority="2"/>
</failoverdomain>
</failoverdomains>
<resources>
<ip address="x.x.x.192" monitor_link="1"/>
<lvm lv_name="lvol0" name="VG_DB_Data" vg_name="vgpda_oradata"/>
<lvm lv_name="lvol0" name="VG_DB_Archive" vg_name="vgpda_archive"/>
<oracledb home="/u02/app/oracle/product/10.2.0/db_1" name="PROVDA" type="10g" user="oracle"/>
<clusterfs device="/dev/mapper/vgapp01-lvol1" force_unmount="1" fsid="22778" fstype="gfs2" mountpoint="/home/app" name="GFS_app01" self_fence="0"/>
<clusterfs device="/dev/mapper/vglog01-lvol1" force_unmount="1" fsid="28279" fstype="gfs2" mountpoint="/home/app/logs" name="GFS_log01" self_fence="0"/>
<fs device="/dev/mapper/vgpda_oradata-lvol0" force_fsck="1" force_unmount="1" fsid="41502" fstype="ext3" mountpoint="/db02/oradata/PROVDA" name="FS_DB_Data" self_fence="0"/>
<fs device="/dev/mapper/vgpda_archive-lvol0" force_fsck="1" force_unmount="1" fsid="32677" fstype="ext3" mountpoint="/db02/oradata/PROVDA/archive" name="FS_DB_Archive" self_fence="0"/>
<ip address="x.x.x.193" monitor_link="1"/>
<ip address="x.x.x.194" monitor_link="1"/>
</resources>
<service autostart="1" domain="FD_DB" exclusive="0" name="DB" recovery="relocate">
<lvm ref="VG_DB_Data"/>
<lvm ref="VG_DB_Archive"/>
<fs ref="FS_DB_Data"/>
<fs ref="FS_DB_Archive"/>
<ip ref="x.x.x.192"/>
<oracledb ref="PROVDA"/>
</service>
<service autostart="1" domain="FD_GFS_tertioapp01" exclusive="0" max_restarts="0" name="GFS_tertioapp01" recovery="restart" restart_expire_time="0">
<clusterfs fstype="gfs" ref="GFS_app01"/>
<clusterfs fstype="gfs" ref="GFS_log01"/>
</service>
<service autostart="1" domain="FD_GFS_tertioapp02" exclusive="0" max_restarts="0" name="GFS_tertioapp02" recovery="restart" restart_expire_time="0">
<clusterfs fstype="gfs" ref="GFS_app01"/>
<clusterfs fstype="gfs" ref="GFS_log01"/>
</service>
<service autostart="1" domain="FD_GFS_tertioapp03" exclusive="0" max_restarts="0" name="GFS_tertioapp03" recovery="restart" restart_expire_time="0">
<clusterfs fstype="gfs" ref="GFS_app01"/>
<clusterfs fstype="gfs" ref="GFS_log01"/>
</service>
<service autostart="1" domain="FDapp_ip" exclusive="0" max_restarts="10" name="App_IP" recovery="restart" restart_expire_time="60">
<ip ref="x.x.x.193"/>
</service>
<service autostart="1" domain="FDtibco_ip" exclusive="0" name="Tertio_IP" recovery="relocate">
<ip ref="x.x.x.194"/>
</service>
</rm>
<totem token="65000"/>
<quorumd interval="5" label="quorumdisk" min_score="1" reboot="0" tko="10" votes="1">
<heuristic interval="2" program="/bin/ping -c1 -t1 x.x.x.250" score="1"/>
</quorumd>
</cluster>
#################################################################################################### ##
 
Old 02-14-2010, 06:39 AM   #2
elcody02
Member
 
Registered: Jun 2007
Posts: 52

Rep: Reputation: 17
What does syslog tell you when you changed the cluster.conf (on all nodes).

If unsure just advance the version and of cluster.conf and do a ccs_tool update /etc/cluster/cluster.conf and attach the relevant syslog entries (clurgmgrd and whatever ccs and cman/openais tell).

Everything else looks quite ok from a first review.
 
Old 02-14-2010, 08:32 AM   #3
hostmaster
Member
 
Registered: Feb 2007
Posts: 55

Original Poster
Rep: Reputation: 17
Syslog only shows "Update of cluster.conf complete (version xx -> xx)." and nothing else. It was working fine before and new services were added without any issue. Nothing has changed and suddenly it does not recognize any newly added services. I have noticed that before when I was adding services successfully the following was logged

#########################################################################
Feb 2 18:37:10 tertioapp02 ccsd[4368]: Update of cluster.conf complete (version 36 -> 37).
Feb 2 18:37:22 tertioapp02 clurgmgrd[5194]: <notice> Reconfiguring
Feb 2 18:37:36 tertioapp02 clurgmgrd[5194]: <notice> Initializing service:App_IP

#########################################################################
But now nothing such (" tertioapp02 clurgmgrd[5194]: <notice> Reconfiguring ") appears. Is there a way to send signals to clurgmgrd daemon to reconfigure ?
 
Old 02-15-2010, 04:41 AM   #4
hostmaster
Member
 
Registered: Feb 2007
Posts: 55

Original Poster
Rep: Reputation: 17
Restarted the whole cluster and now this issue has gone. I think its a bug and will contact RHEL support for that if it happens again.
 
  


Reply

Tags
rhcs, rhel54


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
how to create services in cluster on redha 5 clustering?? prince amir Linux - Enterprise 3 09-26-2007 12:52 AM
RHEL with Cluster services: can I use LVM? mjgsantos Linux - Enterprise 7 08-15-2006 08:02 AM
Need: HA Qmail Cluster Services & Webmail mlewis Linux - Enterprise 0 04-15-2006 06:47 AM
Need: HA Qmail Cluster Services & Webmail mlewis Linux - General 2 04-15-2006 06:37 AM
Monitoring a cluster thru web services nedian123 Linux - Networking 1 12-22-2004 02:40 PM


All times are GMT -5. The time now is 06:21 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration