LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 04-22-2013, 06:21 AM   #1
ravindert
Member
 
Registered: Oct 2011
Posts: 68

Rep: Reputation: Disabled
Redhat Cluster Fencing


Hi ,

Thanks!!!!! in Advance

Can you please help in finding the below questions regarding the fencing in cluster. Currently i have joined the new company where i have to manage the redhat cluster with following configuration files for the redhat cluster. So to support the cluster i have to provide the solution when the fencing occurs in the cluster.

Please find below the current configuration files (cluster.conf) of the redhat cluster.


<?xml version="1.0"?>
<cluster name="adc3cl01" config_version="7">
<cman expected_votes="1" two_node="1">
<multicast addr="224.0.0.1"/>
</cman>
<clusternodes>
<clusternode name="adc3bs21a" nodeid="1">
</clusternode>

<clusternode name="adc3bs08a" nodeid="2">
</clusternode>
</clusternodes>
<rm>
<failoverdomains>
<failoverdomain name="bre-cluster" ordered="0" restricted="1">
<failoverdomainnode name="adc3bs21a" priority="1"/>
<failoverdomainnode name="adc3bs08a" priority="1"/>
</failoverdomain>
<failoverdomain name="vmsm1-cluster" ordered="0" restricted="1">
<failoverdomainnode name="adc3bs21a" priority="1"/>
<failoverdomainnode name="adc3bs08a" priority="1"/>
</failoverdomain>
<failoverdomain name="vmsr1-cluster" ordered="0" restricted="1">
<failoverdomainnode name="adc3bs21a" priority="1"/>
<failoverdomainnode name="adc3bs08a" priority="1"/>
</failoverdomain>
<failoverdomain name="mfdf1-cluster" ordered="0" restricted="1">
<failoverdomainnode name="adc3bs21a" priority="1"/>
<failoverdomainnode name="adc3bs08a" priority="1"/>
</failoverdomain>
<failoverdomain name="mfdr1-cluster" ordered="0" restricted="1">
<failoverdomainnode name="adc3bs21a" priority="1"/>
<failoverdomainnode name="adc3bs08a" priority="1"/>
</failoverdomain>
<failoverdomain name="mfdm1-cluster" ordered="0" restricted="1">
<failoverdomainnode name="adc3bs21a" priority="1"/>
<failoverdomainnode name="adc3bs08a" priority="1"/>
</failoverdomain>
</failoverdomains>
<resources>
<script file="/etc/init.d/bre" name="bre-script"/>
<script file="/etc/init.d/vmsm1" name="vmsm1-script"/>
<script file="/etc/init.d/vmsr1" name="vmsr1-script"/>
<script file="/etc/init.d/mfdf1" name="mfdf1-script"/>
<script file="/etc/init.d/mfdr1" name="mfdr1-script"/>
<script file="/etc/init.d/mfdm1" name="mfdm1-script"/>
<ip address="172.26.0.98" monitor_link="0"/>
<ip address="172.26.0.99" monitor_link="0"/>
<ip address="172.26.0.144" monitor_link="0"/>
<ip address="172.26.0.141" monitor_link="0"/>
<ip address="172.26.0.142" monitor_link="0"/>
<ip address="172.26.0.143" monitor_link="0"/>
</resources>
<service autostart="1" domain="bre-cluster" name="BRE" recovery="relocate">
<script ref="bre-script"/>
<ip ref="172.26.0.98"/>
</service>
<service autostart="1" domain="vmsm1-cluster" name="VMS_M1" recovery="relocate">
<script ref="vmsm1-script"/>
<ip ref="172.26.0.99"/>
</service>
<service autostart="1" domain="vmsr1-cluster" name="VMS_R1" recovery="relocate">
<script ref="vmsr1-script"/>
<ip ref="172.26.0.144"/>
</service>
<service autostart="1" domain="mfdf1-cluster" name="MFD_F1" recovery="relocate">
<script ref="mfdf1-script"/>
<ip ref="172.26.0.141"/>
</service>
<service autostart="1" domain="mfdm1-cluster" name="MFD_M1" recovery="relocate">
<script ref="mfdm1-script"/>
<ip ref="172.26.0.142"/>
</service>
<service autostart="1" domain="mfdr1-cluster" name="MFD_R1" recovery="relocate">
<script ref="mfdr1-script"/>
<ip ref="172.26.0.143"/>
</service>
</rm>
</cluster>


As i cannot see any where the fencing parameter exist but as per the old administrator who is managing the cluster says there is fening exist so can you please help what kind of fencing exist as per the above file. And also please answered my below query.

-> How we can know what triggers the fencing
-> How we can clear the fencing state and how we can remove the faulty node after the false (problem which forced node to get fenced) part of the faulty node repaired.
-> How can i check that fencing is happening when there is some problem with on of the node in cluster. Like is there any command to check the fencing and any log file exist for the fencing etc..

Thanks a lot!!!!!!!!
Ravinder

Last edited by ravindert; 04-22-2013 at 06:26 AM.
 
Old 04-22-2013, 07:32 AM   #2
hyperdaz
Member
 
Registered: Sep 2004
Location: UK
Distribution: CentOS 5.5
Posts: 44

Rep: Reputation: 1
I'll start by saying my RHEL Cluster information might be a little old last administered it on RHEL4.x

Fencing can be done via power supplies or via network switches or fibre switches.. as well as other methods..

Here's a good place to start...
https://access.redhat.com/site/docum..._Overview.html

Looks like in this case you have some VMs and the fencing could be done on the virtual switches? is the whole environment virtual? vmware or other?

The folloing scripts will probably give more information
<resources>
<script file="/etc/init.d/bre
<script file="/etc/init.d/vmsm1
<script file="/etc/init.d/vmsr1
<script file="/etc/init.d/mfdf1
<script file="/etc/init.d/mfdr1
<script file="/etc/init.d/mfdm1

As well as knowing what thse link to VMs or switches?
<ip address="172.26.0.98
<ip address="172.26.0.99
<ip address="172.26.0.144
<ip address="172.26.0.141
<ip address="172.26.0.142
<ip address="172.26.0.143

Do you know if what each cluster is serving gfs2 filesystem or something else?
 
Old 04-22-2013, 09:32 PM   #3
ravindert
Member
 
Registered: Oct 2011
Posts: 68

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by hyperdaz View Post
I'll start by saying my RHEL Cluster information might be a little old last administered it on RHEL4.x

Fencing can be done via power supplies or via network switches or fibre switches.. as well as other methods..

Here's a good place to start...
https://access.redhat.com/site/docum..._Overview.html

Looks like in this case you have some VMs and the fencing could be done on the virtual switches? is the whole environment virtual? vmware or other?

The folloing scripts will probably give more information
<resources>
<script file="/etc/init.d/bre
<script file="/etc/init.d/vmsm1
<script file="/etc/init.d/vmsr1
<script file="/etc/init.d/mfdf1
<script file="/etc/init.d/mfdr1
<script file="/etc/init.d/mfdm1

As well as knowing what thse link to VMs or switches?
<ip address="172.26.0.98
<ip address="172.26.0.99
<ip address="172.26.0.144
<ip address="172.26.0.141
<ip address="172.26.0.142
<ip address="172.26.0.143

Do you know if what each cluster is serving gfs2 filesystem or something else?


Thanks Dear

Last edited by ravindert; 04-23-2013 at 02:41 AM.
 
Old 04-23-2013, 01:19 AM   #4
ravindert
Member
 
Registered: Oct 2011
Posts: 68

Original Poster
Rep: Reputation: Disabled
redhat cluster fencing state

Thanks

Last edited by ravindert; 04-23-2013 at 01:20 AM.
 
Old 04-23-2013, 02:39 AM   #5
ravindert
Member
 
Registered: Oct 2011
Posts: 68

Original Poster
Rep: Reputation: Disabled
Thanks a lott!!!!! hyperdaz for your feed backup.

Firstly , yes these two cluster servers are setup on the virtual machines running on the Citrix xen server.

So can you please tell me how i can check what are setting exist in the xen server for the fencing at virtual switch level??

Secondly below are the content of one of the bre,vmsm1,vmsr1 scripts etc

[root@adc3bs21a ~]# cat /etc/init.d/bre
#!/bin/bash
BRE_INST=prd1
BRE_HOME=/app/bre/instances/$BRE_INST
BRE_USER=bre

case $1 in
start)
su - bre -c "$BRE_HOME/clusterScripts/startup.sh" 2> /dev/null >&2
RETVAL=$?
;;
stop)
su - bre -c "$BRE_HOME/clusterScripts/shutdown.sh" 2> /dev/null >&2
RETVAL=$?
;;
status)
su - bre -c "$BRE_HOME/clusterScripts/sanityDispatch.sh" 2> /dev/null >&2
RETVAL=$?
if [ $RETVAL -eq 0 ]; then
echo "BRE is running";
else
echo "BRE is not running";
fi
;;
*)
echo "Usage: $0 (start|stop|status)"
exit 1
;;
esac
exit $RETVAL


===============



[root@adc3bs21a ~]# cat /app/bre/instances/prd1/clusterScripts/startup.sh
#!/bin/bash

exec ` dirname "$0" `/../scripts/bre.sh start
[root@adc3bs21a ~]#


=================

[root@adc3bs21a ~]#cat /app/bre/instances/prd1/scripts/bre.sh

#!/bin/bash

SCRIPT_HOME=` dirname "$0" `
SCRIPT_HOME=` (cd "$SCRIPT_HOME" && pwd) `
THIS_SCRIPT="$SCRIPT_HOME"/` basename "$0" `

#-------------------------------------------------------------------------------
# Bootstrap configuration
#-------------------------------------------------------------------------------
BOOTSTRAP="$SCRIPT_HOME"/bootstrap.sh

if ! . "$BOOTSTRAP" ; then
echo " Please place a file called "
echo " $BOOTSTRAP, "
echo " containing a line like this: "
echo " CONFIG_DIR=/path/to/bre/config/dir"
exit 1
fi

CONFIG_BASE=scriptConfig.sh
SCRIPT_CONFIG="$CONFIG_DIR"/"$CONFIG_BASE"
if ! . "$SCRIPT_CONFIG" ; then
echo " Please ensure that "
echo " $BOOTSTRAP "
echo " points to an existing directory, which contains $CONFIG_BASE."
exit 1
fi


export JAVA_HOME CATALINA_HOME ANT_HOME

CATALINA_BIN_DIR="$CATALINA_HOME"/bin
CATALINA_STARTUP_SCRIPT="$CATALINA_BIN_DIR"/startup.sh
CATALINA_SHUTDOWN_SCRIPT="$CATALINA_BIN_DIR"/shutdown.sh

LAUNCH_JVM_PRPTY="-Dbre.instanceName=""$INSTANCE_NAME"
LONG_GREP_PATTERN="Dbre.instanceName=""$INSTANCE_NAME"
SHORT_GREP_PATTERN="bre.instanceName=""$INSTANCE_NAME"

main() {
if [ -n "$EXP_CMD" ] ; then
# Make sure that EXP_CMD is intentional
if [ -n "$1" ] ; then
echo "Error varibale EXP_CMD is defined for a normal web/cli " >&2
echo "invocation." >&2
exit 1
fi
# Another instance of this script calling itself
local CMD="$EXP_CMD"
else
local CMD="$1"
if ! shift 1 ; then
usage
exit 1
fi
fi

set -e

case "$CMD" in
start) start ;;
stop) stop "$@" ;;
restart) restart ;;
status) status ;;

redeploy) redeploy ;;

updateData) updateData ;;
migrateSchema) migrateSchema ;;
importData) importData "$@" ;;

rebuild) rebuild ;;
build) build ;;
reinstall) reinstall ;;

passgen) passgen ;;

*) usage ; exit 1 ;;
esac
}

usage() {
local ME=` basename "$0" `
cat <<-END

Usage:
/path/to/$ME <command>

Where command is one of:
start Start instance
stop [-f] Stop instance. With option -f an unsuccessfull clean
shutdown if followed by a forced one.
restart Re-start instance
status Check whether instance is running

redeploy Same as: updateData, rebuild

updateData Same as: migrateSchema, importData
migrateSchema Update database schema to match source
importData Import data file - takes zero, tree or four parameters:
<mode> - one of:
cli = command-line interface (default)
web = initaited from web-based manageent console
<initiator> - Name of invoking user. When mode is
cli, value defaults to un*x user name.
<description> - Operation description. When mode is
cli, defaults to last element of SVN branch name.
<datafile> - file to import. Required when mode==web,
ignored otherwise.

rebuild Same as: build, reinstall
build Build source
reinstall Stop instance, re-create context path directory,
re-install binaries and start instance

passgen Generates passwords for users.xml configuration file

END
}

#-------------------------------------------------------------------------------
# redeploy function
#-------------------------------------------------------------------------------
:
redeploy() {
updateData && rebuild
}

#-------------------------------------------------------------------------------
# start function
#-------------------------------------------------------------------------------
start() {
sectionTitle "Start"
echo "Starting BRE instance $INSTANCE_NAME..."
export JAVA_OPTS="$JAVA_OPTS -Xms${MIN_RAM} "
export JAVA_OPTS="$JAVA_OPTS -Xmx${MAX_RAM} "
export JAVA_OPTS="$JAVA_OPTS -Dbre.config=$CONFIG_DIR "
export JAVA_OPTS="$JAVA_OPTS $LAUNCH_JVM_PRPTY "
export JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote "

exec "$CATALINA_STARTUP_SCRIPT"
}

#-------------------------------------------------------------------------------
# stop function
#-------------------------------------------------------------------------------

tryStop() {
stop -i "$@"
}

stop() {
# Used by tryStop: shutdown is stil attempted just-in-case, but
# the condition, where the process is not detected is ignored.
local IGNORE_NOT_RUNNING="false"
if [ "$1" == "-i" ] ; then
IGNORE_NOT_RUNNING=true
shift
fi

local FORCE_OPT
case $# in
0) FORCE_OPT=false ;;
1)
case "$1" in
'-f') FORCE_OPT=true ;;
*) usage ; exit 1 ;;
esac
;;
:
*) usage ; exit 1 ;;
esac

sectionTitle "Stop"

local RETRIES=4
local TIMEOUT
local NOT_RUNNING=false

while [ "$RETRIES" -gt 0 ] ; do

RETRIES=$[ $RETRIES - 1 ]

if issueStopRequest ; then
# shutdown request accepted
TIMEOUT=25 # patiently wait for process to exit
RETRIES=0 # no need to sent req again
echo Shutdown initiated.
else
# shutdown request not accepted

if [ `countPIDs` -eq 0 ] ; then
# not running in the first place
TIMEOUT=0 # no need to wait
RETRIES=0 # no need to retry
NOT_RUNNING=true # not runnig
printNotRunningMsg
elif [ "$RETRIES" -gt 0 ] ; then
# process still running, will retry
TIMEOUT=5 # use shorter timeout
printRetryMsg "$RETRIES"
else
# process still running, retries exhausted
TIMEOUT=0 # no need to wait
printGiveupMsg "$FORCE_OPT"
fi
fi

if waitForProcessToStop "$TIMEOUT" ; then
echo "Done."
return 0
else
if [ "$RETRIES" -gt 0 ] ; then
echo
fi
:
fi
done

local CONDITION

if [ "$NOT_RUNNING" == true ] ; then
CONDITION="BRE $INSTANCE_NAME does not seem to be runnig."
if [ "$IGNORE_NOT_RUNNING" == true ] ; then
echo "$CONDITION"
echo "Proceeding to next step..."
return 0
else
failure "$CONDITION"
fi
elif [ "$FORCE_OPT" != true ] ; then
failure "Could not stop BRE instance $INSTANCE_NAME""."
else
echo
echo "Will forcefully shutdown the following processe(s):"
fullProcessInfo

local PIDS

PIDS=`getPIDs`
echo
echo "Sending SIGTERM to PID(s) $PIDS..."
if kill $PIDS && waitForProcessToStop 10 ; then
echo "Terminated."
return 0
else
PIDS=`getPIDs`
echo
echo "Sending SIGKILL to PID(s) $PIDS..."
if kill -9 $PIDS && waitForProcessToStop 7 ; then
echo "Killed."
return 0
else
CONDITION="Could not kill BRE instance $INSTANCE_NAME."
CONDITION="$CONDITION Was this script invoked by the same user?"
failure "$CONDITION"
fi
fi
fi
}

issueStopRequest() {
echo "Dispatching stop request to BRE instance $INSTANCE_NAME..."

# The calling function will wait for the process to exit. So it is
# not strictly interested in the return status, but it does need to
# be notified of the connection refused condition.

local RESULT=0

"$CATALINA_SHUTDOWN_SCRIPT" 2>&1 | gawk '
BEGIN {
STATUS=0
}
/java\.net\.ConnectException: Connection refused/ {
STATUS=1
}
# default
{
print
}
END {
exit STATUS
}
' || RESULT=1

return "$RESULT"
}

waitForProcessToStop() {
local TIMEOUT="$1"

while [ "$TIMEOUT" -gt 0 ] ; do
local PIDS=`getPIDs`

if [ `echo "$PIDS" | countArgs` -eq 0 ] ; then
return 0
fi

echo -n "Waiting for BRE process $PIDS to exit... "
echo "timeout in ""$TIMEOUT""s..."
sleep 1
TIMEOUT=$[ $TIMEOUT - 1 ]
done

# On fall-through
return 1
}

printNotRunningMsg() {
cat <<-END

================================================================
NB: Error above typically occurs on an attempt to stop a BRE
instance, which is not running. If you are reinstalling on the
inactive node in production, then this message is OK.

Otherwise, please investigate.
================================================================

END
}

printRetryMsg() {
local RETRIES="$1"
cat <<-END

================================================================
BRE seems to be running, but stop request has has been refused.
This condition may also result from an attempt to stop the
instance while it is still starting.

Shutdown request will be sent again shortly.

Remaining retries: $RETRIES
================================================================

END
}

printGiveupMsg() {
local FORCE_OPT="$1"

local NEXT_STEP
if [ "$FORCE_OPT" == "true" ] ; then
NEXT_STEP="Given the -f option, forced shutdown will be attempted next."
else
NEXT_STEP="For forced shutdown please use the stop command with"
NEXT_STEP="$NEXT_STEP option -f"
fi

:
cat <<-END

================================================================
NB: BRE seems to be running, but has refused all stop requests.
The instance may be taking a long time to start, listening for
shutdown requests on a different port or experiencing a more
serious issue.

Please investigate.

$NEXT_STEP
================================================================
END
}

#-------------------------------------------------------------------------------
# restart function
#-------------------------------------------------------------------------------
restart() {
tryStop
start
}

#-------------------------------------------------------------------------------
# status function
#-------------------------------------------------------------------------------
status() {
local PIDS=`getPIDs`
local COUNT=`echo $PIDS | countArgs`
if [ "$COUNT" -gt 0 ] ; then
echo "Instance $INSTANCE_NAME is running with PID" $PIDS"."
else
echo "Instance $INSTANCE_NAME does not seem to be running."
fi
}

#-------------------------------------------------------------------------------
# data functions
#-------------------------------------------------------------------------------
updateData() {
migrateSchema
importData
}

migrateSchema() {
:
sectionTitle "Schema Migration"

# Liquibase return code is unreliable. Use gawk to track actual outcome:
( cd "$SRC_DIR" && "$ANT_EXEC" migrateDatabase ) \
| gawk '
BEGIN {
STATUS=1
}
/\[java\] Migration successful/ {
STATUS=0
}
/BUILD SUCCESSFUL/ {
/* supress this line if liquibase failed */
if(STATUS != 0) {
next
}
}
# default
{
print
}
END {
exit STATUS
}
' \
|| failure "Schema migration unsuccessful."
}

importData() {
if [ -n "$EXP_MODE" ] ; then
local MODE="$EXP_MODE"
else
local MODE="$1"
if [ -z "$MODE" ] ; then
MODE="cli"
fi
fi
if [ "$MODE" == "cli" ] ; then
sectionTitle "Data Import"
fi
local INITIATOR="$2"
local DESCRIPTION="$3"
local DATAFILE="$4"

local DATE_PREFIX=` date +'%Y-%m-%d_%H-%M-%S' `
:
local PROCESS_DIR="$COMMON_PROCESSES"/"$DATE_PREFIX"_import
local TIME_ZONE=`date +'%z' | gawk '{ print substr($0, 0, 3) }'`':00'
local ISO_DATETIME=` date +'%Y-%m-%dT%H:%M:%S' `"$TIME_ZONE"
#
local PROCESS_DATA_FILE="$PROCESS_DIR"/"$PROCESS_DATA"
local PROCESS_LOG_FILE="$PROCESS_DIR"/"$PROCESS_LOG"
local PROC_STATUS_FILE="$PROCESS_DIR"/"$PROCESS_STATUS"

checkDir COMMON_PROCESSES "$COMMON_PROCESSES"
checkDir IMPORT_UPLOAD "$IMPORT_UPLOAD"

case "$MODE" in
cli)
echo Resolving data import parameters...
# Resolve initiator
if [ -z "$INITIATOR" ] ; then
INITIATOR=`whoami`
fi
# Resolve description
if [ -z "$DESCRIPTION" ] ; then
checkSvnExec
DESCRIPTION=`svnDescription`
fi
# Resolve datafile
if [ -n "$DATAFILE" ] ; then
echo "Warning datafile parameter ignored in mode '$MODE'." >&2
fi
DATAFILE="$VERSIONED_DATA"
#
# Launch

(
export EXP_CMD="importData"
export EXP_MODE="initiate"
export EXP_ACTAUL_MODE="$MODE"
export EXP_DATETIME="$ISO_DATETIME"
export EXP_INITIATOR="$INITIATOR"
export EXP_DESCRIPTION="$DESCRIPTION"
export EXP_PROCESS_DIR="$PROCESS_DIR"
export EXP_PROCESS_DATA="$PROCESS_DATA"
#
:
echo "Importing data..."
lock "$COMMON_LOCK" "echo '$PROC_FAILURE' >'$PROC_STATUS_FILE'"
#
# For CLI make sure that extra properties are up-to-date
installExtraProperties
#
mkdir "$PROCESS_DIR"
cp -a "$DATAFILE" "$PROCESS_DATA_FILE"
script -c "$THIS_SCRIPT 2>&1" "$PROCESS_LOG_FILE"
#
# script ignores output code
# so success has to be asserted explicityly:
if ! grep '^BUILD SUCCESSFUL' \
<"$PROCESS_LOG_FILE" >/dev/null
then
failure "Data import unsuccessful."
else
echo "$PROC_SUCCESS" >"$PROC_STATUS_FILE"
fi
unlock "$COMMON_LOCK"
)
;;
web)
# Vaidate initiator
if [ -z "$INITIATOR" ] ; then
echo "Initaitor must be specified, when mode=='$MODE'." >&2
exit 1
fi
# Vaidate description
if [ -z "$DESCRIPTION" ] ; then
echo "Description must be specified, when mode=='$MODE'." >&2
exit 1
fi
# Vaidate datafile
if [ -z "$DATAFILE" ] ; then
echo "Datafile must be specified, when mode=='$MODE'." >&2
exit 1
fi

mkdir "$PROCESS_DIR"
(
:
export EXP_CMD="importData"
export EXP_MODE="initiate"
export EXP_ACTAUL_MODE="$MODE"
export EXP_DATETIME="$ISO_DATETIME"
export EXP_INITIATOR="$INITIATOR"
export EXP_DESCRIPTION="$DESCRIPTION"
export EXP_PROCESS_DIR="$PROCESS_DIR"
export EXP_PROCESS_DATA="$PROCESS_DATA"
#
lock "$COMMON_LOCK" "echo '$PROC_FAILURE' >'$PROC_STATUS_FILE'"
mv "$DATAFILE" "$PROCESS_DATA_FILE"
"$THIS_SCRIPT"
echo "$PROC_SUCCESS" >"$PROC_STATUS_FILE"
unlock "$COMMON_LOCK"
) >"$PROCESS_LOG_FILE" 2>&1 &
;;
initiate)
#
# TODO: may wish to define
# -Dlog4j.configuration=file:"$CONFIG_DIR"/import_log4j.config
#
set -x
(
cd "$SRC_DIR"
"$ANT_EXEC" \
-Dbre.import.mode="$EXP_ACTAUL_MODE" \
-Dbre.import.datetime="$EXP_DATETIME" \
-Dbre.import.initiator="$EXP_INITIATOR" \
-Dbre.import.description="$EXP_DESCRIPTION" \
-Dbre.import.processDir="$EXP_PROCESS_DIR" \
importData
)
;;
simple)
cat >&2 <<-END
Although 'simple' is a valid mode in build.xml, it designates
that the import was not initaiated from this script and hence
can not be used here. Please specify 'cli' or 'web' instead.
END
exit 1
;;
*)
echo "Mode '$MODE' is not valid. Please specify 'cli' or 'web'." >&2
exit 1
;;
:
esac
}

checkDir() {
local VAR="$1"
local DIR="$2"

if ! [ -d "$DIR" ] ; then
local MSG
MSG=`printf "Not found or not a directory:\n%s='%s'" "$VAR" "$DIR" `
failure "$MSG"
fi
}

checkSvnExec() {
if ! [ -x "$SVN_EXEC" ] ; then
local LINE1="Not found or not an executable:"
local MSG
MSG=`printf "%s\nSVN_EXEC='%s'" "$LINE1" "$SVN_EXEC" `
failure "$MSG"
fi
}

svnDescription() {
local SVNINFO=` "$SVN_EXEC" info "$SRC_DIR" `
local BRANCH=` echo "$SVNINFO" | grep '^URL:' | sed -e 's/.*\///' `
local REVISION=` echo "$SVNINFO" | grep '^Revision:' | sed -e 's/.* //' `
echo "$BRANCH r$REVISION"
}

#-------------------------------------------------------------------------------
# build/install functions
#-------------------------------------------------------------------------------
rebuild() {
build
reinstall
}

build() {
sectionTitle "Build"
( cd "$SRC_DIR" ; "$ANT_EXEC" clean dist ) || failure "Build failed."
}

reinstall() {
tryStop
:
sectionTitle "Install"
resetContextPathDir
installExtraProperties
installBinaries
start
}


resetContextPathDir() {
echo Deleting and recreating context path directory...
( cd "$WEBAPP_ROOT" \
|| failure "Web application parent directory is inaccessible."
rm -rf "$CONTEXT_PATH"
mkdir -p "$CONTEXT_PATH"
) || failure "Could not recreate web application context directory."
}

installExtraProperties() {
echo "Generating $EXTRA_PROPERTIES_FILE..."
if ! cat >"$EXTRA_PROPERTIES_FILE" <<-END

#
bre.deployment.script=$THIS_SCRIPT
bre.common.lock=$COMMON_LOCK
bre.common.lockInfo=$COMMON_LOCK_INFO
bre.common.processes=$COMMON_PROCESSES
bre.import.upload=$IMPORT_UPLOAD
bre.process.data=$PROCESS_DATA
bre.process.log=$PROCESS_LOG
bre.process.status=$PROCESS_STATUS
bre.process.metadata=$PROCESS_METADATA
bre.process.success=$PROC_SUCCESS
bre.process.failure=$PROC_FAILURE
END
then
failure "Could not generate $EXTRA_PROPERTIES_FILE."
fi
}

:
installBinaries() {
echo Unpacking binaries...
( cd "$WEBAPP_ROOT"/"$CONTEXT_PATH" && unzip -q "$WAR_FILE" ) \
|| failure "Could not install binaries."
}

passgen() {
local PASS1
local PASS2
local DONE="no"
while [ "$DONE" != "yes" ] ; do
read -s -p "Please enter new password: " PASS1
echo
read -s -p "Please re-enter new password: " PASS2
echo
if [ "$PASS1" != "$PASS2" ] ; then
echo "The two passwords must match. Please re-try."
else
DONE="yes"
fi
done
md5sum <<<"$PASS1"
}


fullProcessInfo() {

:
ps -ef \
| grep "$LONG_GREP_PATTERN" \
| grep -v 'grep .*'"$SHORT_GREP_PATTERN"
}

getPIDs() {
fullProcessInfo \
| gawk '
BEGIN { first=1 ; result="" }
{
if (first) {
first=0
}
else {
result=result" "
}
result=result$2
}
END { print result }
'
}

countPIDs() {
getPIDs | countArgs
}

countArgs() {
gawk '
BEGIN { count=0 }
{ count+=NF }
END { print count }
'
}


lock() {
local LOCK_NODE="$1" # Should be a directory with absolute path supplied.
# The directory must not exist,
# but its parent must exist and the user must
# have permissions to create the lock directory.
# Only the last lock will be deleted automatically
# on exit
:

local CLEANUP_CMD="$2" # When the shell exits, the lock is automatically
# deleted. If the second argument is present,
# then it is also executed on exit. This is useful
# when eg temporary directories are created.

ensureNotRoot

# Ensure that the lock is deleted on exit
#
local EXIT_ACTION="echo 'Removing lock $LOCK_NODE, if it exists.' >&2"
EXIT_ACTION="$EXIT_ACTION ; test -d '$LOCK_NODE' && rmdir '$LOCK_NODE'"
EXIT_ACTION="$EXIT_ACTION ; exit 1 "
#
if [ -n "$CLEANUP_CMD" ] ; then
EXIT_ACTION="$CLEANUP_CMD ; $EXIT_ACTION"
fi
#
trapAllExit "$EXIT_ACTION"

if mkdir "$LOCK_NODE" ; then
# The only valid operation on this dirtectory should be rmdir.
# In particular we want no children and no shells with
# current working directory in the lock directory.
#
chmod 200 "$LOCK_NODE"
return 0
else
trapAllExit "$CLEANUP_CMD" # Perform only CLEANUP_CMD on exit

if [ -e "$LOCK_NODE" ] ; then
local OWNER=` ls -ld "$LOCK_NODE" | gawk -- '{print $3}' `
echo "A lock owned by user '$OWNER' seems to exist already." >&2
fi
failure "Unable to obtain lock."
fi
}

unlock() {
local LOCK_NODE="$1"

rmdir "$LOCK_NODE"
local RET=$?

trapAllExitCancel
:

if [ "$RET" -ne "0" ] ; then
echo "Warning could not remove lock $LOCK_NODE!" >&2
return 1
else
return 0
fi
}

trapAllExit() {
local CMD="$1"

# Hookup CMD to the EXIT event
#
trap "$CMD" "EXIT"

# Re-route all "interesting" interrupts to EXIT
# May need to include others, see "man kill" for more info.
#
local SIG
for SIG in ALRM HUP INT KILL PIPE TERM QUIT ; do
trap "echo -n '$SIG' ; exit 1" "$SIG"
done
}

trapAllExitCancel() {
local SIG
for SIG in EXIT ALRM HUP INT KILL PIPE TERM QUIT ; do
trap "$SIG"
done
}

ensureNotRoot() {
if [ `whoami` == "root" ] ; then
echo "You are logged in as root!" >&2
echo "PLEASE LOGIN AS YOURSELF and try again." >&2
echo "Aborting." >&2
exit 1
else
return 0
fi
}

#------------------------------------------------------------------------------
# pretty printing
:
#------------------------------------------------------------------------------
sectionTitle() {
local TEXT="$@"
echo
separator
echo " $TEXT"
separator
}

failure() {
local TEXT="$@"
failureHeader "!!! FAILURE !!!" >&2
echo "$TEXT" >&2
echo "Aborting." >&2
exit 1
}

failureHeader() {
local TEXT="$1"
local WIDTH=`gawk <<<"$TEXT" '{ print length() }'`

local MARGIN="2"

local OUTER_WIDTH=$[$MARGIN + $WIDTH + $MARGIN + 2]

echo
repeatChar '*' "$OUTER_WIDTH"
printf '*%*s%-*s%*s*\n' "$MARGIN" "" "$WIDTH" "" "$MARGIN" ""
printf '*%*s%-*s%*s*\n' "$MARGIN" "" "$WIDTH" "$TEXT" "$MARGIN" ""
printf '*%*s%-*s%*s*\n' "$MARGIN" "" "$WIDTH" "" "$MARGIN" ""
repeatChar '*' "$OUTER_WIDTH"
}

separator() {
repeatChar "-" 80
}

repeatChar() {
local CHAR="$1"
local COUNT="$2"

BLANKS=` printf '%*s' "$COUNT" "" `
echo "${BLANKS// /$CHAR}"
}

:
separator
echo " $TEXT"
separator
}

failure() {
local TEXT="$@"
failureHeader "!!! FAILURE !!!" >&2
echo "$TEXT" >&2
echo "Aborting." >&2
exit 1
}

failureHeader() {
local TEXT="$1"
local WIDTH=`gawk <<<"$TEXT" '{ print length() }'`

local MARGIN="2"

local OUTER_WIDTH=$[$MARGIN + $WIDTH + $MARGIN + 2]

echo
repeatChar '*' "$OUTER_WIDTH"
printf '*%*s%-*s%*s*\n' "$MARGIN" "" "$WIDTH" "" "$MARGIN" ""
printf '*%*s%-*s%*s*\n' "$MARGIN" "" "$WIDTH" "$TEXT" "$MARGIN" ""
printf '*%*s%-*s%*s*\n' "$MARGIN" "" "$WIDTH" "" "$MARGIN" ""
repeatChar '*' "$OUTER_WIDTH"
}

separator() {
repeatChar "-" 80
}

repeatChar() {
local CHAR="$1"
local COUNT="$2"

BLANKS=` printf '%*s' "$COUNT" "" `
echo "${BLANKS// /$CHAR}"
}

main "$@"
(END)


Thirdly, These are virtual ip address mentioned in the cluster.conf files except that they are not configured any where.

Fourth, These cluster is using the ocfs file system.

So can you please look into this and reply my three questions regarding the fencing.

Thanks a ton!!!!!!!!!
Ravinder
 
Old 04-23-2013, 08:27 PM   #6
ravindert
Member
 
Registered: Oct 2011
Posts: 68

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by ravindert View Post
Thanks Dear
Thanks a lott!!!!! hyperdaz for your feed backup.

Firstly , yes these two cluster servers are setup on the virtual machines running on the Citrix xen server.

So can you please tell me how i can check what are setting exist in the xen server for the fencing at virtual switch level??

Secondly below are the content of one of the bre,vmsm1,vmsr1 scripts etc

[root@adc3bs21a ~]# cat /etc/init.d/bre
#!/bin/bash
BRE_INST=prd1
BRE_HOME=/app/bre/instances/$BRE_INST
BRE_USER=bre

case $1 in
start)
su - bre -c "$BRE_HOME/clusterScripts/startup.sh" 2> /dev/null >&2
RETVAL=$?
;;
stop)
su - bre -c "$BRE_HOME/clusterScripts/shutdown.sh" 2> /dev/null >&2
RETVAL=$?
;;
status)
su - bre -c "$BRE_HOME/clusterScripts/sanityDispatch.sh" 2> /dev/null >&2
RETVAL=$?
if [ $RETVAL -eq 0 ]; then
echo "BRE is running";
else
echo "BRE is not running";
fi
;;
*)
echo "Usage: $0 (start|stop|status)"
exit 1
;;
esac
exit $RETVAL


===============



[root@adc3bs21a ~]# cat /app/bre/instances/prd1/clusterScripts/startup.sh
#!/bin/bash

exec ` dirname "$0" `/../scripts/bre.sh start
[root@adc3bs21a ~]#


=================

[root@adc3bs21a ~]#cat /app/bre/instances/prd1/scripts/bre.sh

#!/bin/bash

SCRIPT_HOME=` dirname "$0" `
SCRIPT_HOME=` (cd "$SCRIPT_HOME" && pwd) `
THIS_SCRIPT="$SCRIPT_HOME"/` basename "$0" `

#-------------------------------------------------------------------------------
# Bootstrap configuration
#-------------------------------------------------------------------------------
BOOTSTRAP="$SCRIPT_HOME"/bootstrap.sh

if ! . "$BOOTSTRAP" ; then
echo " Please place a file called "
echo " $BOOTSTRAP, "
echo " containing a line like this: "
echo " CONFIG_DIR=/path/to/bre/config/dir"
exit 1
fi

CONFIG_BASE=scriptConfig.sh
SCRIPT_CONFIG="$CONFIG_DIR"/"$CONFIG_BASE"
if ! . "$SCRIPT_CONFIG" ; then
echo " Please ensure that "
echo " $BOOTSTRAP "
echo " points to an existing directory, which contains $CONFIG_BASE."
exit 1
fi


export JAVA_HOME CATALINA_HOME ANT_HOME

CATALINA_BIN_DIR="$CATALINA_HOME"/bin
CATALINA_STARTUP_SCRIPT="$CATALINA_BIN_DIR"/startup.sh
CATALINA_SHUTDOWN_SCRIPT="$CATALINA_BIN_DIR"/shutdown.sh

LAUNCH_JVM_PRPTY="-Dbre.instanceName=""$INSTANCE_NAME"
LONG_GREP_PATTERN="Dbre.instanceName=""$INSTANCE_NAME"
SHORT_GREP_PATTERN="bre.instanceName=""$INSTANCE_NAME"

main() {
if [ -n "$EXP_CMD" ] ; then
# Make sure that EXP_CMD is intentional
if [ -n "$1" ] ; then
echo "Error varibale EXP_CMD is defined for a normal web/cli " >&2
echo "invocation." >&2
exit 1
fi
# Another instance of this script calling itself
local CMD="$EXP_CMD"
else
local CMD="$1"
if ! shift 1 ; then
usage
exit 1
fi
fi

set -e

case "$CMD" in
start) start ;;
stop) stop "$@" ;;
restart) restart ;;
status) status ;;

redeploy) redeploy ;;

updateData) updateData ;;
migrateSchema) migrateSchema ;;
importData) importData "$@" ;;

rebuild) rebuild ;;
build) build ;;
reinstall) reinstall ;;

passgen) passgen ;;

*) usage ; exit 1 ;;
esac
}

usage() {
local ME=` basename "$0" `
cat <<-END

Usage:
/path/to/$ME <command>

Where command is one of:
start Start instance
stop [-f] Stop instance. With option -f an unsuccessfull clean
shutdown if followed by a forced one.
restart Re-start instance
status Check whether instance is running

redeploy Same as: updateData, rebuild

updateData Same as: migrateSchema, importData
migrateSchema Update database schema to match source
importData Import data file - takes zero, tree or four parameters:
<mode> - one of:
cli = command-line interface (default)
web = initaited from web-based manageent console
<initiator> - Name of invoking user. When mode is
cli, value defaults to un*x user name.
<description> - Operation description. When mode is
cli, defaults to last element of SVN branch name.
<datafile> - file to import. Required when mode==web,
ignored otherwise.

rebuild Same as: build, reinstall
build Build source
reinstall Stop instance, re-create context path directory,
re-install binaries and start instance

passgen Generates passwords for users.xml configuration file

END
}

#-------------------------------------------------------------------------------
# redeploy function
#-------------------------------------------------------------------------------
:
redeploy() {
updateData && rebuild
}

#-------------------------------------------------------------------------------
# start function
#-------------------------------------------------------------------------------
start() {
sectionTitle "Start"
echo "Starting BRE instance $INSTANCE_NAME..."
export JAVA_OPTS="$JAVA_OPTS -Xms${MIN_RAM} "
export JAVA_OPTS="$JAVA_OPTS -Xmx${MAX_RAM} "
export JAVA_OPTS="$JAVA_OPTS -Dbre.config=$CONFIG_DIR "
export JAVA_OPTS="$JAVA_OPTS $LAUNCH_JVM_PRPTY "
export JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote "

exec "$CATALINA_STARTUP_SCRIPT"
}

#-------------------------------------------------------------------------------
# stop function
#-------------------------------------------------------------------------------

tryStop() {
stop -i "$@"
}

stop() {
# Used by tryStop: shutdown is stil attempted just-in-case, but
# the condition, where the process is not detected is ignored.
local IGNORE_NOT_RUNNING="false"
if [ "$1" == "-i" ] ; then
IGNORE_NOT_RUNNING=true
shift
fi

local FORCE_OPT
case $# in
0) FORCE_OPT=false ;;
1)
case "$1" in
'-f') FORCE_OPT=true ;;
*) usage ; exit 1 ;;
esac
;;
:
*) usage ; exit 1 ;;
esac

sectionTitle "Stop"

local RETRIES=4
local TIMEOUT
local NOT_RUNNING=false

while [ "$RETRIES" -gt 0 ] ; do

RETRIES=$[ $RETRIES - 1 ]

if issueStopRequest ; then
# shutdown request accepted
TIMEOUT=25 # patiently wait for process to exit
RETRIES=0 # no need to sent req again
echo Shutdown initiated.
else
# shutdown request not accepted

if [ `countPIDs` -eq 0 ] ; then
# not running in the first place
TIMEOUT=0 # no need to wait
RETRIES=0 # no need to retry
NOT_RUNNING=true # not runnig
printNotRunningMsg
elif [ "$RETRIES" -gt 0 ] ; then
# process still running, will retry
TIMEOUT=5 # use shorter timeout
printRetryMsg "$RETRIES"
else
# process still running, retries exhausted
TIMEOUT=0 # no need to wait
printGiveupMsg "$FORCE_OPT"
fi
fi

if waitForProcessToStop "$TIMEOUT" ; then
echo "Done."
return 0
else
if [ "$RETRIES" -gt 0 ] ; then
echo
fi
:
fi
done

local CONDITION

if [ "$NOT_RUNNING" == true ] ; then
CONDITION="BRE $INSTANCE_NAME does not seem to be runnig."
if [ "$IGNORE_NOT_RUNNING" == true ] ; then
echo "$CONDITION"
echo "Proceeding to next step..."
return 0
else
failure "$CONDITION"
fi
elif [ "$FORCE_OPT" != true ] ; then
failure "Could not stop BRE instance $INSTANCE_NAME""."
else
echo
echo "Will forcefully shutdown the following processe(s):"
fullProcessInfo

local PIDS

PIDS=`getPIDs`
echo
echo "Sending SIGTERM to PID(s) $PIDS..."
if kill $PIDS && waitForProcessToStop 10 ; then
echo "Terminated."
return 0
else
PIDS=`getPIDs`
echo
echo "Sending SIGKILL to PID(s) $PIDS..."
if kill -9 $PIDS && waitForProcessToStop 7 ; then
echo "Killed."
return 0
else
CONDITION="Could not kill BRE instance $INSTANCE_NAME."
CONDITION="$CONDITION Was this script invoked by the same user?"
failure "$CONDITION"
fi
fi
fi
}

issueStopRequest() {
echo "Dispatching stop request to BRE instance $INSTANCE_NAME..."

# The calling function will wait for the process to exit. So it is
# not strictly interested in the return status, but it does need to
# be notified of the connection refused condition.

local RESULT=0

"$CATALINA_SHUTDOWN_SCRIPT" 2>&1 | gawk '
BEGIN {
STATUS=0
}
/java\.net\.ConnectException: Connection refused/ {
STATUS=1
}
# default
{
print
}
END {
exit STATUS
}
' || RESULT=1

return "$RESULT"
}

waitForProcessToStop() {
local TIMEOUT="$1"

while [ "$TIMEOUT" -gt 0 ] ; do
local PIDS=`getPIDs`

if [ `echo "$PIDS" | countArgs` -eq 0 ] ; then
return 0
fi

echo -n "Waiting for BRE process $PIDS to exit... "
echo "timeout in ""$TIMEOUT""s..."
sleep 1
TIMEOUT=$[ $TIMEOUT - 1 ]
done

# On fall-through
return 1
}

printNotRunningMsg() {
cat <<-END

================================================================
NB: Error above typically occurs on an attempt to stop a BRE
instance, which is not running. If you are reinstalling on the
inactive node in production, then this message is OK.

Otherwise, please investigate.
================================================================

END
}

printRetryMsg() {
local RETRIES="$1"
cat <<-END

================================================================
BRE seems to be running, but stop request has has been refused.
This condition may also result from an attempt to stop the
instance while it is still starting.

Shutdown request will be sent again shortly.

Remaining retries: $RETRIES
================================================================

END
}

printGiveupMsg() {
local FORCE_OPT="$1"

local NEXT_STEP
if [ "$FORCE_OPT" == "true" ] ; then
NEXT_STEP="Given the -f option, forced shutdown will be attempted next."
else
NEXT_STEP="For forced shutdown please use the stop command with"
NEXT_STEP="$NEXT_STEP option -f"
fi

:
cat <<-END

================================================================
NB: BRE seems to be running, but has refused all stop requests.
The instance may be taking a long time to start, listening for
shutdown requests on a different port or experiencing a more
serious issue.

Please investigate.

$NEXT_STEP
================================================================
END
}

#-------------------------------------------------------------------------------
# restart function
#-------------------------------------------------------------------------------
restart() {
tryStop
start
}

#-------------------------------------------------------------------------------
# status function
#-------------------------------------------------------------------------------
status() {
local PIDS=`getPIDs`
local COUNT=`echo $PIDS | countArgs`
if [ "$COUNT" -gt 0 ] ; then
echo "Instance $INSTANCE_NAME is running with PID" $PIDS"."
else
echo "Instance $INSTANCE_NAME does not seem to be running."
fi
}

#-------------------------------------------------------------------------------
# data functions
#-------------------------------------------------------------------------------
updateData() {
migrateSchema
importData
}

migrateSchema() {
:
sectionTitle "Schema Migration"

# Liquibase return code is unreliable. Use gawk to track actual outcome:
( cd "$SRC_DIR" && "$ANT_EXEC" migrateDatabase ) \
| gawk '
BEGIN {
STATUS=1
}
/\[java\] Migration successful/ {
STATUS=0
}
/BUILD SUCCESSFUL/ {
/* supress this line if liquibase failed */
if(STATUS != 0) {
next
}
}
# default
{
print
}
END {
exit STATUS
}
' \
|| failure "Schema migration unsuccessful."
}

importData() {
if [ -n "$EXP_MODE" ] ; then
local MODE="$EXP_MODE"
else
local MODE="$1"
if [ -z "$MODE" ] ; then
MODE="cli"
fi
fi
if [ "$MODE" == "cli" ] ; then
sectionTitle "Data Import"
fi
local INITIATOR="$2"
local DESCRIPTION="$3"
local DATAFILE="$4"

local DATE_PREFIX=` date +'%Y-%m-%d_%H-%M-%S' `
:
local PROCESS_DIR="$COMMON_PROCESSES"/"$DATE_PREFIX"_import
local TIME_ZONE=`date +'%z' | gawk '{ print substr($0, 0, 3) }'`':00'
local ISO_DATETIME=` date +'%Y-%m-%dT%H:%M:%S' `"$TIME_ZONE"
#
local PROCESS_DATA_FILE="$PROCESS_DIR"/"$PROCESS_DATA"
local PROCESS_LOG_FILE="$PROCESS_DIR"/"$PROCESS_LOG"
local PROC_STATUS_FILE="$PROCESS_DIR"/"$PROCESS_STATUS"

checkDir COMMON_PROCESSES "$COMMON_PROCESSES"
checkDir IMPORT_UPLOAD "$IMPORT_UPLOAD"

case "$MODE" in
cli)
echo Resolving data import parameters...
# Resolve initiator
if [ -z "$INITIATOR" ] ; then
INITIATOR=`whoami`
fi
# Resolve description
if [ -z "$DESCRIPTION" ] ; then
checkSvnExec
DESCRIPTION=`svnDescription`
fi
# Resolve datafile
if [ -n "$DATAFILE" ] ; then
echo "Warning datafile parameter ignored in mode '$MODE'." >&2
fi
DATAFILE="$VERSIONED_DATA"
#
# Launch

(
export EXP_CMD="importData"
export EXP_MODE="initiate"
export EXP_ACTAUL_MODE="$MODE"
export EXP_DATETIME="$ISO_DATETIME"
export EXP_INITIATOR="$INITIATOR"
export EXP_DESCRIPTION="$DESCRIPTION"
export EXP_PROCESS_DIR="$PROCESS_DIR"
export EXP_PROCESS_DATA="$PROCESS_DATA"
#
:
echo "Importing data..."
lock "$COMMON_LOCK" "echo '$PROC_FAILURE' >'$PROC_STATUS_FILE'"
#
# For CLI make sure that extra properties are up-to-date
installExtraProperties
#
mkdir "$PROCESS_DIR"
cp -a "$DATAFILE" "$PROCESS_DATA_FILE"
script -c "$THIS_SCRIPT 2>&1" "$PROCESS_LOG_FILE"
#
# script ignores output code
# so success has to be asserted explicityly:
if ! grep '^BUILD SUCCESSFUL' \
<"$PROCESS_LOG_FILE" >/dev/null
then
failure "Data import unsuccessful."
else
echo "$PROC_SUCCESS" >"$PROC_STATUS_FILE"
fi
unlock "$COMMON_LOCK"
)
;;
web)
# Vaidate initiator
if [ -z "$INITIATOR" ] ; then
echo "Initaitor must be specified, when mode=='$MODE'." >&2
exit 1
fi
# Vaidate description
if [ -z "$DESCRIPTION" ] ; then
echo "Description must be specified, when mode=='$MODE'." >&2
exit 1
fi
# Vaidate datafile
if [ -z "$DATAFILE" ] ; then
echo "Datafile must be specified, when mode=='$MODE'." >&2
exit 1
fi

mkdir "$PROCESS_DIR"
(
:
export EXP_CMD="importData"
export EXP_MODE="initiate"
export EXP_ACTAUL_MODE="$MODE"
export EXP_DATETIME="$ISO_DATETIME"
export EXP_INITIATOR="$INITIATOR"
export EXP_DESCRIPTION="$DESCRIPTION"
export EXP_PROCESS_DIR="$PROCESS_DIR"
export EXP_PROCESS_DATA="$PROCESS_DATA"
#
lock "$COMMON_LOCK" "echo '$PROC_FAILURE' >'$PROC_STATUS_FILE'"
mv "$DATAFILE" "$PROCESS_DATA_FILE"
"$THIS_SCRIPT"
echo "$PROC_SUCCESS" >"$PROC_STATUS_FILE"
unlock "$COMMON_LOCK"
) >"$PROCESS_LOG_FILE" 2>&1 &
;;
initiate)
#
# TODO: may wish to define
# -Dlog4j.configuration=file:"$CONFIG_DIR"/import_log4j.config
#
set -x
(
cd "$SRC_DIR"
"$ANT_EXEC" \
-Dbre.import.mode="$EXP_ACTAUL_MODE" \
-Dbre.import.datetime="$EXP_DATETIME" \
-Dbre.import.initiator="$EXP_INITIATOR" \
-Dbre.import.description="$EXP_DESCRIPTION" \
-Dbre.import.processDir="$EXP_PROCESS_DIR" \
importData
)
;;
simple)
cat >&2 <<-END
Although 'simple' is a valid mode in build.xml, it designates
that the import was not initaiated from this script and hence
can not be used here. Please specify 'cli' or 'web' instead.
END
exit 1
;;
*)
echo "Mode '$MODE' is not valid. Please specify 'cli' or 'web'." >&2
exit 1
;;
:
esac
}

checkDir() {
local VAR="$1"
local DIR="$2"

if ! [ -d "$DIR" ] ; then
local MSG
MSG=`printf "Not found or not a directory:\n%s='%s'" "$VAR" "$DIR" `
failure "$MSG"
fi
}

checkSvnExec() {
if ! [ -x "$SVN_EXEC" ] ; then
local LINE1="Not found or not an executable:"
local MSG
MSG=`printf "%s\nSVN_EXEC='%s'" "$LINE1" "$SVN_EXEC" `
failure "$MSG"
fi
}

svnDescription() {
local SVNINFO=` "$SVN_EXEC" info "$SRC_DIR" `
local BRANCH=` echo "$SVNINFO" | grep '^URL:' | sed -e 's/.*\///' `
local REVISION=` echo "$SVNINFO" | grep '^Revision:' | sed -e 's/.* //' `
echo "$BRANCH r$REVISION"
}

#-------------------------------------------------------------------------------
# build/install functions
#-------------------------------------------------------------------------------
rebuild() {
build
reinstall
}

build() {
sectionTitle "Build"
( cd "$SRC_DIR" ; "$ANT_EXEC" clean dist ) || failure "Build failed."
}

reinstall() {
tryStop
:
sectionTitle "Install"
resetContextPathDir
installExtraProperties
installBinaries
start
}


resetContextPathDir() {
echo Deleting and recreating context path directory...
( cd "$WEBAPP_ROOT" \
|| failure "Web application parent directory is inaccessible."
rm -rf "$CONTEXT_PATH"
mkdir -p "$CONTEXT_PATH"
) || failure "Could not recreate web application context directory."
}

installExtraProperties() {
echo "Generating $EXTRA_PROPERTIES_FILE..."
if ! cat >"$EXTRA_PROPERTIES_FILE" <<-END

#
bre.deployment.script=$THIS_SCRIPT
bre.common.lock=$COMMON_LOCK
bre.common.lockInfo=$COMMON_LOCK_INFO
bre.common.processes=$COMMON_PROCESSES
bre.import.upload=$IMPORT_UPLOAD
bre.process.data=$PROCESS_DATA
bre.process.log=$PROCESS_LOG
bre.process.status=$PROCESS_STATUS
bre.process.metadata=$PROCESS_METADATA
bre.process.success=$PROC_SUCCESS
bre.process.failure=$PROC_FAILURE
END
then
failure "Could not generate $EXTRA_PROPERTIES_FILE."
fi
}

:
installBinaries() {
echo Unpacking binaries...
( cd "$WEBAPP_ROOT"/"$CONTEXT_PATH" && unzip -q "$WAR_FILE" ) \
|| failure "Could not install binaries."
}

passgen() {
local PASS1
local PASS2
local DONE="no"
while [ "$DONE" != "yes" ] ; do
read -s -p "Please enter new password: " PASS1
echo
read -s -p "Please re-enter new password: " PASS2
echo
if [ "$PASS1" != "$PASS2" ] ; then
echo "The two passwords must match. Please re-try."
else
DONE="yes"
fi
done
md5sum <<<"$PASS1"
}


fullProcessInfo() {

:
ps -ef \
| grep "$LONG_GREP_PATTERN" \
| grep -v 'grep .*'"$SHORT_GREP_PATTERN"
}

getPIDs() {
fullProcessInfo \
| gawk '
BEGIN { first=1 ; result="" }
{
if (first) {
first=0
}
else {
result=result" "
}
result=result$2
}
END { print result }
'
}

countPIDs() {
getPIDs | countArgs
}

countArgs() {
gawk '
BEGIN { count=0 }
{ count+=NF }
END { print count }
'
}


lock() {
local LOCK_NODE="$1" # Should be a directory with absolute path supplied.
# The directory must not exist,
# but its parent must exist and the user must
# have permissions to create the lock directory.
# Only the last lock will be deleted automatically
# on exit
:

local CLEANUP_CMD="$2" # When the shell exits, the lock is automatically
# deleted. If the second argument is present,
# then it is also executed on exit. This is useful
# when eg temporary directories are created.

ensureNotRoot

# Ensure that the lock is deleted on exit
#
local EXIT_ACTION="echo 'Removing lock $LOCK_NODE, if it exists.' >&2"
EXIT_ACTION="$EXIT_ACTION ; test -d '$LOCK_NODE' && rmdir '$LOCK_NODE'"
EXIT_ACTION="$EXIT_ACTION ; exit 1 "
#
if [ -n "$CLEANUP_CMD" ] ; then
EXIT_ACTION="$CLEANUP_CMD ; $EXIT_ACTION"
fi
#
trapAllExit "$EXIT_ACTION"

if mkdir "$LOCK_NODE" ; then
# The only valid operation on this dirtectory should be rmdir.
# In particular we want no children and no shells with
# current working directory in the lock directory.
#
chmod 200 "$LOCK_NODE"
return 0
else
trapAllExit "$CLEANUP_CMD" # Perform only CLEANUP_CMD on exit

if [ -e "$LOCK_NODE" ] ; then
local OWNER=` ls -ld "$LOCK_NODE" | gawk -- '{print $3}' `
echo "A lock owned by user '$OWNER' seems to exist already." >&2
fi
failure "Unable to obtain lock."
fi
}

unlock() {
local LOCK_NODE="$1"

rmdir "$LOCK_NODE"
local RET=$?

trapAllExitCancel
:

if [ "$RET" -ne "0" ] ; then
echo "Warning could not remove lock $LOCK_NODE!" >&2
return 1
else
return 0
fi
}

trapAllExit() {
local CMD="$1"

# Hookup CMD to the EXIT event
#
trap "$CMD" "EXIT"

# Re-route all "interesting" interrupts to EXIT
# May need to include others, see "man kill" for more info.
#
local SIG
for SIG in ALRM HUP INT KILL PIPE TERM QUIT ; do
trap "echo -n '$SIG' ; exit 1" "$SIG"
done
}

trapAllExitCancel() {
local SIG
for SIG in EXIT ALRM HUP INT KILL PIPE TERM QUIT ; do
trap "$SIG"
done
}

ensureNotRoot() {
if [ `whoami` == "root" ] ; then
echo "You are logged in as root!" >&2
echo "PLEASE LOGIN AS YOURSELF and try again." >&2
echo "Aborting." >&2
exit 1
else
return 0
fi
}

#------------------------------------------------------------------------------
# pretty printing
:
#------------------------------------------------------------------------------
sectionTitle() {
local TEXT="$@"
echo
separator
echo " $TEXT"
separator
}

failure() {
local TEXT="$@"
failureHeader "!!! FAILURE !!!" >&2
echo "$TEXT" >&2
echo "Aborting." >&2
exit 1
}

failureHeader() {
local TEXT="$1"
local WIDTH=`gawk <<<"$TEXT" '{ print length() }'`

local MARGIN="2"

local OUTER_WIDTH=$[$MARGIN + $WIDTH + $MARGIN + 2]

echo
repeatChar '*' "$OUTER_WIDTH"
printf '*%*s%-*s%*s*\n' "$MARGIN" "" "$WIDTH" "" "$MARGIN" ""
printf '*%*s%-*s%*s*\n' "$MARGIN" "" "$WIDTH" "$TEXT" "$MARGIN" ""
printf '*%*s%-*s%*s*\n' "$MARGIN" "" "$WIDTH" "" "$MARGIN" ""
repeatChar '*' "$OUTER_WIDTH"
}

separator() {
repeatChar "-" 80
}

repeatChar() {
local CHAR="$1"
local COUNT="$2"

BLANKS=` printf '%*s' "$COUNT" "" `
echo "${BLANKS// /$CHAR}"
}

:
separator
echo " $TEXT"
separator
}

failure() {
local TEXT="$@"
failureHeader "!!! FAILURE !!!" >&2
echo "$TEXT" >&2
echo "Aborting." >&2
exit 1
}

failureHeader() {
local TEXT="$1"
local WIDTH=`gawk <<<"$TEXT" '{ print length() }'`

local MARGIN="2"

local OUTER_WIDTH=$[$MARGIN + $WIDTH + $MARGIN + 2]

echo
repeatChar '*' "$OUTER_WIDTH"
printf '*%*s%-*s%*s*\n' "$MARGIN" "" "$WIDTH" "" "$MARGIN" ""
printf '*%*s%-*s%*s*\n' "$MARGIN" "" "$WIDTH" "$TEXT" "$MARGIN" ""
printf '*%*s%-*s%*s*\n' "$MARGIN" "" "$WIDTH" "" "$MARGIN" ""
repeatChar '*' "$OUTER_WIDTH"
}

separator() {
repeatChar "-" 80
}

repeatChar() {
local CHAR="$1"
local COUNT="$2"

BLANKS=` printf '%*s' "$COUNT" "" `
echo "${BLANKS// /$CHAR}"
}

main "$@"
(END)


Thirdly, These are virtual ip address mentioned in the cluster.conf files except that they are not configured any where.

Fourth, These cluster is using the ocfs file system.

So can you please look into this and reply my three questions regarding the fencing.

Thanks a ton!!!!!!!!!
Ravinder
 
Old 04-24-2013, 09:42 AM   #7
hyperdaz
Member
 
Registered: Sep 2004
Location: UK
Distribution: CentOS 5.5
Posts: 44

Rep: Reputation: 1
Thanks for the information i'll try and digest..

If you can do some googling for something like "xen server fencing RHEL Cluster" might give some helpful hints on where to look, as with all things there are many ways to do everything.

on your VMs/Nodes see if you have any of these scripts hopefully you have all three.

fence_cxs_redhat.py, XenAPI.py, fencing.py

find / -iname "fence_cxs_redhat.py"
find / -iname "XenAPI.py"
find / -iname "fencing.py"

If you do you can look at all options availible with
fence_cxs_redhat.py -h

Before I go further what sort of environment is this Production? Development can you test/break things without any issues?

Cheers
Hdaz
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Redhat Cluster Fencing Method for San Boot Nodes Ahmedqme Red Hat 0 03-31-2012 06:20 AM
New to cluster fencing and STONITH Halex Linux - Server 1 08-03-2010 01:42 PM
RedHat Cluster suite ( HALVM) fencing& service Q. sklemer Linux - Enterprise 1 01-12-2010 06:37 AM
RedHat CLuster fencing general question Josephwpc Linux - Newbie 4 02-18-2009 10:30 PM
Cluster Fencing??? your_shadow03 Linux - Server 3 10-10-2008 11:48 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 03:18 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration