Moving /var from ext3 to LVM makes system logger unable to start?

quanta · 06-17-2010, 09:53 AM

Hi all,

My co-worker install CentOS with /var (ext3) is 1TB:

Code:

/dev/mapper/VolGroup00-LogVol00
                      890G  149G  696G  18% /
/dev/sdb1             902G  373M  855G   1% /var
/dev/sda1              99M   35M   60M  37% /boot
tmpfs                 5.9G     0  5.9G   0% /dev/shm

I do the following tasks to move sdb1 to VolGroup00:
- backup /var
- fdisk sdb with filesystem 8e (LVM)
- pvcreate /dev/sdb1
- vgextend VolGroup00 /dev/sdb1
- lvextend /dev/VolGroup00/LogVol00 /dev/sdb1
- resize2fs LogVol00

After that I moved /var back to / folder and reboot but the system hangs at "Starting system logger". Uninstall and re-install sysklogd package don't make any change. What I don't understand is I can start syslogd from command line:

Code:

/sbin/syslogd -m 0

or run the init script in debug mode:

Code:

sh -x /etc/init.d/syslog start

How can I debug this case?

blue_print · 06-17-2010, 10:03 AM

Is the new LVM partition properly mounted on "/var"
If so, are you able to see the log directory under "/var"
Have you copied the backup (you have taken before) to the new "/var"?

quanta · 06-17-2010, 10:30 AM

Quote:

Originally Posted by blue_print

Is the new LVM partition properly mounted on "/var"
If so, are you able to see the log directory under "/var"

No, all I have is only one Logical Volume mounted to /:

Code:

Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                      1.8T  109G  1.6T   7% /
/dev/sda1              99M   35M   60M  37% /boot
tmpfs                 5.9G     0  5.9G   0% /dev/shm

Do you mean I should create a new logical volume and mount it to /var?

Quote:

Originally Posted by blue_print

Have you copied the backup (you have taken before) to the new "/var"?

Yes, obviously.

quanta · 06-17-2010, 09:40 PM

More details:

Code:

# /etc/init.d/syslog restart
Shutting down kernel logger:                               [FAILED]
Shutting down system logger:                               [FAILED]
Starting system logger: (hangs)

Code:

# /sbin/syslogd -m 0
# ps -ef | grep syslog
root      8140     1  0 09:34 ?        00:00:00 /sbin/syslogd -m 0
root      8142  8091  0 09:34 pts/0    00:00:00 grep syslog

Code:

# /etc/init.d/syslog restart
Shutting down kernel logger:                               [FAILED]
Shutting down system logger:                               [  OK  ]
Starting system logger: (hangs)

Code:

# sh -x /etc/init.d/syslog start
+ . /etc/init.d/functions
++ TEXTDOMAIN=initscripts
++ umask 022
++ PATH=/sbin:/usr/sbin:/bin:/usr/bin
++ export PATH
++ '[' -z '' ']'
++ COLUMNS=80
++ '[' -z '' ']'
+++ /sbin/consoletype
++ CONSOLETYPE=pty
++ '[' -f /etc/sysconfig/i18n -a -z '' ']'
++ . /etc/profile.d/lang.sh
+++ sourced=0
+++ for langfile in /etc/sysconfig/i18n '$HOME/.i18n'
+++ '[' -f /etc/sysconfig/i18n ']'
+++ . /etc/sysconfig/i18n
++++ LANG=en_US.UTF-8
++++ SYSFONT=latarcyrheb-sun16
+++ sourced=1
+++ for langfile in /etc/sysconfig/i18n '$HOME/.i18n'
+++ '[' -f /root/.i18n ']'
+++ '[' -n '' ']'
+++ '[' 1 = 1 ']'
+++ '[' -n en_US.UTF-8 ']'
+++ export LANG
+++ '[' -n '' ']'
+++ unset LC_ADDRESS
+++ '[' -n '' ']'
+++ unset LC_CTYPE
+++ '[' -n '' ']'
+++ unset LC_COLLATE
+++ '[' -n '' ']'
+++ unset LC_IDENTIFICATION
+++ '[' -n '' ']'
+++ unset LC_MEASUREMENT
+++ '[' -n '' ']'
+++ unset LC_MESSAGES
+++ '[' -n '' ']'
+++ unset LC_MONETARY
+++ '[' -n '' ']'
+++ unset LC_NAME
+++ '[' -n '' ']'
+++ unset LC_NUMERIC
+++ '[' -n '' ']'
+++ unset LC_PAPER
+++ '[' -n '' ']'
+++ unset LC_TELEPHONE
+++ '[' -n '' ']'
+++ unset LC_TIME
+++ '[' -n '' ']'
+++ unset LC_ALL
+++ '[' -n '' ']'
+++ unset LANGUAGE
+++ '[' -n '' ']'
+++ unset LINGUAS
+++ '[' -n '' ']'
+++ unset _XKB_CHARSET
+++ consoletype=pty
+++ '[' -z pty ']'
+++ '[' -n '' ']'
+++ '[' -n '' ']'
+++ '[' -n en_US.UTF-8 ']'
+++ case $LANG in
+++ '[' xterm = linux ']'
+++ unset SYSFONTACM SYSFONT
+++ unset sourced
+++ unset langfile
++ '[' -z '' ']'
++ '[' -f /etc/sysconfig/init ']'
++ . /etc/sysconfig/init
+++ BOOTUP=color
+++ GRAPHICAL=yes
+++ RES_COL=60
+++ MOVE_TO_COL='echo -en \033[60G'
+++ SETCOLOR_SUCCESS='echo -en \033[0;32m'
+++ SETCOLOR_FAILURE='echo -en \033[0;31m'
+++ SETCOLOR_WARNING='echo -en \033[0;33m'
+++ SETCOLOR_NORMAL='echo -en \033[0;39m'
+++ LOGLEVEL=3
+++ PROMPT=yes
+++ AUTOSWAP=no
++ '[' pty = serial ']'
++ '[' color '!=' verbose ']'
++ INITLOG_ARGS=-q
++ __sed_discard_ignored_files='/\(~\|\.bak\|\.orig\|\.rpmnew\|\.rpmorig\|\.rpmsave\)$/d'
+ RETVAL=0
+ case "$1" in
+ start
+ '[' -x /sbin/syslogd ']'
+ '[' -x /sbin/klogd ']'
+ '[' -f /etc/sysconfig/syslog ']'
+ . /etc/sysconfig/syslog
++ SYSLOGD_OPTIONS='-m 0'
++ KLOGD_OPTIONS=-x
++ SYSLOG_UMASK=077
+ '[' -z 077 ']'
+ umask 077
+ echo -n 'Starting system logger: '
Starting system logger: + daemon syslogd -m 0
+ local gotbase= force= nicelevel corelimit
+ local pid base= user= nice= bg= pid_file=
+ nicelevel=0
+ '[' syslogd '!=' syslogd ']'
+ '[' -z '' ']'
+ base=syslogd
+ __pids_var_run syslogd ''
+ local base=syslogd
+ local pid_file=/var/run/syslogd.pid
+ pid=
+ '[' -f /var/run/syslogd.pid ']'
+ return 3
+ '[' -n '' -a -z '' ']'
+ corelimit='ulimit -S -c 0'
+ '[' -n '' ']'
+ '[' color = verbose -a -z '' ']'
+ '[' -z '' ']'
+ /bin/bash -c 'ulimit -S -c 0 >/dev/null 2>&1 ; syslogd -m 0'
+ '[' 0 -eq 0 ']'
+ success 'syslogd startup'
+ '[' color '!=' verbose -a -z '' ']'
+ echo_success
+ '[' color = color ']'
+ echo -en '\033[60G'
                                                           + echo -n '['
[+ '[' color = color ']'
+ echo -en '\033[0;32m'
+ echo -n '  OK  '
  OK  + '[' color = color ']'
+ echo -en '\033[0;39m'
+ echo -n ']'
]+ echo -ne '\r'
+ return 0
+ return 0
+ RETVAL=0
+ echo

+ echo -n 'Starting kernel logger: '
Starting kernel logger: + daemon klogd -x
+ local gotbase= force= nicelevel corelimit
+ local pid base= user= nice= bg= pid_file=
+ nicelevel=0
+ '[' klogd '!=' klogd ']'
+ '[' -z '' ']'
+ base=klogd
+ __pids_var_run klogd ''
+ local base=klogd
+ local pid_file=/var/run/klogd.pid
+ pid=
+ '[' -f /var/run/klogd.pid ']'
+ return 3
+ '[' -n '' -a -z '' ']'
+ corelimit='ulimit -S -c 0'
+ '[' -n '' ']'
+ '[' color = verbose -a -z '' ']'
+ '[' -z '' ']'
+ /bin/bash -c 'ulimit -S -c 0 >/dev/null 2>&1 ; klogd -x'
+ '[' 0 -eq 0 ']'
+ success 'klogd startup'
+ '[' color '!=' verbose -a -z '' ']'
+ echo_success
+ '[' color = color ']'
+ echo -en '\033[60G'
                                                           + echo -n '['
[+ '[' color = color ']'
+ echo -en '\033[0;32m'
+ echo -n '  OK  '
  OK  + '[' color = color ']'
+ echo -en '\033[0;39m'
+ echo -n ']'
]+ echo -ne '\r'
+ return 0
+ return 0
+ echo

+ '[' 0 -eq 0 ']'
+ touch /var/lock/subsys/syslog
+ return 0
+ exit 0
# ps -ef | grep syslog
root      8161     1  0 09:35 ?        00:00:00 syslogd -m 0
root      8167  8091  0 09:35 pts/0    00:00:00 grep syslog

start() function in init script:

Code:

start() {
	[ -x /sbin/syslogd ] || exit 5
	[ -x /sbin/klogd ] || exit 5

	# Source config
	if [ -f /etc/sysconfig/syslog ] ; then
		. /etc/sysconfig/syslog
	else
		SYSLOGD_OPTIONS="-m 0"
		KLOGD_OPTIONS="-2"
	fi

	if [ -z "$SYSLOG_UMASK" ] ; then
	      SYSLOG_UMASK=077;
	fi
	umask $SYSLOG_UMASK

 	echo -n $"Starting system logger: "

	daemon syslogd $SYSLOGD_OPTIONS
	RETVAL=$?
	echo
	echo -n $"Starting kernel logger: "
	daemon klogd $KLOGD_OPTIONS
	echo
	[ $RETVAL -eq 0 ] && touch /var/lock/subsys/syslog
	return $RETVAL
}