[SOLVED] CentOS server boot looping after installing updates

aschoessler · 06-25-2013, 09:47 AM

Hello,

I installed updates over the weekend on a CentOS 6 server. During the transaction test, Yum said that there was not enough space on /boot to install a kernel update. The Googles advised me to run rpm -e against old kernels to remove them. There were three kernels installed, and I did this for the two oldest ones, which I determined by looking at the names of them.

After that, I ran yum update -y and rebooted. It installed the new kernel fine. However, after the reboot, the server never came back online. I went physically to the server yesterday morning and found that it is boot looping right after saying "Loading GRUB..."

I can't imagine these events are not related. I downloaded a Fedora Core 18 live DVD and have booted that. I mounted the software RAID and verified my backup is good. However, I really, really, do not want to start over with this server.

I mounted /boot and looked at the Grub configuration. There are two kernels listed and both are represented in /boot. I tried changing the default to the second one, but it does the same thing.

I have looked about and found instructions for reinstalling Grub; however, I am not able to determine what version of Grub I have. There seem to be great differences between Grub 1 and 2. The ways of determining this seem to imply that you are in the OS, which I cannot boot.

Any advice would be much appreciated.

paulshwang · 06-25-2013, 10:34 AM

Might be a bit easier to locate the issue if you post the contents of your /boot/grub/grub.conf file and an ls -la of your /boot directory.

aschoessler · 06-25-2013, 11:21 AM

Sorry, should have included that... Here is that information:

Code:

[root@localhost grub]# cat grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd1,0)
#          kernel /vmlinuz-version ro root=/dev/md2
#          initrd /initrd-[generic-]version.img
#boot=/dev/md0
default=0
timeout=5
splashimage=(hd1,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.32-358.11.1.el6.x86_64)
        root (hd1,0)
        kernel /vmlinuz-2.6.32-358.11.1.el6.x86_64 ro root=UUID=ab10904d-2c95-4e89-a8bc-959b1550cc0f rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_MD_UUID=92eb0b4d:2d3b70d4:224e355e:7d609599 rd_MD_UUID=841ab1ea:e0104a72:9185df00:438e16c5 SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_NO_LVM rd_NO_DM rhgb quiet
        initrd /initramfs-2.6.32-358.11.1.el6.x86_64.img
title CentOS (2.6.32-358.2.1.el6.x86_64)
        root (hd1,0)
        kernel /vmlinuz-2.6.32-358.2.1.el6.x86_64 ro root=UUID=ab10904d-2c95-4e89-a8bc-959b1550cc0f rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_MD_UUID=92eb0b4d:2d3b70d4:224e355e:7d609599 rd_MD_UUID=841ab1ea:e0104a72:9185df00:438e16c5 SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_NO_LVM rd_NO_DM rhgb quiet
        initrd /initramfs-2.6.32-358.2.1.el6.x86_64.img

/boot is mounted in the Live DVD as /oboot.

Code:

[root@localhost oboot]# ls -la
total 45051
dr-xr-xr-x.  5 root root     1024 Jun 23 15:42 .
dr-xr-xr-x. 19 root root     4096 Jun 24 16:10 ..
-rw-r--r--.  1 root root   104086 Jun 11 23:59 config-2.6.32-358.11.1.el6.x86_64
-rw-r--r--.  1 root root   104085 Mar 12 20:51 config-2.6.32-358.2.1.el6.x86_64
drwxr-xr-x.  3 root root     1024 Sep 28  2012 efi
drwxr-xr-x.  2 root root     1024 Jun 25 12:17 grub
-rw-r--r--.  1 root root 16258736 Jun 23 15:42 initramfs-2.6.32-358.11.1.el6.x86_64.img
-rw-r--r--.  1 root root 16166430 Mar 28 21:58 initramfs-2.6.32-358.2.1.el6.x86_64.img
drwx------.  2 root root    12288 Sep 28  2012 lost+found
-rw-r--r--.  1 root root   185867 Jun 11 23:59 symvers-2.6.32-358.11.1.el6.x86_64.gz
-rw-r--r--.  1 root root   185828 Mar 12 20:52 symvers-2.6.32-358.2.1.el6.x86_64.gz
-rw-r--r--.  1 root root  2407976 Jun 11 23:59 System.map-2.6.32-358.11.1.el6.x86_64
-rw-r--r--.  1 root root  2407544 Mar 12 20:51 System.map-2.6.32-358.2.1.el6.x86_64
-rwxr-xr-x.  1 root root  4046224 Jun 11 23:59 vmlinuz-2.6.32-358.11.1.el6.x86_64
-rw-r--r--.  1 root root      171 Jun 11 23:59 .vmlinuz-2.6.32-358.11.1.el6.x86_64.hmac
-rwxr-xr-x.  1 root root  4043920 Mar 12 20:51 vmlinuz-2.6.32-358.2.1.el6.x86_64
-rw-r--r--.  1 root root      170 Mar 12 20:51 .vmlinuz-2.6.32-358.2.1.el6.x86_64.hmac

paulshwang · 06-25-2013, 11:51 AM

Nothing really stands out as an issue on your conf file. Next step may be to try reinstalling grub.. Since CentOS is basically the same as RHEL, it should be version 0.97

aschoessler · 06-25-2013, 05:49 PM

Quote:

Originally Posted by paulshwang

Nothing really stands out as an issue on your conf file. Next step may be to try reinstalling grub.. Since CentOS is basically the same as RHEL, it should be version 0.97

Perfect, thanks. I used a tool called Rescatux to reinstall legacy Grub. It said it couldn't properly complete the reinstallation, but when I rebooted the server, it did boot into the OS and we're back up and running.