LinuxQuestions.org - RHEL4 Strange kernel panic on boot. Kernel Update no solution

Hi,

I have the following strange thing with a RHEL4 installation.
Since last week, the system did a reboot and now something is really fucked up. During boot we get the following messages (don't care about 'strange' typo's, my colleague typed it 'blind' from the screen)

Code:

scanning logical volumes

reading all physical volumes this may take a while

hda: packet command error:status=0x51 { Drivev Ready SeekComplete Error}

hda packet command error: error=0x54

ide failed opcode was 100

FOund volume group "Volgroup00" using metadata type lvm2

Activating logical volumes 

2 logical volumes in volume group VolGroup00" is now active

creating root device

Mounting root filesystem

kjournald starting. Commit interval 5secndes

EXT3-fs: mounted filesystem with ordered data mode

switchting to new root

Kernel panic -not syncing: Attempted to kill init!

In above: hda = cdrom
HDD is sda.

The strange thing is that we never see a 'could not mount blabla' or similar messages. First we thought it was a failing kernel update by plesk, but even after manually updating the kernel with RHN RPM's, still the same message. Booting with rescue mode and then chroot the system works. After that we even can start things like plesk and so on.

We double checked things with another RHEL4 install, and at least two things were odd:
1: the working machine has /dev/dm-0 and /dev/dm-1, the broken one doesn't
2: some files on /dev didn't have group root, but 252

We tried to recreate the /dev/dm-X nodes with [vgmknodes -v], output:

Code:

file descriptor 20 left open

file descriptor 21 left open

finding all logical volumes

fixing up missing size 148.88 gb for PV /dev/sda2

A fdisk /dev/sda shows: /dev/sda2 XX XXX XXXXX Linux LVM (I removed the numbers because this line is from another machine, but rest was identical)

We have a copy of the boot partition so if one need more info please let me know.

grub.conf:

Code:

default=0

timeout=5

splashimage=(hd0,0)/grub/splash.xpm.gz

hiddenmenu

title Red Hat Enterprise Linux ES (2.6.9-78.0.8.ELsmp)

        root (hd0,0)

        kernel /vmlinuz-2.6.9-78.0.8.ELsmp ro root=/dev/VolGroup00/LogVol00

        initrd /initrd-2.6.9-78.0.8.ELsmp.img

last part of init extracted from initrd-2.6.9-78.0.8.ELsmp.img:

Code:

echo Making device-mapper control node

mkdmnod

echo Scanning logical volumes

lvm vgscan --ignorelockingfailure

echo Activating logical volumes

lvm vgchange -ay --ignorelockingfailure VolGroup00

echo Creating root device

mkrootdev /dev/root

umount /sys

echo Mounting root filesystem

mount -o defaults --ro -t ext3 /dev/root /sysroot

mount -t tmpfs --bind /dev /sysroot/dev

echo Switching to new root

switchroot /sysroot

umount /initrd/dev

I hope someone can give me some insight in this issue, because my ideas are depleted. Ow BTW !!:: Almost at the same time we have another machine (CentOS) in another Datacenter with the identical issue!! That's just to strange in my eyes...

And for those who are reading this today: Already have a happy newyear! :D

Thanks!
Onno.