Everything started two days ago, after I've compiled new kernel and updated OpenRC (gentoo init).
As far as I can tell everything starts with BIOS, which loads GRUB from MBR, which in turn loads kernel into RAM, passing root=/dev/sda3 (in my case) parameter to it. Kernel init goes well, but when it tries to mount root filesystem it fails to do so with output like that:
Clearly, it tries to mount some binary garbage instead of /dev/sda3, so I've checked GRUB - "root=/dev/sda3" there, no problem. Reinstalled it (from another machine - "root (hd1,0); setup (hd1)") / tried passing different partition - no effect, random garbage (usually 10+ non-ASCII chars) gets appended to any passed parameter.
Then I've blamed it on new kernel I'm trying to boot (2.6.25-gentoo-r1), but booting 2.6.23 / 2.6.24 (all of them worked for months before without any issues, especially issues like THAT) gave exactly the same picture.
Next I thought it could be something with FS/HDD (Seagate ST3250624AS, 250GB, sda1 - ext2-boot-100MB, sda2 - swap-1GB, sda3 - reiserfs3.6-root-X0GB, sda4 - reiserfs3.6-storage-200+GB), but fsck showed no errors on any partitions. I've mounted them all and tried relevant files like kernel, grub.conf, fstab - nothing wrong with them at all.
Okay, it's gotta be RAM (Hynix DDR2 512MB PC-5300) failure then, so I took two modules from two separate 100% working machines (Corsair 512MB PC-5300 and Samsung 1GB PC-6400) and it's all the same with both of them, so the memory is not an issue, right?
Prehaps the memory bus selectively corrupts the data then on this MB (ASUSTeK P5RD2-VM, ATI Xpress 200 chipset), or CPU (Pentium Dualcore 820, 2x2.8GHz) got broken somehow? Nope, tried connecting this HDD to Intel D201GLY2 Platform (SiS 662/964, Celeron 220 Conroe-L CPU) and booting compiled for that hardware kernel, which works flawlessly with another gentoo root, to exactly the same effect.
At this point it started getting rediculous in my eyes, I've tried removing / creating device nodes on rootfs (/dev/sda3) but, obviously, it solved nothing - rootfs wasn't even accessed by the kernel...
Pretty much the same setup (same kernel, init) works on three more machines (from which I've tried to borrow memory, MB, kernel) with no issues except one pictured here: http://stelur.ru/public/panic_02.jpg
, but simple reboot solved that one, and since it was on encrypted partition which was mounted just fine, I guess it's unrelated.
Sooo... well, summary:
It's not GRUB, it's not kernel, it's not FS, it's not hardware, and there's nothing else related with the problem, so I assume it's gotta be a kernel issue, for the sake of sanity.
I'll try to boot from initrd as a workaround, but if anyone can shed any light on this situation, I'll be grateful, for reading all this, at the very least