mount: could not find filesystem '/dev/root'
 

Here's the backstory: I've got a 1And1 server which was pre-installed with Fedora Core 6, and it's running kernel 2.6.16. We've been experiencing random server hangs (doesn't ssh, doesn't ping, serial console is unresponsive). So, I want to try a new kernel, but only once have I ever been able to build a new kernel for it and have it work (and I don't remember what I did on the one that kinda worked).

The problem I usually run into is the kernel will boot, detect the NVidia SATA controller fine, detect the MD mirrored drives fine, and then I'll get a "mount: could not find filesystem '/dev/root'" and then the kernel panics and restarts the machine.

I have copy/pasted the output from a successful 2.6.16 boot and from the unsuccessful 2.6.31 boot and I've begun comparing them, line-by-line. Except for slight changes in the working of some of the status/debugging messages, the detection of the MD devices is identical. It finds /dev/md1, /dev/md5, /dev/md6, and /dev/md7, so I don't think it's an issue of the kernel not seeing the SATA controller.

The root filesystem is on /dev/md1, and it's ext3. The MD, ext2, and ext3 drivers are compiled into the kernel (not as modules), so even a problem with the initrd shouldn't be a deal-killer at this point.

From the unsuccessful boot:
Meanwhile, the successful-booting 2.6.16 kernel finds the ext3 filesystem just fine:

And, just for kicks, /etc/fstab looks like:

Anybody have any clue what I'm missing here?

If you're doing your own kernel, have you tried running without the initrd ?. What does your boot stanza look like ?.

I am thinking your new kernel is missing either the raid stuff or you don't have ext3 file system compiled into the kernel. If you don't want it compiled in then you going to have to include it in your initrd image. Personally I see no point in using initrd images.

Oddly, I have deliberately set the config to have all RAID and ext2/3 compiled in yet, without initrd, I get a different set of errors. Now, I don't get any of the "considering", "adding", "created", "bind" messages from the MD modules, even though the kernel reports registering them.

I'm just getting more and more confused....

This is odd. It is loading all of the raid stuff but it seams its not reassembling your raid array. If these were usb drives or something I would say its because the devices are not getting time to load up before the raid detection starts, but I don't think this is your issue.
Did you use raidtools or mdadm to setup your raid?

Your initrd is new as well, I presume? Or do you only have one version? I'm pretty sure you have to make a new initrd for each kernel.

As per the suggestions of the first replies, I took the initrd out of the boot stanza altogether. That's why I said that it's odd that, even though I've indicated that the raid and ext2/3 drivers should be compiled into the kernel, the ability of the kernel to recognize the raid partitions changes if I remove the initrd from the boot stanza.

Yes, well, I read that. But your initrd might be doing more than loading drivers, are you successful with the old kernel and without the initrd?

They came pre-made from 1And1. When you sign up for a root-server, they just give you one configured like this. I think it's two 160GB SATA's with something like 4 partitions on each, paired to make 4 MD's. So, /dev/sda1 and /dev/sdb1 make up /dev/md1... etc.

What I find *also* strange is that, even when the MD driver failed to set up the md devices, shouldn't the kernel be able to see the *individual* partitions? For example, shouldn't I be able to rip out the whole /dev/sdb drive and have the kernel still be able to use /dev/sda1 instead of /dev/md1? (Maybe it will, and I just need to change the "root=" part of the boot stanza to be root=/dev/sda1 instead of root=/devb/md1).

Old kernel doesn't use initrd.

Here's a related question:

Is there a reliable way to "upgrade" or "update" a .config file from a previous kernel version? I *have* the config file that was used for the working 2.6.16 kernel. However, in the past, I've tried just using a previous .config in a newer kernel and the build would fail. I discovered that, at the least, I'd have to run either "make config" or "make menuconfig" to get any new/changed .config directives into the file... but I still recall having trouble even then.

At present, I'm on the verge of just diff'ing the old config with the current one to make a to-do list of stuff I need to change with menuconfig. Hopefully, there's a slicker way these days.

@ jemenake I thought it was make oldconfig or something like that to merge the old one into the new one.

I know this is old but I took me a lot of research and headache to find the solution:

Enable CONFIG_SYSFS_DEPRECATED_V2 in the kernel source configuration.
If you run “make menuconfig”, browse to:
General Setup —> and enable (Y) the following:
and
—OR—

Edit your .config file with a text editor and add/modify entry:
Mods: I'm replying this thread so other people searching in Google (like me) will find it.