Debian squeeze won't start after reboot
 

After my Dell R210-II server with Debian Squeeze had been running for 4 months without any problems, today I suddenly could not get a shell prompt after a successful login using SSH. Also using the virtual console from iDRAC6 I could not see any prompt.

But all other services like httpd, mysqld and so on were still running OK.

Because I have knowledge to administer the server, but do not have qualifications to debug such a problem any other way, I decided to reboot the server. Also because I did not change any configuration options on the server since weeks

But now the server (single OS install) won’t start anymore… Here is an explanation what happens:

System starts

On PXE menu I select (local) and press enter

Then there is a message:
-----------------------------------------
Booting from local disk...
GRUB loading.
Welcome to GRUB!
-----------------------------------------

After that I can select which OS I want to load and I select
-----------------------------------------
Debian GNU/Linux, with Linux 2.6.32-5-amd64
-----------------------------------------

But then the system reports the following:

-----------------------------------------
[0.527313] pci 0000:02:03.0: BAR 6: address space collision of device [0xffff0000-0xffffffff]
[0.707719] i8042.c: No controller found.
Loading, please wait...
/init: /scripts/init-top/ORDER: line 1: /scripts/init-top/all_generic_ide: Permission denied
/init: /scripts/init-top/ORDER: line 3: /scripts/init-top/blacklist: Permission denied
/init: /scripts/init-top/ORDER: line 5: /scripts/init-top/keymap: Permission denied
/init: /scripts/init-top/ORDER: line 7: /scripts/init-top/udev: Permission denied
/init: /scripts/local-top/ORDER: line 1: /scripts/local-top/mdadm: Permission denied
error sending message: Connection refused
udevadm[79]: error sending message: Connection refused

Gave up waiting for root device. Common problems:
- Boot args (cat /proc/cmdline)
- Check rootdelay= (did the system wait long enough?)
- Check root= (did the system wait for the right device?)
- Missing modules (cat proc/modules; ls /dev)
ALERT! /dev/disk/by-uuid/800d8e8b-daae-4b9e-811f-9057565aa7f3 does not excist.
Dropping to shell!

BusyBox v1.71.1 (Debian 1:1.17.1-8 built-in shell (ash)
Enter 'help' for a list of built in commands.
/bin/sh: can't access tty; job control is turned off (initramfs)
-----------------------------------------

It seems like the same problem was as discussed at http://forums.debian.net/viewtopic.php?f=17&t=73085 but I cannot boot any linux. So working from command prompt (shell) is not an option without physically going to the datacenter

Is there anybody who can help?

Thanx in advance,
Alvin

For some reason you seem to have access to the system during boot, so there's some connectivity... In the Grub menu, press 'e' to get a rudimentary shell so you can change the boot options of the chosen grub-entry. Your first task is to comment out the UUID line of the root fs (the line that states "root=") by putting a # in front of it. Then create a new line under it (go the the line below it, press enter then move the cursor up one line) and enterAssuming /sda1 is the root partition, that is. If not, change sda1 to the appropriate partition. Check to see you did it right (syntax!) then press b to boot. You may need to do this more then once if you get it wrong... Make notes of what you've changed so you can revert it back if it fails. Once the system boots, run the "update-grub" command. Mind that your problems might come from failing hardware, so they could be more persistent then you'd like...

Thanx for your reply Dutch Master... The initial config looks like:

I have tried a lot of things, like

and


but also


But none with any success :(
With last one I kept getting error that the drive was not found and after pressing a key I got back into config...

I am just playing a bit, and now I have tried:

and now I get a message:

But changing the (hd0,1) to all kinds of possible entries, like sd0,1, sda,1, sda0,1 and so on, sometimes it returns "no such disk" and all other times "Filesystem is unknown"

Your root filesystem resides on a RAID, named md0, and it seems it won't load the drivers for it. You need to comment out this line:
and replace it with
on a new line. Hopefully it works...

Unfortunately the same error as before editing the GRUB config.

Did you comment out the line starting with "search"? It tells Grub to look for that disk, but as it can't find it it won't proceed further. Try this, if it fails I'm afraid you need to plan a road trip to the data-centre soon... :(

Again, no success :(

But what could possibly be the problem? Is it a driver issue maybe? Seeing the first 2 error lines:

I came across somewhere where there was mention of some drivers of the Dell R210-II not being correctly installed with Debian Squeeze. Can this be the problem?
Is a kernel update the remedy for this?

Or is the boot sector corrupt?

Good catch, I didn't realise there could be a problem with that... :( According to this page it works: http://kmuto.jp/debian/hcl/DELL/PowerEdge+R210

I found one particularly interesting answer on a question (http://serverfault.com/questions/358...d-dell-servers) quite insightful: how many kernels do you have, are there any newer kernels then the 2.6 version you now use mentioned in the Grub menu? If there are, try the highest numbered one. If there's no other version, you may want to compile a new kernel from scratch. That sounds more difficult then it is, but still shouldn't be performed too light-heartedly though. But it does require a working system... Mind that the Backports repository holds a 3.2 kernel that could be useful, but again: a working system is mandatory... :(

You marked it as solved, what happened?

Thanx for your reply Dutch Master... We have solved the problem by upgrading the kernel to backport version 3.2.0-0. After the reboot we still got the 'address collision' error, but that did not stop the booting process. Via iDRAC we updated the NIC drivers and now the 'address collision' error is also gone.

Still it is uncertain what suddenly caused this mess...

By the way, the GRUB config is now:

So exactly like before, just the version has been upgraded.

Anyway, thanx for your effort trying to help me.