LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices



Reply
 
Search this Thread
Old 11-02-2009, 09:01 AM   #1
websissy
Member
 
Registered: Jul 2008
Posts: 49

Rep: Reputation: 15
Unhappy Help! Reboot Fails aft Debian Etch upgrade now at GRUB prompt


Help!! I'm a debian server admin who is preparing to upgrade from etch to Lenny and was told I should upgrade from Etch from last August to the final release of Etch from May '09 first.

So, I made a full boot drive backup with dd. Then I ran the "mock upgrade" using aptitude first and everything went well. Then I ran the real upgrade (128 pacakges were upgraded) and everything seemed to go well too. (Famous last words!)

In the middle of the upgrade I got a notice that I'd need to reboot after the upgrade had finished in order to let Apache complete its rebuild (No surprise there!). So when the upgrade completed successfully with no horrible errors or warnings, I double-checked to make sure all updates had been installed and then I tried to reboot the system. ARggghhhhh! The bleeping reboot failed.

The server is remote (1,200 miles away). So I requested KVM access from the server center and find I'm at the GRUB loader's prompt...

YIKES! I have NO idea what went wrong or what the exact name of the system bootstrap file is. I do know it's on sda1 (hd0,0) in Grub's parlance) and I know my full save (dd) backup is on sdb1 (Is that hd1,0 to grub?). I also know the Etch release I was running ended in ".18". I'm fairly sure it was like vmlinuz-2.6.18 and that it ended with -amd64.

What the heck do I do NOW??

Last edited by websissy; 11-02-2009 at 09:52 AM.
 
Old 11-02-2009, 09:12 AM   #2
sarajevo
Member
 
Registered: Apr 2005
Distribution: Debian, OpenBSD,Fedora,RedHat
Posts: 228
Blog Entries: 1

Rep: Reputation: 31
Quote:
Originally Posted by websissy View Post
Help!! I'm a debian server admin who is preparing to upgrade from etch to Lenny and was told I should upgrade from Etch from last August to the final release of Etch from May '09 first.

So, I made a full boot drive backup with dd. Then I ran the "mock upgrade" using aptitude first and everything went well. Then I ran the real upgrade (128 pacakges were upgraded) and everything seemed to go well too. (Famous last words!)

In the middle of the upgrade I got a notice that I'd need to reboot after the upgrade had finished in order to let Apache complete its rebuild (No surprise there!). So when the upgrade completed successfully with no horrible errors or warnings, I double-checked to make sure all updates had been installed and then I tried to reboot the system. ARggghhhhh! The bleeping reboot failed.

The server is remote (1,200 miles away). So I requested KVM access from the server center and find I'm at the GRUB loader's prompt...

YIKES! I have NO idea what went wrong or what the exact name of the system bootstrap file is. I do know it's on sda1 (hd0,0) in Grub's parlance) and I know my full save (dd) backup is on sdb1 (Is that hd1,0 to grub?). I also know the Etch release I was running ended in ".18". I'm fairly sure it was like 2.6.xx.18

What the heck do I do NOW??
if you can over KVM boot to some live CD system and see what is in /boot, what kind of kernel is placed there during upgrade.

Then try to compare it with value in menu.lst, what values for kernel boot are there, still old ones, or ....

What kind of error you see ?

This link http://www.mondorescue.org/ describes mondo backup solution....I recomend it, do backup, test does backup works--if yes, perfom upgrades/changes



Regards

Last edited by sarajevo; 11-02-2009 at 09:16 AM.
 
Old 11-02-2009, 09:32 AM   #3
websissy
Member
 
Registered: Jul 2008
Posts: 49

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by sarajevo View Post
if you can over KVM boot to some live CD system and see what is in /boot, what kind of kernel is placed there during upgrade.

Then try to compare it with value in menu.lst, what values for kernel boot are there, still old ones, or ....

What kind of error you see ?

This link http://www.mondorescue.org/ describes mondo backup solution....I recomend it, do backup, test does backup works--if yes, perfom upgrades/changes



Regards
This server center gives very little support. The only set of boot CDs they are likely to have there would be the original OS install CD's and they might well be for Lenny now and not for Etch. Is it possible get into recovery mode from those CDs? Clearly, I don't want to do a fresh OS install! What I need to do now is get a good look at the root directory on that boot drive and on the backup drive as well.

I will also look at the mondorescue.org site. But I assumed the full-drive dd copy would produce a bootable backup. In fact, I'm not sure it DIDN'T produce a bootable backup. I'm just not clear about how to convince GRUB to boot from that drive.

Confucious say: You never know what you should know until you need to know it! :-(

Thanks.

Last edited by websissy; 11-02-2009 at 09:38 AM.
 
Old 11-02-2009, 09:37 AM   #4
fpmurphy
Member
 
Registered: Jan 2009
Location: /dev/ph
Distribution: Fedora, Ubuntu, Redhat, Centos
Posts: 294

Rep: Reputation: 61
Quote:
Is that hd1,0 to grub?
yes.

You are getting as far of the grub prompt so that is good news.

Here is a pointer to the GRUB manual. You are going to need it as a reference.

What is the output of the following grub command?
Code:
cat /grub/grub.conf
 
Old 11-02-2009, 10:55 AM   #5
websissy
Member
 
Registered: Jul 2008
Posts: 49

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by fpmurphy View Post
yes.

You are getting as far of the grub prompt so that is good news.

Here is a pointer to the GRUB manual. You are going to need it as a reference.

What is the output of the following grub command?
Code:
cat /grub/grub.conf
Well, having burrowed deep into the manure pile that is grub manual in the past, I had hoped to avoid that horrid experience again. But I did manage to find a great grub tutorial (I KNOW that's a contradiction in terms, but it's true.) at:

http://www.troubleshooters.com/linux/grub/grub.htm

This helped me figure out the location and name of the vmlinuz file; but when I tried to reboot using that information (from hd0 which is actually sda1) I got the following kernel panic:

Quote:
Originally Posted by bootloader
VFS: Cannot open root device "hda1" or unknown-block((0,0)
Please append a correct "root=" option
Kernel panic - not syncing: VFS: unable to mount root fs on unknown-block(0,0)
Before getting this error, I used these commands in grub:

Quote:
Originally Posted by grub
grub> root (hd0,0)
grub> kernel /boot/vmlinuz-2.6.18-6-amd64 root=/dev/hda1
grub> boot
I also tried this again with the following commands

Quote:
Originally Posted by bootloader
root (hd0,0)
kernel /boot/vmlinuz-2.6.18-6-amd64 root=/dev/sda1
boot
And it produced the same kernel panic error with a reference to sda1 rather than hda1

Does anyone know what I'm doing wrong with the device names?

I forgot on the restart to look up the output of the "cat /grub/grub.conf" command. I'll try that on this next restart. And I gues I'll have to go dig out that grub manual after all... sigh!

Thanks again!

Last edited by websissy; 11-02-2009 at 11:06 AM.
 
Old 11-02-2009, 11:43 AM   #6
websissy
Member
 
Registered: Jul 2008
Posts: 49

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by fpmurphy View Post
yes.
What is the output of the following grub command?
Code:
cat /grub/grub.conf
This command doesn't seem to work. When I try it all I get is "File Not Found"

I've also tried

find /grub/gru<tab>

in an effort to use autocomplete to figure out the file name. But it isn't working either. This IS a debian box. That MAY mean that the file name for grub.conf has changed here.

Is there another way to locate it?
 
Old 11-02-2009, 03:35 PM   #7
yancek
Guru
 
Registered: Apr 2008
Distribution: PCLinux, Slackware
Posts: 5,112

Rep: Reputation: 816Reputation: 816Reputation: 816Reputation: 816Reputation: 816Reputation: 816Reputation: 816
If you're looking for the configuration file it should be menu.lst which is in the Grub directory so your command should be:

cat /boot/grub/menu.lst

Your vmlinuz file should be in the /boot directory.
 
Old 11-02-2009, 04:00 PM   #8
fpmurphy
Member
 
Registered: Jan 2009
Location: /dev/ph
Distribution: Fedora, Ubuntu, Redhat, Centos
Posts: 294

Rep: Reputation: 61
Quote:
cat /boot/grub/menu.lst
Yes, you are right. I was thinking of Fedora/Redhat/Centos for some reason.

The following will not work until you add details of your initial ram disk using initrd.
Code:
root (hd0,0)
kernel /boot/vmlinuz-2.6.18-6-amd64 root=/dev/sda1
boot
 
Old 11-03-2009, 12:23 AM   #9
websissy
Member
 
Registered: Jul 2008
Posts: 49

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by yancek View Post
If you're looking for the configuration file it should be menu.lst which is in the Grub directory so your command should be:

cat /boot/grub/menu.lst

Your vmlinuz file should be in the /boot directory.
You're right, Yanek. The grub directory was inside the boot directory. I knew it was near the top but wrongly remembered it as being in the root directory. That's why I couldn't find it. I figured that out later... shortly before I discovered my server has been rootkit-ed...

So now I have a totally different batch of fish to fry. :-(

Thanks for the help!
 
Old 11-03-2009, 12:34 AM   #10
websissy
Member
 
Registered: Jul 2008
Posts: 49

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by fpmurphy View Post
Yes, you are right. I was thinking of Fedora/Redhat/Centos for some reason.

The following will not work until you add details of your initial ram disk using initrd.
Code:
root (hd0,0)
kernel /boot/vmlinuz-2.6.18-6-amd64 root=/dev/sda1
boot
Yes, fpmurphy. You were right. The problem I was having was I was missing the need for the initrd command.

I'm sure I read somewhere (I believe it was in this excellent grub tutorial: http://www.troubleshooters.com/linux/grub/grub.htm) that in many cases the initrd command wasn't needed after it had been provided once. Yet in my case, the system repeatedly fails to boot if this grub command is NOT included.

It clearly wasn't required for the old kernel to boot.

Does anyone know of a way to get the info provided in initrd embedded into the kernel or must I always include it in the bootup from this day forward. If so, can someone explain why?

For the moment, I'm ignoring the fact that I'm still trying to decide what to do about that &^%$#@! rootkit and want to complete my knowledge on how to eliminate the kernel panic caused by failing to include the initrd command in grub.

Thanks!

Last edited by websissy; 11-03-2009 at 08:17 AM.
 
Old 11-03-2009, 12:33 PM   #11
fpmurphy
Member
 
Registered: Jan 2009
Location: /dev/ph
Distribution: Fedora, Ubuntu, Redhat, Centos
Posts: 294

Rep: Reputation: 61
Quote:
Does anyone know of a way to get the info provided in initrd embedded into the kernel or must I always include it in the bootup from this day forward. If so, can someone explain why?
You can build a kernel that does not need an initrd to boot by compiling all the modules into the kernel but most people do not bother. Doing so eliminates a certain amount of splash functionaility and just makes it harder to do kernel upgrades since you will have to roll your own kernel each time.

Assume than you always need an initrd line in your (grub) configuration stanza.
 
Old 11-04-2009, 10:13 AM   #12
websissy
Member
 
Registered: Jul 2008
Posts: 49

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by fpmurphy View Post
You can build a kernel that does not need an initrd to boot by compiling all the modules into the kernel but most people do not bother. Doing so eliminates a certain amount of splash functionaility and just makes it harder to do kernel upgrades since you will have to roll your own kernel each time.

Assume than you always need an initrd line in your (grub) configuration stanza.
Thanks a bunch, fpmurphy. Am I correct in concluding that compiling a new kernel also automatically produces a matching intrd image file too? Is that the deal? That's what seems to be the case. I'm just confirming my gradually growing understanding...

Thanks again!
 
Old 11-05-2009, 12:10 AM   #13
fpmurphy
Member
 
Registered: Jan 2009
Location: /dev/ph
Distribution: Fedora, Ubuntu, Redhat, Centos
Posts: 294

Rep: Reputation: 61
Quote:
Am I correct in concluding that compiling a new kernel also automatically produces a matching intrd image file too?
If you build a kernel RPM, yes. If you just build a kernel, no.
 
  


Reply

Tags
debian, etch, failed, grub, upgrade


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
When GRUB fails then command prompt appears then what to do sumeet inani Linux - Newbie 1 07-04-2009 04:47 PM
upgrade from etch to testing fails kpachopoulos Debian 2 04-06-2008 05:12 PM
correct grub.conf, reboot and cannot get grub prompt... sportivo888 Linux - Newbie 1 12-11-2007 12:47 AM
ubuntu 7.10 aft upgrade f/7.04 , rdesktop, ssh, refuse connections rocket777 Ubuntu 1 10-21-2007 07:46 PM


All times are GMT -5. The time now is 12:32 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration