LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 12-14-2015, 04:32 AM   #1
barla.rkrishna
LQ Newbie
 
Registered: Dec 2015
Posts: 9

Rep: Reputation: Disabled
Question servers frequently going to grub state


Hello Folks,

We have bunch of servers with different OS versions of RHEL and SLES. Recently all our servers went to grub state all of a sudden one after the other. We just tried re-installing OS on all the machines followed by post OS installation of some tools like telnet,ftp,xinetd,etc.But all our efforts were not fruitful, all the servers again going to grub state after a day or so. we tried this experiment like 3-4 times but again observing the same behavior. Even we were not able to recover the machines from grub rescue state because of missing partitions. We tried thinking in all dimensions but still not root caused the issue. Could any one throw some light in root causing the issue and the possible issues that are hidden in this.Also i want to know in which scenarios a server goes to grub state.

Note: servers going to grub state just even after installing OS(with out post OS installation).

Thanks,
Ramakrishna.
 
Old 12-14-2015, 04:37 AM   #2
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,557

Rep: Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252
would be nice to describe what do you mean by "all dimensions". Also what do you mean by grub state. What happened to the partitions? I think that can be a power problem, but you gave almost no information, so I can only guess.
 
1 members found this post helpful.
Old 12-14-2015, 04:50 AM   #3
barla.rkrishna
LQ Newbie
 
Registered: Dec 2015
Posts: 9

Original Poster
Rep: Reputation: Disabled
Thanks for the quick reply.

We have ethernet card(2-port) inserted in the machine. We have tried the below experiments

1)Installed OS with ethernet card in it
2)So just to eliminate the suspicion on card we have removed the card and installed the OS.
3)Changed the power cables and line.

grub means the system crashed and were you have some limited commands or means to recover. I have gone through the grub rescue steps available in different sites but couldn't able to recover the server.

Let me know if you need more info.

Thanks,
Ramakrishna
 
Old 12-14-2015, 05:39 AM   #4
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
If your partitions go away, I can see three possible causes:
  • Bad hardware - interfaces, cables, switches, disks, power, ....
  • Something in the OS overwrites the partitions.
  • Something outside overwrites the partitions. Are you sharing disks?
When a server crashes, is there any panic message on the screen?
Can you retrace the recovery steps you took and how and at what point the recovery failed?
Any other context you can share?
 
Old 12-14-2015, 06:18 AM   #5
barla.rkrishna
LQ Newbie
 
Registered: Dec 2015
Posts: 9

Original Poster
Rep: Reputation: Disabled
Not able to see any kernel panic or anything on screen, i could only see the grub screen.

i followed the steps from the below link

http://askubuntu.com/questions/11959...own-filesystem

Attached the screen shot of grub screen and steps followed in recovering.
Attached Thumbnails
Click image for larger version

Name:	grub_screen.png
Views:	26
Size:	223.7 KB
ID:	20278   Click image for larger version

Name:	steps.jpg
Views:	25
Size:	123.7 KB
ID:	20279  
 
Old 12-14-2015, 07:30 AM   #6
berndbausch
LQ Addict
 
Registered: Nov 2013
Location: Tokyo
Distribution: Mostly Ubuntu and Centos
Posts: 6,316

Rep: Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002Reputation: 2002
If you have RHEL and SLES systems, you better find instructions for RHEL and SLES rather than Ubuntu, as system initialization is an area where distros differ.
By the way, there is also a big difference between RHEL6 and 7 - RHEL 6 still uses legacy grub, RHEL 7 grub2.

Here are instructions for Centos that might apply to your case: https://wiki.centos.org/TipsAndTricks/ReinstallGRUB.

I suspect that your partitions are fine, but they are not on the disk where grub looks for them. In other words, your grub installation might be wrong. Use the install disk to boot into rescue mode to check this. Any LiveCD (or USB stick) would be similarly helpful.
 
1 members found this post helpful.
Old 12-14-2015, 08:40 AM   #7
barla.rkrishna
LQ Newbie
 
Registered: Dec 2015
Posts: 9

Original Poster
Rep: Reputation: Disabled
I have mounted iso file and tried installing grub and it went fine. Then unmounted iso and did reboot. But it is again booting to rescue saying boot device not found.
Attached Thumbnails
Click image for larger version

Name:	boot_failed.png
Views:	19
Size:	111.9 KB
ID:	20281  
 
Old 12-15-2015, 03:29 AM   #8
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,557

Rep: Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252Reputation: 7252
that can be a simple misconfiguration, either you have disabled something in bios or you have a wrong /etc/fstab (or something similar).
Did you configure any raid related settings?
 
Old 12-15-2015, 09:00 AM   #9
yancek
LQ Guru
 
Registered: Apr 2008
Distribution: Slackware, Ubuntu, PCLinux,
Posts: 10,411

Rep: Reputation: 2468Reputation: 2468Reputation: 2468Reputation: 2468Reputation: 2468Reputation: 2468Reputation: 2468Reputation: 2468Reputation: 2468Reputation: 2468Reputation: 2468
In post 5 you have two images indicating "no such partition" and unknown filesystem. Your partitions have changed for some reason and Grub is unaware of the change. Since you have only Linux filesystems, I'm surprised you get an "unknown filesystem" error. Might need fsck?? The image in post 7 indicates a UUID for a partition pointed to by Grub does not exist which could be a result of changing partitions without updating Grub. Posting more detailed information would be the best next step. If you go to the site below, download and burn the boot repair to a CD and boot with it you should be able to get detailed info by selecting the "Create Bootinfo Summary" option and posting the link. Do not try to run any repair.

https://help.ubuntu.com/community/Boot-Repair
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How do you keep your servers running in stable and secure state? lazycai Linux - Server 4 08-05-2010 08:31 PM
iptables -I INPUT -p tcp --dport 22 -i eth0 -m state --state NEW -m recent --set deadeyes Linux - Server 14 07-29-2009 05:30 AM
how grub knows there is a system in suspend/hibernate state ? marozsas Linux - General 9 07-27-2009 03:35 PM
Can't access BIOS on IBM T41 - Grub is in error state 15 (<- My doing!) ontornorg Linux - Newbie 3 03-28-2008 06:28 PM
Bad page state in process apache2 (Server reboots frequently) Zeest Linux - Server 2 05-18-2007 12:13 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 03:30 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration