LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Red Hat (https://www.linuxquestions.org/questions/red-hat-31/)
-   -   Red Hat 6.5 - takes an hour to reboot (https://www.linuxquestions.org/questions/red-hat-31/red-hat-6-5-takes-an-hour-to-reboot-4175536283/)

paulfoel 03-10-2015 07:33 AM

Red Hat 6.5 - takes an hour to reboot
 
Apart from boot.log (which has no times in) and /var/log/messages, what can I look at to determine why this server is taking so long to reboot?

paulfoel 03-10-2015 07:36 AM

Code:

Mar 10 06:30:35 server1 kernel: Kernel logging (proc) stopped.
Mar 10 06:30:35 sever1 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="1317" x-info="http://www.rsyslog.com"] exit
ing on signal 15.
Mar 10 07:25:04 server1 kernel: imklog 5.8.10, log source = /proc/kmsg started.
Mar 10 07:25:04 server1 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="1317" x-info="http://www.rsyslog.com"] start

Extract from /var/log/messages. As you can see, there are 55 mins where nothing appears to happen. Any way to increase log level for this?

linosaurusroot 03-10-2015 07:40 AM

It's not so much log level but you're waiting for the new logging to start.

paulfoel 03-10-2015 08:41 AM

Quote:

Originally Posted by linosaurusroot (Post 5329819)
It's not so much log level but you're waiting for the new logging to start.

So how can I determine whats going on for these 55 minutes?

linosaurusroot 03-10-2015 09:29 AM

Quote:

Originally Posted by paulfoel (Post 5329846)
So how can I determine whats going on for these 55 minutes?

Are you present with the server and screen and keyboard?

paulfoel 03-10-2015 11:14 AM

Quote:

Originally Posted by linosaurusroot (Post 5329867)
Are you present with the server and screen and keyboard?

I have remote access. Its a live server so I can't reboot just now.

Its scheduled to reboot once a week at 6-30am.

linosaurusroot 03-10-2015 12:04 PM

Do you have a similar system for practice?
Has this slow boot problem only appeared recently?

John VV 03-10-2015 03:02 PM

is "bootchart" installed ?
http://www.bootchart.org/

use that to get a log and a SVG image you can look at


in more newer OS's this is built in ( RHEL7)
systemd-analyze

paulfoel 03-11-2015 04:37 AM

Quote:

Originally Posted by linosaurusroot (Post 5329938)
Do you have a similar system for practice?
Has this slow boot problem only appeared recently?

No test system. Not sure how long its been like this - probably a long time. I've only had visibility of this server for the past few weeks.

paulfoel 03-11-2015 04:44 AM

Quote:

Originally Posted by John VV (Post 5330015)
is "bootchart" installed ?
http://www.bootchart.org/

use that to get a log and a SVG image you can look at


in more newer OS's this is built in ( RHEL7)
systemd-analyze

Thanks. Had a look at this - looks good.

BUT, I wont get authorisation to install this on a live server unfortunately. Is there any way to check whats going on without installing additional tools?

linosaurusroot 03-11-2015 10:10 AM

Quote:

Originally Posted by paulfoel (Post 5330298)
No test system. Not sure how long its been like this - probably a long time. I've only had visibility of this server for the past few weeks.

Convince the boss you need a lifelike test system, make one from new and see if it has this boot problem. If it doesn't swap them.

frankbell 03-11-2015 09:34 PM

What are specs of this system and what is the usage environment?

One of the members of my LUG is an accomplished Linux sysadmin who has recently acquired (within the past year) some very brawny servers that run RHEL (I forget the manufacturer). It's been a while since he demonstrated them at a LUG meeting, but they have 64 GB RAM and serve about 150 users in a thin-client environment.

They take 20 minutes to boot to login from power on. That is their normal.

linosaurusroot 03-12-2015 05:32 AM

Quote:

Originally Posted by frankbell (Post 5330710)
They take 20 minutes to boot to login from power on. That is their normal.

That's a good point. I've seen several minutes (~10?) in bios-type activity before starting RH. If this is confirmed by OP it may be unavoidable until the next hardware change.

GaWdLy 03-12-2015 09:03 PM

Somebody needs to sit in front of it to see what takes it so long. It's pretty likely that there's an automated fsck going on (look in /etc/fstab to see). Larger servers take up to ~20 minutes to test RAM, engage and test RAIDs, but many servers need to fsck one filesystem or another at boot-time, and that could take quite a bit of time.

You really won't know until someone hooks a console up to it and reports back to you.

speirs1 03-13-2015 06:10 PM

You said you only have ssh access? Does the server have a BMC? iLo, Drac etc. Redirect serial out to the Virtual Serial Port linking the BMC and server. You can then ssh to the BMC and view everything as if you were sitting at a console. Also remove "rhgb quiet" from the kernel boot parameter if you haven't already. /etc/grub/grub.conf

Red Hat has some nice articles on the VSP thing. Step by step even for the Dell/HP side.


All times are GMT -5. The time now is 10:42 PM.