investigating random restart

saiyen2002 · 11-02-2013, 01:20 PM

I am running Centos 6 2.6.32-358.11.1.el6.x86_64 on a ProLiant DL120 G7. The machine is less then a year old

The machine just randomly restarted without any cause twice in 1 day.
I have soo far checked the messages, which doesn't show anything before the restart, just when the logging started after the restart.
Below is the messages of when it restarted on both occasions

Code:

Nov  2 11:11:26 voip3 ntpd[1750]: synchronized to 217.70.19.12, stratum 2
Nov  2 11:59:06 voip3 ntpd[1750]: synchronized to 84.2.44.19, stratum 2
Nov  2 16:32:15 voip3 kernel: imklog 5.8.10, log source = /proc/kmsg started.
Nov  2 16:32:15 voip3 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="1180" x-info="http://www.rsyslog.com"] start
Nov  2 16:32:15 voip3 kernel: Initializing cgroup subsys cpuset
Nov  2 16:32:15 voip3 kernel: Initializing cgroup subsys cpu
Nov  2 16:32:15 voip3 kernel: Linux version 2.6.32-358.11.1.el6.x86_64 (mockbuild@c6b7.bsys.dev.centos.org) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Wed Jun 12 03:34:52 UTC 2013

Code:

Nov  2 11:11:26 voip3 ntpd[1750]: synchronized to 217.70.19.12, stratum 2
Nov  2 11:59:06 voip3 ntpd[1750]: synchronized to 84.2.44.19, stratum 2
Nov  2 16:32:15 voip3 kernel: imklog 5.8.10, log source = /proc/kmsg started.
Nov  2 16:32:15 voip3 rsyslogd: [origin software="rsyslogd" swVersion="5.8.10" x-pid="1180" x-info="http://www.rsyslog.com"] start
Nov  2 16:32:15 voip3 kernel: Initializing cgroup subsys cpuset
Nov  2 16:32:15 voip3 kernel: Initializing cgroup subsys cpu
Nov  2 16:32:15 voip3 kernel: Linux version 2.6.32-358.11.1.el6.x86_64 (mockbuild@c6b7.bsys.dev.centos.org) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Wed Jun 12 03

The machine is running asterisk, which had not been taking any calls prior to the restart and also shows last entry in its log file 4 minutes before the restart.
The secure log file show last entry only 3 minutes before the restart

the last command shows exact time of reboots on that day

Code:

athiq    pts/0        host-78-146-106- Sat Nov  2 16:40   still logged in
reboot   system boot  2.6.32-358.11.1. Sat Nov  2 16:32 - 17:32  (01:00)
reboot   system boot  2.6.32-358.11.1. Sat Nov  2 08:55 - 17:32  (08:37)
athiq    pts/1        192.168.0.35     Wed Oct 16 13:33 - 13:48  (00:15)

I have installed OpenIPMI tools in the hope that should this happen again, i will have better debugging tools.

In the meantime, is there anything else i can do to investigate this? Any measure to put in place so i can get better logging data if this does happen again ?

Thanks,

saiyen2002

smeezekitty · 11-02-2013, 03:07 PM

If there is no logged messages before, then there is a good chance it is hardware related.
It could be a kernel bug but something that serious would no doubt be found already.

That appears to be some sort of server module which I don't know much about. But in a normal desktop or even laptop a common cause for reboots is overheating or bad power.

Beryllos · 11-04-2013, 12:04 AM

I had a very similar problem with a Dell Optiplex 755 that I bought secondhand. I had installed Debian some recent version. It rebooted at random intervals, usually within a few hours uptime. Log files showed nothing suspicious prior to reboot. A few times it rebooted within minutes of a cold start, which suggests the trouble was not due to overheating. Dell's built-in diagnostics checked out okay. I tried replacing the memory and later the power supply (swapping these parts from a nearly identical 755 which never crashes). The last thing I tried was installing Windows XP

. The random reboots continued just the same, so that proved it was not software but hardware. Rather than going on replacing different parts, now I'm shopping around for another cheap computer, and I'll keep the old one for spare parts.