Disk access lights constantly blinking

bdb4269 · 04-10-2006, 05:58 PM

I have three redhat servers and 2 of them's disk access lites blink normally, when (and how much) you would expect them to. The third ones disk access lights blink all the time. I'd estimate atleast once a second every second, even if there in nothing going on at all.

I have no idea where to start looking to figure out why - and I'm not even sure if I should worry about it or not.

Can anybody help?

J.W. · 04-11-2006, 02:58 AM

This command will list each process that's currently running

Code:

ps ax

I suspect that you may either be running something by default or you've got a runaway process

bdb4269 · 04-11-2006, 12:45 PM

Quote:

Originally Posted by J.W.

This command will list each process that's currently running

Code:

ps ax

I suspect that you may either be running something by default or you've got a runaway process

I've looked at all the running processes through webmin and have eliminated everything that I knew shouldn't be running and it is still blinking. It doesn't appear that the remaining processes are doing to much (judging by CPU time) though I realize that alone doesn't necessarily rule any of them out as the culprit. I've posted the list below in case anything looks suspicious. Is there a way I can check individual processes disk access?

On a different train of thought - These servers have 2 disk RAID1 arrays - I was looking in the adaptec RAID array manager utility, on the server in question, trying to see if any thing looked hinky and when I was looking in the help it said one of the actions you could perform (through a right click context menu) on a physical device was called "Blink". Which it said would make the light corresponding to that physical device blink. Though it did say that not all options would always be available. Which seemed to be the case because it didn't appear on the context menu for the drives and I couldn't find any other way to enable/disable this blinking thing. But it makes me wonder if somehow this blinking is "activated" on the drives and I just can't find the option to disable it.

When I look at the lights when there is absolutely nothing going on and the others blinks have settled down enough that I can distinctly pick out the constant blink (I realize that sounds kind of weird) it is very steady - not the usual seeming random blinks of normal disk activity - it literally just blinks at perfectly (as far as I can make out) spaced intervals of about 1 second. This of course makes me even more suspicious that this blinking option could be the cause -- but it seems that if the blinking was enabled I should be able to find the option to disable it (there is not much to the adaptec utility if you get my meaning -- too bare for obscure options to be very "buried") -- which makes me think that it's not the cause.

What do you think? Is there any process that might access the disk every second when it refreshes or something?

Also - everything that is running on this server is also running on the other two except the other two also have other things running (one of the other two has a LOT more running) and both of their disk access lights blink normally.

Any ideas where I should go from here?

Code:

Latest Process List: 
root (root)
    Process ID 	CPU 	Started 	Command
    4910 	19.0 % 	11:50 	/usr/libexec/webmin/proc/index_user.cgi
    4909 	2.5 % 	11:50 	/usr/bin/perl /usr/libexec/webmin/miniserv.pl /etc/webmin/miniserv.conf
    1935 	0.5 % 	Apr10 	./jre/bin/java -Djava.compiler=NONE -cp /usr/StorMan/RaidMan.jar com.ibm.sysmgt. ...
    2475 	0.3 % 	Apr10 	/usr/X11R6/bin/X :0 -auth /var/gdm/:0.Xauth vt7
    1   	0.0 % 	Apr10 	init
    2 	        0.0 % 	Apr10 	[migration/0]
    3 	        0.0 % 	Apr10 	[migration/1]
    4 	        0.0 % 	Apr10 	[migration/2]
    5 	        0.0 % 	Apr10 	[migration/3]
    6 	        0.0 % 	Apr10 	[keventd]
    7 	        0.0 % 	Apr10 	[ksoftirqd/0]
    8 	        0.0 % 	Apr10 	[ksoftirqd/1]
    9 	        0.0 % 	Apr10 	[ksoftirqd/2]
    10 	        0.0 % 	Apr10 	[ksoftirqd/3]
    13 	        0.0 % 	Apr10 	[bdflush]
    11 	        0.0 % 	Apr10 	[kswapd]
    12 	        0.0 % 	Apr10 	[kscand]
    14 	        0.0 % 	Apr10 	[kupdated]
    15 	        0.0 % 	Apr10 	[mdrecoveryd]
    22 	        0.0 % 	Apr10 	[ahd_dv_0]
    23 	        0.0 % 	Apr10 	[ahd_dv_1]
    24 	        0.0 % 	Apr10 	[scsi_eh_0]
    25 	        0.0 % 	Apr10 	[scsi_eh_1]
    27 	        0.0 % 	Apr10 	[scsi_eh_2]
    28 	        0.0 % 	Apr10 	[aacraid]
    30 	        0.0 % 	Apr10 	[scsi_eh_2]
    33 	        0.0 % 	Apr10 	[kjournald]
    88 	        0.0 % 	Apr10 	[khubd]
    413 	0.0 % 	Apr10 	[kjournald]
    1609 	0.0 % 	Apr10 	syslogd -m 0
    1613 	0.0 % 	Apr10 	klogd -x
    1623 	0.0 % 	Apr10 	irqbalance
    1632 	0.0 % 	Apr10 	mdmpd
    1748 	0.0 % 	Apr10 	/usr/sbin/sshd
    1762 	0.0 % 	Apr10 	xinetd -stayalive -pidfile /var/run/xinetd.pid
    1772 	0.0 % 	Apr10 	/opt/arkeia/bin/arkeiad
    1791 	0.0 % 	Apr10 	sendmail: accepting connections
    1810 	0.0 % 	Apr10 	gpm -t ps/2 -m /dev/psaux
    1828 	0.0 % 	Apr10 	crond
    1860 	0.0 % 	Apr10 	smbd -D
    1864 	0.0 % 	Apr10 	smbd -D
    1865 	0.0 % 	Apr10 	nmbd -D
    1874 	0.0 % 	Apr10 	winbindd
    1877 	0.0 % 	Apr10 	winbindd
    1894 	0.0 % 	Apr10 	rhnsd --interval 240
    1908 	0.0 % 	Apr10 	mdadm --monitor --scan -f
    1928 	0.0 % 	Apr10 	sh /usr/StorMan/StorAgnt.sh
    1937 	0.0 % 	Apr10 	smbd -D
    1943 	0.0 % 	Apr10 	smbd -D
    1950 	0.0 % 	Apr10 	smbd -D
    1961 	0.0 % 	Apr10 	/sbin/mingetty tty1
    1962 	0.0 % 	Apr10 	/sbin/mingetty tty2
    1963 	0.0 % 	Apr10 	/sbin/mingetty tty3
    1964 	0.0 % 	Apr10 	/sbin/mingetty tty4
    1965 	0.0 % 	Apr10 	/sbin/mingetty tty5
    1966 	0.0 % 	Apr10 	/sbin/mingetty tty6
    1967 	0.0 % 	Apr10 	/usr/bin/gdm-binary -nodaemon
    2024 	0.0 % 	Apr10 	/usr/bin/gdm-binary -nodaemon
    2059 	0.0 % 	Apr10 	smbd -D
    2118 	0.0 % 	Apr10 	smbd -D
    2775 	0.0 % 	Apr10 	/usr/bin/perl /usr/libexec/webmin/miniserv.pl /etc/webmin/miniserv.conf
    3999 	0.0 % 	04:02 	cupsd
    4584 	0.0 % 	09:07 	smbd -D
    4744 	0.0 % 	11:06 	smbd -D
    4745 	0.0 % 	11:06 	smbd -D
    4746 	0.0 % 	11:06 	smbd -D
    4841 	0.0 % 	11:49 	sshd: bbeetle [priv]
    4916 	0.0 % 	11:50 	sh -c ps --cols 500 -eo user,ruser,group,rgroup,pid,ppid,pgid,pcpu,vsz,nice,etim ...
    4917 	0.0 % 	11:50 	ps --cols 500 -eo user,ruser,group,rgroup,pid,ppid,pgid,pcpu,vsz,nice,etime,time ...

named (Named)
    Process ID 	CPU 	Started 	Command
    1732 	0.0 % 	Apr10 	/usr/sbin/named -u named

smmsp
    Process ID 	CPU 	Started 	Command
    1800 	0.0 % 	Apr10 	sendmail: Queue runner@01:00:00 for /var/spool/clientmqueue

ftp (FTP User)
    Process ID 	CPU 	Started 	Command
    1819 	0.0 % 	Apr10 	proftpd: (accepting connections)

xfs (X Font Server)
    Process ID 	CPU 	Started 	Command
    1851 	0.0 % 	Apr10 	xfs -droppriv -daemon

daemon (daemon)
    Process ID 	CPU 	Started 	Command
    1884 	0.0 % 	Apr10 	/usr/sbin/atd

gdm
    Process ID 	CPU 	Started 	Command
    2484 	0.0 % 	Apr10 	/usr/bin/gdmgreeter

pofcorn · 03-16-2007, 03:04 PM

Hi,

I had pretty much the same problem as you on a IBM xseries 366 at work. The SAS hard drives are connected to a ServeRAID 8i card and configured in RAID1. The leds were constantly blinking. Updating the card's firmware fixed the problem.

One thing to note is that it wasn't Linux related. As soon as the server finished booting up the 2 RAID cards (the other being a ServeRAID 6m), it started blinking, even before the OS loaded.

Also, the blinking was like it was constantly reading/writing to the disks, not like you described with a steady 1 second interval. Still, might be worth looking into.