[SOLVED] Unable to find process for lrpid indicated by ipcs

abudinsz · 11-09-2012, 09:56 PM

Seems I have a ghost process reading from a SysV queue on RedHat. I am looking at readers and writers to a queue with the following command:

Code:

ipcs -q -i 53674024

The output I get is the following:

Code:

Message Queue msqid=53674024
uid=500	gid=500	cuid=500	cgid=500	mode=0660
cbytes=0	qbytes=262144	qnum=0	lspid=31975	lrpid=32061
send_time=Sat Nov 10 03:29:56 2012  
rcv_time=Sat Nov 10 03:29:56 2012  
change_time=Sat Nov 10 03:17:13 2012

Next I try to find the process listed in lrpid with the following:

Code:

ps ax | grep 32061

The result is no such pid on the system. I keep rerunning the ipcs command and can see that the rcv_time is being updated so we are not talking about a process that existed earlier and is no longer around.

I should mention that the expected results for the ipcs command should indicate a different lrpid. And indeed continuously rerunning the command using watch I can see my correct application pid show up. But then the 'ghost' one shows up again. Am I missing something here? Are there other options to ps that I'm unaware of?

System and environment info:
2.6.18-238.12.1.el5 smp (rhel 5.8)
I'm running shell as root but application is a daemon running as different user.

tronayne · 11-10-2012, 04:24 PM

Have you tried

Code:

ipcs -p -q
<or>
icps -c -q

That should show you the PID and creator, the -p and -c are exclusive (use one or the other) or try -a, all.

You will see that kind of behavior when a application is using message queues -- the queue is set up and will exist until the parent dies (and if the parent process has code for removing the queue on exit, which doesn't always happen with just an exit -- it should but sometimes things don't do what they should, eh?). From your description it appears that the queue is active; a message is placed, another process is picking it up and the time stamp is updated. Your ghost may be just be an inactive queue on its way to the great byte bucket in the sky? Without examining the application it might be difficult to figure out exacting what is happening and why.

You may want to try

Code:

ps -efl

"ef" means "everything" and the l means long. There are other options that will show you a whole lot stuff; take a look at the manual page and see if there are one or two that make sense to you.

abudinsz · 11-15-2012, 02:06 PM

After a little more playtime with ps and ipcs a friend of mine and I found the solution. The ipcs app will actually print the lwpid (light weight process id) in the lrpid/lspid columns. These are unique across not just process but threads. Think of an application thread as an lwp. So the bottom line is that the -L option on ps will display all lwpids as well as their actual pid (process id). Now my grep was fruitful. It turns out that another thread in my application was also pending on the same queue (that is a whole other story...)

To use data from my orginal post the correct command would be:

Code:

ps -eL | grep 32061

This now showed me the actual application that was reading from the queue.