How can a program run but not be in the process list?
 

I wrote a program that watches (using inotify) for new images from
remote webcams then makes thumbnails of them. It runs all the time.
Occasionally, so they tell me, it stops making thumbnails. It keeps a
log of every inotification and the program never stops on its own but
the script they use to restart the program looks for the program in
the process list and starts it only if it finds it absent. The only
way this makes sense is if the program continues but its process
either has disappeared from the process list or no longer is
associated with the program. How can this happen?

Don't know about that, but you could probably replace that script with one that forks a copy of the webcam script then waits on the child-pid, reaping it (and then promptly restarting it) if it dies for any reason. Presumably the parent-process never would die. You can also use inittab to do this sort of thing ... just be sure to start the process as nobody or somesuch. (The "init" process basically does nothing but wait around for its children, being of course the "proud papa" of the whole system ...)

runner
 

Easy, just take out the code that searches for the process.

I didn't report a problem: I asked a question, to wit: can a program continue to execute after it either no longer has a process or the process list no longer associates the process with the program?

A process is the running instance of a program. It requires a pid to be scheduled.

That would mean that the reports people give me mean that a program can continue to run but no longer be identified (presumably in /proc/$PID) with the program that called it?

No way that I can think of. But anything's possible.
I would be more inclined to doubt the info you are being given, or at least demand incontrovertible proof that all that is alleged is actually what is happening.
Not implying they are lying, possibly just misinterpreting the data.

A program can fork itself and even replace itself, see the exec... cmds http://linux.die.net/man/3/execl

The OP is the author - hopefully [s]he would be aware of the likelihood of any such practices.

Yes, I wrote the program. It makes an entry in a log every time inotify informs it of a new webcam photo. That log shows no sign of the program not working: notifications are logged every 10 or 15 or 30 minutes (different cameras have different frequencies) for months straight and no errors are reported. The people on site tell me that a user informs them occasionally that the thumbnail is not a miniature of the current photo, that not only is it not but that it is black (the webcams run all night) at a time that the sun has been up for hours. There's a lag of a second or so, but not hours. They restart with a script I have written that looks for the program running. That program finds the thumbnail-maker for camera in question not in the process list.

I have rewritten the program to store its PID in /var/run/ and the script to look for that file and report about the PID. So I should find out next time it happens.

You might also add another script that runs every minute by cron, picks up that PID from /var/run, shows it from "ps" and also gathers its /proc/PID info. Then if something odd happens, you have some idea of what and when.

Normally a program does not just fall out of the process list. More likely it strangled itself from some bad memory access, and the program name in virtual memory was a victim of this, leading it have some other string for the program name in the "ps" list.

Beyond this thread, you might need to go through debug cycles on the program (which I'm guessing is your intention once you catch it in the misdeed).