-   Linux - Software (
-   -   Java Daemon Dies Quietly (

dswanson99 03-03-2010 12:21 PM

Java Daemon Dies Quietly
I have a java processing running as a system service using a shell script to launch it. It runs under a user other than root via sudo -u.

The process runs just fine but seems to die sometimes for no apparent reason. When the process is killed normally (via service stop), it fires a shutdown hook in java and you can see the process going away.

What I see in the log now is nothing the process just dies. When I checked the system.log I found at the same time this:

messages:Mar  2 19:43:22 beta_web gconfd (sifagent-16786): starting (version 2.14.0), pid 16786 user 'sifagent'
messages:Mar  2 19:43:22 beta_web gconfd (sifagent-16786): Resolved address "xml:readonly:/etc/gconf/gconf.xml.mandatory" to a read-only configuration source at position 0
messages:Mar  2 19:43:22 beta_web gconfd (sifagent-16786): Resolved address "xml:readwrite:/home/sifagent//.gconf" to a writable configuration source at position 1
messages:Mar  2 19:43:22 beta_web gconfd (sifagent-16786): Resolved address "xml:readonly:/etc/gconf/gconf.xml.defaults" to a read-only configuration source at position 2

The user is just for this service, so they didn't have a home. Previously the same gconfd message appeared but with an error about not having a writable home. I fixed that and part but the apparent connection between the gconfd messages in system.log and the java process having died persist.

Any ideas?


kbp 03-03-2010 05:01 PM

Are there any application specific logs ? You should also be able to set up debug level logging in the JAVA_OPTS

dswanson99 03-04-2010 11:33 AM

Yes there are, but nothing in there at stop time. When the process is shutdown in an orderly fashion (via kill) its shutdown hook fires and it does log - so its not getting stopped via service or a direct garden variety kill as nothing gets logged.

Its happened two more times - both with the gconfd activity for the daemon user at the same instant.

dswanson99 03-04-2010 01:20 PM

I tried changing the daemon user's shell to noshell to see if that stopped the gconfd activity, but no luck. The daemon process just died a few hours after making that change.

dswanson99 03-09-2010 09:02 AM

Any ideas?

kbp 03-09-2010 10:09 PM

I'm not convinced that the gconf stuff is related... can you enable debug level logging ?

dswanson99 03-10-2010 02:22 PM

I'm not convinced either, but its the only lead at this point. The only thing that seems clear is that the gconfd activity happens at the same time (every time) and only when it (java process dies) so its either a symptom or disease.

The logging level isn't external to the install so I have to have the developer rebuild and see what that yields. I'll post those results when I get them.

kbp 03-10-2010 02:52 PM

I'm not sure what platform you're running this on, but is selinux present or active ?

dswanson99 03-10-2010 02:58 PM

RHEL 5.3 and selinux is in permissive.

kbp 03-10-2010 05:28 PM

Which jdk are you using ?

dswanson99 03-11-2010 08:21 AM

OpenJDK 64-Bit Server VM (build 1.6.0-b09, mixed mode)

We've got a version of the app now that logs everything from the 3rd party libraries that we're using, hopefully that will point out something.

dswanson99 03-30-2010 07:50 AM

Just wanted to say thanks for your help. We still don't know anything but the inquiry has moved from the server to the java code, so I'm off the hook for now!

All times are GMT -5. The time now is 06:01 PM.