Process won't stay running in the background after exit, even if nohuped or disowned.

nbritton · 10-31-2016, 06:08 PM

I'm stumped, I've tried at least 20 incantations and all of them work when I run them manually, but when I run them via an ansible script they don't do anything.

I'm trying to kick off racadm serveraction powercycle from an ansible script. Ideally I want it to run as a daemon (first sleep for 10 seconds, then run) in the background and return control back to the script so it can exit in order to properly close the ssh connection. It doesn't seem to matter if I use disown, nohup, setsid, or detach all I/O. It works perfect if I run the script manually, but as soon as I have ansible kick it off it never runs. It enters the if control block, the echos work but forking it never does anything.

Code:

if [[ ${POWERCYCLE} == "1" ]] || test -f /tmp/POWERCYCLE; then
    echo "The system must be power cycled for the new firmware changes to take effect.";
    echo "racadm serveraction powercycle has been triggered, restarting in 10 seconds.";
    #{ (sleep 10 && /opt/dell/srvadmin/sbin/racadm serveraction powercycle) &>/dev/null </dev/null & } 2>/dev/null;
    #{ nohup sh -c '(sleep 10 && /opt/dell/srvadmin/sbin/racadm serveraction powercycle) &>/dev/null </dev/null' & } 2>/dev/null;
    #(nohup /opt/dell/srvadmin/sbin/racadm serveraction powercycle </dev/null &>/dev/null &) 2>/dev/null;
    #sh -c '( echo "Hello" &) & disown; exit' 2>/dev/null
    sh -c '( /opt/dell/srvadmin/sbin/racadm serveraction powercycle </dev/null &>/dev/null &)' & disown 2>/dev/null;
fi

Turbocapitalist · 11-01-2016, 02:31 AM

Which distro are you running, including version? The ones infected with systemd have trouble keeping processes alive if you don't hop through a lot of special hoops unique to systemd.

Edit: If that is indeed the problem than the work-around is something like this:
systemd-run --user --scope yourscript

more discussion here and other places:
https://bugs.debian.org/cgi-bin/bugr...cgi?bug=825394

NevemTeve · 11-01-2016, 02:51 AM

@OP: create a script and add debugging into it:

Code:

#!/bin/sh

set -xv

exec >>/tmp/somescript.log 2>&1

date

trap '' HUP

...

pan64 · 11-01-2016, 02:58 AM

I would say the environment is not the same, so when you run it from ansible something was missing (most probably a library or environment variable).
I suggest you to check if there was an error message during execution. I would try to redirect stdout and stderr into files. Suppressing them is not a good idea (if you want to find the reason). Also it may have some log files.

jpollard · 11-15-2016, 01:47 PM

It is possible you are running into a systemd issue - it tends to kill any user process during logout as part of cleaning up the session. And "nohup" doesn't help as it is also part of the session.

I understand there is supposed to be a configuration option to change this back to something more standard...