LinuxQuestions.org - [C++] How to log the segmentation faults/errors (which crash the program) by remote logging library?

It is possible to catch segmentation faults and bus errors in the process itself by installing SIGSEGV and SIGBUS signal handlers. They are extremely tricky to write correctly, but it is possible. They cannot be used to fix any problems, though; the process is already dying when the signals are caught.

While you can use write() and send() and their variants in a signal handler (and even open(), socket(), connect() and so on; see Async-signal-safe functions section in man 7 signal for complete listing), you need to be extremely careful. All dynamic memory structures may be shot at this point (except for addresses to static and global variables, and parameters to the signal handler). Especially C and C++ libraries' internal data structures get mangled very easily, so you just cannot rely on them at all! To obtain information about the problem, set the SA_SIGINFO flag when installing the handlers. Also, I'd recommend setting up an alternate stack, too, because the default stack may be completely shot also. Because you cannot rely on any library function to work, your code should use syscalls directly. You might also wish to use an atomically modified global variable to see if the fault handler has already been executed, and if so, explicitly exit: in some cases the signal handler may end up being called in an endless loop.

Indeed, I think you'd absolutely have to write the signal handler code in C, because otherwise the C++ runtime might foil your attempt anyway. (Even C-looking code tends to use C++ runtime under the hood; if the C++ runtime state is shot, then even basic C++ code is likely to fail to work.) Simply write these handlers in C, and use signaturevoid __attribute__ ((constructor)) my_initialization_function(void) for the initialization function, so that it is automatically called when the library is dynamically loaded (or if statically compiled, before main() is executed). But, like I said, this is very hard to do right, and all you might get extra is a pointer to the memory location causing the problem.

If adding such code into a process or library is not feasible, you can do something very similar by using a process supervisor (similar to e.g. daemontools or runit); a parent process to the actual supervised process. In that case the supervised process usually logs to standard output, and the supervisor transmits the log to the remote end. If the supervised process exits or dies, the parent==supervisor will be sent a SIGCHLD signal, and reaping the supervised process using wait() or waitpid() will be able to determine the reason by checking the result using the WIFSIGNALED() and WTERMSIG() macros. WCOREDUMP() will tell if there is a core dump, so I guess it would be possible for the supervisor to check the core dump (if available) for the details of the demise of the supervised process, but otherwise the supervisor process only knows why the process died, not which code or memory location caused the death.