[RFC] libxl: fork/wait handling API proposal

[RFC] libxl: fork/wait handling API proposal

[Ian Jackson]

Here is a first cut of a new API proposal.  There is no implementation
as yet.

libxl will also need to use pthread_atfork to ensure proper tidying up
of things on forks, but this is hidden inside the library and does not
need to be in the API.  I have some WIP code for this but it's not
ready for the light of day.

Ian.

diff --git a/tools/libxl/libxl_event.h b/tools/libxl/libxl_event.h
index f889115..17ffd81 100644
--- a/tools/libxl/libxl_event.h
+++ b/tools/libxl/libxl_event.h
@@ -369,6 +369,125 @@ void libxl_osevent_occurred_fd(libxl_ctx *ctx, void *for_libxl,
  */
 void libxl_osevent_occurred_timeout(libxl_ctx *ctx, void *for_libxl);
 
+
+/*======================================================================*/
+
+/*
+ * Subprocess handling.
+ *
+ * Unfortunately the POSIX interface makes this very awkward.
+ *
+ * There are two possible arrangements for collecting statuses from
+ * wait/waitpid.
+ *
+ * For naive programs:
+ *
+ *     libxl will keep a SIGCHLD handler installed whenever it has an
+ *     active (unreaped) child.  It will reap all children with
+ *     wait(); any children it does not recognise will be passed to
+ *     the application via an optional callback (and will result in
+ *     warnings to stderr if no callback is provided).
+ *
+ *     libxl may have children whenever:
+ *
+ *       - libxl is performing an operation which can be made
+ *         asynchronous; ie one taking a libxl_asyncop_how, even
+ *         if NULL is passed indicating that the operation is
+ *         synchronous; or
+ *
+ *       - events of any kind are being generated, as requested
+ *         by libxl_evenable_....
+ *
+ * For programs which run their own children alongside libxl's:
+ *
+ *     The application must install a SIGCHLD handler and reap (at
+ *     least) all of libxl's children and pass their exit status
+ *     to libxl by calling libxl_childproc_exited.
+ *
+ *     An application which does this must call
+ *     libxl_childproc_setmode.
+ * 
+ * An application which fails to call setmode, or which passes 0 for
+ * hooks, must not:
+ *   - have any child processes running while it uses any libxl
+ *     operation which might create or use child processes (see
+ *     above);
+ *   - install a SIGCHLD handler; or
+ *   - reap any children.
+ */
+
+
+typedef enum {
+    /* libxl owns SIGCHLD whenever it has a child */
+    libxl_sigchld_owner_libxl,
+
+    /* Application promises to call libxl_childproc_exited but
+     * NOT from within a signal handler. */
+    libxl_sigchld_owner_mainloop,
+} libxl_sigchld_owner;
+
+typedef struct {
+    libxl_sigchld_owner chldowner;
+
+    /* All of these are optional: */
+
+    /* Called by libxl instead of fork.  Should behave exactly like
+     * fork, including setting errno etc.  May NOT reenter into libxl.
+     * Application may use this to discover pids of libxl's children,
+     * for example.
+     */
+    pid_t (*fork_replacement)(void *user);
+
+    /* With libxl_sigchld_owner_libxl, called by libxl when it has
+     * reaped a pid.  (Not permitted with _owner_app.)
+     *
+     * Should return 0 if the child was recognised by the application
+     * (or if the application does not keep those kind of records),
+     * ERROR_NOT_READY if the application knows that the child is not
+     * the application's; if it returns another error code it is a
+     * disaster as described for libxl_event_register_callbacks.
+     * (libxl will report unexpected children to its error log.)
+     *
+     * If not supplied, the application is assumed not to start
+     * any children of its own.
+     *
+     * This function is NOT called from within the signal handler.
+     * Rather it will be called from inside a libxl's event handling
+     * code and thus only when libxl is running, for example from
+     * within libxl_event_wait.  (libxl uses the self-pipe trick
+     * to implement this.)
+     *
+     * childproc_exited_callback may call back into libxl, but it
+     * is best to avoid making long-running libxl calls as that might
+     * stall the calling event loop while the nested operation
+     * completes.
+     */
+    int (*childproc_exited_callback)(pid_t, int status, void *user);
+} libxl_childproc_hooks;
+
+/* hooks may be 0 in which is equivalent to &{ libxl_sigchld_owner_libx, 0, 0 }
+ *
+ * May not be called when libxl might have any child processes, or the
+ * behaviour is undefined.  So it is best to call this at
+ * initialisation.
+ */
+void libxl_childproc_setmode(libxl_ctx *ctx, const libxl_childproc_hooks *hooks,
+                             void *user);
+
+/* May be called only by an application which has called setmode with
+ * chldowner == libxl_sigchld_owner_mainloop.  If pid was a process started
+ * by this instance of libxl, returns 0 after doing whatever
+ * processing is appropriate.  Otherwise silently returns
+ * ERROR_NOT_READY.  No other error returns are possible.
+ *
+ * May NOT be called from within a signal handler which might
+ * interrupt any libxl operation.  The application will almost
+ * certainly need to use the self-pipe trick (or a working pselect or
+ * ppoll) to implement this.
+ */
+int libxl_childproc_exited(libxl_ctx *ctx, pid_t, int status);
+
+
 #endif
 
 /*