In-kernel process exit hooks?

[Paul Goyette <paul%whooppee.com@localhost>]

Please see references [1], [2], and [3] for details on this issue ...

Based on internal implementation of filemon(4), there is an ordering 
requirement imposed on the sequence of events that occur when a process 
using /dev/filemon exits.  In particular, the file descriptor on which 
the device is open needs to be closed prior to closing the descriptor to 
which avent/activity records are being logged.  (Otherwise, because of 
an extra reference on the log file, fd_close() will hang indefinitely.)

It has been suggested to implement a filemon(4)-specific "exit hook" to 
ensure that the /dev/filemon device gets closed first.  Instead of a 
hook that is specific to filemon(4), I would propose that a generic hook 
mechanism be added.  This way, if any future ordering requirements are 
found, we won't need another feature-specific mechanism.

We already have an exit_hook implementation, but unfortunately the hooks 
are calls too late in the exit processing.  Particularly, the hooks are 
called after all files are closed (via fd_free()) and after the working 
directory of the process has been free()d (via cwdfree()).

As far as I can tell, there are only two current users of the exithook 
mechanism (in kern/sys_aio.c and kern/sysv_sem.c).  It is not clear to 
me if either of these users would be affected if the execution of the 
exithooks were to happen _before_ the call to fd_free().

I'd rather not have two sets of exithooks if it can be avoided.  We 
could perhaps add a "phase" argument to the current exithook routines, 
using it to select between phase-specific LIST_HEADs.  Similar to

(in kern/kern_hook.c starting at line 222)

static hook_list_t exithook_pre_close_list =
	LIST_HEAD_INITIALIZER(exithook_pre_close_list);
static hook_list_t exithook_post_close_list =
	LIST_HEAD_INITIALIZER(exithook_post_close_list);
static hook_list_t *exithook_table[] = {
	&exithook_pre_close_list,
	&exithook_post_close_list
};

extern krwlock_t exec_lock;

void *
exithook_establish(void (*fn)(struct proc *, void *), void *arg, int ph)
{
	void *rv;

	KASSERT(ph >= 0 && ph < __arraycount(exithook_table));
	rw_enter(&exec_lock, RW_WRITER);
	rv = hook_establish(exithook_table[ph], (void (*)(void *))fn, arg);
	rw_exit(&exec_lock);
	return rv;
}

and with similar changes for exithook_disestablish() and doexithooks().

Then, in kern/kern_exit.c we simply have (at line 278+)

	...
	/*
	 * Close open files, release open-file table and free signal
	 * actions.  This may block!
	 */
	doexithooks(p, EXITHOOK_PRE_CLOSE);
	fd_free();
	cwdfree(p->p_cwdi);
	p->p_cwdi = NULL;
	doexithooks(p, EXITHOOK_POST_CLOSE);
	sigactsfree(p->p_sigacts);
	...


Comments?  Alternative approaches?



[1] http://mail-index.netbsd.org/source-changes-d/2016/01/06/msg008307.html
[2] http://mail-index.netbsd.org/tech-kern/2016/01/05/msg019896.html
[3] http://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=50627

+------------------+--------------------------+------------------------+
| Paul Goyette     | PGP Key fingerprint:     | E-mail addresses:      |
| (Retired)        | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com   |
| Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org |
+------------------+--------------------------+------------------------+