I have written the code for two extra options to systrace that I
think will help when systrace comes across less than usual situations.

They are:

Randomly cause system calls to fail.
 * This can be used to explore code paths that may only be taken
   rarely.  When used with automatic policy generation, this helps
   the policy to converge on a programs actual behaviour.

   $ systrace -A cat /etc/myname
   $ cp .systrace/bin_cat .systrace/bin_cat.orig
   $ for x in $(jot 10); do systrace -A -r 10 cat /etc/myname; done
   ...
   $ diff -u .systrace/bin_cat.orig .systrace/bin_cat
+	netbsd-fsread: filename eq "/usr/share/nls/C/libc.cat" then permit
+	netbsd-fsread: filename eq "/usr/share/nls/nls.alias" then permit
+	netbsd-fsread: filename eq "/<non-existent filename>: /usr/share/nls/libc/C" then permit
+	netbsd-__sigprocmask14: permit
+	netbsd-getpid: permit
+	netbsd-kill: pidname eq "/bin/cat" and signame eq "SIGABRT" then permit
+	netbsd-kill: pidname eq "<unknown>" and signame eq "SIGABRT" then permit
+	netbsd-__sigaction_sigtramp: permit

   The last kill rule appears to be the result of getpid failing.
   getpid() will never fail, so it seems to me that maybe systrace
   should be told about system calls that cannot fail? The same
   failure could occur in automatic mode, with some code like this:
	kill(getpid(), SIGKILL);
   so if getpid returns -1, and the process is running as root...
   other nasty things with get*id()

 * Also useful for blackbox testing of the error handlers of
   programs.  Using this, I noticed that in malloc(3) the return
   value of sysconf(_SC_PAGESIZE) is not checked to be -1, and is
   also assumed to be in the form 2^n. If the return value is not
   in the form 2^n and/or is -1, malloc_init will loop indefinitely.

 * Selection of returned error is also random. Currently there is a fixed
   list of fairly general errors (EPERM, EINTR, EINVAL, EAGAIN, ENOENT)

Terminating a process when a system call not in its policy is
attempted (only for unsupervised processes.) May help with policy
probing attacks, and the problem noted above with kill.

Is either these two features worth having? Comments?

My current diff is at:
ftp://ftp.NetBSD.org/pub/NetBSD/misc/cb/systr-erratic.diff