dtrace and ZFS update

[Chuck Silvers <chuq%chuq.com@localhost>]

hi folks,

I've been working on updating netbsd's copy of the dtrace and zfs code to rebase
from the existing ancient opensolaris version to a recent freebsd version.
most of the freebsd changes are pretty close to what netbsd needs,
so that seems like a more useful upstream for us.  I have things working
well enough now that I want to share the code in preparation for committing.

this update improves upon our existing dtrace/zfs code in several ways:
 - picks up all the upstream zfs fixes and enhancements from the last decade
 - zfs now supports mmap on netbsd, so you can run executables stored in zfs
 - dtrace fbt probes can now be used in kernel modules (such as zfs)

here's the diff, which is based on netbsd HEAD as of
"Tue Oct 10 20:14:48 UTC 2017":
http://ftp.netbsd.org/pub/NetBSD/misc/chs/diff.cddl.20171012

note that the patch needs to be applied with "patch -E" since it
adds and removes various files.

here's a summary of what's in the diff:

 - freebsd's dtrace/zfs code as of r315983 (2017-03-26), adapted for netbsd.
   many netbsd changes in our current version to fix warnings have been removed
   or redone to be reduce the difference from the new upstream.
 - a few updates to our copy of freebsd's libproc.
   I plan to do a wholesale resync of libproc to current freebsd
   rather than just picking up the few bits in this diff.
 - build system support for using -fno-omit-frame-pointer everywhere
   and disabling other compiler optimizations that confuse dtrace.
   I'm ok with leaving these changes disabled by default,
   or perhaps making them conditional on MKDTRACE.
 - sample kernel config changes for a couple evbarm configs (the ones I tested).
 - module/ksyms enhancements for dtrace integration.
 - genfs API enhancements to support zfs.
 - an option to have mutexes not become no-ops during a panic.
 - uvm_aobj API change to support 64-bit aobj sizes (eg. for tmpfs).

the plan is that items listed separately above will be committed separately
but in quick succession, to allow one kernel version increment for all of
the ones which require that.


here's a list of known issues with the patch:

 - unloading the zfs module fails even with no zpools imported
   if you've done much with zfs since it was loaded.
   there's some refcounting problem that I haven't tracked down yet.

 - the module refcounting for active fbt probes is bogus.
   currently module refcounting is protected by kernconfig_lock(),
   but taking that lock down in the bowels of dtrace seems likely
   to create deadlocks.  I plan to do something fancier but haven't
   gotten to it yet.

 - the dtrace uregs[] stuff is probably still wrong.

 - the CTF typeid overflow problem is still there (more on this below).

 - a bunch of features are still not supported:
   - the ".zfs" virtual directory, eg. ".zfs/snapshot/foo@bar"
   - zvols
   - ZFS ACLs (aka. NFSv4 ACLs)
   - NFS exporting a ZFS file system
   - setting dtrace probes in application code
   - using ZFS as the root fs
   - new crypto hashes SHA512_256, skein, and edonr
     (the last one is not in freebsd yet either)
   - zio delay injection (used for testing zfs)
   - dtrace support for platforms other than x86 and arm


I don't intend to fix any of these issues before initial commit of
these changes, since I don't think any of them are regressions from
what we have currently.

there a few other bits still in the diff that are just for debugging or
otherwise need to be cleaned up, but not very many.  I'll clean them up
before committing.


about the CTF typeid overflow issue I mentioned above:

CTF is the Compact Type Format data that dtrace uses to know about
function argument types and return values.  CTF encodes a type ID
as a 15-bit value, but unfortunately a netbsd GENERIC kernel has
more than 15 bits worth of types, so the values overflow the field.
we don't really want to change this format unilaterally since it is
currently the same on all dtrace-supporting operating systems, so we
need to work around the problem for now by removing various subsystems
from the kernel config used with dtrace, so that the number of types
is within what CTF supports.

to tell if your kernel will work with dtrace, run
	ctdump /path/to/kernel | grep "total number of types"

if the number of types is less than 32768 then you're fine,
otherwise you'll need to change your kernel config to remove
some drivers or file systems that you won't use.

this error is another symptom of the typeid overflow problem:
dtrace: failed to compile script sleep.d: "/usr/lib/dtrace/psinfo.d", line 46: failed to copy type of 'pr_arglen': Type information is in parent and unavailable

note that dtrace scripts that do not reference any argument or return types
will work even if the kernel has too many types for CTF (but you'll need to
remove the psinfo.d file mentioned above to remove that reference).

I hope to integrate a solution for this CTF overflow issue before
the first netbsd release with this new dtrace code in it (netbsd 9.0),
but it's not clear what that solution will end up being yet.


anyone interested in zfs or dtrace on netbsd please give this a try
and let me know about any problems that you find.
any comments on the diff are also welcome.

-Chuck