Patch: rework kernel random number subsystem (*nearly final*)

[Thor Lancelot Simon <tls%panix.com@localhost>]

On Fri, Oct 21, 2011 at 05:15:55PM -0400, Thor Lancelot Simon wrote:
> 
> I have placed a patch at http://www.panix.com/~tls/rnd1.diff which
> implements many changes to the generation and use of randomness
> in the kernel (I previously sent it to these lists directly but
> it seems to be too large).
> 
> It is (most of) the first step in a three step process I envision for major
> overhaul of this subsystem:
> 
>       1) Provide infrastructure needed to separate entropy
>          harvesting from random stream generation.  Clean up
>          interfaces between existing kernel components that
>          deal with random number generation and consumption.

There's a new patch at http://www.panix.com/~tls/rnd4.diff which
I intend to commit as soon as I find and fix one remaining bug.  This
version of the patch adds some infrastructure the new random/urandom
pseudodevices will need and addresses several nasty race conditions
around rekeying by changing the rndpool mutex from IPL_SOFTCLOCK to
IPL_VM.

It also fixes several miscellaneous bugs (including a severe one in
rnd_add_data that has been around for a long time) and addresses KNF
issues in my previous patches.  There are also a few performance tweaks.

The mutex change to IPL_VM should not have much performance impact as
it's never held during copy in/out and my next set of changes will
eliminate any direct access to the entropy pool (thus any taking of
the mutex) by the pseudodevices -- that is, from userspace consumers.

The bug which remains is this:

        * With a LOCKDEBUG, RND_VERBOSE kernel, run the "arandom"
          test program, which reads as much data with
          sysctl kern.arandom as it can, in a tight loop.

        * If you don't have a hardware RNG, you will see many rekeyings
          of the arc4random generator.  The entropy pool will drain, and
          soon you will see "forced" rekeyings.

        * While you run the test program, the system will be fine.  If
          you kill the test program and wait a minute or two, the system
          will lock up hard.

Sources to the test programs are at http://www.panix.com/~tls/[au]random.c .

Heck if I know where it is; if anyone else sees it, I'd love to know.
Rekeying of the 'kern.urandom' generator (with a source change to force
that much more often) causes no such symptom so I assume it must be in
the libkern arc4random code somewhere.

Thor