Re: getrandom and getentropy

[Taylor R Campbell <riastradh%NetBSD.org@localhost>]

> Date: Sun, 3 May 2020 21:43:15 +0000
> From: nia <nia%NetBSD.org@localhost>
> 
> On Sun, May 03, 2020 at 09:28:37PM +0000, Taylor R Campbell wrote:
> > But on closer inspection, it's not clear there's a consensus on what
> > the semantics is supposed to be -- apparently _what_ everyone seems to
> > implement is slightly different:
> > 
> > - OpenBSD originally defined getentropy to be block-never, as in
> >   getrandom(GRND_INSECURE)
> > 
> > - glibc, FreeBSD, and illumos implement block-at-boot, as in
> >   getrandom(0)
> 
> The thing is, I'm not convinced there's a meaningful difference between
> the first two. As you said, /systems/ should be engineered to block
> until enough entropy is available. block-on-boot beaviour shouldn't be
> hit because the system should wait until entropy is acquired before any
> application software with serious entropy needs gets initialized.

OK, I did a little more research into exactly what goes on behind the
scenes in a few different systems.  Here are four different models:

- NetBSD-current
  . RESEED SCHEDULE:
    o reseed when 256 bits of entropy have been counted
    o reseed on rndctl -L (done early in userland, /etc/rc.d/random_seed)
    o reseed on write to /dev/random or sysctl -w kern.entropy.consolidate=1
  . ENTROPY COUNTING: driver estimate for HWRNG samples only
  . BLOCKING CRITERION: when 256 bits of entropy have been counted
  . WHAT'S READY IN USERLAND: all samples gathered before userland
    (including HWRNG if available), and seed on disk, after
    /etc/rc.d/random_seed has run (very early on in rc, just after the
    local file systems needed to find the seed are mounted)
  . SOURCE REFERENCE:
    https://nxr.NetBSD.org/xref/src/sys/kern/kern_entropy.c
    https://nxr.NetBSD.org/xref/src/sys/kern/subr_cprng.c
    https://nxr.NetBSD.org/xref/src/sys/dev/random.c

- OpenBSD
  . RESEED SCHEDULE:
    o reseed with whatever we have before reaching userland
    o reseed after every 10min
    o reseed after every 160000 bytes of output
    o reseed after write to /dev/random
  . ENTROPY COUNTING: none
  . BLOCKING CRITERION: none
    o Relies on gathering full entropy immediately, under the premise
      that either there's a HWRNG or there's enough entropy in timing
      &c. measurements early at boot.
  . WHAT'S READY IN USERLAND: all samples gathered before userland
    (including HWRNG if available), and seed on disk, after an early
    point in rc
  . SOURCE REFERENCE:
    https://cvsweb.openbsd.org/src/sys/dev/rnd.c?rev=1.204&content-type=text/x-cvsweb-markup
    https://cvsweb.openbsd.org/src/etc/rc?rev=1.543&content-type=text/x-cvsweb-markup

- FreeBSD (assuming default configuration with Fortuna)
  . RESEED SCHEDULE:
    o according to Fortuna model in Ferguson/Schneier/Kohno's book
    o initial seeding depends on having enough physical bytes of data
      from any source (no entropy estimate involved)
  . ENTROPY COUNTING: none
  . BLOCKING CRITERION: block until initially seeded
    o Relies on gathering full entropy within the first 64 bytes of
      data gathered (or kern.random.fortuna.minpoolsize).
    o May not have been initially seeded by the time userland starts.
  . WHAT'S READY IN USERLAND: not necessarily anything, as far as I can tell
  . SOURCE REFERENCE:
    https://svnweb.freebsd.org/base/head/sys/dev/random/randomdev.c?revision=356096&view=markup
    https://svnweb.freebsd.org/base/head/sys/dev/random/fortuna.c?revision=358379&view=markup

- Linux >=5.6 (more or less the NetBSD<=9.0 model too)
  . RESEED SCHEDULE:
    o reseed when 128 bits of entropy have been counted
    o reseed every 5min after initially seeded
    o reseed on ioctl(RNDRESEEDCRNG)
  . ENTROPY COUNTING:
    o based on value of samples with `delta' estimator for timing samples
    o driver estimate for HWRNG samples
  . BLOCKING CRITERION: block until first reseed
    o Relies on accurate `delta' estimator or HWRNG.
  . WHAT'S READY IN USERLAND: not necessarily anything, as far as I can tell
  . SOURCE REFERENCE;
    https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/char/random.c?h=v5.6

In all of these models, userland getentropy(...) and getrandom(..., 0)
(where they are implemented) will generally not return anything until
it has been initialized:

- In OpenBSD and NetBSD, this is because they incorporate all of the
  samples at boot and a seed from disk early on.

- In FreeBSD and Linux, this is because they block until first seeded,
  although the blocking criterion is rather different in FreeBSD and
  Linux (Fortuna's data- and driver-independent schedule, vs Linux's
  delta estimator).

So, although OpenBSD's getentropy never blocks while FreeBSD's and
Linux's may block, I think it is possible to interpret the semantics
reasonably consistently across OpenBSD, FreeBSD, and Linux as follows
(and I think this is what joerg was getting at somewhat obliquely
earlier):

- On FreeBSD, getentropy blocks until the system has been seeded with
  64 physical bytes of input, irrespective of where they came from or
  what, just to make sure the pool is seeded with something.  In other
  words, it doesn't guarantee anything about the entropy of the
  output -- only that _some_ input has been fed in, on Fortuna's
  schedule.

- On Linux (glibc), getentropy (i.e., getrandom(..., 0)) blocks until
  the system has been seeded with enough to fool the delta estimator,
  irrespective of the actual entropy of the underlying process.  In
  other words, it doesn't guarantee anything about the entropy of the
  output -- only that _some_ input has been fed in, enough to fool the
  delta estimator.  (In contrast, on Linux, reading from /dev/urandom
  might read from an essentially uninitialized pool, although it will
  print a warning in that case.)

- (I started to look into illumos too but I got lost in the KCF
  abstractions.)

(Obviously, it is possible to run a kernel with an /etc/rc script that
skips loading the seed from disk altogether; I'm not considering weird
customized installations like that.  System engineers who futz with
this are responsible for getting the details right.)

In NetBSD-current, there's a distinction between:
- incorporating what samples we have, and
- confidently achieving full entropy.
This distinction is made between sources that maybe might have
entropy, like interrupt timings, but we can't honestly put a lower
bound on the amount, and hardware random number generators, which are
engineered with specific physical designs to have a certain advertised
minimum amount of entropy.

Right now, reading from /dev/urandom on NetBSD generally gives at
least as good a result as you get on any of the other systems -- the
pool will have been seeded with whatever we can get, including cycle
counts sampled at various times during kernel startup and a seed on
disk or HWRNG samples if available; it's never reading from an
_uninitialized_ pool.

However, reading from /dev/random might wait _longer_ on NetBSD, for
what it can more confidently call full entropy based either on
automatic counting from HWRNG drivers, or on intervention by the
operator.


Given that, I think it is reasonable to implement getentropy(...)  as
an alias for getrandom(..., GRND_INSECURE) == read from /dev/urandom
== sysctl kern.arandom (as nia@ just committed the other day), which
is consistent with the somewhat nuanced interpretation of the
semantics above, and to provide getrandom(...,0) as I originally
suggested alongside it.