Re: Initial entropy with no HWRNG

[Taylor R Campbell <campbell+netbsd-tech-crypto%mumble.net@localhost>]

> Date: Tue, 12 May 2020 20:46:47 -0400 (EDT)
> From: darkstar%city-net.com@localhost
> 
> At the very least the system should pop up a message early in boot
> if sufficient randomness is not available and refuse to continue
> without manual intervention.

I understand the security motivation for this perspective, but this is
an excellent recipe to make people conclude `welp, I guess NetBSD is
just flaky, since Linux works fine on the same machine' and give up in
frustration.

There are some systems that are just between a rock and a hard place,
with no HWRNG, no disk, &c.  You could say `NetBSD just shouldn't
support such systems on security grounds'.  It's not clear to me that
this is the right answer either; such systems still have uses even if
it's not safe to have them doing much of anything on the public
internet today.

The middle ground we have right now is that you get a warning on the
console -- with some careful design and rate limits in an attempt to
mitigate warning fatigue -- and reads from /dev/random block, for
applications that are engineered to gracefully handle blocking.

(This is not to say that the warnings are perfect -- I am planning to
tweak them to make them clearer if I can but I haven't gotten to that
yet.)

> A remote or less secure local network random source could be used as
> an additional source to convert "hopefully good enough" into "good
> as long as either the starting randomness was actually ok or that
> one connection was not observed (and the remote system not
> compromised)".

This is why, as a compromise, /etc/rc.d/random_seed will sometimes
accept the entropy estimate in a stored seed on a networked file
system -- specifically, if that networked file system is also /, the
root file system.

> A few other issues:  The current data structure is entropy estimation,
> data, SHA1 hash of data.  It could be helpful to have a way for non-netbsd
> systems to generate a file that could be used automatically with widely
> available utilities (say a text SHA1 hash in a different file instead of
> binary hash after the data and assume full entropy unless read only). 

{ printf '\0\0\0\0' && head -c 512 < /dev/urandom; } > seed.tmp
{ cat seed.tmp && openssl dgst -binary -sha1 < seed.tmp; } > seed

A two-liner is not ideal but it'll work.

I might have designed the file format slightly differently if I were
doing it from scratch.  But I don't want to change it unless there's a
compelling reason to justify changing it and putting in all the effort
to implement and test compatibility logic.  The idea is that _if_ you
have a private persistent disk, then the seed file is all you need --
and as such it is precious; a mistake here could persistently ruin an
installation's security.

> Strictly speaking it would be a bit better to load the random data,
> overwrite the file with new data, and only when that succeeds assign an
> entropy value to the loaded data.

This is a generally reasonable approach.  I chose not to do this way
for specific reasons, though -- particularly, to ensure that on
systems with an acceptable seed, having an updatable seed is enough to
avoid triggering warnings that might lead to warning fatigue.  That
way, I hope, the warnings will be meaningful.

How does this work?

1. In order to mitigate iterative-guessing attacks -- where we
   iteratively add (say) 64 bits of entropy to the pool at a time
   before revealing a hash of the pool, so the adversary only has to
   do a few 2^64-cost brute force searches in sequence -- the kernel
   gathers pending samples into per-CPU pools until they collectively
   have full entropy (~256 bits), and then it atomically consolidates
   them into the pool that actually serves /dev/u?random.

   Otherwise, the system _avoids_ frequently consolidating entropy.
   (rndctl -S does it, and we run that once a day via /etc/security,
   but that's the most frequently that it happens automatically.)

2. The operator can choose to consolidate entropy manually, by issuing
   `sysctl -w kern.entropy.consolidate=1', if they have a reason to
   believe what's in the pending pools is enough on its own to thwart
   an attack.  If the kernel doesn't agree, it will print a warning to
   the console.

3. There are two other points when the system chooses to consolidate
   entropy, neither of which is a regular occurrence:

   - on write to /dev/u?random, since the man page has advertised for
     some years now that `writing to either /dev/random or
     /dev/urandom influences subsequent output of both devices,
     guaranteed to take effect at next open'; and

   - on loading the seed with rndctl -L, which typically happens once
     at boot.

Consolidating entropy when the seed is loaded with rndctl -L means
that by the time the bulk of userland has started, all of the samples
that have been fed into the kernel during boot, and the seed, are
guaranteed to influence the subsequent output of /dev/u?random -- and
sysctl kern.arandom and arc4random.  And not only does that transition
happen atomically, but also _if_ it collectively added up to full
entropy by the system's count (e.g., if the seed had full entropy),
then a warning will not be printed.

What if we did it the way you suggested -- as rndctl did for a short
while (never released, just in current)?  For concreteness, let's
suppose the sequence were:

1. oldseed := read from /var/db/entropy-file
2. newseed := hash(oldseed, read from /dev/urandom)
3. write newseed to /var/db/entropy-file.tmp
4. rename /var/db/entropy-file.tmp to /var/db/entropy-file
   zero the entropy if any of the above failed
5. ioctl(RNDADDDATA, oldseed)   // consolidates & updates system entropy count

There are two problems with this sequence:

(a) The act of reading from /dev/urandom in step (2) would trigger a
    WARNING: consolidating entropy too early.

(b) newseed would fail to incorporate all of the samples already in
    the pending pools, so we would fail to write them out to disk

I briefly considered addressing problem (b) by issuing sysctl
kern.entropy.consolidate=1 before step (2), but that would leave
problem (a).

As a compromise, I chose the sequence that is in tree right now:

1. create /var/db/entropy-file.tmp and remember if this failed
2. oldseed := read from /var/db/entropy-file
   zero the entropy estimate in oldseed if anything looks fishy:
   - checksum failed
   - creating /var/db/entropy-file.tmp failed
   - entropy estimate looks unreasonable in either byte order
3. ioctl(RNDADDDATA, oldseed)    // consolidates & updates system entropy count
4. newseed := hash(oldseed, read from /dev/urandom)
5. write newseed to /var/db/entropy-file.tmp
6. rename /var/db/entropy-file.tmp to /var/db/entropy-file

This way, we detect a read-only file system early, _and_ we only
consolidate entropy and read from /dev/urandom after the seed has been
entered in order to avoid printing a warning in the event that the
system is OK.

Of course, it is possible that creating the file works while writing
the file fails, e.g. if the /var partition is almost completely full
(in which case the system is kind of hosed anyway), but I think this
strikes a reasonable balance between effectively detecting the
read-only case and avoiding warning fatigue in the read/write case.

> Currently it is assumed that being able to open the file with write
> permission means it will actually be possible to write it.  I'm not
> sure if there are potentially relevent cases where that is incorrect
> (the more likely case where this assumption causes trouble is a disk
> image duplicated behind the scenes that can be impossible to detect
> from the running OS).

If there are noteworthy cases where

(a) we can successfully issue open(O_WRONLY|O_CREAT|O_TRUNC) to create
    a new file on disk, but

(b) we can't actually write or update the seed, and

(c) the system is not otherwise basically hosed,

I would be curious to hear about them and weigh them against what I
think are the majority of cases, where either open() fails with EROFS
or open() and write() and rename() all succeed.

> Currently there is a race condition between reading the data and
> writing the new file, although it seems difficult to both trigger
> that and use the randomness in a way that could cause trouble if the
> same value is used twice.

Can you be more specific, or are you just referring to the same
scenarios as above?

> Considering the importance of the file and inconvenience on some
> systems if something happens to it there should be a backup that is
> not written at the same time.  To help with read only root it might
> make sense to support a tiny randomness partition (or at least
> mounting some partition before rndctl).

The file is stored on /var by default, and read in after the critical
local file systems -- typically at least /var -- have been mounted.
It is not necessary to have read/write / in order to keep a seed.