Re: Initial entropy with no HWRNG

[darkstar%city-net.com@localhost]

On Wed, 13 May 2020, Taylor R Campbell wrote:
>> Date: Tue, 12 May 2020 20:46:47 -0400 (EDT)
>> From: darkstar%city-net.com@localhost
>>
>> At the very least the system should pop up a message early in boot
>> if sufficient randomness is not available and refuse to continue
>> without manual intervention.
>
> I understand the security motivation for this perspective, but this is
> an excellent recipe to make people conclude `welp, I guess NetBSD is
> just flaky, since Linux works fine on the same machine' and give up in
> frustration.

The message or documentation that it points to can explain this situation
and how to fix it correctly or easily do what other systems do.  Ideally,
NetBSD should be the go-to example of what all systems should do and it
seems close to that at this point.  Ideally, all systems should refuse to
boot to an insecure state without being very obvious that it is insecure.

It can be argued just how insecure such a system might be.  In practice,
almost the only chance of an actual exploited vulnerability is producing
the exact same output multiple times.  While this is hopefully unlikely in
practice it is hard to say just how unlikely it is and the same arguments
apply as for not estimating the entropy of those sources.  The same might
apply to some extent to an audio workaround as well, although I think the
large amount of work done on lossless audio compression (with limited
success) gives some confidence there if the microphone in question meets
minimal standards, and amplifier noise also seems like something that is
likely to have been heavily studied.  Thor's suggestion of password-like
entry of the initial seed (or to secure tranfer of a seed) would avoid the
need to rely on audio as a backup, although it might often be less
convenient.

To counter an argument on the other thread, I don't think the system being
unable to detect every possible case where there is an issue should mean
it ignores when it knows there is a chance of an issue.

> There are some systems that are just between a rock and a hard place,
> with no HWRNG, no disk, &c.  You could say `NetBSD just shouldn't
> support such systems on security grounds'.  It's not clear to me that
> this is the right answer either; such systems still have uses even if
> it's not safe to have them doing much of anything on the public
> internet today.

My main concern is to make certain that someone who is using such a
machine knows that and not just because they happened to notice a console
message scrolling by.  To the extent that randomness is the issue, it is
at least fairly easy to fix and make it fully secure.  When the issue is
lack of maintinance of a port, users should also be aware of that but that
can be done in a more obvious way for everyone using a port.  Some ports
also always have issues with randomness, but even amd64 has a few systems
that don't have a sufficient hardware randomness source.

> The middle ground we have right now is that you get a warning on the
> console -- with some careful design and rate limits in an attempt to
> mitigate warning fatigue -- and reads from /dev/random block, for
> applications that are engineered to gracefully handle blocking.

If it is made a mandatory step in the install procedure then it doesn't
contribute to warning fatigue at all, just to installer complexity.  As
you say elsewhere that is an issue as well, however IMO setting up an
insecure system without realizing it is about the worst possible outcome
so any complexity needed to prevent that is just necessary.  Additionally,
there is ongoing complexity in explaining the difference between
/dev/random and /dev/urandom and just dealing with it in the installer
(and verifying during startup) means that there is no need for this
difference.  Since currently /dev/urandom or equivalent is frequently
used, fixing the issue after seeing the warning when the system is booted
doesn't guarantee that it hasn't already caused trouble that will persist
after the issue is fixed.

>> A few other issues:  The current data structure is entropy estimation,
>> data, SHA1 hash of data.  It could be helpful to have a way for
>> non-netbsd
>> systems to generate a file that could be used automatically with widely
>> available utilities (say a text SHA1 hash in a different file instead of
>> binary hash after the data and assume full entropy unless read only).
>
> { printf '\0\0\0\0' && head -c 512 < /dev/urandom; } > seed.tmp
> { cat seed.tmp && openssl dgst -binary -sha1 < seed.tmp; } > seed
>
> A two-liner is not ideal but it'll work.

That file does not claim to have any entropy, though, does it?  It looks
like if the entropy estimate is too large it tries to byteswap so the
entropy could be entered in any endianness and I agree that kind of thing
would work.

I was also thinking about Windows systems that don't (unless they started
recently) ship with openssl but can generate a text hash easily (at least
windows 7 and later, not sure about earlier, via certutil -hashfile or
powershell's get-filehash).  Although someone who knows PowerShell could
likely get around that and I'm not sure there is an easy equivalent of
reading from /dev/random without non-trivial PowerShell usage so maybe it
doesn't matter and if someone can produce the right PowerShell script to
create the seed file that could just be distributed.  I would guess there
is some type of key generator that could be used with a simple command
line and would likely be easier (absent a script) to copy to /dev/random
than get into the right format, but again maybe the easiest is a
PowerShell script that just works.

> I might have designed the file format slightly differently if I were
> doing it from scratch.  But I don't want to change it unless there's a
> compelling reason to justify changing it and putting in all the effort
> to implement and test compatibility logic.  The idea is that _if_ you
> have a private persistent disk, then the seed file is all you need --
> and as such it is precious; a mistake here could persistently ruin an
> installation's security.

That makes sense.  In the model I am suggesting it does not make sense to
have files with partial entropy or to have files larger than 64 bytes
(256-bit data and hash, or 52 bytes with SHA1) and the current file being
over 512 bytes seems slightly unfortunate for theoretical block splitting
reasons but I guess not really worth the trouble of changing.

>> Strictly speaking it would be a bit better to load the random data,
>> overwrite the file with new data, and only when that succeeds assign an
>> entropy value to the loaded data.
>
> This is a generally reasonable approach.  I chose not to do this way
> for specific reasons, though -- particularly, to ensure that on
> systems with an acceptable seed, having an updatable seed is enough to
> avoid triggering warnings that might lead to warning fatigue.  That
> way, I hope, the warnings will be meaningful.

Thanks for explaining and I agree it must not generate a warning when
loading a seed.  One other way this could be done is to edit the file in
place and zero the entropy estimation before entering the seed.  Is it
necessary to do the new file and rename dance in this case?  Not doing
that would eliminate the file system full issue.  An fsync of that before
entering the data then fsync of the newly generated data seems like it
would be most reliable at a performance cost.

> If there are noteworthy cases where
>
> (a) we can successfully issue open(O_WRONLY|O_CREAT|O_TRUNC) to create
>     a new file on disk, but
>
> (b) we can't actually write or update the seed, and
>
> (c) the system is not otherwise basically hosed,
>
> I would be curious to hear about them and weigh them against what I
> think are the majority of cases, where either open() fails with EROFS
> or open() and write() and rename() all succeed.

I can't think of anything besides the filesystem full case you mentioned,
mostly just trying to figure out the assumptions being made.  Ideally, at
the point this goes wrong (even if the system is otherwise hosed) the boot
would fail rather than attempting to continue (although if the seed has
not been used before then it might be better to continue to allow it to be
fixed remotely).

>> Currently there is a race condition between reading the data and
>> writing the new file, although it seems difficult to both trigger
>> that and use the randomness in a way that could cause trouble if the
>> same value is used twice.
>
> Can you be more specific, or are you just referring to the same
> scenarios as above?

I just mean the same as above but where the system crashes before the seed
is updated even if it might have been able to write had it not crashed. 
It seems possible if something was blocking on /dev/random while the seed
is being updated that it could potentially use the random data before the
crash, although hopefully the startup scripts ensure that this doesn't
happen and if it does it could be corrected there (plus there isn't even a
theoretical issue as long as the internal state isn't exactly the same
when the seed is loaded the second time, although as above it seems not
great to rely on that being the case).

>> Considering the importance of the file and inconvenience on some
>> systems if something happens to it there should be a backup that is
>> not written at the same time.  To help with read only root it might
>> make sense to support a tiny randomness partition (or at least
>> mounting some partition before rndctl).
>
> The file is stored on /var by default, and read in after the critical
> local file systems -- typically at least /var -- have been mounted.
> It is not necessary to have read/write / in order to keep a seed.

Oops, I looked at rd.d/rndctl's dependencies not random_seed.

With the current method it seems slightly complicated to figure out when
to write a backup file since there might not be full entropy at
installation, however with the method I am suggesting it would be easy to
create a backup seed file at install time that is never used automatically
but available in case something happens to the main seed file.

Thanks for all the work you have put into this and however this particular
issue is decided it is a huge improvement.

-M