Re: Cert validation in pkg_add

[Taylor R Campbell <riastradh%NetBSD.org@localhost>]

> Date: Fri, 22 Dec 2023 17:20:13 +0000
> From: Taylor R Campbell <riastradh%NetBSD.org@localhost>
> 
> It is a mistake for pkg_add not to authenticate every byte it receives
> over the network by default -- it would be a surprising failure of
> security expectations for pkg_add to fail to do that.
> [...]
> The attached patch is much smaller, but
> 
> (a) it fails to guarantee that it has authenticated every byte over
>     the network even if you ask for `pkg_add https://...' without any
>     security overrides; and
> 
> (b) it is more likely to incur wider fallout because it changes the
>     semantics of the library, and the conditions that make it behave
>     differently depend not just on the local configuration but also on
>     the behaviour of remote servers.
> [...]
> If that's really what you want, we can commit it instead.

Just to be clear, I'm not raising these as objections -- I don't want
them to get in the way of having `pkg_add https://...' validate certs
in the most important case, when it _doesn't_ redirect to http/ftp.

We just have to make _some_ tentative decision about the edge cases
(https redirects to non-https or vice versa) in order to commit
anything, so I'm laying out my reasoning around the edge cases on
security and compatibility grounds.

But I'm not objecting to the smaller patch, as long as a fix for the
important case is handled promptly.

(The example I gave of https://cdn.netbsd.org redirecting to http
isn't relevant for the particular URLs that pkg_add uses.  Also,
currently no released version of NetBSD can use https://cdn.netbsd.org
with pkg_add anyway because of the SNI issue!)