Re: Cert validation in pkg_add

[Greg Troxel <gdt%lexort.com@localhost>]

(pmc hat off)

I don't think where you are ending up is reasonable.

First, I asked "does TNF intend continue to offer ftp and http access"
and the answer was unsurprisingly "there are no plans to discontinue it"
and that it can be helpful for people with slow machines.  (We are of
course intentionally welcoming to retrocomputers.)


What we are doing at a fundamental level is fixing a bug, which is that
https fetches are expected to validate server certificates, more or less
following RFC5280.  I looked at TLS

  https://datatracker.ietf.org/doc/rfc8446/

and did not quickly find a SHOULD/MUST about clients and successful pkix
validation -- only that it's out of scope and see RFC580.  However, I
think it's reasonable to accept that in 2023, any use of TLS by default
somewhere between STRONGLY SHOULD (to make up a new IETF term) and MUST
perform pkix validation, so I'm not going to worry about whether RFC8446
actually says that.

I think it's reaosnable to be timely in fixing a bug, period, but also
when it avoids problems because of other, not yet fixed problems.  But I
don't want to do things that aren't justifiable by themselves, as part
of working around other problems (lack of signed packages).

I also see it as core UNIX and NetBSD doctrine to let people do what
they want, even if it's something someone else thinks they shouldn't do.
Besides basic respect for users, I am mindful that people use software
in ways that we don't understand, and I don't think it's ok to say "I
don't understand your situation but if you are doing X you are wrong".
Your patch isn't doing this -- it has an easy disable, but I think this
needs to be kept in mind.

So that leads me to be ok with pkg_add doing https validation by default
and a setting to disable it.

Taylor R Campbell <riastradh%NetBSD.org@localhost> writes:


> On the one hand, not validating https on redirect from http -- and,
> conversely, _refusing_ https to http redirect -- may be surprising.

Indeed, both of those are completely surprising.

> On the other hand:
>
> - If user-specified http is redirected to server-specified https, the
>   adversary has already had the opportunity to intercept http by the
>   time pkg_add could begin to act on the redirect.
>
>   So pkg_add can't guarantee authentication if you use http URLs,
>   whether or not it validates certs on redirect to https.  But
>   validating certs in this case might break existing setups.

pkg_add does not have a guarantee of authentication when fetching
packages and we should not pretend that it does.  If someone configures
http or ftp, that's what they have done.  That's just like choosing to
use NFS or SMB.  There's no reason anyone who does that should expect
any cryptographic integrity/authentication, and we shouldn't invent a
requirement for it.

It is fair to say that if someone configures a https URL then fetches
*to that URL* should have pkix validation.

> - pkg_add automates the installation (and, via INSTALL scripts,
>   execution) of software from the remote host the moment you run it.
>
>   So if user-specified https is redirected to server-specified http,
>   there's no opportunity like an interactive web browser provides for
>   a user to check for `https' or a lock icon in the address bar after
>   a page is done loading, but before entering their social security
>   bank account password's maiden name.
>
>   Thus, if pkg_add were to quietly honor https to http redirect, it
>   would quietly render users more vulnerable than an interactive web
>   browser would.

I think this is a stretch.  When you specify an https URL, you are
saying

   This is the location I want to get bits from.

   Implicitly, PKIX validation should happen on fetches, so the bits are
   really from that server (broadly construed, server might have CDN).

I view any https server giving a redirect to anything non-https as a
bug.  I am not aware of any reports of places serving packages over
https that return redirects to http.  If there are we should report that
to them and ask them to fix.

> So I think it is reasonable for the contract to be: If you specify
> https://... URLs, then pkg_add guarantees it only installs packages
> retrieved over authenticated transport.  (Otherwise, no change.)

This means an http: URL that redirects to https will not get validation,
too.

I see this as ending up way too complicated.  Instead, simply

  Validate certifications for all https connections if
  HTTPS_VALIDATE_CERTS=yes.

  Omit validation if HTTPS_VALIDATE_CERTS=no.

  Set the default value to yes for NetBSD 10.

  Set the default value to yes for any system where it is known that as
  part of proper base system setup that trust anchors are configured.

  Probably, just set it to yes everywhere, because it is a bug to fetch
  https without validatin, unless the user has asked not to validated.
  <pmc>This is probably a bit much for 2023Q4 pullup.</pmc>

  Make no changes at all to ftp/http.  There is no bug to be fixed.


Implicit in the above is that I still object to INSECURE_TRANSPORT as
being about "you should not do this" vs "let's fix the bug of not
validating".