Re: Handling system-wide TLS certification bundles for openssl builtin and www/curl

["Dr. Thomas Orgis" <thomas.orgis%uni-hamburg.de@localhost>]

Hi Greg,

it seems to me that we're not quite on the same page yet. I'll try to
clarify in my inline responses.

First thing: The main point of the patch is to un-break curl with system
openssl on CentOS, or any distro that uses a certificate bundle. It
wouldn't find certificates if not told explicitly where. And currently,
the bundled form is just ignored by the builtin detection.

Then there is the issue whether pkgsrc openssl can/should use the
system's certificates.

am Tue, 23 Feb 2021 08:47:59 -0500
schrieb Greg Troxel <gdt%lexort.com@localhost>: 

> This looks like a behavior change always, checking magic paths that
> happen to be there on Centos, but on all systems.

I figured that if /etc/pki/tls exists, it is there for a reason. And
this check is the last step before the ‘most likely place’ fallback. So
you prefer that part to be in an OPSYS check. How specific? Is Linux
enough?

Same for the bundle check?


>  And I don't see that
> it is something you ask for by a variable in mk.conf

I don't see a point for a choice bettween broken curl and working curl.
Who would choose broken? In this setup with system openssl and bundled
certificates, curl without the patch will not do HTTPS.

> I think this needs stepping back and considering:
> 
>   When is pkgsrc using pkgsrc openssl and when it is using base ssl, and
>   do we think those decisions are right?

When I choose to prefer system openssl, I very much think that pkgsrc
should follow that choice as long as it is possible. I understand that
it might refuse when the system has say, openssl 0.9.8, and things are
incompatible with it.

>   What's the grand plan for the configured set of trust anchors, for
>   openssl, for other ssl libraries, and for programas that use one of
>   those but provide their own set instead of using default validation?

Yes, a grand plan wouldn't be bad. My wish is that I can continue to
manage certificates with the base Linux distro, as this is clearly part
of the security updates that are maintained for it. Also, as long as
feasible, I want to work with the system openssl.

Note that my use does not incorporate high-level desktop stuff like
Firefox or other browsers that may have an own opinion about trust
anchors.

>   If someone wants to use an old branch of pkgsrc for a long time (I get
>   the reasons), then perhaps they need to do security maintenance on
>   that branch, essentially turning it into a LTS.

I think you didn't really get my reasons yet;-) I am not interested in
maintenance of old pkgsrc branches. I install mostly scientific
software from a pkgsrc release and then will never replace the built
binaries unless they are actually broken. The rule is that there is no
updating of what is installed. For updates, you switch to another
version of pkgsrc using environment modules. There can be dozens of
different instances of pkgsrc on the same system.

This is a multi-user HPC install where people may need to work with a
fixed set of application software for extended time periods. And it's
different people working in different environments at the same time.
Environment modules make this easy, no need for everyone running
separate VMs/containers or such (although, we do have modules for
container runtimes, too, of course;-).

While I want to keep the user software itself fixed, I do keep the
minimal system packages from CentOS updated, as this is used for
running server processes and the actual user shells. I am not insane
enough to expect to be able to keep that unchanged for several years!
Part of this essential runtime is openssl, and of course also a
maintained list of TLS certificates.

So, no LTS pkgsrc. My model is instances of pkgsrc on a stable LTS base
OS, to both get fresh software and keep old software around.

> In your case, if you build pkgsrc with system openssl, then I'd expect
> the cert dir to point to the system place.  If you build pkgsrc with
> pkgsrc openssl, then I'd expect it to point there.

This is what happens right now. I still think it would be sensible for
my use-case, in case pkgsrc has to bring its own openssl, that I can
point it to the system trust store. Right now, I copy the system certs
on each package build (I will add packages to an existing prefix, just
not update). But that is suboptimal as they get out of sync unless I do
a cron job for that.

The issue that my patch fixes is that for the case of using system
openssl, pkgsrc's curl will _not_ use the system certificates! So I
does fit your logic. The case of bundled certs is just not handled.

> So as I see it what maybe should happen is some sort of variable to
> configure pkgsrc openssl, maybe other TLS implementations, and things
> that pass a dir to the validator, to point to some user-defined place.
> 

> And perhaps, the handling of the default setting of the cert dir for
> system openssl on some systems is wrong and should be fixed.

This is what I am addressing. So would it be fine if the patch is
wrapped in an OS check?

> I think it's important not to flip users from pkgsrc certs to system
> certs without them asking for it.  Part of the point of pkgsrc is to get
> pkgsrc behavior everywhere.

Nowhere did I attempt to flip anything. First, I'm fixing the situation
where www/curl sees _no_ certs at all while trying to use system ones.
Second, I did indeed suggest a next step where there could be an option
for pkgsrc's openssl to use the system certificate directory. I imagine
that I would add this as an option that is off by default. Do you have
an issue with that?


Alrighty then,

Thomas

-- 
Dr. Thomas Orgis
HPC @ Universität Hamburg