"Dr. Thomas Orgis" <thomas.orgis%uni-hamburg.de@localhost> writes:
> it seems to me that we're not quite on the same page yet. I'll try to
> clarify in my inline responses.
indeed we are not. I think a lot of that is me wanting to be very clear
about what's going on, such that someone who has zero idea what your
flavor fo Linux does about certificates will follow.
> First thing: The main point of the patch is to un-break curl with system
> openssl on CentOS, or any distro that uses a certificate bundle. It
You say 'unbreak curl', but that's to me unclear. Perhaps you mean
"provide curl with a set of trust anchors so that it can validate
certificates". Sorry if that seems difficult but sometimes people use
"works" to mean "file is downloaded" vs "server certificate is properly
validated and if not valid file is not downloaded so I tend to border on
pedantic for these discussions.
The big thing that was not clear is if your pkgsrc builds are using the
system openssl or if they are using pkgsrc openssl. I'm guessing they
all use system openssl.
I further guess that your entire message and request for change is only
about having pkgsrc use system openssl's trust anchors better, when a
package is going to depend on system openssl anyway.
> wouldn't find certificates if not told explicitly where. And currently,
> the bundled form is just ignored by the builtin detection.
OK, so now this is becoming clearer. openssl's builtin.mk decides based
on sometimes OS type and sometimes not, sometimes based on what
directories exist in the filesystem, how to configure SSLDIR. And that
right now expects $SSLDIR/certs, which are expected to have a set of
trust anchors, in the form that openssl will naturally use.
> Then there is the issue whether pkgsrc openssl can/should use the
> system's certificates.
That's another can of worms, and I think for now should be an explicit
mk.conf setting. It's going to be tricky because mozilla-rootcerts may
then not install to system, and it might be using SSLDIR right now. But
I think we can defer that.
> am Tue, 23 Feb 2021 08:47:59 -0500
> schrieb Greg Troxel <gdt%lexort.com@localhost>:
>
>> This looks like a behavior change always, checking magic paths that
>> happen to be there on Centos, but on all systems.
>
> I figured that if /etc/pki/tls exists, it is there for a reason. And
> this check is the last step before the ‘most likely place’ fallback. So
> you prefer that part to be in an OPSYS check. How specific? Is Linux
> enough?
>
> Same for the bundle check?
Now that I'm understanding better, I'm not sure. Is the notion of
/etc/pki/tls (which is bizarre as these trust anchors are as I
understand it for all pkix validation, not necessarily just tls/ssl)
something that appears on just Redhat/Centos? Is anyone aware of it
happening on other than Linux?
>> And I don't see that it is something you ask for by a variable in
>> mk.conf
>
> I don't see a point for a choice bettween broken curl and working curl.
> Who would choose broken? In this setup with system openssl and bundled
> certificates, curl without the patch will not do HTTPS.
That's not really my point. It's about changing behavior based on
directories existing, without explicit request, and pkgsrc in general
tries not to respond to the environment but to control it, most of the
time.
>> I think this needs stepping back and considering:
>>
>> When is pkgsrc using pkgsrc openssl and when it is using base ssl, and
>> do we think those decisions are right?
>
> When I choose to prefer system openssl, I very much think that pkgsrc
> should follow that choice as long as it is possible. I understand that
> it might refuse when the system has say, openssl 0.9.8, and things are
> incompatible with it.
OK - I get it now that you are only talking about the case where a
package is choosing base system penssl.
>> What's the grand plan for the configured set of trust anchors, for
>> openssl, for other ssl libraries, and for programas that use one of
>> those but provide their own set instead of using default validation?
>
> Yes, a grand plan wouldn't be bad. My wish is that I can continue to
> manage certificates with the base Linux distro, as this is clearly part
> of the security updates that are maintained for it. Also, as long as
> feasible, I want to work with the system openssl.
That makes sense too - for things that use base system openssl they
should just use it. Really curl is out of bounds here wanting this
configured at build time instead of just doing openssl default
validation like everything else, but maybe I'm missing something.
> Note that my use does not incorporate high-level desktop stuff like
> Firefox or other browsers that may have an own opinion about trust
> anchors.
Sure, understood that some things choose to ignore system trust anchor
config.
>> If someone wants to use an old branch of pkgsrc for a long time (I get
>> the reasons), then perhaps they need to do security maintenance on
>> that branch, essentially turning it into a LTS.
>
> I think you didn't really get my reasons yet;-) I am not interested in
> maintenance of old pkgsrc branches. I install mostly scientific
> software from a pkgsrc release and then will never replace the built
> binaries unless they are actually broken. The rule is that there is no
> updating of what is installed. For updates, you switch to another
> version of pkgsrc using environment modules. There can be dozens of
> different instances of pkgsrc on the same system.
I thought you were trying to use pkgsrc openssl and perhaps pkgsrc
trust anchors.
> This is a multi-user HPC install where people may need to work with a
> fixed set of application software for extended time periods. And it's
> different people working in different environments at the same time.
> Environment modules make this easy, no need for everyone running
> separate VMs/containers or such (although, we do have modules for
> container runtimes, too, of course;-).
Sure, but you are running in to the intractable problem: I want all my
software to stay frozen but I also want it up to date. Granted you seem
to have a plan to reconcile those.
>> In your case, if you build pkgsrc with system openssl, then I'd expect
>> the cert dir to point to the system place. If you build pkgsrc with
>> pkgsrc openssl, then I'd expect it to point there.
>
> This is what happens right now. I still think it would be sensible for
> my use-case, in case pkgsrc has to bring its own openssl, that I can
> point it to the system trust store. Right now, I copy the system certs
> on each package build (I will add packages to an existing prefix, just
> not update). But that is suboptimal as they get out of sync unless I do
> a cron job for that.
I can see you wanting to do that, and for now I think it should be opt
in behavior.
>> I think it's important not to flip users from pkgsrc certs to system
>> certs without them asking for it. Part of the point of pkgsrc is to get
>> pkgsrc behavior everywhere.
>
> Nowhere did I attempt to flip anything. First, I'm fixing the situation
> where www/curl sees _no_ certs at all while trying to use system ones.
> Second, I did indeed suggest a next step where there could be an option
> for pkgsrc's openssl to use the system certificate directory. I imagine
> that I would add this as an option that is off by default. Do you have
> an issue with that?
I don't have a conceptual problem with a variable that if defined,
causes pkgsrc openssl to use the system trust anchors. However I expect
that to be quite tricky to get right.
So back to your proposed patch, and trying to summarize what's up:
As I understand it "/etc/ssl" is not so much "most likely" in some
sort of just barely winning out of total chaos as the standard path
for openssl if you don't try to mess with it. And sometimes it is
openssl.
The minor annoyance is that some linux systems have a gratuitously
different path for /etc/ssl. Actually what you posted makes it look
like /etc/ssl/certs is still the cert dir, just arrived at by a
symlink to a different logical path.
The big change is that Centos 7 does not install trust anchors in
single files with symlinks by hashes as has been the longstanding
convention, but is instead installing a bundle file. (Presumably
programs that use openssl and don't try to specify validation use this
as default and nobody every notices.)
As I see it, a given system would either have a pile of certificates
or a bundle; having both seems messy.
pkgsrc has never had a notion of detecting and configuring a cert
bundle.
curl can be built to use trust anchor from either a cert dir
(containing a set of files with 1 cert each and hashed symlinks) or a
cert bundle (single file with many certs)
curl's pkgsrc Makefile has no idea about cert bundles
So:
I don't see why the new path has to be used given that /etc/ssl/certs
gets to a dir with a bundle file. But I guess the symlinks in
/etc/ssl is limited to certs and not to the other dirs expected. This
really feels like a bug in Centos to me, not that it really matters.
If there's a bundle, there isn't really a SSLCERTS dir. My immediate
reaction is that if we set SSLCERTBUNDLE we should not set SSLCERTS --
one or the other. But it may be less of a change to the world to
leave it be.
It's fine to have curl use bundle if set, use SSLCERTS if set, and if
not probably not pass either arg. I don't see that it should fail to
build because of this.
Regarding SSLCERTS, I find the following in Makefile* and options.mk:
./chat/jabberd2/Makefile: -e "s|<cachain>@sysconfdir@|<cachain>${SSLCERTS}|g" \
./chat/jabberd2/Makefile: -e "s|<pemfile>@sysconfdir@|<pemfile>${SSLCERTS}|g" \
./chat/jabberd2/Makefile: -e "s|cachain='@sysconfdir@|cachain='${SSLCERTS}|g" \
./chat/jabberd2/Makefile: -e "s|pemfile='@sysconfdir@|pemfile='${SSLCERTS}|g" \
./chat/profanity/Makefile:SUBST_VARS.paths= SSLCERTS
./mail/alpine/Makefile:CONFIGURE_ARGS+= --with-ssl-certs-dir=${SSLCERTS:Q}
./mail/cone/Makefile:CONFIGURE_ARGS+= --with-certdb=${SSLCERTS}
./mail/courier-mta/Makefile:#CONFIGURE_ARGS+= --with-certdb=${SSLCERTS}
./mail/imap-uw/options.mk:MESSAGE_SUBST+= SSLCERTS=${SSLCERTS:Q}
./mail/imap-uw/options.mk:EXTRASPECIALS+= SSLCERTS=${SSLCERTS:Q}
./mail/prayer/Makefile:MAKE_ENV+= SSLCERTS=${SSLCERTS:Q}
./mail/re-alpine/Makefile:CONFIGURE_ARGS+= --with-ssl-certs-dir=${SSLCERTS:Q}
./net/mosquitto/Makefile:SUBST_VARS.paths= SSLCERTS VARBASE
./net/vsftpd/options.mk:SUBST_SED.ssl+= -e 's,/usr/share/ssl/certs/vsftpd.pem,${SSLCERTS}/vsftpd.pem,g'
./net/s6-networking/options.mk:SUBST_VARS.paths= SH SETENV SSLCERTS
./security/libguardtime/Makefile:CONFIGURE_ARGS+= --with-cadir=${SSLCERTS}
./wip/spamassassin-cvs/Makefile:SA_SSLCERTS= ${SSLCERTS}
./wip/spamassassin-cvs/Makefile:SA_SSLCERTS= '$$LOCAL_RULES_DIR/certs'
./wip/spamassassin-cvs/Makefile:SUBST_VARS.sa2= PREFIX PKG_SYSCONFDIR SA_SSLCERTS
./www/curl/Makefile:CONFIGURE_ARGS+= --with-ca-path=${SSLCERTS}
./www/dillo/options.mk:SUBST_SED.sslcerts= -e 's,"/etc/ssl/certs,"${SSLCERTS},'
./www/netsurf/Makefile:SUBST_SED.ssl-certs= -e 's,/etc/ssl/certs,${SSLCERTS},'
and probably all of those will have trouble too.
So, for now I think it's better to leave SSLCERTS defined.
I'll edit your patch in with an ifdef linux and adjust the comments and
post it.
Attachment:
signature.asc
Description: PGP signature