pkgsrc-Users archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: [External] Re: RHEL/Centos 7 builtin vs pkgsrc zlib clash for Makefiles that include devel/zlib/

On 2020-08-15 17:12, Lai, Peter PW wrote:
-----Original Message-----
From: Jason Bacon []
Sent: Saturday, August 15, 2020 3:07 PM
To: Lai, Peter PW <>;
Subject: [External] Re: RHEL/Centos 7 builtin vs pkgsrc zlib clash for Makefiles
that include devel/zlib/

On 2020-08-15 10:56, Lai, Peter PW wrote:
-----Original Message-----
From: Jason Bacon []
Sent: Friday, August 14, 2020 5:23 PM
To: Lai, Peter PW <>;
Subject: [External] Re: RHEL/Centos 7 builtin vs pkgsrc zlib clash
for Makefiles that include devel/zlib/

On 2020-08-10 12:05, Lai, Peter PW wrote:
-----Original Message-----
From: Jason Bacon []
Sent: Monday, August 10, 2020 10:00 AM
To: Lai, Peter PW <>;
Subject: [External] Re: RHEL/Centos 7 builtin vs pkgsrc zlib clash
for Makefiles that include devel/zlib/

On 2020-08-10 05:50, Lai, Peter PW wrote:

* RHEL 7.7 (Centos 7 for those playing at home I guess) in Azure.
* Toolchain: Software collection (SCL) devtoolset-9

* Yum installs native zlib-devel as a dependency for openssl-devel

* openssl-devel was installed in order to support PREFER_NATIVE=

In this environment because zlib-devel is installed, bootstrap by
picks up native zlib (/usr/include/zlib.h). When devel/libxml2 is
then built, it is built against native zlib ( => /lib64/
Next, some packages that also include
"../../devel/zlib/" such
nginx is built, it pulls in devel/zlib. This is weird because RHEL
7 includes zlib
1.2.7 which satisfies zlib>=1.2.3 in buildlink3.
What happens after is that libxml2 and downstream libxml2
(e.g. databases/postgresql) break because they see the pkgsrc zlib
no version information available (required by

# ldd /mnt/datadisk01/pkg/lib/
/mnt/datadisk01/pkg/lib/ no version information
available (required by /mnt/datadisk01/pkg/lib/
/mnt/datadisk01/pkg/lib/ no version information
available (required by /mnt/datadisk01/pkg/lib/

I worked around this by adding zlib to PREFER_NATIVE when
nginx, but is there a better way to resolve this? This looks
somewhat related to the continuing "pkg_install fails when
archivers/xz is installed" thread because RPATH manipulation could
be involved, but I am opening a separate thread in case it is not.
This does look like the same issue that we've been discussing
regarding xz and it's a great example of the kind of problems that
are avoided by using PREFER_PKGSRC=yes (and/or
What's your motive for using native openssl instead of pkgsrc?  The
only issue I've seen with the latter is the need to manually
security/mozilla-rootcerts* or equivalent to make curl, etc. work
If there's some other disadvantage to pkgsrc openssl, maybe we
should try to correct it.
The RHEL version of OpenSSL is backported/patched 1.0.2k with the
module compiled (and validated through vendor affirmed basis as per;!!
41nFt2k-C-GdOu-Q44Q$  ). So provisional compliance with FIPS 140-2
level 1 can be met for binaries linked to wrapper module when the
system & kernel are fips enabled and the binary uses the required
calling convention to autoconfigure or manually configure FIPS
operation mode: We fork the pkgsrc repo and patch Makefiles for
downstream builds' configure knobs where applicable to enable the
calling of openssl's fips mode for those apps.
I don't think this is worth correcting on the pkgsrc/NetBSD level for
folks who need this niche requirement but I sent the message because
the xz thread seemed related when dealing with a non-openssl-related
base vs pkgsrc library issue...
I bootstrapped on CentOS 7 using my modified bootstrap script
(attached) with zlib-devel and openssl-devel preinstalled, so pkgsrc
should be able to use native versions of both.  After installing
libxml2, I verified that it's using pkgsrc zlib.  Packages that
depend on openssl are not picking up the Yum installation, though,
because it's one revision too old.  Yum's zlib meets the current
requirements, though, so as a test I added it to PREFER_NATIVE and
checked dependencies for libarchive before and after.  This confirmed
that my mk.conf configuration is working as expected.
So I'm not sure why your installation was using pkgsrc zlib.  Did you
have anything in PREFER_* besides PREFER_NATIVE=openssl?

Linux centosdev.ceas  bacon ~ 1072: (pkgsrc): yum list installed|grep
zlib.x86_64 1.2.7-18.el7                     @anaconda
zlib-devel.x86_64 1.2.7-18.el7                     @base

Linux centosdev.ceas  bacon ~ 1073: (pkgsrc): yum list installed |
grep openssl
openssl.x86_64 1:1.0.2k-19.el7                  @anaconda
openssl-devel.x86_64 1:1.0.2k-19.el7                  @base
openssl-libs.x86_64 1:1.0.2k-19.el7                  @anaconda

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc-test/archivers/libarchive
1074: (pkgsrc): grep PREFER ~/Pkgsrc/pkg-test/etc/mk.conf
PREFER_PKGSRC=        yes PREFER_NATIVE=        openssl

[ Install pkgsrc libxml2 ]

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc-test/archivers/libarchive
1075: (pkgsrc): ldd ~/Pkgsrc/pkg-test/lib/|grep libz => /home/bacon/Pkgsrc/pkg-test/lib/

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc-test/archivers/libarchive
1076: (pkgsrc): bmake show-depends

[ Add zlib to PREFER_NATIVE ]

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc-test/archivers/libarchive
1078: (pkgsrc): grep PREFER ~/Pkgsrc/pkg-test/etc/mk.conf
PREFER_PKGSRC=        yes PREFER_NATIVE=        openssl zlib

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc-test/archivers/libarchive
1079: (pkgsrc): bmake show-depends

Libxml2 will pick up native libz fine when built first. The problem
was www/nginx which pulled in pkgsrc libz while still linking to
native openssl. After installing pkgsrc libz to satisfy nginx, libxml2
broke. Adding zlib to PREFER_NATIVE resolved the issue. I didn't have
anything else in PREFER_NATIVE except for openssl
So in the absence of a PREFER_PKGSRC setting and PREFER_NATIVE set to a
list of specific packages, dependencies not in the PREFER_NATIVE list will be
linked preferentially from pkgsrc over native if both are installed, otherwise
to whichever is available. This by itself seems fairly reasonable to me, with
some obvious risks especially on Linux. The problem here is that it doesn't
account for pre-installed packages linked against a native dependency before
the pkgsrc equivalent dependency was installed.

Bookstrapping with --prefer-pkgsrc (which will soon be the default for
Linux) should avoid this issue by installing the pkgsrc dep regardless of
whether there's a native equivalent available.

I don't see a simple way to address this issue without setting
PREFER_PKGSRC during bootstrap.  Trying to avoid these issues using
RPATH/RUNPATH or equivalent is looking a bit messy, if it's even feasible.

I still see a slight potential for problems if someone explicitly installs a pkgsrc
package listed in PREFER_NATIVE, or if a bug causes it to be installed as a dep,
but I'm not sure that's worth trying to prevent.


My disconnect here is why would www/nginx have preferred pkgsrc libz when not explicitly stated by any makefile? It just includes zlib/ in the exact way libxml2 does and none of nginx nor libxml2 prefers a zlib in any other configuration than default (which at the time was use native if found).
I think because your PREFER_NATIVE is different during and after bootstrap.  Setting PREFER_NATIVE=list after bootstrap implies PREFER_PGKSRC for other packages:

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc/textproc/libxml2 1008: (pkgsrc): grep PREFER_NATIVE ~/Pkgsrc/pkg/etc/mk.conf
#PREFER_NATIVE=        openssl

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc/textproc/libxml2 1010: (pkgsrc): bmake show-depends

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc/textproc/libxml2 1011: (pkgsrc): vi ~/Pkgsrc/pkg/etc/mk.conf

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc/textproc/libxml2 1012: (pkgsrc): grep PREFER_NATIVE ~/Pkgsrc/pkg/etc/mk.conf
PREFER_NATIVE=        openssl

Linux centosdev.ceas  bacon ~/Pkgsrc/pkgsrc/textproc/libxml2 1013: (pkgsrc): bmake show-depends

So setting both PREFER_PKGSRC and PREFER_NATIVE during bootstrap should prevent problems like this.

Home | Main Index | Thread Index | Old Index