pkgsrc-Users archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: pkg_install fails when archivers/xz is installed

On 2020-07-29 17:26, Greg Troxel wrote:
Jason Bacon <> writes:

The point here is not to add system lib dirs, but to exclude
${PREFIX}/lib.  It's my understanding that RPATH overrides all other
search paths, so this would ensure that pkg_install binaries don't get
fooled into using the wrong libs by LD_LIBRARY_PATH and such.
That is not my understanding at all.  I expect LD_LIBRARY_PATH, any
RPATH values, and system libs, in that order.  I would suggest that you
experimentally validate this rather than make assumptions.
This was not an assumption.

From the ld man page:

           The linker uses the following search paths to locate required
           shared libraries:

           1.  Any directories specified by -rpath-link options.

           2.  Any directories specified by -rpath options.  The difference
               between -rpath and -rpath-link is that directories specified by
               -rpath options are included in the executable and used at
               runtime, whereas the -rpath-link option is only effective at
               link time. Searching -rpath in this way is only supported by
               native linkers and cross linkers which have been configured
               with the --with-sysroot option.

           3.  On an ELF system, for native linkers, if the -rpath and
               -rpath-link options were not used, search the contents of the
               environment variable "LD_RUN_PATH".

           4.  On SunOS, if the -rpath option was not used, search any
               directories specified using -L options.

           5.  For a native linker, search the contents of the environment
               variable "LD_LIBRARY_PATH".


Verification for GNU ld, showing that -rpath overrides use of liblzma in $LD_LIBRARY_PATH.

Linux centosdev.ceas  bacon ~ 596: (pkgsrc): cat lzma-test.c
#include <stdio.h>
#include <sysexits.h>
#include <lzma.h>

int     main(int argc,char *argv[])

    lzma_allocator  *allocator;

    return EX_OK;

Linux centosdev.ceas  bacon ~ 597: (pkgsrc): echo $LD_LIBRARY_PATH

Linux centosdev.ceas  bacon ~ 598: (pkgsrc): cc -o lzma-test lzma-test.c
/tmp/cc2n17BS.o: In function `main':
lzma-test.c:(.text+0x17): undefined reference to `lzma_index_init'
collect2: error: ld returned 1 exit status

Linux centosdev.ceas  bacon ~ 599: (pkgsrc): cc -o lzma-test lzma-test.c -llzma

Linux centosdev.ceas  bacon ~ 600: (pkgsrc): ldd lzma-test =>  (0x00007ffd9a5c6000) => /home/bacon/Pkgsrc/pkg/lib/ (0x00007fa404594000) => /lib64/ (0x00007fa4041c6000) => /lib64/ (0x00007fa403faa000) => /lib64/ (0x00007fa403da2000)
    /lib64/ (0x00007fa4047b9000)

Linux centosdev.ceas  bacon ~ 601: (pkgsrc): cc -o lzma-test lzma-test.c -Wl,-rpath=/usr/lib64 -llzma

Linux centosdev.ceas  bacon ~ 602: (pkgsrc): ldd lzma-test =>  (0x00007ffc65bc2000) => /usr/lib64/ (0x00007fb4a5967000) => /usr/lib64/ (0x00007fb4a5599000) => /usr/lib64/ (0x00007fb4a537d000)
    /lib64/ (0x00007fb4a5b8d000)

Sure, but these people are doing things not following best practices, to
put it politely, and lots of things are going to break.  It's perfectly
possible to install things with --prefix=$HOME/stuff and not need
LD_LIBRARY_PATH becuase correct software puts $HOME/stuff/lib in RPATH
in those binaries.  I have done this (on NetBSD) with no difficulty.

I don't feel responsible for accomodating such practices, or that we
need to or even should contort to deal with this case.  Are you saying
that pkgsrc binaries should have some magic mechanism to override the
normal OS handling and protect people from bad judgement?  This just
doesn't make sense to me.
It's not a matter of whose fault it is, but whether we want to deal with PRs that could have been avoided.  If I can easily prevent end-users from shooting themselves in the foot without impinging on their freedom, I will always choose to do so.  Not just for their sake, but it saves my valuable time solving the same problems over and over or trying to convince people not to set LD_LIBRARY_PATH in their .bashrc.  Lots of HPC users do things like this, and yes my pkgsrc binaries are immune to it as long as they use rpath properly.
As I've stated before on this list, I'm leery of having my pkgsrc
installations rely on anything outside their control.  On CentOS
systems, for example, most Yum packages never get upgraded until you
upgrade the major version of the OS.  They only get bug fixes and
security patches.  So while pkgsrc is keeping up with the latest
releases, "base" liblzma is standing still for the several years the
system is in service and eventually it may be too old for pkgsrc to
I realize you don't like this, but I don't think it is at all rational
to expect to build software on an OS and then allow random OS updates
and expect that built software to still work.  Software in general is
built against and ABI and if you change the ABI you have to start over.
What you want is basically impossible, if any base system libs or
binaries are useed in the resulting pkgsrc build.

In pkgsrc we have pkg_rolling-replace to rebuild things when
depdendencies change.  If you update the base OS then you more or less
need new packages.  It's only more or less because in NetBSD we leave
the old shlibs.  In $RANDOM_LINUX, I have no idea what the rules are.

This is exactly my point.  The fewer dependencies come from outside pkgsrc, the less often OS or other non-pkgsrc updates will break pkgsrc.  This is especially true if those dependencies come from something like Debian packages in a non-LTS distro, where ABIs are changing all the time.  A Debian update to lzma won't affect pkgsrc users who use pkgsrc lzma.  That's one reason I always set PREFER_PKGSRC=yes.  With this set, pkgsrc binaries use very little from base besides libc.
On Debian bases systems, it could mean a library getting upgraded
behind our backs by apt upgrade, resulting in an ABI/API change that
suddenly breaks a pkgsrc binary that's been working fine until now.
That is why it is only sound to use pkgsrc binaries on Linux which are
built against a configuration-managed set of base systems libraries.  I
don't know why you are insisting that it should be possible to work
against some arbitrary, shifting base.  If you mean there is some FHS
that specifies a fixed ABI (with fixed shlib versions), and to build for
that defined ABI, ok, but then you need to back off from "this lib is on
my system" to "this lib is in the specification."
I don't know how you reached this conclusion, which is basically the opposite of what I'm saying.  I steer clear a shifting base as much as possible by using dependencies from pkgsrc instead.  On RHEL/CentOS, ABIs are preserved as much as possible through an entire major version of the OS to maintain compatibility for commercial software, so aging libraries is the problem there, rather than a shifting base.  But even on bleeding-edge linux distros, changes to the compiler and libc are far less frequent than other random libraries, so updates are unlikely to cause trouble when pkgsrc builds most other dependencies itself.

I am talking about freetype2 or something like that.  If you don' have a
stable base system, you need to configure things to depend on pkgsrc
whenever that is possible.  If you want to propose changing pkgsrc to
basically never trust Linux base system versions of things, if it is at
all avoidable, I think that's an entirely reasonable thing to propose.

I don't think any core pkgsrc change is necessary since this is already fairly well-supported by PREFER_PKGSRC/PREFER_NATIVE.  I would only aim to educate people about their use.


Home | Main Index | Thread Index | Old Index