Jason Bacon <outpaddling%yahoo.com@localhost> writes:
> On 2020-07-29 17:26, Greg Troxel wrote:
>> Jason Bacon <outpaddling%yahoo.com@localhost> writes:
>>
>>> The point here is not to add system lib dirs, but to exclude
>>> ${PREFIX}/lib. It's my understanding that RPATH overrides all other
>>> search paths, so this would ensure that pkg_install binaries don't get
>>> fooled into using the wrong libs by LD_LIBRARY_PATH and such.
>> That is not my understanding at all. I expect LD_LIBRARY_PATH, any
>> RPATH values, and system libs, in that order. I would suggest that you
>> experimentally validate this rather than make assumptions.
> This was not an assumption.
>
> From the ld man page:
>
> The linker uses the following search paths to locate required
> shared libraries:
>
> 1. Any directories specified by -rpath-link options.
>
> 2. Any directories specified by -rpath options. The difference
> between -rpath and -rpath-link is that directories
> specified by -rpath options are included in the
> executable and used at runtime, whereas the -rpath-link
> option is only effective at link time. Searching -rpath
> in this way is only supported by native linkers and
> cross linkers which have been configured
> with the --with-sysroot option.
>
> 3. On an ELF system, for native linkers, if the -rpath and
> -rpath-link options were not used, search the contents
> of the environment variable "LD_RUN_PATH".
>
> 4. On SunOS, if the -rpath option was not used, search any
> directories specified using -L options.
>
> 5. For a native linker, search the contents of the environment
> variable "LD_LIBRARY_PATH".
Wow, this is really different between BSD and Linux. Is that about the
run-time linker from the binary, or about ld(1) and how it finds things
at link time?
From ld.elf_so on NetBSD 8:
The linker will search for libraries in three lists of paths:
1. A user defined list of paths as specified in LD_LIBRARY_PATH and
ld.so.conf(5).
The use of ld.so.conf should be avoided as the setting of a global
search path can present a security risk.
2. A list of paths specified within a shared object using a DT_RPATH
entry in the dynamic section. This is defined at shared object link
time.
3. The list of default paths which is set to /usr/lib.
> Verification for GNU ld, showing that -rpath overrides use of liblzma
> in $LD_LIBRARY_PATH.
I wonder what is in the binary, and how these args flow to things in the
ELF object.
> It's not a matter of whose fault it is, but whether we want to deal
> with PRs that could have been avoided. If I can easily prevent
> end-users from shooting themselves in the foot without impinging on
> their freedom, I will always choose to do so. Not just for their
> sake, but it saves my valuable time solving the same problems over and
> over or trying to convince people not to set LD_LIBRARY_PATH in their
> .bashrc. Lots of HPC users do things like this, and yes my pkgsrc
> binaries are immune to it as long as they use rpath properly.
I think part of the hard question here is that shared library runtime
search path schemes differ widely. I thought it was part of the ELF
spec and find the Linux behavior very surprising -- but I was never 100%
clear.
> This is exactly my point. The fewer dependencies come from outside
> pkgsrc, the less often OS or other non-pkgsrc updates will break
> pkgsrc. This is especially true if those dependencies come from
> something like Debian packages in a non-LTS distro, where ABIs are
> changing all the time. A Debian update to lzma won't affect pkgsrc
> users who use pkgsrc lzma. That's one reason I always set
> PREFER_PKGSRC=yes. With this set, pkgsrc binaries use very little
> from base besides libc.
I agree; if your base OS is unstable, it is best to use everything from
pkgsrc that you can.
Perhaps we should revisit the pkgsrc defaults for Linux, pushing them
more in the PREFER_PKGSRC direction.
>>> On Debian bases systems, it could mean a library getting upgraded
>>> behind our backs by apt upgrade, resulting in an ABI/API change that
>>> suddenly breaks a pkgsrc binary that's been working fine until now.
>> That is why it is only sound to use pkgsrc binaries on Linux which are
>> built against a configuration-managed set of base systems libraries. I
>> don't know why you are insisting that it should be possible to work
>> against some arbitrary, shifting base. If you mean there is some FHS
>> that specifies a fixed ABI (with fixed shlib versions), and to build for
>> that defined ABI, ok, but then you need to back off from "this lib is on
>> my system" to "this lib is in the specification."
> I don't know how you reached this conclusion, which is basically the
> opposite of what I'm saying. I steer clear a shifting base as much as
> possible by using dependencies from pkgsrc instead. On RHEL/CentOS,
> ABIs are preserved as much as possible through an entire major version
> of the OS to maintain compatibility for commercial software, so aging
> libraries is the problem there, rather than a shifting base. But even
> on bleeding-edge linux distros, changes to the compiler and libc are
> far less frequent than other random libraries, so updates are unlikely
> to cause trouble when pkgsrc builds most other dependencies itself.
I don't think it's opposite really. I really meant "whatever pkgsrc
relies on must be configured managed, or it is unsound". You are
accepting that, I think, and making the point that therefore one should
rely on as little as possible, and that the closer you get to core libc
etc. the less likely it is to change.
>> I am talking about freetype2 or something like that. If you don' have a
>> stable base system, you need to configure things to depend on pkgsrc
>> whenever that is possible. If you want to propose changing pkgsrc to
>> basically never trust Linux base system versions of things, if it is at
>> all avoidable, I think that's an entirely reasonable thing to propose.
>
> I don't think any core pkgsrc change is necessary since this is
> already fairly well-supported by PREFER_PKGSRC/PREFER_NATIVE. I would
> only aim to educate people about their use.
If the consensus is that on Linux, because of old libs (FC derivatives)
or unstable libs, people should steer hard to PREFER_PKGSRC, we should
set the defaults that way. As pkgsrc maintainers, we curate decisions
in the best interests of most users, in addition to providing knobs for
others. You have made what feels like a strong argument that using base
libs that could have been avoided isn't generally a good idea.
Attachment:
signature.asc
Description: PGP signature