Incorrect logic in mk/pkgformat/pkg/metadata.mk

["Aleksej Lebedev" <root%zta.lk@localhost>]

Hi, everyone.

I already wrote about a problem in the logic in mk/pkgformat/pkg/metadata.mk that sometimes brings spurious dependencies on shared libraries from ${DESTDIR}.

I couldn't fix the problem, but because these dependencies appeared only when we (in TomTom) build our own packages that are simply re-packaged BLOBS (like various components of androd-ndk, etc) we "solved" the problem by adding CHECK_SHLIBS_SUPPORTED=no.

There is still (at least) one more problem with metadata.mk. I am quite confident that it's a bug - not intended behavior.
When REQUIRES list is generated as part of "package" target, ldd gets executed on all binaries that belong to the package and all the dependencies are added to REQIRES list and then printed to +BUILD_INFO.

Now because the rpath passed to the linker is pointing to ${PREFIX}/lib, not to ${DESTDIR}${PREFIX}/lib, ldd never find the libraries from the package itself. And that would be OK, because libraries from the package should not belong to REQUIRES list (and should not be listed in +BUILD_INFO). However, if there happen to be a system library with the same name it does get picked up by ldd and put to +BUILD_INFO as a result.

I did the following experiment (I am using Ubuntu, but this should be reproducible anywhere, except maybe MacOS):
1. Installed libpython36.so.1.0: 
    $ apt-get install python3-dev
    $ sudo ln -s libpython3.6m.so /usr/lib/x86_64-linux-gnu/libpython3.6.so.1.0 # because original one has "m" suffix.
2. Build the python36 package:
    $ cd ~/pkgsrc/lang/python36 && ~/pkg/bin/bmake package
3. Extracted +BUILD_INFO from it and printed all the REQUIRES entries:
    $ ~/pkg/bin/bsdtar -C ~/tmp -xzf /home/a/pkgsrc/packages/All/python36-3.6.8nb1.tgz +BUILD_INFO
    $ grep ^REQUIRES ~/tmp/+BUILD_INFO

The result is:

REQUIRES=/home/a/pkg/lib/libbz2.so.0
REQUIRES=/home/a/pkg/lib/libcrypto.so.1.0.0
REQUIRES=/home/a/pkg/lib/libdb4-4.8.so.8
REQUIRES=/home/a/pkg/lib/libffi.so.6
REQUIRES=/home/a/pkg/lib/liblzma.so.5
REQUIRES=/home/a/pkg/lib/libssl.so.1.0.0
REQUIRES=/home/a/pkg/lib/libz.so.1
REQUIRES=/lib/x86_64-linux-gnu/libcrypt.so.1
REQUIRES=/lib/x86_64-linux-gnu/libc.so.6
REQUIRES=/lib/x86_64-linux-gnu/libdl.so.2
REQUIRES=/lib/x86_64-linux-gnu/libexpat.so.1
REQUIRES=/lib/x86_64-linux-gnu/libm.so.6
REQUIRES=/lib/x86_64-linux-gnu/libnsl.so.1
REQUIRES=/lib/x86_64-linux-gnu/libpthread.so.0
REQUIRES=/lib/x86_64-linux-gnu/librt.so.1
REQUIRES=/lib/x86_64-linux-gnu/libutil.so.1
REQUIRES=/lib/x86_64-linux-gnu/libz.so.1
REQUIRES=/usr/lib/x86_64-linux-gnu/libpython3.6.so.1.0

It's interesting to note that if I install this package and rebuild it again, the dependency on /usr/lib/x86_64-linux-gnu/libpython3.6.so.1.0 disappears - this is because when ldd from metadata.mk gets executed on ${DESTDIR}${PREFIX}/bin/python3.6 it picks up for ${PREFIX}/pkg/lib/libpython3.6.so.1.0  before /usr/lib/x86_64-linux-gnu/libpython3.6.so.1.0.

This makes package creation unstable.

I know how to fix the problem conceptually, but didn't quite figure out all the subtleties of writing awk code in-lined in shell executed from bmake.

I will probably try to factor out the awk code from bmake. I see that this was already done in some cases.

In any case, if somebody who is more familiar with this code than me can fix this, I will say a big thank you to them and send a nice postcard from the Netherlands ;-).

--
Aleksej Lebedev