lang/python27: (patch) RPATH breaking build with previous installed python27

[Matthias Ferdinand <mf+ml.pkgsrc-users%netzwerkagentursaarland.de@localhost>]

Hi,

upgrading python27 from 2.7.3nb3 to 2.7.6nb2 failed on my Linux
installations (except for some older than ~6 years).

On Linux at least, RPATH has precedence over LD_LIBRARY_PATH. Pkgsrc
uses RPATH to direct its binaries to the pkgsrc installed libraries
instead of the system libraries.

This breaks pythons use of LD_LIBRARY_PATH for the build process,
resulting in the use of a previously installed libpython2.7.so instead
of the just compiled current version. The build succeeds if no previous
python27 is installed (package removed or libpython2.7.so renamed).

See here for a nice summary of the RPATH vs. LD_LIBRARY_PATH problem:
http://postgresql.1045698.n5.nabble.com/LD-LIBRARY-PATH-versus-rpath-td2017872.html

Attached is a patch for Makefile.pre.in. To work around the issue, it
creates a copy of the ./python binary, removes the RPATH setting using
chrpath, and uses the resulting binary for further local python invocations.

The patch probably lacks in elegance, and possibly in portability (does
chrpath exist for every pkgsrc platform?). The pkgsrc Makefile also
needs an additional USE_TOOLS+=chrpath for this patch.

Regards
Matthias

-----------------------------------------------------------------------

Error message from the build:

LD_LIBRARY_PATH=/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6: 
./python -E -S -m sysconfig --generate-posix-vars
Traceback (most recent call last):
  File 
"/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6/Lib/site.py", line 
548, in <module>
    main()
  File 
"/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6/Lib/site.py", line 
530, in main
    known_paths = addusersitepackages(known_paths)
  File 
"/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6/Lib/site.py", line 
266, in addusersitepackages
    user_site = getusersitepackages()
  File 
"/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6/Lib/site.py", line 
241, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File 
"/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6/Lib/site.py", line 
231, in getuserbase
    USER_BASE = get_config_var('userbase')
  File 
"/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6/Lib/sysconfig.py", 
line 516, in get_config_var
    return get_config_vars().get(name)
  File 
"/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6/Lib/sysconfig.py", 
line 468, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File 
"/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6/Lib/sysconfig.py", 
line 352, in _init_posix
    from _sysconfigdata import build_time_vars
ImportError: No module named _sysconfigdata
*** Error code 1


Looking at the ./python binary with objdump and ldd:

root@pkgsrc1004-32:/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6# 
objdump -x ./python | grep RPATH
  RPATH                /opt/pkgsrc/lib

root@pkgsrc1004-32:/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6# 
LD_LIBRARY_PATH=/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6: ldd 
python
        linux-gate.so.1 =>  (0xb7776000)
        libpython2.7.so.1.0 => /opt/pkgsrc/lib/libpython2.7.so.1.0 (0xb7606000)
        libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb75e7000)
        libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb75e2000)
        libutil.so.1 => /lib/tls/i686/cmov/libutil.so.1 (0xb75de000)
        libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb75b8000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7457000)
        /lib/ld-linux.so.2 (0xb7777000)

Removing the RPATH entry allows LD_LIBRARY_PATH to take over:

root@pkgsrc1004-32:/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6# 
chrpath -d python
root@pkgsrc1004-32:/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6# 
LD_LIBRARY_PATH=/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6: ldd 
python
        linux-gate.so.1 =>  (0xb77a8000)
        libpython2.7.so.1.0 => 
/opt/pkgsrc/work/pkgsrc/lang/python27/default/Python-2.7.6/libpython2.7.so.1.0 
(0xb7635000)
        libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7616000)
        libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7611000)
        libutil.so.1 => /lib/tls/i686/cmov/libutil.so.1 (0xb760d000)
        libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb75e7000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7486000)
        /lib/ld-linux.so.2 (0xb77a9000)

--- Makefile.pre.in.older       2014-02-16 23:25:41.192051768 +0100
+++ Makefile.pre.in     2014-02-16 23:32:47.610633449 +0100
@@ -191,8 +191,13 @@
 
 PYTHON=                python$(EXE)
 BUILDPYTHON=   python$(BUILDEXE)
+PKGSRCBUILDPYTHON= ./pkgsrcbuild-python$(BUILDEXE)
 
-PYTHON_FOR_BUILD=@PYTHON_FOR_BUILD@
+$(PKGSRCBUILDPYTHON):  $(BUILDPYTHON)
+               cp $(BUILDPYTHON) $(PKGSRCBUILDPYTHON)
+               chrpath -d $(PKGSRCBUILDPYTHON)
+
+PYTHON_FOR_BUILD=$(PKGSRCBUILDPYTHON) -E
 _PYTHON_HOST_PLATFORM=@_PYTHON_HOST_PLATFORM@
 HOST_GNU_TYPE=  @host@
 
@@ -405,7 +410,7 @@
 
 # Default target
 all:           build_all
-build_all:     $(BUILDPYTHON) oldsharedmods sharedmods gdbhooks
+build_all:     $(PKGSRCBUILDPYTHON) $(BUILDPYTHON) oldsharedmods sharedmods 
gdbhooks
 
 # Compile a binary with gcc profile guided optimization.
 profile-opt: