Subject: Re: make replace and library dynamic loading
To: Xavier HUMBERT <xavier.humbert@xavhome.fr.eu.org>
From: Frederick Bruckman <fredb@immanent.net>
List: tech-pkg
Date: 02/26/2003 12:07:52
On Wed, 26 Feb 2003, Xavier HUMBERT wrote:

> I've been bitten by "make replace" breaking dynamic libraries dependencies.

Yes, that's what "make replace" does. It's broken by design. :-{

> I updated expat, from which depends php-xml, then after upgrading php,
> got this error in the logfile :
>
> > [26-Feb-2003 14:30:41] PHP Warning:  Unable to load dynamic library
> >       '/usr/pkg/lib/php/20020429/xml.so' - Shared object "libexpat.so.1"
> >       notfound in Unknown on line 0
>
> Fortunately, I do have a log file....
>
> And, in fact, if I check the library (this is ldd *after* I fix the problem) :
>
> > # ldd /usr/pkg/lib/php/20020429/xml.so
> > /usr/pkg/lib/php/20020429/xml.so:
> >          -lexpat.4 => /usr/pkg/lib/libexpat.so.4
>
> What hurts me, is that php-xml is linked against a versioned library,
> while there is a symlink which is precisely done for this purpose :
>
> > ls -l /usr/pkg/lib/libexpat.so
> > lrwxr-xr-x  1 root  wheel      15 Feb 12 15:00
> >                       /usr/pkg/lib/libexpat.so@ -> libexpat.so.4.0
>
> So I'm asking if :
>
> -- there is a way to check your system for those problems (with find and ldd ?)
> -- why use -llibname.vers in make file, instead relying on symlinks mechanisms ?

You don't understand. The soname change signifies that the new shared
library *is* *not* *compatible* with binaries linked against the old.
If you try to abuse the symlinks to make it seem like it is, you'll
get a binary that dumps core (or worse! -- undefined behaviour).

You linked against '-lexpat', which expanded to libexpat.so. The ELF
linker looked inside that shared library (which happened to be a
symlink) to see that the soname was "-lexpat.so.1", and stuck that in
the binary. When you run the binary, the run-time linker looks for
libexpat.so.1. This is what you're supposed to have:

	/usr/pkg/lib/libexpat.so -> libexpat.so.4
	/usr/pkg/lib/libexpat.so.1 -> libexpat.so.1.0
	/usr/pkg/lib/libexpat.so.4 -> libexapt.so.4.0
	/usr/pkg/lib/libexpat.so.1.0
	/usr/pkg/lib/libexpat.so.4.0

With that, you can build and link against the new shared library,
while still running old binaries against the old.

You get such a set-up naturally, when you simply install on-top of the
old. The package system works against you, and "make replace" doesn't
help. I've made a start towards letting the package system do the
right things with shared libraries here:

    ftp://ftp.netbsd.org/pub/NetBSD/misc/fredb/pkg_hack.tar.gz

It's not particularly well-documented or well-supported however --
what you see is what you get. I'll give a hint though: download an old
"expat" binary package, expat-1.95.2, and "pkg_hack -a
expat-1.95.2.tgz". If you want to build new packages properly against
the new shared library, follow on with "pkg_hack -a
expat-1.95.6nb1.tgz". You should end up with something like this:

fredb@tautology-> ll /usr/pkg/lib/libexpat.*
-rw-r--r--  1 root  wheel  166550 Feb 16 11:31 /usr/pkg/lib/libexpat.a
-rwxr-xr-x  1 root  wheel     766 Feb 16 11:31 /usr/pkg/lib/libexpat.la
lrwxr-xr-x  1 root  wheel      15 Feb 16 11:31 /usr/pkg/lib/libexpat.so -> libexpat.so.4.0
lrwxr-xr-x  1 root  wheel      15 Nov 24 02:45 /usr/pkg/lib/libexpat.so.1 -> libexpat.so.1.0
-rwxr-xr-x  1 root  wheel  142774 Nov 24 02:45 /usr/pkg/lib/libexpat.so.1.0
lrwxr-xr-x  1 root  wheel      15 Feb 16 11:31 /usr/pkg/lib/libexpat.so.4 -> libexpat.so.4.0
-rwxr-xr-x  1 root  wheel  146999 Feb 16 11:31 /usr/pkg/lib/libexpat.so.4.0

fredb@tautology-> for s in /usr/pkg/lib/libexpat.*; do echo -n $s'	'; pkg_info -Fe $s; done
/usr/pkg/lib/libexpat.a expat-1.95.6nb1
/usr/pkg/lib/libexpat.la        expat-1.95.6nb1
/usr/pkg/lib/libexpat.so        expat-1.95.6nb1
/usr/pkg/lib/libexpat.so.1      expat-SO-1.95.2
/usr/pkg/lib/libexpat.so.1.0    expat-SO-1.95.2
/usr/pkg/lib/libexpat.so.4      expat-1.95.6nb1
/usr/pkg/lib/libexpat.so.4.0    expat-1.95.6nb1


Frederick