Subject: Re: "-Ldir -lname" vs. "dir/libname.so"
To: Todd Vierling , Johnny Lam <jlam@jgrind.org>
From: Nick Hudson <skrll@netbsd.org>
List: tech-toolchain
Date: 03/14/2002 15:37:04
On Wednesday 13 March 2002 10:10 pm, Todd Vierling wrote:
[...]
> Is this buildlink or libtool that does the relinking?  If buildlink, then
> libtool is broken and needs to be fixed (since the package with libtool
> won't work out-of-the-box).  In either case, relinking the binaries is
> completely unnecessary for NetBSD, as well as any other ELF-compliant
> system.

Actually I don't believe this is true. Also, libtool (the way it works today) 
relinks for all libraries with uninstalled dependencies.

> The correct way to record a dependency while everything is still in a build
> tree goes something like below, where a program and libraries are being
> linked in a dependency tree "prog -> libbar -> libfoo -> libsomelib".
> 
> Here, I'm presuming that libfoo is going in /usr/pkg/foo/lib, and libbar in
> /usr/pkg/bar/lib (so that you understand which -Rs and -Ls point to what
> library).  -lsomelib is assumed already installed in /usr/pkg/some/lib.
> 
> ==========
> [ELF]
> 
> 	cd libfoo
> 	cc -shared -o libfoo.so.0.0 -Wl,-soname,libfoo.so.0 ... \
> 		-R/usr/pkg/some/lib -L/usr/pkg/some/lib -lsomelib
> 	ln -s libfoo.so.0.0 libfoo.so.0
> 	ln -s libfoo.so.0.0 libfoo.so
> 
> 	cd ../libbar
> 	cc -shared -o libbar.so.0.0 -Wl,-soname,libbar.so.0 ... \
> 		-R/usr/pkg/foo/lib -L../libfoo -lfoo \
> 		-Wl,-rpath-link,/usr/pkg/some/lib

Two things to note about this are

	- The -Wl,-rpath-link part is no longer needed. The 2.11.2 based ld and the
	  patched 2.9.3 ld (in 1.5.3) automatically look for NEEDED libraries in the
	  RPATH of the dependent library.

	- The uninstalled libbar won't contain the necessary RPATH to find the
	  uninstalled libfoo.

> 	cd ../prog
> 	cc -o prog ... \
> 		-R/usr/pkg/bar/lib -L../libbar -lbar \
> 		-Wl,-rpath-link,/usr/pkg/some/lib \
> 		-Wl,-rpath-link,../libfoo

Simliar problems exist here.

> [a.out]
> 
> 	cd libfoo
> 	ld -Bshareable -o libfoo.so.0.0 ... \
> 		-R/usr/pkg/some/lib -L/usr/pkg/some/lib -lsomelib
> 
> 	cd ../libbar
> 	ld -Bshareable -o libbar.so.0.0 ... \
> 		-R/usr/pkg/foo/lib -L../libfoo -lfoo
> 
> 	cd ../prog
> 	cc -o prog ... \
> 		-R/usr/pkg/bar/lib -L../libbar -lbar
> ==========
> 
> Above, "..." is the list of objects for each binary.
> 
> You'll note that a.out doesn't need anything special, but that's because the
> a.out linker is actually *missing* the recursion logic for dependencies (it
> assumes that -l arguments are linked properly, and lets ld.so sort out the
> dependency logic).
> 
> But under ELF, the situation is different:  the linker searches for the
> dependencies.  While the rpaths are provided above, there's no way to know
> how to find the dependencies at link time without *another* option (ld's
> -rpath-link option, to be exact).

Again, this behaviour is no longer the default. In fact I don't think there's 
an option to revert to this behaviour.

> If -rpath-link is properly specified, there is no reason to relink anything
> at all.  That option allows ld(1) to find all the dependencies when doing an
> ELF link, without recording those temporary paths into the binary.

I believe that this is wrong as libtool supports the ability to use 
uninstalled libraries as part of a debug process. relinking is needed to 
generate the installed versions with the correct RPATH entries. 

I believe there is a much simpler way of supporting this. If libtool were to 
use install paths in uninstalled libraries and binaries then, using the 
normal wrapper scripts, it could simply override the RPATHs using 
LD_LIBRARY_PATH. Obviously this assumes that all platforms have the 
equivalent of LD_LIBRARY_PATH which may not be true.

> ===
> 
> : 		/usr/lib/libssl.so		[system supplied ]
> : 		/usr/lib/libcrypto.so		[  OpenSSL-0.9.5a]
> :
> : 		/usr/pkg/lib/libssl.so		[pkgsrc-supplied ]
> : 		/usr/pkg/lib/libcrypto.so	[  OpenSSL-0.9.6b]
> :
> : 	libbar.so explicitly needs the old OpenSSL libraries, and has
> : 	the following dependencies:
> :
> : 		libbar.so: /usr/pkg/lib/libfoo.so.0.0
> : 		           /usr/lib/libssl.so.?.?
> : 		           /usr/lib/libcrypto.so.?.?
> :
> : 	The relink command for libbar.so does:
> :
> : 		cc -shared -Wl,-soname -Wl,libbar.so.0 -o libbar.so.0.0 \
> : 			-L/usr/pkg/lib -lfoo -lssl -lcrypto
> :
> : But this relink command will link against the wrong OpenSSL libraries.
> 
> What does the original (non-relink) command for libbar do?  (Under the
> assumption, as above, that relinking should not actually be necessary.)
> 
> Basically, if you have "-Wl,-rpath-link,/usr/pkg/lib" in the above, you
> should end up with the pkgsrc libssl and libcrypto, rather than the
> system-supplied version.  ld will use that rpath-link directory to find the
> expected dependencies of libfoo.
> 
> ===
> 
> : If instead, it executed:
> :
> : 		cc -shared -Wl,-soname -Wl,libbar.so.0 -o libbar.so.0.0 \
> : 			/usr/pkg/lib/libfoo.so -lssl -lcrypto
> :
> : then I could control which libraries to link against more accurately.
> 
> Wouldn't this break you even further?  With no -L or -R at all, you'll link
> against the system-supplied ssl/crypto libraries for sure, not the
> pkgsrc-supplied ones.  And given that libfoo.so is in /usr/pkg/lib, why
> would you want to assume that the system-supplied versions are the correct
> ones?

So, going back to the original problem... If the relink looked more like

	cc -shared -Wl,-soname -Wl,libbar.so.0 -o libbar.so.0.0 \
		-L/usr/lib -L/usr/pkg/lib -lfoo -lssl -lcrypto

then the "correct" lib{ssl,crypto} will be used and their sonames put into 
the NEEDED list of libbar. This can be achieved by adding -L/usr/lib to the 
right place in the link line of libbar.

Finally, you want to pickup the old system supplied libraries, but this in 
itself, seems wrong. Why do you want the *old* versions? Which pkg are we 
talking about?

Nick