On Tue, Apr 04, 2000 at 03:33:07PM -0500, erh@nimenees.com wrote:
> On Tue, Apr 04, 2000 at 10:35:40AM -0500, John Darrow wrote:
> > The linker is doing exactly what we tell it to, linking in an object
> > '.libs/libfoo.so' into an executable.  It could be argued that the path
> > should be stripped from the internal table if the object turns out to be
> > a shared library (thus resulting in a runtime-type link).  But see below.

> 	It already ends up being a runtime-type link, just with an
> unusual name.

That's exactly what I said.  Apparently, our ld.new does strip off the
path, as seen from an ELF compile of pkgsrc/editors/pico:

/usr/pkg/bin/pkglibtool-ELF-1.2p2 cc -O2    -Dneb -DJOB_CONTROL -DPOSIX -DMOUSE main.o libpico.la  -ltermcap -o pico
cc -O2 -Dneb -DJOB_CONTROL -DPOSIX -DMOUSE main.o -Wl,--rpath -Wl,/usr/pkg/lib .libs/libpico.so -ltermcap -o .libs/pico
^^^^^^^^^^^^^^^					  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note this part...

[...]
[514] root@jdarrowpiii:ttyp5:/var/pkgsrc/editors/pico:# ldd work/pine4.21/pico/.libs/pico 
work/pine4.21/pico/.libs/pico:
         -lpico.3 => not found
         -ltermcap.0 => /usr/lib/libtermcap.so.0
         -lc.12 => /usr/lib/libc.so.12

(at this point, I haven't done a make install, so libpico isn't found by
the runtime linker yet.  But it _was_ found by the compile-time linker, or
it wouldn't have linked.)

Apparently (from your comments) the old ld doesn't do the same stripping,
but libtool hands it a different command line:

/usr/pkg/bin/pkglibtool-a.out-1.2p2 cc -O2    -Dneb -DJOB_CONTROL -DPOSIX -DMOUSE main.o libpico.la  -ltermcap -o pico
cc -O2 -Dneb -DJOB_CONTROL -DPOSIX -DMOUSE main.o -R/usr/pkg/lib -L.libs -lpico -ltermcap -o .libs/pico

Still, it works.  Or you'd have a lot of angry pico and pine lovers storming
your door...

> > gcc -o foo foo.c -L./.libs -lfoo -Wl,-rpath,/usr/local/lib
> > 
> > On both a.out and elf, this will produce an executable which will just
> > work, without the need of either ldconfig or LD_LIBRARY_PATH.

> 	uh, no.  This doesn't work.  Try it.  You're going to have to
> set LD_LIBRARY_PATH in order for "./foo" to be able to find the
> library.  

I just did.  It works fine.  On both 1.4.2 (a.out, uses gnusrc/usr.bin/ld)
and 1.4V (ELF, uses gnusrc/usr.bin/ld.new), both on i386.  No /testlibs in
/etc/ld.so.conf, no LD_LIBRARY_PATH set.  I'll include the test setup at
the end of this message.

Besides, if it didn't work, then many of the builds in our pkgsrc would
fail (i.e. any of them in which a package includes a shared library which
is also used by another program in the package.)  The package system uses
-R instead of -rpath, but both our ld and our ld.new treat them the same
(if given a directory name - see the source code).

> > The point is that locations like './.libs' are _temporary_ - they are
> > used during the build process, and will probably no longer exist once the
> > program is installed.  We don't want the runtime loader wasting time
> > looking up directories which don't exist.  (It's for reasons like this
> > that the -L does _not_ automatically stuff in a runtime path.)

> 	yes, those type of locations are temporary.  So the completely
> correct solution would be to make whatever piece of software you're building
> set LD_LIBRARY_PATH to the temporary directory.  But this means you're 
> going to have to go changing the build process for everything that
> excepects -L... to do the right thing.  

No, you've got that backwards.  _Nothing_ needs to set LD_LIBRARY_PATH at
all.  Only -L to the build location, and -Wl,-rpath to the runtime location.
Look again at the package system to see how it works.  (A good quick example
is editors/pico.)

> 	Sure, it would be nice if every piece of software knew all the
> quirks of linking against shared libs on NetBSD, but they don't.  It makes
> a lot of sense to me to be compatible with what external software packages
> expect.  It seems to me that the ideal behaviour would be to enable what
> happens with ld on SunOS (according to the gnu ld man page, look under -rpath)
> -L... args add to rpath unless rpath is explicitly set.

Yes, on SunOS, ld adds the -L options to a default runtime search path
unless -rpath is given explicitly.  See that same gnu ld man page for one
reason why that might _not_ be a good thing, and I've already stated another
(useless runtime lookups of directories such as .libs which aren't there.)

Basically, the whole -rpath thing is the ELF way of doing things, and that's
the direction the UNIX world seems to be moving.  Given that there are tools
available (such as libtool) which make it much easier to get things right,
we should be encouraging program authors to doing things the correct way,
instead of putting in wasteful hacks to support those who don't.

(Example code follows signature...)

jdarrow

-- 
John Darrow - Senior Technical Specialist               Office: 630/752-5201
Computing Services, Wheaton College, Wheaton, IL 60187  Fax:    630/752-5968
Alphapage: 6303160707@alphapage.airtouch.com            Pager:  630/316-0707
Email:     John.P.Darrow@wheaton.edu

Note that my example commands are identical between a.out and ELF, with the
exception of using libshfunc.so.0.0 (with version numbers) for a.out and
libshfunc.so (without version numbers) for ELF.  However, the link commands
for the main program are _identical_.

shfunc.h:

int sharedfunc __P((char *));


shfunc.c:

#include <stdio.h>
#include "shfunc.h"

int sharedfunc (char *data) {

	return printf ("parameter is %s...\n", data);
}


main.c:

#include <stdio.h>
#include <err.h>
#include "shfunc.h"

int main (int argc, char *argv[]) {

	int numchars = 0;

	if (argc != 2)
		errx (1, "Invalid number of arguments");

	numchars = sharedfunc(argv[1]);

	printf ("Number of chars printed is %d.\n", numchars);

	return 0;
}


NetBSD 1.4.2 (a.out):

[207] root@abigail:ttyp0:/var/tmp/shlibex:# jls
total 10
578277 2 drwxr-xr-x  2 root  wheel  512 Apr  4 17:08 ./
162240 2 drwxrwxrwt  5 root  wheel  512 Apr  4 17:18 ../
578280 2 -rw-r--r--  1 root  wheel  278 Apr  4 17:08 main.c
578279 2 -rw-r--r--  1 root  wheel  120 Apr  4 17:04 shfunc.c
578278 2 -rw-r--r--  1 root  wheel   31 Apr  4 17:04 shfunc.h
[208] root@abigail:ttyp0:/var/tmp/shlibex:# mkdir .libs
[209] root@abigail:ttyp0:/var/tmp/shlibex:# cc -c -O -fPIC -DPIC shfunc.c
[210] root@abigail:ttyp0:/var/tmp/shlibex:# ld -Bshareable -o .libs/libshfunc.so.0.0 shfunc.o
[211] root@abigail:ttyp0:/var/tmp/shlibex:# mkdir /testlibs
[212] root@abigail:ttyp0:/var/tmp/shlibex:# cc -c -O main.c
[213] root@abigail:ttyp0:/var/tmp/shlibex:# cc -o main main.c -L./.libs -lshfunc -Wl,-rpath,/testlibs
[214] root@abigail:ttyp0:/var/tmp/shlibex:# ldd ./main
./main:
	-lshfunc.0 => not found (0x0)
	-lc.12 => /usr/lib/libc.so.12.40 (0x40019000)
[215] root@abigail:ttyp0:/var/tmp/shlibex:# cp -p .libs/libshfunc.so.0.0 /testlibs
[216] root@abigail:ttyp0:/var/tmp/shlibex:# mv .libs .notlibs       
[217] root@abigail:ttyp0:/var/tmp/shlibex:# ldd ./main
./main:
	-lshfunc.0 => /testlibs/libshfunc.so.0.0 (0x40019000)
	-lc.12 => /usr/lib/libc.so.12.40 (0x4001b000)
[218] root@abigail:ttyp0:/var/tmp/shlibex:# ./main teststring
parameter is teststring...
Number of chars printed is 27.
[219] root@abigail:ttyp0:/var/tmp/shlibex:# uname -a
NetBSD abigail.wheaton.edu 1.4.2 NetBSD 1.4.2 (RACKMOUNT) #0: Mon Mar 27 12:35:34 CST 2000     jdarrow@abigail.wheaton.edu:/var/src/sys/arch/i386/compile/RACKMOUNT i386

NetBSD 1.4V (ELF):

[487] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# jls
total 10
325256 2 drwxr-xr-x  2 root  wheel  512 Apr  4 17:25 ./
379456 2 drwxrwxrwt  5 root  wheel  512 Apr  4 17:00 ../
325269 2 -rw-r--r--  1 root  wheel  278 Apr  4 17:08 main.c
325265 2 -rw-r--r--  1 root  wheel  120 Apr  4 17:04 shfunc.c
325260 2 -rw-r--r--  1 root  wheel   31 Apr  4 17:04 shfunc.h
[488] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# mkdir .libs
[489] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# cc -c -O -fPIC -DPIC shfunc.c
[490] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# ld -Bshareable -o .libs/libshfunc.so shfunc.o
[491] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# mkdir /testlibs
[492] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# cc -c -O main.c
[493] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# cc -o main main.c -L./.libs -lshfunc -Wl,-rpath,/testlibs
[494] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# ldd ./main
./main:
         -lshfunc => not found
         -lc.12 => /usr/lib/libc.so.12
[495] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# cp -p .libs/libshfunc.so /testlibs
[496] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# mv .libs .notlibs
[497] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# ldd ./main
./main:
         -lshfunc => /testlibs/libshfunc.so
         -lc.12 => /usr/lib/libc.so.12
[498] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# ./main teststring
parameter is teststring...
Number of chars printed is 27.
[499] root@jdarrowpiii:ttyp5:/var/tmp/shlibex:# uname -a
NetBSD jdarrowpiii.wheaton.edu 1.4V NetBSD 1.4V (JDARROW) #0: Tue Mar 21 15:04:28 CST 2000     jdarrow@jdarrowpiii.wheaton.edu:/var/src/sys/arch/i386/compile/JDARROW i386