[ On , January 27, 2005 at 22:38:57 (-0500), Ian Lance Taylor wrote: ]
> Subject: Re: static vs. dynamic runtime linking, again (was: PAM and su -K)
>
> "Greg A. Woods" <woods@weird.com> writes:
> 
> > All AT&T UNIX System V Release 4 ports used only '-L',
> 
> This turns out not to be the case.  The SVR4 linker does not default
> the runtime search path to the -L arguments.

I didn't just rely on my possibly foggy memories of porting vast amounts
of software to SysVr4 over a period of about three years in the early
1990's to make those claims -- I re-checked the old manuals I happen to
have right handy here too and I'm pretty darn sure they fully
corroborate my memories, and they seem to do so, although that old
documentation is pretty sparse in places, esp. compared to modern
SunOS-5 manuals, and there never was a proper manual page in the AT&T
releases for the runtime dynamic linker.

First off note that the original SysVr4 link editor, as released by AT&T
and USL up to and including the last 4.2 releases, did not even have a
"-R" option to allow it to do otherwise (at least not for the i386
port), (though it appears now in UnixWare-7, as I see from SCO's online
manuals)....  It did have LD_RUN_PATH though to record a specific search
path in the output object.

However everything else about shared library searching refers only to
the '-L' options, LD_LIBRARY_PATH, and the default "LIBPATH" (which the
FILES section of the manual page describes as "usually
/usr/ccs/lib:/usr/lib") and all these other references are done without
stating any explicit difference between the roles of the link editor and
the runtime dynamic linker.  There's even suggestion that
LD_LIBRARY_PATH works at runtime too, though I seem to remember that
feature getting squashed well before 4.1 in a security patch from most
vendors since it was a major problem for setuid and setgid binaries.

Presumably LD_RUN_PATH overrode the stored path generated from the '-L'
options, though I always avoided the LD_* environment variables like a
plague, especially since some software build systems I worked with had a
tendency to occasionally use "make -e" when building.

There was also the "ld -YP" option to change the default search list,
but I don't know if it affected the runtime linker's search or not
(though it probably should have if it didn't).

In any case Suns running SunOS-5 are the only systems I remember ever
having any runtime library path problems with. 

Note that most of the systems I ported software to during that time were
official porting-base ports that were supposed to be very true to the
original AT&T/USL releases (e.g. including the Commodore Amiga release
for m68k, Fujitsu's own internal SPARC port, NEC's port, Convergent's
port, Pyramid's port, etc.).

Unfortunately I don't have SVR4 source handy any more, or even a running
system, for real proof either way....  :-)


This was all different than the way SunOS-5 does things now, which is,
as you no doubt know, described best by this from the "Solaris 1.x to
Solaris 2.x Transition guide":

  Specifying Library Search Paths

           In SunOS release 4.x, directories specified with the -L
           option were searched at link time and the information
           retained for use at execution time.  This behavior is now
           divided between the -L and -R options.  The -L option
           specifies the directories to search at link time; the new -R
           option tells the linker the search paths to be retained for
           use at run time.  See "Search Path Rules," in the next
           section for more information.

I think that it was, and obviously still is for some of us, a rather bad
idea to break backward compatability like that though.


My main point though was that everyone was showing der Mouse to the
documentation, but the documentation did not clearly say anything about
this rather drastic change of behaviour -- certainly nothing like the
Sun manual I quote from above.  Indeed given some of the other comments
made on this thread it seems there are several other folks who are are
also quite confused over exactly how it all works and how it's supposed
to be used.  And as I showed the current documentation is drastically
out of date in key areas related to this issue.

Even worse the libraries not being found were part of the installed
system and were in a standard system directory.  Having to add one '-L'
to find them is bad enough, but having to say the same thing twice to
even get a working result is just not right.

Worst of all though is this b.s. about "this is how ELF works and how
ELF has always worked everywhere" -- this has nothing to do with ELF per
se, it's just fallout from how the ELF feature of allowing a runtime
search path to be specified via the link editor that's separate from the
link editor's own search path has been presented to the developer.

I.e. it's just a bloody user interface issue, not something fundamental
to ELF, and the user interface we've ended up with SUCKS badly because
it breaks backward compatability for libraries in non-default
directories, _including_, in this case, those in the _standard_
system-supplied /usr/X11R6/lib directory!!!!

I thought this was NetBSD where one main goal "is emphasizing correct
design and well written code".  Separating '-L' and '-R' without
providing a decent default value for the new '-R' option when it's not
explicitly specified, and thus offering proper backwards compatability
for existing software build systems, is simply _not_ correct design.


> SunOS5 is Solaris,

Well, actually, no, SunOS-5 is a component of Solaris.  :-)

Solaris (2.x and newer) is/was the marketing name for a package
containing the base SunOS-5.x plus a windowing system, and it still is
despite the recent dropping of the "2." prefix, and/or leap from 2 to 7,
or however you might perceive it.

And the name "Solaris 1" was actually created (or at least published)
after the "Solaris-2" moniker was already firmly entrenched and it was
used to refer to the final SunOS-4.1.4 (plush OpenWindows) releases....

And you of all people should know this stuff -- you were around then too.


>  Perhaps you mean SunOS4, which did.
> SunOS4 used a.out, not ELF,

No, and I'm very well aware of SunOS-4 and its use of a.out.  I ran my
own SunOS-4 systems up until about 7 years ago and I've still got a
colleague who insists on porting some of my software to his still in use
SunOS-4 system.

(just for fun note that System V manuals document ELF in the a.out(4)
manual page :-)


> but the SVR4/Solaris linker, and ELF in
> general, is based on the SunOS4 linker.

Chickens and eggs.  :-)  Having spoken with UNIX Systems Labs folks and
AT&T developers many times, as well as a few Sun developers too, about
these kinds of things, there was often little agreement over who
originated what, unless it was the obvious things like RFS vs. NFS.


FYI, I am _very_ well aware of all the issues surrounding link-time and
run-time searching for dynamic objects, and I'm still firmly with der
Mouse on the fact that any dynamic link editor which does not default
the run-time path of a dynamic object to be effectively the same as its
link-time path is severely broken (e.g. by forming the runtime link path
from the '-L' parameters in the order they were given).  I just haven't
made as much of an issue about it very often before because I have been
aware of these issues and I've always been able to specify the right
"-rpath" or whatever.  Sun did not make the right choice on this, and
sadly it seems GNU binutils followed Sun along for the ride, and now
experienced developers like der Mouse are still tripping over this same
issue in NetBSD.

There's no valid reason been given not to default the search path to one
made up of '-L' parameters when no '-rpath' is given and every reason to
do so, especially since all new and updated software will likely build
with '-rpath' (or '-R') anyway, especially if it builds its own shared
libraries too.  This of course can only help with already installed
libraries that are not installed in the default system search path, such
as those in /usr/X11R6/lib on NetBSD, but it will definitely be better
than the current, broken, situation.


-- 
						Greg A. Woods

H:+1 416 218-0098  W:+1 416 489-5852 x122  VE3TCP  RoboHack <woods@robohack.ca>
Planix, Inc. <woods@planix.com>          Secrets of the Weird <woods@weird.com>