Subject: Re: Reimplementing the /usr/contrib directory hierarchy
To: Igor Sobrado <sobrado@string1.ciencias.uniovi.es>
From: Andrew Brown <atatat@atatdot.net>
List: tech-userlevel
Date: 01/02/2003 13:23:37
i'll try to keep this short...  :)

>Some movements happened in the past:
>
>  - named homedir has been moved from /var/named to /var/chroot/named

yes and no.  named typically always used /etc/namedb as its actual
home (ie, the place all its data was stored) unless you were running
it in chrooted configuration (though some people may have used
/var/named instead).  the /var/chroot/named directory is more in line
with the "standard" place for chrooted daemons to live.

note that named would then use /etc/namedb under *that* directory
(unless you, again, changed it).

>  - MPC6xx files (powerpc) were moved to their own directory

i can't comment on this one.  i never noticed and it was probably also
driven by a more uniform layout.

>  - setkey(8), and sysctl(8) were moved from /usr/sbin to /sbin

these changes were driven by the need to use these programs during the
boot process (when only / can be assumed to be mounted and /usr may
not be present).

>and even more complex ones:
>
>  - dlopen(3), dlclose(3), dlsym(3), and dlerror(3) were moved to libc.

that only makes things easier for you, since you no longer need to
include another library to use them.

>All those changes were good and I do not remember complains about it.

because they made simple sense.  moving stuff from one place to an
obviously better place doesn't tend to incur much debate.  making a
new place and putting stuff in it doesn't (to me, at least) have the
same amount of obvious sense.

>/usr/contrib allows maintainers to minimize the impact of running
>software not controlled by the NetBSD Foundation.  That software can
>change in an incompatible way.  I think that the key is that it is
>mostly software that currently is in strong evolution.

it can also add needless complexity.  let's suppose, for example, that
the openssl materials were moved to /usr/contrib.  that means there'd
be a /usr/contrib/bin (with the openssl binary in it), a
/usr/contrib/include (for the include files), and a /usr/contrib/lib
(for the libraries).  applications needing those libraries would need
to be relinked at the very least, since they wouldn't be in /usr/lib
any more.  applications needing to be recompiled would need to add an
include path and an rpath for the linker.  one might even be tempted
to put the contents of /etc/openssl under /usr/contrib/etc/openssl in
your scheme.  that would probably require even more pain on the part
of recompiling applications that weren't merely simple users of the
openssl api.

>On the other hand, /usr/contrib is clean and elegant.  This directory
>layout was available in the 4.4BSD/Lite and 4.4BSD/Lite2 releases,
>that is not a new feature but a part of 4.4BSD, and it is currently
>being used by some of the most important Unices (for example BSD/OS,
>and HP-UX).

but solaris uses /opt.  why should we not use opt instead?  or maybe a
/thirdparty directory?

>In other words, /usr/contrib directory can be in the default user's PATH
>environment variable (minimizing this way users confusion), but allowing
>users that do not want to use that third-parties software to remove
>/usr/contrib from their PATHs.

i can't think of anyone who's not going to use at least *some* third
party software, even they are their own third party.

>IMHO, what looks really confusing to me is installing customized versions
>of that software in /usr/local.  In this case, the operating system
>behaviour will depend on what directory is found first, and it will be
>worst that providing those packages in another directory.

if you're installign third party software in /usr/local, then you need
to adjust the environment accordingly.  that means either removing (or
making non-executable) system binaries that you have duplicated, or
noting which one should be in preferred use.  i don't like the
nslookup binary that comes with bind9 (which i installed over the
default bind distribution that accompanies netbsd), so i replaced it
(after installing bind9) with the nslookup from bind8.  no big deal.

>Most important points are that:
>
>  - /usr/contrib was standard, it was a part of the 4.4BSD/Lite
>    directory layout, and it is currently being used in some OSes.

"was" is the important word there.  

>  - Software has been changed (most times between /usr/sbin and /sbin)
>    in the past without receiving users complains on this matter.

typically because those who need to use it already have both
components in their path.  if i add any /sbin directory to my path, i
typically add all of them (/usr/sbin, /usr/local/sbin, /usr/pkg/sbin,
etc).

>Let me publish the list of software provided in two of those operating
>systems:
>
>  - 4.4BSD/Lite2, the original version released by the Regents of the
>    University of California in 1994:
>
>      - MH-6.8 in /usr/contrib/mh-6.8
>          (currently nmh in /usr/contrib/nmh-1.0.4 looks more reasonable)

we don't ship mh, but you can get it from pkgsrc which makes it live
in /usr/pkg.  unless you decide otherwise.  you can quite easily set
LOCALBASE to /usr/contrib if you so desire.

>      - kermit-5A, perl-4.036, rcs, flex-2.5.2, gzip-1.2.4, X11R5 stuff, ...

pkgsrc, pkgsrc, pkgsrc and /usr/bin, pkgsrc and /usr/bin, pkgsrc and
/usr/bin, /usr/X11R6.  the last one there is already properly
segregated.  i don't think that putting /usr/X11R6 under /usr/contrib
would be a win in anyone's book.

>Please, do not discard that proposal so fast!

i'm not.  i just wish to be convinced.

>There is a lot of possibilities provided by this small change that need
>to be considered carefully.
>
>Let me give another example: some users complain in the newsgroups about
>problems building the NetBSD kernel.  Sometimes they have changed the GCC
>release provided with NetBSD with an updated one.  We run into a problem:
>NetBSD users want the most recent stable releases of gcc and g++,
>but those releases are not automatically incorporated to NetBSD because
>those releases need to be *carefully* tested building not only the kernel
>but also the entire system.  NetBSD provides gcc with "cc" as an alias (link)
>to it.  I read in a lot of posts that "those users should install new
>compilers in /usr/local, renaming the binaries to something like "gcc-new".

we now have some wonderful cross-building toolchain machinery that you
can use to avoid this exact problem.  it should be noted that while we
are working towards making our code into something a "stock" gcc can
compile with no difficulty, this is not done yet.  as such, if you
install a stock gcc (or some other compiler), you should expect that
it might not be able to do the job.

>IMHO, it is a bad workaround.  Suppose now that gcc is installed in
>/usr/contrib, and that the Makefiles define 'CC = /usr/contrib/bin/gcc'
>(yes, a hardcoded path to the approved gcc release).  In this case, those
>users will be able to install the most recent gcc releases on their
>workstations and servers without breaking the system building process.

sure, but there's no guarantee that you will be able to build the
system with that compiler.

>Two ways to serve the old gcc to those users that want it as default
>compiler: either providing /usr/contrib/bin in the PATH, or setting
>up links to /usr/contrib/bin/gcc at /usr/bin/[g]cc.

another way would be for you to install each gcc compiler that you
want into a separate tree, eg, /usr/gcc-3.1, /usr/gcc-3.2, etc, and
then set your path accordingly.

>IMHO, less complains from users!  Now, their are able to run the latest
>stable releases of gcc and g++, and the operating system will continue
>using fully-tested older stable releases of gcc, for critical tasks
>like building a new kernel or an entire world.

from the perl man page:

       The Perl motto is "There's more than one way to do it."
       Divining how many more is left as an exercise to the
       reader.

anything that you can come up with that works for you is fine.

one way that i have used on occasion is to install as much stuff as i
possibly can and then add symlinks or tiny scripts in a single
directory so that when i invoke "foo", i get precisely the
implementation and version of foo that i want.  this can be especially
useful on, say, solaris where you have have packages installed under
/opt and /usr/local, and you want some things from /usr/bin, but some
from /usr/ccs/bin, some from /usr/xpg4/bin, and even some from
/usr/ucb.

-- 
|-----< "CODE WARRIOR" >-----|
codewarrior@daemon.org             * "ah!  i see you have the internet
twofsonet@graffiti.com (Andrew Brown)                that goes *ping*!"
werdna@squooshy.com       * "information is power -- share the wealth."