Subject: Re: pkg/17427
To: None <tech-pkg@netbsd.org>
From: David Burgess <burgess@neonramp.com>
List: tech-pkg
Date: 07/26/2002 17:25:34
> On Fri, 19 Jul 2002, David Burgess wrote:
>
>> If you can come up with a reason that required some thought on why
>> being able to update and upgrade system components based on the
>> systems in pkgsrc, I'd love to hear it.  Blowing me off is not going
>> to make this issue go away.
>
> Issues that are at all controversial don't belong in a PR -- they need
> to be discussed in tech-pkg first.

Sorry - I got the order wrong.  I thought we documented our requirements
and then argued about implementation.
>
> The problem is this: there's no consensus to do whatever it is you're
> asking for, therefore, the choices are 1) take no action, leaving the
> PR open forever, 2) deny it. We have way too many PR's already, that no
> one has the heart to outright deny, cluttering up the database and
> making it hard to pick useful tasks out of it.
>

There is a third option - include a mechanism that allows an individual
decide.  If we make whatever it is optional, then the guy at the end of
the line can decide.
One more important thing - I'm not recommending that every package make
it's way into /usr.  That's the way Linux people think, and I'm opposed to
it.  I'm specifically looking for a way to upgrade software, in place,
that are already part of the basic install.  Specifically, I'm talking
about system libraries, system executables, and the rest of the short list
of things that we pull in from other places.
I'm not sure what the replacement/supplimentary mechanism would be right
now, but there are at least three different ways (in order of simplicity)
that I can think of:
1)  Include a mechanism in the Install target of pkg.mk.conf that deletes
(or moves to *.old) the live system's executables from the Packing List. 
This is "pretty much" what I do now - I install the package in the
/usr/pkg tree and then delete the old executables by searching for 'bin'
in the packing list and deleting everything in the /usr tree.  A
syntacically correct version of the script "rm -f /usr/{`grep bin PLIST`}"
does this pretty simply.  Wrap an "#IfDef KILL_OS_EXES" around it and
we're there.  As a matter of fact, there is lots of prior art on this -
most of the packages we install do this (rename the old to whatever.old)
with their executables.  We could even extend the "uninstall" target to
'undelete' the files from the .old exile.
2)  Include a facility in the package Makefile that actually overwrites
the existing code base.  This would mean something like a flag that says
"Use /usr instead of /usr/pkg, /var instead of /usr/pkg/var, and /etc
instead of /usr/pkg/etc".  I've tried some attempts at this in the past,
and the closest I got was building a /usr/etc symlink to /etc and running
with $DESTDIR=/usr.  The same thing needs to happen if /usr/pkg/var files
are used, but you get the idea.
3)  Include a mechanism that overwrites the /usr/src source tree from the
installation script.  This way, the /usr/src tree agrees with the /usr/*
directories.  This is my least favorite, but is conceptually the best,
since the fidelity of the installed software base is based on the source
tree, not two competing sources.
> What I suggest you do, is make a clear appeal in this forum (tech-pkg)
> for exactly what it is you want pkgsrc to do. Listen to the pro's and
> con's, extend and revise your position, and *then* file another PR,
> summarizing the major points and referring to said discussion in the
> PR. This is way more likely to lead to some satisfactory conclusion
> than simply filing your position and hoping for the best.

As it stands now, there is no record in the PR database that making
something like this happen is a requirement.  If there's a better place to
put requirements, please let me know.
The justification for this is that not everything honors the $PATH I
carefully craft.  Putting /usr/pkg/bin before /bin confuses people and
they fire up their editor to "fix" my obvious error.  Unlike lots of other
people, I have dozens of servers, thousands of people on my servers, and
dozens of thousands of possible combinations.  I've also got an operations
staff that needs to have some consistency in how to invoke most of the
programs they use.  Having to remember that /usr/pkg/sbin/makemap is the
way to make a new sendmail database on half of the servers, and
/usr/sbin/makemap works on the rest is killing me, especially when
"makemap" is still on the system and first in the path.
Of course, we haven't even started in on the OpenSSL Library fiasco yet
(as a concrete example).  The OpenSSL library in the current system
doesn't include MD4 - no big deal, until one of the programs in pkgsrc
calls for it.  To solve the problem, I install the version from
/usr/pkgsrc (which is, IIRC, the same version).  From there, it's just one
mysterious library mismatch after another.  My IMP installations fell
victim to this one a few weeks ago and I'm still not sure how to fix it.
Next, we get into Cron jobs - As a default, the crontab path doesn't
include /usr/pkg until the end, so any management jobs I start up have to
be fully qualified to work.  This works fine until the next upgrade, when
I want to switch to the installed routine.  Of course, this really isn't a
problem since the installation procedure wipes out the root crontab.  User
crontabs are even more problematic, since I'm not likely to get their
concurrence on whether I'm repacing the named support programs they know
and love (for example).
Next, we get into the scripts that everyone loves to run.  Thank Heaven no
one has put PERL or PHP into the base install - that would KILL me.
Basically, I'm willing to talk about this at anyone's leisure.  Lots of
folks run one or two servers, so the current system isn't too bad.  My
staff and I run 24 NetBSD servers and service 4000+ accounts as an ISP.  I
also run anywhere from 30 to 300 packages per server (especially the web
servers).  Lately, every time I try to upgrade anything, I end up with at
least one version conflict on at least one program.  It just makes my life
miserable to have to hand massage the system whenever I install a package.
Anyway, that's a buffet of issues - who's next?

-- 
Dave Burgess
CTO, Nebraska On-Ramp
Chief Engineer, Mitec Internet Services
Bellevue, NE 68123