Subject: Re: /etc/default ickiness...
To: NetBSD Userlevel Technical Discussion List <tech-userlevel@netbsd.org>
From: Greg A. Woods <woods@weird.com>
List: tech-userlevel
Date: 09/23/2000 17:23:59
I'm going to be tedious and pedantic here again....  as always, delete
now if you don't care....

(note though that herein I hopefully clarify my arguments against
adding commented-out settings in /etc/rc.conf)

[ On Saturday, September 23, 2000 at 02:04:46 (-0700), John Nemeth wrote: ]
> Subject: Re: /etc/default ickiness...
>
>      I read the changelog, which is generally pretty complete, so if
> there was something that I needed, I would most likely catch it, and
> could deal with it, assuming I hadn't already.

I would bet that you're "one in a million" there.  The processes and
procedures suggested for upgrades must designed in such a way that
reading the changelog (be it /usr/src/CHANGES or netbsd-source-changes
postings) is not a requirement, nor even remotely something the admin is
advised to do!  (That's not to say that release notes shouldn't be
included which highlight those changes which may require special
handling by admins.)

> } The trick is in reducing the amount of information that you have to
> } manage from being *everything* to being just those items that make your
> } system unique from the distribution.
> 
>      This isn't feasible.  I work with too many operating systems, not
> to mention versions of operating systems.  I have to know exactly
> what's going on on a given system, not just how it differs from a
> default install.  NetBSD is far better then most systems for this, but
> not quite perfect.

Ah, John, we're, or at least I'm, talking *only* about NetBSD here.
Indeed the current scheme of /etc/default[s]/rc.conf and /etc/rc.conf
does *exactly* allow you to reduce the specified configuration items to
just and only those which make a given system unique from the virgin
distribution.  I.e. it's not only "feasible", it's already been done!

> } Meanwhile the trick employed by NetBSD with rc.conf (or in the past by
> } rc.conf.local) is immediately and instantly usable by any human to
> } understand what's been done to configure a particular system.
> 
>      As a general rule, I'm more interested in how a given system is
> currently configured then in how its configuration has changed.

Exactly.  From a certain point of view one can argue that in fact the
changes to the configuration from the default distribution are exactly
those items that define how the system is currently configured (at least
within the confines of what's currently defined in /etc/default[s]).

For example changes to scripts (or other programs) which have to be
reapplied after a system is configured are indeed configuration items by
definition.  Similarly the packages which have to be added again if
indeed the upgraded system doesn't have shared system libraries and/or
doesn't support the old executable format very well, etc. are also
"configuration items".  However these things are well and far outside
the scope of /etc/default[s].

> } > } There's literally no real difference between your /etc/orig and the
> } > } existing /etc/default (except that one could in theory make changes to
> } > 
> } >      Actually, there is a real difference.
> } 
> } What I meant was that fundamentally the result is exactly the same.  Two
> } systems configured with each method will behave identically.
> } 
> } The differences are not operational but rather in the techniques used to
> } manage them.
> 
>      We're not talking about operational status, but rather how to
> manage them, so this is a moot point.

I knew that using that phrase was going to cause a misunderstanding! :-)
I should have chosen my words more carefuly.  So, are you agreeing with
me, or disagreeing with me here?  :-)

Just to be pedantic and To reword again:  My original point, i.e. the
reason I said "there's literally no real difference", was that the end
result in the way the system operates (i.e. in what programs it starts
at boot and what options and parameters it sets for them) is exactly the
same no matter how you manage the configuration items that specify these
settings.

(maybe "fundamental" would have been better than "real"?)

> } The advantage of the /etc/default scheme is that you can swap original
> } /etc/default files around underneath the delta file (under controlled
> } upgrade procedures of course) and not necessarily have to do anything
> } else to retain the uniqueness of the system instance.
> 
>      Yes, you would.  Not checking out the changes to the defaults
> would be extremely sloppy.

Of course.  HOWEVER, there's every possibility that you won't find
anything of concern in the differences between old and new /etc/default
files and as a sresult "you will not have to do anything else to retain
the uniqueness of the system instance."

> } Certainly there are changes that can be introduced by the change of
> } system defaults, but those changes won't always have a negative effect
> } on the way a given instance of a system fulfills its requirements.  Say
> } for example that SSH becomes a built-in tool and perhaps the system
> } didn't have an add-on SSH on it already, so in that case the
> } introduction of a new daemon is probably a very positive and beneficial
> } change in the default behaviour of the system.
> 
>      On the other hand, if I had sshd started from rc.local and
> suddenly rc.conf decided that it wanted to start sshd then there would
> be a problem.  If rc.conf decided to do so with different keys or a
> different configuration file, then there could be a big problem.

Ah, please read what I wrote:  "and perhaps the system didn't have an
add-on SSH on it already"....

> } In any case simple deductive logic proves my point.  Without an
> } /etc/default style of system it's impossible to just upgrade and go
> } because the upgraded /etc/rc.conf file must always be re-edited to
> } re-introduce local configuration changes.  Therefor one must conclude
> 
>      There are a lot more files in /etc then just rc.conf.  I always do
> a recursive diff of the entire /etc directory.  This means that adding
> a few lines to rc.conf is in the noise of the overall process.

I don't agree.  There are very few other files that cannot be initially
correct for common cases and which still need upgrading and merging with
new releases.  I.e. files which contain multiple configuration items of
which only some subset might be changed for a specific system instance
and where new items or important new defaults might be added in a new
distribution version of the file.  *maybe* inetd.conf is one of these.
*maybe* /etc/group and /etc/master.passwd too.  At the moment I can't
think any any others where local changes are not really just bug fixes
that should be part of the new release.  If you generate your own custom
releases of NetBSD with all your site-specific defaults and all of these
"bug fixes" already "built in" then this is even more likely.  If you
know of other files that are affected by upgrades and may need "merging"
then let's talk about them separately because I'll bet they are not the
kind of files that can be put in /etc/default[s], at least not without
making changes to the way the programs that use them work.

In the mean time those items that are given defaults by files now in
/etc/default[s] are the majority of items that must be changed to make a
given system do what it's supposed to do *and* which given that they're
all now collected into a very few files which also need "merging" upon
an upgrade (since it is almost certain that those files will contain new
items and important new default settings which must be used if the local
configuration does not override them).  Given the syntax with which
these items are now defined the current scheme is probably the very best
stand-alone scheme for managing the "delta" between a virgin
distribution and a useful unique system instance.

> } that with at least the possibility of being able to upgrade and go
> } without having to reconfigure anything must in fact make the
> 
>      I have never seen this case in practice.  One could also say that
> I have a possibility of winning the lottery, assuming I was actually
> silly enough to buy lottery tickets.

You're talking about apples and oranges because in the past this scheme
has not been available to you, at least not with NetBSD.  I can guess
from my experience with upgrading systems already using these schemes
(eg. those already on the -current and 1.5-release branches) that this
is indeed *always* going to be the case for patch releases in the
future.  Maybe even for some systems on point releases.  Since we've
never yet seen a "major" release I don't know how to predict for it
yet...  :-)

Indeed lots of people have already upgraded through patch releases
without taking anything at all from the /etc set.  Didn't you yourself
even say you've done this?  This is just a more striking example of
upgrading a system without having to reconfigure it.  Adding
/etc/default[s] to the picture simply makes it easier to take changes
from the /etc set still without having to reconfigure the whole system
from scratch.

>      Okay, I see what's happening.  Although the lines for creating
> $$.oldconf and $$.newconf are exactly the same, you're changing
> /etc/default/rc.conf between them, and since that is sourced by
> /etc/rc.conf it means that these lines will have different outputs.
> This means that you're depending on a hidden side effect.  A very nasty
> thing to do.

The whole damn thing depends on side effects in the way the shell works!
I thought replacing a file in the middle was the least "unobvious" thing
I did there!  :-)

>  Obviously, a production version of this script would need
> to be heavily commented.

"Production"!?!?!?  ;-)  I scribbled that down off the top of my head
just to illustrate a point.  I seriously doubt anything like it would be
used in any production upgrade system.

-- 
							Greg A. Woods

+1 416 218-0098      VE3TCP      <gwoods@acm.org>      <robohack!woods>
Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>