Subject: Re: more work in rc.d [was Re: rc, rc.shutdown proposed change]
To: None <tech-userlevel@netbsd.org>
From: Greg A. Woods <woods@most.weird.com>
List: tech-userlevel
Date: 03/17/2000 21:44:50
[ On Friday, March 17, 2000 at 03:44:09 (-0500), der Mouse wrote: ]
> Subject: Re: more work in rc.d [was Re: rc, rc.shutdown proposed change]
>
> > [T]his is entirely about making a better system that's more flexible,
> > more reliable, easier to manage, and more maintainable.  In all of
> > those senses it is a step forward (and a big leap forward on some
> > fronts!).
> 
> More opinion being stated as fact.  I think I've been careful to keep
> my opinions marked as opinions; please do me the courtesy of doing
> likewise.  (If I've slipped up on this, please call me on it.)

Pardon my so-called opinions, *but* I think there's been a lot of at
least semi-expert testimonial appear over the past few debates on this
topic to speak to the fact that my opinions are at least well founded
opinions if indeed not exactly facts on their own, yet.  So far all I've
really heard in terms of concrete claims by those such as yourself who
oppose the new scheme is that they don't like change and they especially
don't like changes that are forced on them by surprise.  Change is the
only constant and this change was discussed in at least those forums
where any discussion on it would be worth anything.

Certainly there are people who get to like doing things in one
particular way and are very uncomfortable with change.  I'm not trying
to say that their opinions are not important but I am saying that no
matter what one does to improve something they care about, the change
itself is all they're going to see and it is not going to be met with
much enthusiasm regardless of how solid the proof is that real benefit
results.  Some people would indeed rather chop their own wood and grind
their own flour no matter how much extra work it takes to do things that
way.

It is also important to remember that all other "ground shaking" changes
in -current are usually only mentioned after they've been committed in
current-users anyway.  Certainly if this kind of change had been slipped
last-minute into a release (eg. 1.4.2) there'd be reason to complain but
I can't see any reason whatsoever in complaining about the "surprise" of
such a change in -current.  If you really don't like it then obviously
you can complain -- just don't complain that it was forced down your
throat.  As an "outside developer" I can't say I'm entirely happy about
this approach either, but it is a long established tradition by now.

> More reliable?  It's new code, therefore almost axiomatically buggier.
> There's more of it -> almost axiomatically buggier.  It involves more
> steps -> more pieces that can break -> almost axiomatically buggier.

Yes, indeed there are bugs in the first incarnations of this system.
However by design they will be more reliable in the longer run.  They
can hardly be anything but, especially given the fact that they are
broken apart into separate components that isolate the effects of
failure in one from the others.  I'm sure the inevitable "newness" bugs
will be squashed and most of the wrinkles will be at least smoothed over
(if not ironed right out) long before this stuff gets to the public in
the form of an official release.

That said of course it would be nice if the current failure handling
mechanism was slightly enhanced so that a service could be marked as
critical and that total system failure would only occur when a critical
step fails, but boot to multiuser could continue if a non-critical one
fails.

> Easier to manage?  This is entirely a judgement call by the individual
> sysadmin.  As I said, I've used monolithic systems and I've used
> split-up systems, and on the latter I constantly find myself feeling
> I'm fighting the system, rather than working with it.  (One could argue
> that it's not the split-up nature of the scripts but the particular
> implementation thereof.  In theory this could turn out to be
> correct...and in theory, I could inherit six million dollars tomorrow.)

You've agreed several times now to the fact that automated management is
now at least possible and clearly that makes the overall management
of the system much easier too (at least if you buy the premise that once
you've automated some part of a larger task then the entire task becomes
that much easier to perform).

Several people have also pointed out that it's easier to give more naive
people one-step instructions for doing various tasks.  Regardless of
whether that's true or not I know that it's better to have a single
consistent and easy-to-remember way of doing common things to often
disparate programs and subsystems.  I can also assure you from long
experience that it's much easier to tell people to do something like
"/etc/rc.d/inetd reload" than it is to tell them to read the manual page
to learn what signal triggers a reload and to send that signal to the
running "inetd" process; and it's still easier than telling them to do
something more mechanical such as "kill -1 `ps -a | grep inetd | awk
'{print $1}'`"; and I think it's even easier and more productive than
telling them to do "kill -1 $(cat /var/run/inetd.pid)".

If you feel like you're fighting this system then you probably are, but
what's important is to ask the question "why?".  If the only answer you
can come up with is "because it's different from what I'm accustomed to"
then I think you can guess what I'd suggest.  Any other answer will no
doubt be better applied to refining the new system than to being used as
an argument to kick it out again.

> More maintainable?  Certainly not by me.  Even on the reasonably
> objective merit of sheer size, "wc /etc/rc" on my monolithic (pre-rc.d)
> system is 473 lines.  "wc rc.d/*" in src/etc in my supped-to-current
> tree reports 2086 lines.  337 of those are in rc.d/network, it's true -
> but pre-rc.d netstart is only 181 lines.  That you can sit there and
> with a straight face try to claim that over two thousand lines spread
> over 74 files is more maintainable than 654 lines in two files is
> utterly astonishing to me.  (There is an rc.subr in the pre-rc.d
> system.  It's all of 15 lines long.  If you want to count it in, things
> get even worse: 669 lines in three files, versus 2569 lines in 75.)

There's a heck of a lot more to maintainability than just counting lines
of code.  The new system is, in my *opinion* thats based on many years
of experience, much easier to understand, especially from the point of
view of someone who only needs to add, change, or delete one particular
component.  Obviously there needs to be better documentation developed
for the new system but that's supposed to be an opportunity not a
road-block.

Indeed given the additional functionality that's presented by the new
system I would have to say that it's rather elegant and one could in
fact be pleasantly surprised at the relatively small increase in lines
of code!  You can argue all you want that the additional functionality
wasn't necessary for your purposes and that as a result the additional
code is bloat and thus a maintenance headache for you, and you may very
well be right for your specific circumstances, but I don't think very
many other people will agree with you for the general case.

The other issue with maintainability here is the fact that monolithic
blobs of code are impossible to localise on a microsopic internal basis
(shell scripts or not) without destroying the possibility that they can
be automatically upgraded.  Now at least localisations can be isolated
to much smaller components and far less impact will be felt on upgrades
as a result.  (Having just gone through a one-year leap forward in a
"production" -current system that happened to include conversion to the
new rc.d system I can assure you that courtesy of the retention of the
rc.conf style of control the new rc.d system is the least of your
worries in merging your localisations into a new /etc.  It is a tiny
fraction of the total effort required for any significant general
purpose machine and it is in fact entirely and easily automatable!)

> Ah yes, once again "you don't need to care about that, it just works,
> trust us".  I'll thank you to stop trying to tell me what I do and
> don't need to know about my systems!

It never fails to make me wonder why people will get all up in arms
about silly sys-admin details and yet take for granted the thousands of
obscure lines of code that make some of the most intricate and important
internals of the system function the way they do....  Certainly you can
see and feel the sys-admin things much more closely, and if your job is
to maintain systems then such things might affect you far more than the
rewrite of a filesystem or the VM system but I still don't see the
difference.  In each case there's a damn good chance that any such
change will have been very closely considered by persons who've taken
into account as wide a set of requirements and ways of doing things as
they can without getting totally stuck on inevitable conflicts and other
road-blocks.

Being "*BSD-like" is not, IMO, about the sys-admin crud on top of the
system -- it's about the innards and the APIs!  The *BSD crowd, when I
was not de facto a member, always seemed to be so caught up in the
innards of their systems that they never paid enough attention to making
the system administration tasks "production quality".  Now before anyone
tries to claim that "production" and "research" systems are not
compatible in their goals let me remind you that much of what's done on
a research system has nothing at all to do with the sys-admin details in
the first place and indeed the more attention that is paid to good
production-quality sys-admin the more reliable and productive the entire
system can be even when it's only used as an OS research platform.  Also
please remember that research into what it takes to build a high-quality
production system is just about as important as research into networking
protocols or filesystems.

> There didn't used to *be* any repetitive details - or at least no more
> than any repeated if clause is repetitive, and if that's repetitive, so
> is much of what is even now present in rc.d/*, like dotting rc.subr and
> rc.conf, assigning to command and name, etc.

Please don't try to count sourcing files containing shared components as
"repetitive" in that sense.  That's mis-direction of the worst kind.

Perhaps you're also forgetting the repetitive manual tasks that were
implied by the old system too....

Given the trade-off between what's now possible in terms of additional
functionality (status/stop/reload/etc) and of course the far more
flexible dependency controls I think the extremely small amount of new
code is well worth having.

-- 
							Greg A. Woods

+1 416 218-0098      VE3TCP      <gwoods@acm.org>      <robohack!woods>
Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>