Subject: managing NetBSD src (overview)
To: None <current-users@NetBSD.ORG>
From: VaX#n8 <vax@linkdead.paranoia.com>
List: current-users
Date: 08/18/1996 16:40:34
I'm interested in how people are integrating NetBSD source and local mods,
patches, and replacements.

I am brainstorming here and would welcome any suggestions.
I have a script I use to fix up a /usr/src, but it could be improved.
Many of these thoughts can be applied to packages as well.

Here's a few situations to consider:

1   You may have a custom version of a program like "file" or "passwd",
    or a more recent version of some critical program (sendmail, gcc).
    Or, you simply have stuff to add to the system, like a kernel config
    file.
    To consider; what if NetBSD catches up and/or surpasses your revision?
    Also, what about making sure that relevant patches from the standard
    version get applied to yours?
1.1 put it in /usr/local and be sure to call it instead of the standard one
    (annoying, sometimes not very useful, but simplest)
1.2 compile it somewhere else, but install over the standard one
    (obviously you must disable installation of the standard one
    or install new one after standard sources)
1.3 integrate it into the NetBSD build system and
    copy the source in place of the standard one after copying to /usr/src
    This is how I handle kernel configs.
1.4 same as 1.3, but keep on a local branch
    in this case, you might be able to migrate relevant patches to standard
    sources; or, it might be a big hassle to implement if your custom code
    has very different file structure

2   You have your own patches you wish to apply
2.1 apply them to the stock NetBSD source code after copying to /usr/src
2.2 track NetBSD sources in CVS and keep your patches on a local branch

3   You have patches being tracked in CVS to apply (e.g. ipfilter)
3.1 run installation/patching scripts from that module (e.g. kinstall)
    after copying source code into /usr/src
3.2 periodically install them into your local branch;
    depending on setup, this could be annoying and/or time consuming

4   You wish to make certain administrative changes (e.g. removing SUID
    from rdist and other programs, removing bogus programs from build
    process completely)
4.1 implement as a script to operate on /usr/src just before build
4.2 keep as changes on local branch
    (disadvantage; could require some maintenance if the Makefiles change
     much)

Random Thoughts:

When dealing with a build system like this, you can make modifications in
two ways.  You can change the system itself (e.g. the "local branch" stuff
above), which has the advantage of being seamless but the distinct
disadvantage of requiring maintenance that is probably linear with
the volatility of the code your are tracking.  It is not clear exactly how
updates are handled when you are merging two systems (e.g. NetBSD and ipfilter)
in this way.  The other way is to build a meta-build system which wraps
around the code in question to prepare it in various ways.  This has the
disadvantage of being another system to worry about, but the advantage of
being able to merge multiple systems cleanly (e.g. by operating on them in
a certain order) - the way OpenBSD puts a makefile wrapper around GNU tools.
A middle ground is to use some kind of script to do your modifications to
a system, like the way NetBSD now runs a PERL script on the gcc sources.

Many people implement this meta-build system manually (e.g. copy NetBSD
source in, run ipfilter's kinstall, build NetBSD sources, install), which
is error-prone.

Tracking some systems with CVS can be space-consuming, and you may need to
run scripts to move files around to convert from one file structure to
another anyway (c.f. gcc/NetBSD sources).  Plus, it's really nice to
track the sources in their original form.

There appear to be two kinds of updates; "push", and "pull".  In a push
update, you go and grab the source code to the latest package foo, compile
it, and install its binaries (completing the update).  In a "pull" update,
attempting to update the program could trigger FTPing the latest copy
(this is evidenced in FreeBSD's package system), or something as mundane
as exporting the CVS-controlled sources into the current build area.

In some ways, the "push" update corresponds to the "meta-build" scenario
above; there is no mappings from the build system to it's source.
For example, the sys/netinet directory has no knowledge that it should
be mangled by applying ipfilter patches to it. (*)
The target doesn't need to know how it was made.  On the other hand,
something needs to have this knowledge, and in this case it is the admin
or the meta-build scripts.  Integrating it too closely disturbs source
control techniques (for example, you may start importing the NetBSD sources
from a new SUP server, but that fact shouldn't force you to make vast changes
in your CVS-controlled NetBSD sources).  However, it is important, since
most sysadmins think this way (they start with "I want to update sendmail,
how do I do it", rather than "I just downloaded this big tarball for no
reason, I wonder which programs it affects").

Let me know if this made sense to anyone.  I fear I may have expressed
myself more clearly than I think, so hopefully someone can help clarify
things.

(*) You too can anthropomophize with the best of them! Just send $9.99
    and get my 30-minute email, "anthropomorphization with no money down".