Jeremy C. Reed wrote:
> It apparently works for the developer who made the commits (that break for 
> someone else).

This reminds me of a question that has been nagging me for a bit: how
many distinct platforms does a typical developer test on before committing,
or have access to for testing at all? I, for example, have only i386 hw
here.

Perhaps NetBSD should have KNOWN_GOOD_<port> tags added independently
as versions are determined to work on different ports.

The trick would be a good definition of the tag semantics. KNOWN_GOOD
is really a misleading name--it's more like not-yet-known-bad, and
has at least two important parameters: not yet known bad after testing
by how many users and for how long.  Perhaps the tag should be called
something like CONFIDENCE_1W100_macppc, added as soon as a version has
seen 1 week's use under normal workloads by 100 macppc users without noticing
any regression. It sounds like a good candidate for automation, with a
command like send-sr (for sending "success reports" rather than problem
reports :), and a receiving process that, upon receiving 100 1-week
success reports for <port>, would move the CONFIDENCE_1W100_<port> tag,
leaving a CONFIDENCE_1W100_<port>_PREVIOUS tag in its original place.
Should reports of a significant regression cause the tag to move back?
One would think the answer should be yes.

The idea obviously generalizes to a small family of confidence tags that
differ by number of users or duration of testing, and people could match
their level of risk aversion to the tag they choose to sync to.

The scheme would require send-sr to unambiguously know the date/time
(for -D in cvs) of the checkout from which the running system was
built, which could be kept in a file somewhere in the tree that is
updated by a commit-script whenever anything is committed anywhere.
In fact, that seems likely to be functionality we might already have
though I've not stumbled on it yet--do we have such a thing? send-sr
should probably also verify that the checked-out sources all correspond
to the recorded revision date.

I don't know enough about svn to guess if such a scheme would be easier
or harder in svn. It doesn't seem infeasible in cvs.

Thinking about it, I would actually call such a facility useful. Does
anybody else think so?  Any thoughts on what values for duration and
number-of-users would make reasonably meaningful tags?

-Chap