Re: Test failures

[Jukka Ruohonen <jruohonen%iki.fi@localhost>]

On Wed, Mar 07, 2012 at 04:20:08AM -0500, Thor Lancelot Simon wrote:
> I think it is excellent that developers are free to experiment and be
> creative so long as they do not leave the tree in a state in which other
> developers cannot work for long periods of time.  If you want a stable
> branch -- we have several, use them!  From my point of view, if you want
> the appropriate point at which to impose draconian requirements about what
> code can and cannot be checked in based on failure counts from automated
> tests, that point is when the code is designated as stable -- which, the
> way the NetBSD repository and processes are organized, would be when it's
> pulled up to a stable branch or when we declare HEAD frozen before a new
> branch is created.

Well I don't agree that committing working code is a "draconian requirement".
Rather, I would personally welcome automated checks that tell me that my code
has unexpected bugs (or perhaps that tests/rump/etc. need maintenace). As long
as I don't have to personally own that build server to commit.

The current "outsourcing of QA" usually means that people have to debug
things that are outside their area of expertise. As a anecdotal comment:
other people's freedom to experiment interrupted my NetBSD work for two
months by a regression that prevented me from even building a new HEAD. 
As a result my local changes did not make it to 6.0, which makes me sad.

But what is missing from these dicussion is the view from releng@. It is my
understanding that they'd want to have easier branching and pull-ups as well
as shorter integration/stabilization periods during branch time. This kind
of points to the real or perceived problems in HEAD. But yes, I guess the
used version control system is big part of the problem.

- Jukka.