Current-Users archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Test failures

Thor Lancelot Simon <> writes:

> Usually most developers (even I) don't screw up.  This is NetBSD-current,
> not a release branch, and so I believe some amount of "screw up and fix
> it later" is OK if it greatly speeds development.  And I believe that,
> because most developer usually don't screw up, the current situation
> does, in fact, greatly speed development, compared to forcing everyone
> to run all the anita tests before every commit.  I think periodic runs
> of the tests represent a good compromise between speed of development
> and speed of debugging.

I agree with your argument that very strict pre-commit testing and no
requirements for quality would both be worse situations.  And I agree
with your characterization that most people are pretty careful.  But, I
think the question can be framed as whether where we are is the right
place, or if we should be a little more (or a little less) careful,
rather than whether we should have far stricter norms.

As others have pointed out, periods of time when the tree doesn't build
and periods where there are massive test failures seem to lead to
cascading effects.  The build-broken-often problem does seem to be
milder now than last fall.

So perhaps where we are now should be modified by a norm that says:

  If you've made commits more than a few times in the last 6 months that
  break the build (even if it is "just setlists"), you really really
  have to at least build whatever you are working on (and perhaps one of
  i386/amd64/sparc64, if different) before committing.  This is
  especially true if you were not able to fix your previous breaks in
  the last 24 hours.

  Changes that are large in scope and don't obviously have a low risk of
  trouble should at least have a py-anita run or equivalent.

I realize that won't catch everything, but it should make the two
situations that seem to give rise to real trouble rarer, and only put
more work where it is actually needed.

Attachment: pgpGTQhjcW2ON.pgp
Description: PGP signature

Home | Main Index | Thread Index | Old Index