Re: Test failures

[Thor Lancelot Simon <tls%panix.com@localhost>]

On Tue, Mar 06, 2012 at 10:36:04PM +0200, Jukka Ruohonen wrote:
> 
> Fair enough. But most of the failures would be catched if developers would
> actually run the tests before committing.

Only if every developer ran every test before every commit, on, at least,
every Tier I platform.  And, unfortunately, I suspect that is a burden
which would lead many developers to simply not commit.  We're already
asking developers to do full system builds, which take the best part of
an hour; now you'd be asking them to maintain either a fully up-to-date
VM for each Tier I platform, or a private copy of an automated framework
to compile-install-test on such VMs.

Usually most developers (even I) don't screw up.  This is NetBSD-current,
not a release branch, and so I believe some amount of "screw up and fix
it later" is OK if it greatly speeds development.  And I believe that,
because most developer usually don't screw up, the current situation
does, in fact, greatly speed development, compared to forcing everyone
to run all the anita tests before every commit.  I think periodic runs
of the tests represent a good compromise between speed of development
and speed of debugging.

Thor