Re: current build failure automated messages

[Andreas Gustafsson <gson%gson.org@localhost>]

Paul Goyette wrote:
> For me, I'd be interested in another message that detailed changes in 
> the sets of Pass/Fail ATF tests.  The "New test failures" and "Tests No 
> Longer Failing" lines are useful for me.

The "new test failures" part I have already implemented some time ago.
I have been testing it by running it on my personal testbed with the
emails going to myself, and I have been forwarding them to the
relevant committers as needed.  I do intend to deploy it on b5 with
the emails going to current-users eventually.  New test failures
actually happen pretty rarely compared to build breakage anyway.

The hard part of this is filtering out spurious reports from tests
that fail randomly.  My current heuristic is that a test has a "new
failure" if the last three runs all failed and the 20 or so runs
before that all passed, and this seems to be working pretty well.
-- 
Andreas Gustafsson, gson%gson.org@localhost