proposal: Boost test package

[brook%biology.nmsu.edu@localhost (Brook Milligan)]

Over the years it seems that our Boost packages (devel/boost-* and
meta-pkgs/boost) tend to lag (sometimes greatly) behind the releases.
In part this may be a result of the complexity of testing Boost, as at
times a number of patches have been required for multiplatform
support.  Boost, itself, has a testing mechanism, but there is no
systematic and easy means of exercising it on machines running pkgsrc
and thus no efficient way to verify pkgsrc patches prior to either
incorporating them into an updated set of packages or submitting them
upstream.

I would like to suggest the following sort of package, perhaps called
'devel/boost-test-suite' but names are welcome.

- This package would install a script (perhaps
  $PREFIX/sbin/boost-test-suite) and a few bits (perhaps in
  $PREFIX/var/boost) useful for automating the running of the Boost
  test scripts.

- When run (ideally by cron), the script could (depending on options)
  download a set of patches for Boost from a repository and run the
  Boost test script, reporting the results back to the standard place
  (results are summarized at
  http://www.boost.org/development/tests/trunk/developer/summary.html).

This would enable a number of testers with different platforms to
verify both Boost itself and a common set of patches under evaluation
for updated packages.  Upstream developers would be much more likely
to take our patches because they could immediately see their effects
across many different systems.  We could put out a new set of packages
very easily for each release, because the patches would already be
well tested.  Indeed, perhaps we could even eliminate the patches by
getting them incorporated prior to releases.  I have had some success
with the latter, but broader testing would really help.

I have a few questions:

- Is this approach (i.e., a package to run the test suite with/without
  a set of standard patches) a good idea?

- How should the repository of patches be maintained and what is the
  best way to retrieve its contents each time the script runs?  One
  idea is simply to use the patches directory for the package itself
  and download it via anoncvs or something.  Is that appropriate?

- Is there a community willing to install such a package and run the
  script on some (at least) semi-regular basis?  If so, what
  representation of platforms can we cover?

I would like to see our Boost packages better track their releases
with fewer patches.  I hope others agree and have suggestions on how
to improve my ideas to make it work.

I look forward to your ideas.  Thanks for your input.

Cheers,
Brook