Re: Importing Kyua into NetBSD

[Jean-Yves Migeon <jeanyves.migeon%free.fr@localhost>]

Le 01/09/12 22:25, Julio Merino a écrit :
> Hello,
>
> I would like to propose the replacement of atf-run and atf-report in
> the base system with Kyua (which, simply put, is "ATF v2") well early
> into the development of NetBSD 7.x so that there is plenty of time to
> flesh out any issues.  Because this plan requires some background
> information, and because it is composed of multiple steps that deserve
> significant explanation, I have written everything down in this wiki
> page:
>
>      http://wiki.NetBSD.org/kyua/import/
>
> I'll consider this page to be the "living status document" for this
> project, and will therefore assimilate any results of this thread into
> the text.  I'll also keep it up to date with any progress made.
>
> So, let's start the discussion.  Could you please take a look at the
> plan and provide feedback?
>
> Thank you!
>
> PS: Let's give 2 weeks of time for this initial discussion as the
> document says (but of course will extend the period if it proves to be
> necessary).

FWIW the steps look fine to me, however:

1) I would provide the two knobs (MKATF and MKKYUA) for builds:
- saves up build time if required, and allow anyone to build only on or 
the other
- make the transition a bit smoother if you want to deprecate ATF. 
Having KYUA protected with a MKATF knob is somewhat disturbing (at least 
to me :) )

2) Speaking about the "user validation period": what about tracking 
regressions between an ATF run and a KYUA run and comparing results?

ATF uses XML coupled with xsltproc to generate an HTML report, while 
kyua outputs HTML directly. What would be the best way to validate that 
no regression happened after an adjustment? Comparing HTML outputs 
"visually"? Parsing the ATF db and KYUA sqlite db manually? Something else?

--
Jean-Yves Migeon
jeanyves.migeon%free.fr@localhost