Subject: HEADS UP: Automated Testing Framework (ATF) imported
To: None <tech-userlevel@NetBSD.org>
From: Julio M. Merino Vidal <jmmv@NetBSD.org>
List: tech-userlevel
Date: 11/12/2007 15:44:50
Hello,

I am very pleased to announce the initial import of the Automated Testing
Framework (ATF) into the NetBSD tree.  ATF is the project I started working
on during the Google Summer of Project 2007 program, mentored by Martin
Husemann, which reached the 0.1 release when the program ended.  It is now
at version 0.3, and I felt it was mature enough to be integrated.

ATF is developed as a separate component from NetBSD in the hope that it
will be useful to other developers, but is distributed under the The
NetBSD Foundation umbrella.  Given that ATF is just a framework, everything
else that is specific to the NetBSD operating system will be kept under
the NetBSD source tree exclusively.

For more details on the project, please refer to its web page:

	http://www.NetBSD.org/~jmmv/atf/

So what has been done now?

I have imported a verbatim copy of the ATF 0.3 release under dist/atf,
added the necessary reachover Makefiles all over the tree to build and
install it, and converted some subtrees of regress to the new
infrastructure.

An interesting point to mention is the addition of the tests.tgz set as
part of a NetBSD release.  This contains all test programs and accompanying
data files that make up the NetBSD test suite.  These are usable *without*
any developer tool (i.e. without comp.tgz).  sysinst has been modified to
allow its installation and afterboot(8) has been adjusted to tell new
users how to run the tests.

As a short summary, to run the tests on your machine (once installed), you
can do something like the following:

	# cd /usr/tests
	# atf-run | atf-report

Should you want to generate a nice HTML report to serve through an HTTP
server, you will need to have the xsltproc utility shipped as part of the
textproc/libxslt package in pkgsrc.  Then, do:

	# cd /usr/tests
	# atf-run | atf-report -o xml:/var/www/report.xml
	# cd /var/www
	# XML_CATALOGS=/usr/share/xml/catalog xsltproc \
	  -o report.html /usr/share/xsl/atf/tests-results.xsl report.xml
	# cp /usr/share/examples/atf/tests-results.css .

And access your report through the report.html page.  I am thinking of
ways to simplify this in the future.

For more details on how the tools work, start by reading the introductory
atf(7) manual page.  Unfortunately there is not much developer-specific
documentation in there yet, but the test programs I converted can be used
as a starting point; see the code in the new 'tests' subtree.

Be welcome to the all-testing world and...  enjoy ;-)

-- 
Julio M. Merino Vidal <jmmv@NetBSD.org>