[GSoC] Atfify - mid-term report

[Lukasz Strzygowski <qx89l4%gmail.com@localhost>]

Hello Devs,

This is a brief mid-term summary of the GSoC project - atfify [1].

I started my work by creating new tool for ATF, called atf-check, to
replace original function atf_check (atf-sh-api(3)). It's much more
convenient and flexible and it should completely replace old atf_check
in main branch of ATF soon. I'm using it in all tests written in sh.

So far I have converted test suites for:

* tools:
 - awk 
 - basename
 - bzip2
 - config
 - cut
 - dirname
 - grep
 - gzip
 - m4
 - make
 - sdiff
 - xlint

* libraries:
 - libm
 - libposix
 - libprop
 - libpthread
 - librt
 - libskey
 - libutil

* other:
 - header files (bitstring, stdint, bootblock)
 - csu code (support for init/fini sections)

I've also created wrapper test suites for libcrypto and libevent, because
they are 3rd party software.

Right now I'm working on tests for libc. In original schedule I planned to
finish them before Jul 14. I hope I'll finish that part this week, though.
After that I'm going to start converting tests of kernel (src/regress/sys).
For the end of GSoC I'd like to convert most, if not all, of src/regress.

Some random problems that I encountered:

- Test suite for sort(1) in most test cases uses non-stable sort, so even
  if sort's output differs from expected, it still doesn't mean that it's
  incorrect. I decided to switch to stable sort instead (which is default
  in NetBSD's sort). I've also dropped test cases for features that aren't
  present in our sort.

- Calling atf_tc_fail() is not thread-safe. I don't know yet how to call
  ATF_CHECK_* macros from multiple threads in test suite for libpthread
  without using libpthread to protect them.

- Figuring out what some tests check was problematic. Usually
  I looked for explanations in commit messages and public reports.

- My exam session overlapped with SoC, fortunately it's already over.

Downloading sources:

- You can download updated version of ATF from [2]. New branch is called
  org.NetBSD.atf.src.check.
- Tests are available in CVS repository on SF.net.

1. http://netbsd-soc.sourceforge.net/projects/atfify/
2. http://www.netbsd.org/~jmmv/atf/repository.html

Kind regards,
Łukasz
-- 
Łukasz Strzygowski <qx89l4%gmail.com@localhost>