Re: Running tests with different path for Kyuafiles and binaries ?

[Julio Merino <jmmv%NetBSD.org@localhost>]

On Fri, Jun 8, 2012 at 12:48 PM, Jukka Ruohonen <jruohonen%iki.fi@localhost> 
wrote:
> On Fri, Jun 08, 2012 at 10:34:26AM -0400, Julio Merino wrote:
>> Another possibility that just crossed my mind (and that I think makes
>> a lot of sense atually) would be to teach "kyua test" about build
>> directories: allow the caller to specify a destdir/buildroot and use
>> that as the prefix to locate the binaries themselves (not the
>> Kyuafiles).  This would be more in line with make's behavior and some
>> other build tools I have used.  The issue here would be dealing with
>> programs that reference data files in the program's srcdir: if the
>> data file needs some kind of "build", then it will be in the
>> buildroot; otherwise, it will be in the source tree.  But we could
>> deal with that somehow!
>
> As I've ranted about the subject before, I can do it again. No, you usually
> do NOT want your tests to be embedded in the source tree.

I agree, and not.

To reinforce your point: a generic libc posix test suite would be an
example of the willingness to keep the tests separately.  But here you
are saying that the product you are shipping is the test suite itself
and that the tests are the end goal, which is just not true of most
software projects.

Another example would be, sometimes, system (integration) tests.  They
touch so many components that you probably want the tests separate
because there really is no place in the source tree to put them.

But to counter your point: unit tests are generally best keep next to
the source.  Having these separate from the source is a recipe for
getting broken/abandoned tests.

> Whether you are
> doing JUnit or ATF tests, you want a clear separation between the source
> code and the tests. In NetBSD a typical example would be third-party code;
> you don't want to pollute the genuine source code in any way. If you think
> about languages with namespaces or packages, the point should be even
> clearer.

However, this has nothing to do with what Emmanuel asked.  You are
free to organize your source tree and its corresponding test programs
as you wish.  There is nothing in Kyua that forces you to intermix
them.

-- 
Julio Merino / @jmmv