Re: How to integrate atf with autotools

[Julio Merino <jmmv%NetBSD.org@localhost>]

On Sun, Jul 11, 2010 at 5:48 AM, Ivan "Rambius" Ivanov
<rambiusparkisanius%gmail.com@localhost> wrote:
> Hello,
>
> I started a project using autotools as its build system and ATF as a
> testing framework. I am able to build the ATF-based test programs with
> autoconf/automake/configure, etc. I also created an Atffile and when I
> run manually "atf-run | atf-report" the tests are executed. Now I want
> to automate the execution of the atf tests using the default "make
> check" command provided by autotools.
>
> The most obvious solution was to use TESTS_ENVIRONMENT and TESTS
> automake variables.
[...]

Hello Ivan,

As you well say, you should not be running atf-run for every test case
separately.  If you do so, you lose the unified report created by
atf-run!  You must not also add atf-run to TESTS_ENVIRONMENT; doing so
doesn't fit the variable description (as that variable is exclusively
for environment variables) so things may break.

At the moment, integrating atf with autotools is a bit complicated.
There is ticket #23 to remind me of this issue, but I haven't given it
any thought so far.  As far as I know, automake can't be extended by
third-party packages... so it won't be really possible to provide a
clean way for integration.  There is also ticket #13 to come up with a
way of resolving the return code issue of "atf-run | atf-report".

I suggest you to take a look at the atf Makefile.am files to see how
the whole thing works.  You will need to create rules to install the
test programs as well as the Atffiles.  (OK, you don't *need* to
install any tests, but one of the features of atf is that it allows to
do so and for good reasons! Take a look at [1].)

If the atf makefiles are too complex for you to read, you can take a
look at the ones of etcutils, which are way simpler [2].  I think
these makefiles are out of date because they still use the obsolete
atf-compile utility, but they would give you a clear start.

Hope this helps,

1: 
http://www.netbsd.org/~jmmv/atf/docs/faq.html#faq-user-why-install-test-programs
2: http://www.netbsd.org/~jmmv/etcutils/

-- 
Julio Merino