Re: How to integrate atf with autotools

["Ivan \"Rambius\" Ivanov" <rambiusparkisanius%gmail.com@localhost>]

Hello,

On Mon, Jul 12, 2010 at 11:31 PM, Julio Merino <jmmv%netbsd.org@localhost> 
wrote:
> Did you know that you can also create XML/HTML reports with atf right
> now?  atf-report can output to XML, and atf ships with sample XSLT/CSS
> files to process the output and generate a "nice" page.
Yes, I know. I first had to add ATF's dtd to the system catalog -
without doing it I received some warnings by xsltproc:

$ sudo xmlcatmgr add system
http://www.NetBSD.org/XML/atf/tests-results.dtd
/opt/local/share/xml/atf/tests-results.dtd

Then I instruct atf-report to generate xml report and transform it
using xsltproc:

$ atf-run | atf-report -o xml:atf-run.xml
$ xsltproc -o atf-run.xml /opt/local/share/xsl/atf/tests-results.xsl
atf-run.xml


>
>>> [...]
>>> or something like this.  You get the idea I hope :-)
>> Yes, as I said above, this is very cool!
>
> We'll eventually get there :-)  The route is not trivial but that's
> certainly what I'd like to reach not so far in the future.
Now ATF is version 0.10. In which version do you plan to include a
consolidated atf tool?

>> Question 1) check-atf and installcheck-atf targets contain duplicated
>> code. How to remove that duplication? I know it is more of an
>> make/automake question, but as I said early I am new to these tools?
>
> A common, portable trick in Makefiles is to do the following:
>
> [...]
I extended your example to the following:

atf_run_and_report = rm -f $${fifofile}; \
        mkfifo $${fifofile}; \
        cd $${dir}; \
        cat $${fifofile} | tee $${logfile} | atf-report & \
        atf-run >>$${fifofile}; \
        res=$${?}; \
        wait; \
        echo "The verbatim output of atf-run has been saved to" \
             "$${logfile}; exit code was $${res}"; \
        test $${res} -eq 0

check-local: check-atf
.PHONY: check-atf
check-atf:
        logfile=$$(pwd)/check.log; fifofile=$$(pwd)/check.fifo;
dir=${builddir}; $(atf_run_and_report)
CLEANFILES += check.fifo check.log

installcheck-local: installcheck-atf
.PHONY: installcheck-atf
installcheck-atf:
        logfile=$$(pwd)/installcheck.log; fifofile=$$(pwd)/installcheck.fifo;
dir=$(testsdir); ${atf_run_and_report}
CLEANFILES += installcheck.log installcheck.fifo


Now both 'make check' and 'make installcheck' work correctly and the
duplication is removed.

>> Question 2) The variable check_PROGRAMS contains the test programs
>> that will be executed during 'make check'. The variable tests_PROGRAMS
>> contains the test programs that will be installed and executed during
>> 'make installcheck'. In my case the two variables will contain the
>> same information. Is acceptable to assign $(check_PROGRAMS) to
>> $(tests_PROGRAMS):
>>
>> tests_PROGRAMS = $(check_PROGRAMS)
>
> This should be fine.  But note that check_PROGRAMS are only built when
> you do 'make check', not as part of a regular build.  Therefore, I'm
> not sure if 'make install' will work well.
If the test programs are define in check_PROGRAMS variable, they are
not installed with 'make install' - I verified this.

> You can just avoid setting
> check_PROGRAMS altogether because tests_PROGRAMS will be
> unconditionally built anyway.
Yes, this is the best way. Now I declare the test programs in test_PROGRAMS:

testsdir = $(exec_prefix)/tests/cryptoexamples
tests_DATA = Atffile

tests_PROGRAMS = test_seed_prng
test_seed_prng_SOURCES = seed_prng.c test_seed_prng.c

I am also surprised that test_PROGRAMS variable is not mentioned at
all in automake manual.

Regards
Rambius

-- 
Tangra Mega Rock: http://www.radiotangra.com