CVS commit: src/external/bsd/atf/dist

[Antti Kantee <pooka%netbsd.org@localhost>]

Module Name:    src
Committed By:   pooka
Date:           Wed Jun 16 15:17:37 UTC 2010

Modified Files:
        src/external/bsd/atf/dist/atf-c: tcr.c tcr.h
        src/external/bsd/atf/dist/atf-c++: formats.cpp tests.cpp tests.hpp
        src/external/bsd/atf/dist/atf-report: atf-report.cpp
        src/external/bsd/atf/dist/atf-run: atf-run.cpp
        src/external/bsd/atf/dist/tests/atf/atf-c: t_macros.c
        src/external/bsd/atf/dist/tests/atf/atf-report: t_integration.sh

Log Message:
Introduce expected failures to atf.  They can be used to flag tests
which are known to fail, e.g.:

        atf_tc_set_md_var(tc, "xfail", "PR kern/43456");

Expected failures do not count towards the ultimate pass/fail result
from the test run:

pain-rustique:39:~/<2>src/tests/fs/ptyfs> atf-run t_nullpts | atf-report
Tests root: /home/pooka/src/wholesrc2/src/tests/fs/ptyfs

t_nullpts (1/1): 1 test cases
    nullrevoke: Expected failure: PR kern/43456

Summary for 1 test programs:
    0 passed test cases.
    0 failed test cases.
    1 expected failures.
    0 skipped test cases.
pain-rustique:40:~/<2>src/tests/fs/ptyfs> echo $?
0

However, an xfail test which passes will count as a failure, i.e.
xfail inverts test case success/fail.  This way we can get a better
sense from the ultimate verdict of the NetBSD atf run by seeing if
there were any unexpected failures, i.e. new regressions.

This feature will be present in the upcoming atf 0.10 release,
possibly with finer grained control.

patch reviewed by jmmv


To generate a diff of this commit:
cvs rdiff -u -r1.1.1.4 -r1.2 src/external/bsd/atf/dist/atf-c/tcr.c
cvs rdiff -u -r1.1.1.3 -r1.2 src/external/bsd/atf/dist/atf-c/tcr.h
cvs rdiff -u -r1.1.1.3 -r1.2 src/external/bsd/atf/dist/atf-c++/formats.cpp
cvs rdiff -u -r1.1.1.4 -r1.2 src/external/bsd/atf/dist/atf-c++/tests.cpp \
    src/external/bsd/atf/dist/atf-c++/tests.hpp
cvs rdiff -u -r1.1.1.1 -r1.2 \
    src/external/bsd/atf/dist/atf-report/atf-report.cpp
cvs rdiff -u -r1.2 -r1.3 src/external/bsd/atf/dist/atf-run/atf-run.cpp
cvs rdiff -u -r1.1.1.4 -r1.2 \
    src/external/bsd/atf/dist/tests/atf/atf-c/t_macros.c
cvs rdiff -u -r1.1.1.2 -r1.2 \
    src/external/bsd/atf/dist/tests/atf/atf-report/t_integration.sh

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.