[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: failures with NetBSD
On Mar 24, 2008, at 19:35 , Martin Husemann wrote:
On Mon, Mar 24, 2008 at 06:43:51PM +0100, Julio M. Merino Vidal wrote:
before the conversion to ATF). I guess I really have to add an
"Expected failure" output for test cases (it's in the to-do list for
0.6), so that we can clearly mark those that we know are broken and
thus prevent confusion to end users.
I think "expected failure" is confusing - maybe add a "known to fail"
result and allow outputing some helpfull comment (like "PR bin/
I like the idea of the helpful comment. However, I think "expected
failure" is a widespread terminology and we should keep it. I have
seen it already in several test suites. The idea of this feature is
that, in the test case, you do some check that should fail and, if it
does, you abort the test with "expected failure". I.e. you must
still run the test and not omit it. "Known to fail" seems to
indicate that the test is not run, because we already know it will fail.
Which leads us to the need to also have unexpected passes, which are
the contrary of the above and are used to detect test cases that have
been unexpectedly fixed or their problems have been hidden.
Main Index |
Thread Index |