Re: xfail: expected failures

[Julio Merino <jmmv%NetBSD.org@localhost>]

On Sun, Jun 27, 2010 at 9:39 PM, Antti Kantee <pooka%cs.hut.fi@localhost> wrote:
> On Fri Jun 25 2010 at 23:58:21 +0100, Julio Merino wrote:
>> >> Please take a look at this design proposal I have written for the feature:
>> >>
>> >>     http://www.julipedia.org/projects/atf/trac/wiki/DesignXFail
>> >>
>> >> It is a bit different from your current implementation but I think
>> >> covers all use cases that we discussed.  Comments?
>> >
>> > Looks good, but it doesn't cover timeout (which I missed in the original
>> > implementation).  Notably, timeout doesn't work either if you run the
>> > test by hand.
>>
>> What do you mean?  Expect a timeout?
>
> Yea, or, well, more like expect the test case to hang.

So if the test case reaches its defined timeout, report it as passed?
(Is this the same as you "race condition tests" request?)

>> > I find the name atf_tc_expect_pass() confusing, though.
>>
>> How so?  I find it pretty representative compared to the other cases ;-P
>
> Does it have any other meaning that "xfail test will fail here"?  If not,
> why not name it accordingly.  If it does, it's not obvious to me from
> the name.

atf_tc_expect_pass() would be the default behavior; you should not
need to use it (unless you want to perform cleanup steps after a
failed check).  Basically, your test case goes like this:

body() {
   blah, blah blah

   some more setup steps

   atf_tc_expect_fail("See PR foo");
   ATF_CHECK(return_bogus_value());
}

-- 
Julio Merino