Re: xfail: expected failures

[Antti Kantee <pooka%cs.hut.fi@localhost>]

On Sun Jun 27 2010 at 21:46:20 +0100, Julio Merino wrote:
> >> > Looks good, but it doesn't cover timeout (which I missed in the original
> >> > implementation).  Notably, timeout doesn't work either if you run the
> >> > test by hand.
> >>
> >> What do you mean?  Expect a timeout?
> >
> > Yea, or, well, more like expect the test case to hang.
> 
> So if the test case reaches its defined timeout, report it as passed?

I assume you mean "report it as xfailed".  In that case, yes.  Otherwise,
see first sentence.

> (Is this the same as you "race condition tests" request?)

No, race condition tests fail sporadically.

> >> > I find the name atf_tc_expect_pass() confusing, though.
> >>
> >> How so?  I find it pretty representative compared to the other cases ;-P
> >
> > Does it have any other meaning that "xfail test will fail here"?  If not,
> > why not name it accordingly.  If it does, it's not obvious to me from
> > the name.
> 
> atf_tc_expect_pass() would be the default behavior; you should not
> need to use it (unless you want to perform cleanup steps after a
> failed check).  Basically, your test case goes like this:
> 
> body() {
>    blah, blah blah
> 
>    some more setup steps
> 
>    atf_tc_expect_fail("See PR foo");
>    ATF_CHECK(return_bogus_value());
> }

So what happens if you have (note commented xfail):

body
{

  //atf_tc_expect_fail("bug without end FIXED");
  some_call();
  atf_tc_expect_pass();

  ATF_REQUIRE(false);
}

?