Re: xfail: expected failures

[Julio Merino <jmmv%NetBSD.org@localhost>]

On Fri, Jun 25, 2010 at 4:28 PM, Antti Kantee <pooka%cs.hut.fi@localhost> wrote:
> On Tue Jun 22 2010 at 23:07:55 +0100, Julio Merino wrote:
>> On Wed, Jun 16, 2010 at 9:49 PM, Antti Kantee <pooka%cs.hut.fi@localhost> 
>> wrote:
>> [...]
>> > Xfail will be present in atf 0.10 (according to Julio ;).  The current
>> > implementation is a big binary hammer, meaning it is not possible to
>> > specify where the test is expected to fail.  This may change for the
>> > 0.10 release.  If anyone has any other ideas about what they would like to
>> > see in the this department, I guess now would be a good time to speak up.
>>
>> Please take a look at this design proposal I have written for the feature:
>>
>>     http://www.julipedia.org/projects/atf/trac/wiki/DesignXFail
>>
>> It is a bit different from your current implementation but I think
>> covers all use cases that we discussed.  Comments?
>
> Looks good, but it doesn't cover timeout (which I missed in the original
> implementation).  Notably, timeout doesn't work either if you run the
> test by hand.

What do you mean?  Expect a timeout?

It's true that timeout doesn't work without atf-run, so I won't bother
too much if the xfail features don't work 100% (they can't) without
atf-run either.  I think running tests directly through the program
should be discouraged except for debugging reasons.  (But atf-run's
interface needs to improve to make this nice enough.)

> I find the name atf_tc_expect_pass() confusing, though.

How so?  I find it pretty representative compared to the other cases ;-P

-- 
Julio Merino