Re: lib/45143 (add -f to atf-run(1))

[Jukka Ruohonen <jruohonen%iki.fi@localhost>]

On Mon, Aug 15, 2011 at 12:50:03PM +0000, Julio Merino wrote:
>  >   If you have tests that are skipped because they don't work, the
>  >   necessary first step toward making them start working is to run them
>  >   anyway and see what happens.
>  
>  Uh... of course.  But if you are already attempting to debug a failing 
>  test case, it's equally easy for you to edit the code and remove what 
>  you think is faulty.  Using a "force flag" in this scenario provides 
>  zero value.
>  
>  And, "are skipped because they don't work" ?  Can you provide an example 
>  of that?  What kind of "don't work" are you referring to?

This has nothing to do with "broken tests".

But consider an example: tests can be skipped on purpose because these cause
a panic.  Perhaps this occurs on some emulator but not on real hardware, or
perhaps it occurs on arch-X but not on arch-Y.  You don't always have the
sources ready (cf.  dedicated test infrastructure systems or end-user
systems, as advertised in tests(7)). But you still want to see whether you
are affected.  Comparable case: if it is known that a test causes a panic on
some, largely unknown, set of machines, you don't even want to go and try to
"fix" the test in the source tree.

- Jukka.