Re: #29: Implement race condition tests

["atf trac notification" <nobody%julipedia.org@localhost>]

#29: Implement race condition tests
--------------------------+-------------------------------------------------
  Reporter:  jmmv         |       Owner:  jmmv    
      Type:  enhancement  |      Status:  accepted
  Priority:  minor        |   Milestone:  0.11    
 Component:  MISC         |     Version:          
Resolution:               |    Keywords:          
--------------------------+-------------------------------------------------

Comment (by jmmv):

 Alright. While mocking up some code for this, I think that your two
 requests are complementary.

 On the one hand, when you are talking about a "race condition" test you
 really mean an "expected race condition" test.  Correct?  If so, we need
 to extend the xfail mechanism to add one more case, which is to report any
 failures as a race condition error and, if there is no failure, report the
 test as successful.

 On the other hand, the atf-wide mechanism to support how long the test
 should run for can be thought as a "stress test" mechanism.  I.e. run this
 test for X time / iterations and report its results regularly without
 involving xfail at all.

 So, with this in mind:

  * For a test that triggers an unfixed race condition, you set xfail to
 race mode and define the test as a stress test. Any failures are reported
 as expected failures.
  * For a test that verifies a supposedly-fixed race condition, you do
 *not* set xfail to race mode, and only set the test to stress test. Any
 failures are reported as real failures.

 These stress test cases implement a single iteration of the test and atf-
 run is in charge of running the test several times, stopping on the first
 failure.

 Does that make sense?

 Please reply through the web interface.  I don't have hooks set up to
 process email replies to atf-log, which means that the email-only replies
 get lost :-(

-- 
Ticket URL: <http://www.julipedia.org/projects/atf/trac/ticket/29#comment:6>
Automated Testing Framework <http://www.NetBSD.org/~jmmv/atf/>
Automated Testing Framework