Re: ATF tests still failing to complete within 2 hours on amd64

[Alan Barrett <apb%cequrux.com@localhost>]

On Fri, 02 Dec 2011, Thor Lancelot Simon wrote:
> If the machine is in fact working correctly and the code lacks 
> severe bugs that would impact system security, there is a 99.99% 
> chance that the machine will behave normally at any given 
> startup, and a 0.01% chance that it will panic and reboot, 
> then behave normally (okay, okay, the conditional probability 
> that it will _not_ then reboot normally, but rather panic 
> again, is 0.0001%, assuming I counted my zeroes right...) 
> That seems entirely okay to me -- and, given that the test is 
> conditionalized on DIAGNOSTIC, those who prefer an enhanced risk 
> of false negative to the small risk of false positive are free 
> to do what they usually do: turn DIAGNOSTIC off.  What's the 
> problem?

I missed the "conditional on DIAGNOSTIC" before.  I would prefer 
tests for working hardware to be unconditional (or conditional on 
some new option), and tests for working software to be conditional 
on DIAGNOSTIC.

Also, could we improve the accuracy of the test?  For example, 
if the test fails then run another slower but better test, and 
reboot only if both fail?  Or if the first test fails then re-run 
it a few times, and reboot only if at least two of the iterations 
fails?

--apb (Alan Barrett)