Re: CVS commit: src/tests/ipf

[Jukka Ruohonen <jruohonen%iki.fi@localhost>]

On Tue, Mar 27, 2012 at 03:38:50PM +0200, Alan Barrett wrote:
> >Mark the failing tests as broken. XXX: If no one is willing to maintain
> >the ipf tests, these should be removed.
> 
> I object to this.  If ipf fails its tests, then the fact should be 
> made clear in the test reports, not hidden by disabling the tests. 

Indeed.

But only the maintainer knows whether these are "real" bugs or bugs in the
tests. I don't know whether ipf tries to maintain binary or configuration
compatibility, which seems to be the root of the failures. Frankly, I am not
sure even on what is being tested (thus why all tests should be clear and/or
heavily commented).

> I don't know whether the bugs are in ipf or in the tests, but 
> either way, removing or disabling the tests seems to me to be 
> counter-productive.

These are not disabled but marked as "bogus". The reports contain a message
about the supposition that the "test case is probably broken". Apparently
someone else has also reached the same conclusions, given that there was even
a specific function to mark ipf-tests as bogus.

- Jukka.