On Fri, 5 Nov 2010, Brett Lymn wrote:
On Thu, Nov 04, 2010 at 06:46:48PM -0700, Paul Goyette wrote:Your reasoning seems to imply that, if we were to change libcurses to use a different, more optimum character sequence to generate a specific screen appearance, that would be considered a regression. I disagree with that reasoning.well, so do I. I am not sure where that implication came from. If someone changes the code to optimise the character sequence then that is excellent - the test should be updated to reflect the new output.
I guess this is where I disagree. The test, in my opinion, should be verifying that the desired results - visual appearance - are still achieved.
My understanding of regression testing is that you have a test that verifies that correct _results_ are generated, even if you change, fix, modify, or otherwise update the code that produces those results. Thus, the test should not need to be updated to keep in synch with changes in the library's implementation.
errr... I really cannot understand this - we have a chunk of software that should, modulo bugs, react the same every time you feed the same calls into it you should get the same output. People seem to be implying that curses randomly switches outputs around from run to run. Believe me, it doesn't. I have been helping maintain curses for quite a while, I do know a fair bit about the guts of it. What I have done is based on a reasonably good grasp of what goes on inside the library. I am confident the methodology is sound.
Definitely not suggesting that the library will randomly decide which sequences to use from run to run. But I _AM_ suggesting that developers should be free to change their decision of which sequences to use from version to version, and appropriate regression tests should be able to determine that, regardless of which sequences are used, the resulting visual appearance has not changed.
Changing the test procedure to keep it in lock-step with the specific implementation of that-which-is-being-tested does not, in my mind, generate meaningful test results.
Just my humble opinion, and probably not explained very well. But it is about the best I can do.
------------------------------------------------------------------------- | Paul Goyette | PGP Key fingerprint: | E-mail addresses: | | Customer Service | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com | | Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette at juniper.net | | Kernel Developer | | pgoyette at netbsd.org | -------------------------------------------------------------------------