Current-Users archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: atf for libcurses

On Fri, 5 Nov 2010, Brett Lymn wrote:

On Thu, Nov 04, 2010 at 06:46:48PM -0700, Paul Goyette wrote:

Your reasoning seems to imply that, if we were to change libcurses to
use a different, more optimum character sequence to generate a specific
screen appearance, that would be considered a regression.  I disagree
with that reasoning.

well, so do I.  I am not sure where that implication came from.  If
someone changes the code to optimise the character sequence then that
is excellent - the test should be updated to reflect the new output.

I guess this is where I disagree. The test, in my opinion, should be verifying that the desired results - visual appearance - are still achieved.

My understanding of regression testing is that you have a test that verifies that correct _results_ are generated, even if you change, fix, modify, or otherwise update the code that produces those results. Thus, the test should not need to be updated to keep in synch with changes in the library's implementation.

errr... I really cannot understand this - we have a chunk of software
that should, modulo bugs, react the same every time you feed the same
calls into it you should get the same output.  People seem to be
implying that curses randomly switches outputs around from run to
run.  Believe me, it doesn't.  I have been helping maintain curses for
quite a while, I do know a fair bit about the guts of it.  What I have
done is based on a reasonably good grasp of what goes on inside the
library.  I am confident the methodology is sound.

Definitely not suggesting that the library will randomly decide which sequences to use from run to run. But I _AM_ suggesting that developers should be free to change their decision of which sequences to use from version to version, and appropriate regression tests should be able to determine that, regardless of which sequences are used, the resulting visual appearance has not changed.

Changing the test procedure to keep it in lock-step with the specific implementation of that-which-is-being-tested does not, in my mind, generate meaningful test results.

Just my humble opinion, and probably not explained very well. But it is about the best I can do.

| Paul Goyette     | PGP Key fingerprint:     | E-mail addresses:       |
| Customer Service | FA29 0E3B 35AF E8AE 6651 | paul at    |
| Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette at |
| Kernel Developer |                          | pgoyette at  |

Home | Main Index | Thread Index | Old Index