Re: atf for libcurses

[Brett Lymn <blymn%baea.com.au@localhost>]

On Fri, Nov 05, 2010 at 06:20:27PM +0200, Alan Barrett wrote:
> 
> It's deterministic given your constraints and also given a particular
> implementation or curses, but it can legitmately change when the curses
> implementation changes.
> 

correct.

> For example, if some line somewhere on the screen contains the text
> "abcde", and the cursor is on the "b". and you want to move the cursor
> to the "d", then any one of these would be acceptable:
> 
>     cup(x,y) -- move to absolute position of the "e" character
>     "bc"     -- overwrite all chars between old and new position
>     cuf1; cuf1  -- move right two positions
>     TAB      -- if you are lucky and the "d" is on a tabstop
>     etc...
> 

FWIW curses already does these evaluations in the plod() routine.

> If the test expects cuf1;cuf1, and later somebody improves the curses
> library to emit "bc" or TAB instead, then the test would fail, but the
> actual change to the code would be an improvement, not a bug.
> 

Tests failing doesn't have to be a bug - it indicates the output
stream has changed.  I am not sure in this case why that would be a
bad thing, it confirms the change has had the expected effect.  In
fact, I would have expected the test output should have been updated
to ensure that the correct results are emitted.  How else are you
going to validate that your change has had the desired effect?  How
can you confirm that it correctly switches between, say cup(y,x) and
cuf1 when the penalty for single character moves becomes too high?
Just analysing the screen appearance won't tell you this.

> I think everybody understands this, but some people draw the conclusion
> "therefore the tests should not be at this abstraction level", and other
> people draw the conclusion "therefore we will have to deal with updating
> the tests as the code changes".
> 

I understand that checking the appearance is the easy way out but it
does mean that aberrant behaviour will be hidden - such as the bug
with getch() echoing out an assembled key symbol.

One thing that can cut down the maintenance work is the fact that the
test language permits the inclusion of other test fragments using the
"include" directive.  This does mean that common things can be put
into a single file and all the other tests can reference that simply
by including a common test - an example of this is the curses
initialisation codes, they will be the same for every test so I
created a "curses_start" test that validates that startup sequence and
I include that test at the start of every other test.  Includes can be
nested up to a hard coded limit - the only reason there is a limit is
just a simple mechanism for stoppng circular includes blowing up the
test.  My thoughts were to use the include files to set up the basics
for a test and then use that as a basis to perform a unit test on a
function call.  Include files are just other test files, they can be
run in isolation so you can start with the basic tests, validate
those, working your way through more complex scenarios without having
an ever increasing check file size.  Now that I think of it, there is
no scoping on the variables set in an include file which could be good
and bad but, at the moment, it means that you can do something like:

include curses_start # checks curses init sequence
include curses_make_window # creates a window and checks sequence, sets $win1
call OK wborder $win1 .... # draw border around window
compare wborder.chk # check output of wborder call

So wborder.chk only contains the output from the wborder call,
everything else has already been validated.

Yes, it doesn't help if someone makes a major change to the guts of
curses that switches things about in the output stream but at least
the check files are in more managable chunks.

-- 
Brett Lymn
"Warning:
The information contained in this email and any attached files is
confidential to BAE Systems Australia. If you are not the intended
recipient, any use, disclosure or copying of this email or any
attachments is expressly prohibited.  If you have received this email
in error, please notify us immediately. VIRUS: Every care has been
taken to ensure this email and its attachments are virus free,
however, any loss or damage incurred in using this email is not the
sender's responsibility.  It is your responsibility to ensure virus
checks are completed before installing any data sent in this email to
your computer."