Re: Revisiting the migration path to Kyua

[Andreas Gustafsson <gson%gson.org@localhost>]

Julio Merino wrote:
> > Would we not also want the ability to run both?  Will there be a
> > transition period when some tests are only in ATF and others are
> > only in Kyua?
> 
> No, I'd rather not do that because maintaining two different test
> sets will become ugly very very quickly.  Both tools should run the
> exact same tests.

OK, it wasn't clear to me before that both tools would run the same
tests; that sounds good.

> > It might be
> > better for anita to try running both ATF and Kyua, and for the
> > higher-level test harnesses that generate HTML reports to publish the
> > results from whichever one worked, or both.
> 
> Maybe, but as you say there are problems that prevent you from doing
> that right now.  Also, doing that would double the time required to
> run tests, and we know that qemu is super-slow for this... so I'm
> not sure it's a good idea.

Given that both tools run the same tests, I agree that running both
would be wasteful.  Maybe we could add a command line option like you
suggested, but make it fall back to ATF when Kyua is selected but not
available, as the case may be when testing a historic version?

> I'm hoping that once people have an easier way to start testing kyua
> they will, and I'm sure that _more_ problems will be discovered once
> that happens.  I'd rather uncover what these issues are sooner
> rather than later.

Fair enough.

> > but only postpone the growth until the
> > existing tests are migrated, so what problems will it solve?
> 
> With "tests" you mean "machines running tests" here?

I meant "test cases", but never mind, I was incorrectly assuming that
the two tools would run different tests.
-- 
Andreas Gustafsson, gson%gson.org@localhost