[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: Initial Kyua import done
> I am very happy to announce that the initial import of Kyua into the
> NetBSD source tree is complete! I am sure there are some rough edges
> and here is where you come into play.
> First of all, let me clarify that all the integration changes are gated
> by the MKKYUA build setting, which still defaults to no. Unless you
> explicitly set MKKYUA=yes in /etc/mk.conf or in the build.sh command
> line, you will /not/ get a system with Kyua. However, once you set that
> flag, you will transition to the full new setup:
Why don't you just enable it by default? I don't see the point of making it
optional, it's new stuff that gets added, it does not break existing stuff,
> * Kyua will be installed as /usr/bin/kyua. See kyua(1) to get started.
> * The old atf-run and atf-report tools will become compatibility
> wrappers. These should be a reasonable drop-in replacement for most
> use cases, but they are probably not perfect.
> * Your /usr/tests tree will be populated by a bunch of Kyuafiles.
> So what's new compared to the ATF tools? Here are some highlights:
> * Ability to write and run ATF-less tests. It has been a common desire
> around here to develop test programs that do not rely on the ATF
> libraries. With this change, this becomes possible. See kyuafile(5)
> for details.
> * Direct HTML report generation. There is no need to set up a complex
> XML toolchain any more to convert the ATF test reports into HTML
> pages. Kyua can generate plain HTML directly, so it will be feasible
> to serve such content straight from NetBSD's built-in httpd.
> * Historical data. As seen in the various test beds that have appeared
> around ATF, there is a desire to maintain historical data of the test
> results. Kyua does that natively, by recording the results of the
> execution in a SQLite database. Reports can later be extracted from
> this database. There is still a lot of room for improvement here.
> * More flexible metadata and configuration. While this does not provide
> a real advantage today, as soon as the old atf-run and atf-report
> tools are gone we can trivially fix some long-standing issues
> (e.g. the inability to customize test deadlines).
> * Less complexity during test case execution. As you may have noticed
> over the years, the code in atf-run to capture the output of tests and
> deal with interrupts is not particularly robust. There have been
> several problems in this area, and I'm not convinced that they are all
> fixed. The new code works in a different manner and has been more
> carefully thought around these edges.
> * Independent testers. The code that implements the isolation of test
> cases and their controlled execution has been split into a set of
> "testers" that live in /usr/libexec/kyua-*-tester. These tools
> provide scriptable interfaces to interact with tests, with the idea
> that the kyua(1) frontend should end up being pretty straightforward.
> Should you want to write your own trivial script to run tests without
> kyua(1), you could pretty easily do that by interacting with the
> testers directly.
> How can you help? Easy. Just rebuild your system with MKKYUA=yes, read
> through kyua(1), start using it to stress-test your system and report
> any problems you may encounter!
> My immediate next steps include addressing your feedback and working
> with our major test runners to add support to their systems to use the
> new tools (for example, change anita to support running tests with
> Enjoy and thanks for reading.
> 1: http://wiki.NetBSD.org/kyua/
> 2: http://wiki.NetBSD.org/kyua/import/
> 3: https://code.google.com/p/kyua/issues/list for kyua-specific bugs
> and send-pr for NetBSD-specific bugs.
> Julio Merino / @jmmv
micro systems, Wiesendamm 2a, Postfach, 4019 Basel
fon +41 61 383 05 10, fax +41 61 383 05 12, http://www.msys.ch/
Main Index |
Thread Index |