CVS commit: pkgsrc/devel/kyua

["Julio Merino" <jmmv%netbsd.org@localhost>]

Module Name:    pkgsrc
Committed By:   jmmv
Date:           Mon Nov 23 03:39:14 UTC 2015

Modified Files:
        pkgsrc/devel/kyua: DESCR Makefile PLIST distinfo

Log Message:
Update devel/kyua to 0.12:

**Released on November 22nd, 2015.**

This is a huge release and marks a major milestone for Kyua as it finally
implements a long-standing feature request: the ability to execute test
cases in parallel.  This is a big deal because test cases are rarely
CPU-bound: running them in parallel yields much faster execution times for
large test suites, allowing faster iteration of changes during development.

As an example: the FreeBSD test suite as of this date contains 3285 test
cases.  With sequential execution, a full test suite run takes around 12
minutes to complete, whereas on a 4-core machine with a high level of
parallelism it takes a little over 1 minute.

Implementing parallel execution required rewriting most of Kyua's core and
partly explains explains why there has not been a new release for over a
year.  The current implementation is purely subprocess-based, which works
but has some limitations and has resulted in a core that is really complex
and difficult to understand.  Future versions will investigate the use of
threads instead for a simplified programming model and additional
parallelization possibilities.

* Issue #2: Implemented support to execute test cases in parallel when
  invoking `kyua test`.  Parallel execution is *only* enabled when the new
  `parallelism` configuration variable is set to a value greater than `1`.
  The default behavior is still to run tests sequentially because some test
  suites contain test cases with side-effects that might fail when run in
  parallel.  To resolve this, the new metadata property `is_exclusive` can
  be set to `true` on a test basis to indicate that the test must be run on
  its own.

* Known regression: Running `kyua debug` on a TAP-based test program does
  not currently report the output in real time.  The output will only be
  displayed once the test program completes.  This is a shortcoming of
  the new parallel execution engine and will be resolved.

* Removed the external C-based testers code in favor of the new built-in
  implementations.  The new approach feels significantly faster than the
  previous one.

* Fixed the handling of relative paths in the `fs.*` functions available
  in `Kyuafile`s.  All paths are now resolved relative to the location of
  the caller `Kyuafile`.  `Kyuafile.top` has been updated with these
  changes and you should update custom copies of this file with the new
  version.

* Changed temporary directory creation to always grant search
  permissions on temporary directories.  This is to prevent potential
  problems when running Kyua as root and executing test cases that require
  dropping privileges (as they may later be unable to use absolute paths
  that point inside their work directory).

* The cleanup of work directories does not longer attempt to deal with
  mount points.  If a test case mounts a file system and forgets to unmount
  it, the mount point will be left behind.  It is now the responsibility of
  the test case to clean after itself.  The reasons for this change are
  simplicity and clarity: there are many more things that a test case can
  do that have side-effects on the system and Kyua cannot protect against
  them all, so it is better to just have the test undo anything it might
  have done.

* Improved `kyua report --verbose` to properly handle environment
  variables with continuation lines in them, and fixed the integration
  tests for this command to avoid false negatives.

* Changed the configuration file format to accept the definition of
  unknown variables without declaring them local.  The syntax version
  number remains at 2.  This is to allow configuration files for newer Kyua
  versions to work on older Kyua versions, as there is no reason to forbid
  this.

* Fixed stacktrace gathering with FreeBSD's ancient version of GDB.
  GDB 6.1.1 (circa 2004) does not have the `-ex` flag so we need to
  generate a temporary GDB script and feed it to GDB with `-x` instead.

* Issue #136: Fixed the XML escaping in the JUnit output so that
  non-printable characters are properly handled when they appear in the
  process's stdout or stderr.

* Issue #141: Improved reporting of errors triggered by sqlite3.  In
  particular, all error messages are now tagged with their corresponding
  database filename and, if they are API-level errors, the name of the
  sqlite3 function that caused them.

* Issue #144: Improved documentation on the support for custom properties
  in the test metadata.

* Converted the `INSTALL`, `NEWS`, and `README` distribution documents to
  Markdown for better formatting online.


To generate a diff of this commit:
cvs rdiff -u -r1.1 -r1.2 pkgsrc/devel/kyua/DESCR
cvs rdiff -u -r1.7 -r1.8 pkgsrc/devel/kyua/Makefile
cvs rdiff -u -r1.2 -r1.3 pkgsrc/devel/kyua/PLIST
cvs rdiff -u -r1.3 -r1.4 pkgsrc/devel/kyua/distinfo

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.