[pkgsrc/trunk]: pkgsrc/doc/guide/files Added examples to the regression tests...

[rillig <rillig%pkgsrc.org@localhost>]

details:   https://anonhg.NetBSD.org/pkgsrc/rev/5da92773f8bb
branches:  trunk
changeset: 348270:5da92773f8bb
user:      rillig <rillig%pkgsrc.org@localhost>
date:      Sat Jun 11 13:28:09 2016 +0000

description:
Added examples to the regression tests documentation.

diffstat:

 doc/guide/files/regression.xml |  70 ++++++++++++++++++++++++++++-------------
 1 files changed, 47 insertions(+), 23 deletions(-)

diffs (122 lines):

diff -r 709cb0a47b24 -r 5da92773f8bb doc/guide/files/regression.xml
--- a/doc/guide/files/regression.xml    Sat Jun 11 13:19:54 2016 +0000
+++ b/doc/guide/files/regression.xml    Sat Jun 11 13:28:09 2016 +0000
@@ -1,4 +1,4 @@
-<!-- $NetBSD: regression.xml,v 1.2 2006/12/15 13:22:14 martti Exp $ -->
+<!-- $NetBSD: regression.xml,v 1.3 2016/06/11 13:28:09 rillig Exp $ -->
 
 <chapter id="regression"> <?dbhtml filename="regression.html"?>
 <title>Regression tests</title>
@@ -12,13 +12,6 @@
        This chapter describes how regression tests work in pkgsrc and
        how you can add new tests.</para>
 
-<sect1 id="regression.descr">
-<title>The regression tests framework</title>
-
-<para></para>
-
-</sect1>
-
 <sect1 id="regression.run">
 <title>Running the regression tests</title>
 
@@ -43,34 +36,58 @@
 <sect2 id="regression.fun.override">
 <title>Overridable functions</title>
 
-       <para>These functions do not take any parameters. They are all
-       called in <quote>set -e</quote> mode, so you should be careful
-       to check the exitcodes of any commands you run in the
-       test.</para>
+       <para>These functions do not take any parameters. Although they
+       are called in <quote>set -e</quote> mode, they don't stop at the
+       first failing command. See <ulink
+       url="http://stackoverflow.com/q/4072984";>this StackOverflow
+       question</ulink> for details.</para>
 
        <variablelist>
 
-       <varlistentry><term><varname>do_setup()</varname></term>
+       <varlistentry><term><varname>do_setup</varname></term>
        <listitem><para>This function prepares the environment for the
        test. By default it does nothing.</para></listitem>
        </varlistentry>
 
-       <varlistentry><term><varname>do_test()</varname></term>
+       <varlistentry><term><varname>do_test</varname></term>
        <listitem><para>This function runs the actual test. By default,
        it calls <varname>TEST_MAKE</varname> with the arguments
        <varname>MAKEARGS_TEST</varname> and writes its output including
        error messages into the file
-       <varname>TEST_OUTFILE</varname>.</para></listitem>
+       <varname>TEST_OUTFILE</varname>.</para>
+
+       <para>When defining this function, make sure that all output that
+       needs to be checked is written to the correct output file.
+       Example:</para>
+
+<programlisting>
+do_test() {
+       echo "Example output"
+} 1>$TEST_OUTFILE 2>&1
+</programlisting>
+
+       </listitem>
        </varlistentry>
 
-       <varlistentry><term><varname>check_result()</varname></term>
+       <varlistentry><term><varname>check_result</varname></term>
        <listitem><para>This function is run after the test and is
        typically used to compare the actual output from the one that is
        expected. It can make use of the various helper functions from
-       the next section.</para></listitem>
+       the next section. Example:</para>
+
+<programlisting>
+check_result() {
+       exit_status 0
+       output_require "Example"
+       output_require "^[[:alpha:]+[[:space:]][[:alpha:]]{6}$"
+       output_prohibit "no such file or directory"
+}
+</programlisting>
+
+       </listitem>
        </varlistentry>
 
-       <varlistentry><term><varname>do_cleanup()</varname></term>
+       <varlistentry><term><varname>do_cleanup</varname></term>
        <listitem><para>This function cleans everything up after the
        test has been run. By default it does nothing.</para></listitem>
        </varlistentry>
@@ -83,17 +100,24 @@
 
 <variablelist>
 
-       <varlistentry><term><varname>exit_status(expected)</varname></term>
+       <varlistentry><term><varname>exit_status expected</varname></term>
        <listitem><para>This function compares the exitcode of the
-       <command>do_test()</command> function with its first parameter.
+       <command>do_test</command> function with its first parameter.
        If they differ, the test will fail.</para></listitem>
        </varlistentry>
 
-       <varlistentry><term><varname>output_require(regex...)</varname></term>
+       <varlistentry><term><varname>output_require regex...</varname></term>
        <listitem><para>This function checks for each of its parameters
-       if the output from <command>do_test()</command> matches the
+       if the output from <command>do_test</command> matches the
        extended regular expression. If it does not, the test will
-       fail.</para></listitem>
+       fail. Example:</para>
+
+<programlisting>
+output_require "looks fine"
+output_require "^[[:alpha:]+[[:space:]][[:alpha:]]{6}$"
+</programlisting>
+
+       </listitem>
        </varlistentry>
 
        <varlistentry><term><varname>output_prohibit(regex...)</varname></term>