[src/trunk]: src/regress new tests should use atf -> new tests must use atf

[pooka <pooka%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/4a5bf8372069
branches:  trunk
changeset: 757201:4a5bf8372069
user:      pooka <pooka%NetBSD.org@localhost>
date:      Wed Aug 18 21:28:03 2010 +0000

description:
new tests should use atf -> new tests must use atf

diffstat:

 regress/README |  61 ++-------------------------------------------------------
 1 files changed, 3 insertions(+), 58 deletions(-)

diffs (68 lines):

diff -r f8847f564824 -r 4a5bf8372069 regress/README
--- a/regress/README    Wed Aug 18 21:23:48 2010 +0000
+++ b/regress/README    Wed Aug 18 21:28:03 2010 +0000
@@ -1,61 +1,6 @@
-$NetBSD: README,v 1.2 2009/09/14 17:15:32 apb Exp $
+       $NetBSD: README,v 1.3 2010/08/18 21:28:03 pooka Exp $
 
-NOTE: New tests should use the ATF framework; see atf(7)
+New tests must use the ATF framework; see atf(7)
 and the src/tests directory.
 
-------------------------------------------------------------------------
-This part of the source tree contains regression tests. There are special
-make targets and rules to follow. Most of these, however, are currently not
-enforced, and most tests available are not conforming.
-
-We hope to fix this someday. If you add new tests, please try to be conforming.
-
-What is a regression test?
-
-  A regression test is run by a makefile in a test directory (see below).
-  Each makefile may run multiple tests.
-
-What is a test directory?
-
-  A directory in this part of the tree is a regression test directory. It
-  contains a Makefile which implements the additional "regress" target,
-  and runs all it's regression tests during this target.
-
-May the make progress be stopped on failures?
-
-  No, the "make regress" target should succeed, unless some regression 
-  binaries could not be build, disk is full or other catastrophic failures
-  outside of the tested subsystem happen. A failing regression test should
-  log the failure (see below), but not make the target itself fail.
-
-What are the possible results of a regression test?
-
-  A test may either
-
-  - succeed, in which case it logs "PASSED" (see below for logging details)
-  - fail, in which case it logs "FAILED"
-  - not be able to run, in which case it logs "SKIPPED" and the reason
-    for the skip in the comment field (see below)
-
-  Typical reasons for tests to not being run are missing kernel options,
-  or missing privileges (test needs root, "make regress" is invoked by
-  mere mortal or vice versa). A test may not fail because of such
-  environmental issues, it must detect and properly log the problem.
-
-  If a test directory contains tests that may be skipped, it should have
-  a README file explaining the prerequisites (e.g. needed kernel options)
-
-  In future, we will mark affected makefiles and optimize run/skipped test
-  during repeated runs with differing privileges - but currently there is no
-  make framework in place to handle this.
-
-How and when does a test log results?
-
-  If the make/environment variable ${REGRESS_LOG} is defined, the final
-  results (and only those) should be logged to the file named by that
-  variable. We will, in the future, add make targets for this purpose.
-  The log format is line oriented, one line used per test. Each line
-  consists of the directory where the Makefile lives, followed by the
-  test name and the result (see above: PASSED, FAILED, SKIPPED).
-  Following this an optional comment may be added. For SKIPPED tests the
-  comment is not optional. Fields are separated by spaces.
+All exceptions need prior approval from core.