[src/trunk]: src/usr.bin/make/unit-tests make(1): test all combinations of fa...

[rillig <rillig%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/70c98bd4eba5
branches:  trunk
changeset: 946728:70c98bd4eba5
user:      rillig <rillig%NetBSD.org@localhost>
date:      Sun Dec 06 22:36:58 2020 +0000

description:
make(1): test all combinations of failed main and .END targets

Adding an individual test for each of the 16 combinations would have
been too much manual work, and it's not easy to come up with a good
naming scheme for all the tests, keeping them short and expressive at
the same time.

diffstat:

 usr.bin/make/unit-tests/deptgt-end-fail.exp |  146 +++++++++++++++++++++++++++-
 usr.bin/make/unit-tests/deptgt-end-fail.mk  |   64 +++++++++++-
 2 files changed, 201 insertions(+), 9 deletions(-)

diffs (231 lines):

diff -r 525da5f28514 -r 70c98bd4eba5 usr.bin/make/unit-tests/deptgt-end-fail.exp
--- a/usr.bin/make/unit-tests/deptgt-end-fail.exp       Sun Dec 06 21:22:04 2020 +0000
+++ b/usr.bin/make/unit-tests/deptgt-end-fail.exp       Sun Dec 06 22:36:58 2020 +0000
@@ -1,8 +1,148 @@
-: all
-: Making .END out of nothing.
-false
+Test case all=ok all-dep=ok end=ok end-dep=ok.
+: Making all-dep out of nothing.
+: Making all from all-dep.
+: Making end-dep out of nothing.
+: Making .END from end-dep.
+exit status 0
+
+
+Test case all=ok all-dep=ok end=ok end-dep=ERR.
+: Making all-dep out of nothing.
+: Making all from all-dep.
+: Making end-dep out of nothing.
+*** Error code 1 (continuing)
+
+Stop.
+make: stopped in unit-tests
+exit status 1
+
+
+Test case all=ok all-dep=ok end=ERR end-dep=ok.
+: Making all-dep out of nothing.
+: Making all from all-dep.
+: Making end-dep out of nothing.
+: Making .END from end-dep.
+*** Error code 1 (continuing)
+
+Stop.
+make: stopped in unit-tests
+exit status 1
+
+
+Test case all=ok all-dep=ok end=ERR end-dep=ERR.
+: Making all-dep out of nothing.
+: Making all from all-dep.
+: Making end-dep out of nothing.
 *** Error code 1 (continuing)
 
 Stop.
 make: stopped in unit-tests
 exit status 1
+
+
+Test case all=ok all-dep=ERR end=ok end-dep=ok.
+: Making all-dep out of nothing.
+*** Error code 1 (continuing)
+`all' not remade because of errors.
+exit status 0
+
+
+Test case all=ok all-dep=ERR end=ok end-dep=ERR.
+: Making all-dep out of nothing.
+*** Error code 1 (continuing)
+`all' not remade because of errors.
+exit status 0
+
+
+Test case all=ok all-dep=ERR end=ERR end-dep=ok.
+: Making all-dep out of nothing.
+*** Error code 1 (continuing)
+`all' not remade because of errors.
+exit status 0
+
+
+Test case all=ok all-dep=ERR end=ERR end-dep=ERR.
+: Making all-dep out of nothing.
+*** Error code 1 (continuing)
+`all' not remade because of errors.
+exit status 0
+
+
+Test case all=ERR all-dep=ok end=ok end-dep=ok.
+: Making all-dep out of nothing.
+: Making all from all-dep.
+*** Error code 1 (continuing)
+: Making end-dep out of nothing.
+: Making .END from end-dep.
+
+Stop.
+make: stopped in unit-tests
+exit status 1
+
+
+Test case all=ERR all-dep=ok end=ok end-dep=ERR.
+: Making all-dep out of nothing.
+: Making all from all-dep.
+*** Error code 1 (continuing)
+: Making end-dep out of nothing.
+*** Error code 1 (continuing)
+
+Stop.
+make: stopped in unit-tests
+exit status 1
+
+
+Test case all=ERR all-dep=ok end=ERR end-dep=ok.
+: Making all-dep out of nothing.
+: Making all from all-dep.
+*** Error code 1 (continuing)
+: Making end-dep out of nothing.
+: Making .END from end-dep.
+*** Error code 1 (continuing)
+
+Stop.
+make: stopped in unit-tests
+exit status 1
+
+
+Test case all=ERR all-dep=ok end=ERR end-dep=ERR.
+: Making all-dep out of nothing.
+: Making all from all-dep.
+*** Error code 1 (continuing)
+: Making end-dep out of nothing.
+*** Error code 1 (continuing)
+
+Stop.
+make: stopped in unit-tests
+exit status 1
+
+
+Test case all=ERR all-dep=ERR end=ok end-dep=ok.
+: Making all-dep out of nothing.
+*** Error code 1 (continuing)
+`all' not remade because of errors.
+exit status 0
+
+
+Test case all=ERR all-dep=ERR end=ok end-dep=ERR.
+: Making all-dep out of nothing.
+*** Error code 1 (continuing)
+`all' not remade because of errors.
+exit status 0
+
+
+Test case all=ERR all-dep=ERR end=ERR end-dep=ok.
+: Making all-dep out of nothing.
+*** Error code 1 (continuing)
+`all' not remade because of errors.
+exit status 0
+
+
+Test case all=ERR all-dep=ERR end=ERR end-dep=ERR.
+: Making all-dep out of nothing.
+*** Error code 1 (continuing)
+`all' not remade because of errors.
+exit status 0
+
+
+exit status 0
diff -r 525da5f28514 -r 70c98bd4eba5 usr.bin/make/unit-tests/deptgt-end-fail.mk
--- a/usr.bin/make/unit-tests/deptgt-end-fail.mk        Sun Dec 06 21:22:04 2020 +0000
+++ b/usr.bin/make/unit-tests/deptgt-end-fail.mk        Sun Dec 06 22:36:58 2020 +0000
@@ -1,14 +1,66 @@
-# $NetBSD: deptgt-end-fail.mk,v 1.3 2020/12/06 21:22:04 rillig Exp $
+# $NetBSD: deptgt-end-fail.mk,v 1.4 2020/12/06 22:36:58 rillig Exp $
 #
-# Tests for an error in the .END node.
+# Tests for an errors in the main target, its dependencies,
+# the .END node and its dependencies.
 #
 # Before 2020-11-25, an error in the .END target did not print the "Stop.",
 # even though this was intended.  The cause for this was a missing condition
 # in Compat_Run, in the code handling the .END node.
 
-all:
-       : $@
+test: .PHONY
+
+# The default stop-on-error mode is not as interesting to test since it
+# stops right after the first error.
+.MAKEFLAGS: -k
+
+.for all in ok ERR
+.  for all-dep in ok ERR
+.    for end in ok ERR
+.      for end-dep in ok ERR
+.        for target in ${all}-${all-dep}-${end}-${end-dep}
+test: ${target}
+${target}: .PHONY .SILENT
+       echo Test case all=${all} all-dep=${all-dep} end=${end} end-dep=${end-dep}.
+       ${MAKE} -r -f ${MAKEFILE} \
+               all=${all} all-dep=${all-dep} \
+               end=${end} end-dep=${end-dep} \
+               all; \
+       echo "exit status $$?"
+       echo
+       echo
+.        endfor
+.      endfor
+.    endfor
+.  endfor
+.endfor
+
+.if make(all)
 
-.END:
+all all-dep end-dep: .PHONY
+
+CMD.ok=                true
+CMD.ERR=       false
+
+all: all-dep
+       : Making ${.TARGET} from ${.ALLSRC}.
+       @${CMD.${all}}
+
+all-dep:
        : Making ${.TARGET} out of nothing.
-       false
+       @${CMD.${all-dep}}
+
+.END: end-dep
+       : Making ${.TARGET} from ${.ALLSRC}.
+       @${CMD.${end}}
+
+end-dep:
+       : Making ${.TARGET} out of nothing.
+       @${CMD.${end-dep}}
+
+.endif
+
+# XXX: As of 2020-12-06, several of the test cases printed "`all' not remade
+# because of errors.", followed by "exit status 0".  This doesn't match.
+
+# XXX: As of 2020-12-06, '.END' is made if 'all' fails, but if a dependency
+# of 'all' fails, it is skipped.  This is inconsistent.