Source-Changes-HG archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
[src/trunk]: src/usr.bin/make/unit-tests Give each group of tests its own out...
details: https://anonhg.NetBSD.org/src/rev/9bf5e96631ad
branches: trunk
changeset: 331718:9bf5e96631ad
user: apb <apb%NetBSD.org@localhost>
date: Thu Aug 21 13:44:51 2014 +0000
description:
Give each group of tests its own output file.
* Rename each sub-makefile to *.mk;
* Add a *.exp file of expected output for each sub-makefile;
* Remove test.exp, which is replaced by all the other *.exp files.
* Use suffix rules to generate *.rawout and *.out files for
each test case.
* Rewrite the test and accept targets to adapt to the new way.
The old (now removed) test.exp file is almost identical to the
concatenation (in the correct order) of all the new *.exp files. There
are expected differences in makefile names embedded in the output, and
the new "exit status" lines. Some old "*** Error code 1 (ignored)"
lines are also removed (replaced by new "exit status 1" lines).
diffstat:
usr.bin/make/unit-tests/Makefile | 78 +++--
usr.bin/make/unit-tests/comment | 31 --
usr.bin/make/unit-tests/comment.exp | 5 +
usr.bin/make/unit-tests/comment.mk | 31 ++
usr.bin/make/unit-tests/cond1 | 109 --------
usr.bin/make/unit-tests/cond1.exp | 23 +
usr.bin/make/unit-tests/cond1.mk | 109 ++++++++
usr.bin/make/unit-tests/doterror | 20 -
usr.bin/make/unit-tests/doterror.exp | 9 +
usr.bin/make/unit-tests/doterror.mk | 20 +
usr.bin/make/unit-tests/dotwait | 61 ----
usr.bin/make/unit-tests/dotwait.exp | 30 ++
usr.bin/make/unit-tests/dotwait.mk | 61 ++++
usr.bin/make/unit-tests/error | 10 -
usr.bin/make/unit-tests/error.exp | 4 +
usr.bin/make/unit-tests/error.mk | 10 +
usr.bin/make/unit-tests/export | 22 -
usr.bin/make/unit-tests/export-all | 23 -
usr.bin/make/unit-tests/export-all.exp | 4 +
usr.bin/make/unit-tests/export-all.mk | 23 +
usr.bin/make/unit-tests/export-env | 24 -
usr.bin/make/unit-tests/export-env.exp | 9 +
usr.bin/make/unit-tests/export-env.mk | 24 +
usr.bin/make/unit-tests/export.exp | 6 +
usr.bin/make/unit-tests/export.mk | 22 +
usr.bin/make/unit-tests/forloop | 45 ---
usr.bin/make/unit-tests/forloop.exp | 19 +
usr.bin/make/unit-tests/forloop.mk | 45 +++
usr.bin/make/unit-tests/forsubst | 10 -
usr.bin/make/unit-tests/forsubst.exp | 2 +
usr.bin/make/unit-tests/forsubst.mk | 10 +
usr.bin/make/unit-tests/hash | 18 -
usr.bin/make/unit-tests/hash.exp | 9 +
usr.bin/make/unit-tests/hash.mk | 18 +
usr.bin/make/unit-tests/misc | 16 -
usr.bin/make/unit-tests/misc.exp | 1 +
usr.bin/make/unit-tests/misc.mk | 16 +
usr.bin/make/unit-tests/moderrs | 31 --
usr.bin/make/unit-tests/moderrs.exp | 16 +
usr.bin/make/unit-tests/moderrs.mk | 31 ++
usr.bin/make/unit-tests/modmatch | 25 -
usr.bin/make/unit-tests/modmatch.exp | 17 +
usr.bin/make/unit-tests/modmatch.mk | 25 +
usr.bin/make/unit-tests/modmisc | 38 --
usr.bin/make/unit-tests/modmisc.exp | 10 +
usr.bin/make/unit-tests/modmisc.mk | 38 ++
usr.bin/make/unit-tests/modorder | 22 -
usr.bin/make/unit-tests/modorder.exp | 11 +
usr.bin/make/unit-tests/modorder.mk | 22 +
usr.bin/make/unit-tests/modts | 43 ---
usr.bin/make/unit-tests/modts.exp | 33 ++
usr.bin/make/unit-tests/modts.mk | 43 +++
usr.bin/make/unit-tests/modword | 151 -----------
usr.bin/make/unit-tests/modword.exp | 122 +++++++++
usr.bin/make/unit-tests/modword.mk | 151 +++++++++++
usr.bin/make/unit-tests/order | 20 -
usr.bin/make/unit-tests/order.exp | 4 +
usr.bin/make/unit-tests/order.mk | 20 +
usr.bin/make/unit-tests/phony-end | 9 -
usr.bin/make/unit-tests/phony-end.exp | 6 +
usr.bin/make/unit-tests/phony-end.mk | 9 +
usr.bin/make/unit-tests/posix | 24 -
usr.bin/make/unit-tests/posix.exp | 23 +
usr.bin/make/unit-tests/posix.mk | 24 +
usr.bin/make/unit-tests/qequals | 8 -
usr.bin/make/unit-tests/qequals.exp | 2 +
usr.bin/make/unit-tests/qequals.mk | 8 +
usr.bin/make/unit-tests/sunshcmd | 10 -
usr.bin/make/unit-tests/sunshcmd.exp | 4 +
usr.bin/make/unit-tests/sunshcmd.mk | 10 +
usr.bin/make/unit-tests/sysv | 26 --
usr.bin/make/unit-tests/sysv.exp | 7 +
usr.bin/make/unit-tests/sysv.mk | 26 ++
usr.bin/make/unit-tests/ternary | 8 -
usr.bin/make/unit-tests/ternary.exp | 10 +
usr.bin/make/unit-tests/ternary.mk | 8 +
usr.bin/make/unit-tests/test.exp | 394 -------------------------------
usr.bin/make/unit-tests/unexport | 8 -
usr.bin/make/unit-tests/unexport-env | 14 -
usr.bin/make/unit-tests/unexport-env.exp | 2 +
usr.bin/make/unit-tests/unexport-env.mk | 14 +
usr.bin/make/unit-tests/unexport.exp | 4 +
usr.bin/make/unit-tests/unexport.mk | 8 +
usr.bin/make/unit-tests/varcmd | 49 ---
usr.bin/make/unit-tests/varcmd.exp | 9 +
usr.bin/make/unit-tests/varcmd.mk | 49 +++
usr.bin/make/unit-tests/varshell | 18 -
usr.bin/make/unit-tests/varshell.exp | 12 +
usr.bin/make/unit-tests/varshell.mk | 18 +
89 files changed, 1356 insertions(+), 1315 deletions(-)
diffs (truncated from 3068 to 300 lines):
diff -r 790ef114cf7e -r 9bf5e96631ad usr.bin/make/unit-tests/Makefile
--- a/usr.bin/make/unit-tests/Makefile Thu Aug 21 13:31:52 2014 +0000
+++ b/usr.bin/make/unit-tests/Makefile Thu Aug 21 13:44:51 2014 +0000
@@ -1,24 +1,25 @@
-# $NetBSD: Makefile,v 1.41 2014/08/21 13:31:52 apb Exp $
+# $NetBSD: Makefile,v 1.42 2014/08/21 13:44:51 apb Exp $
#
# Unit tests for make(1)
# The main targets are:
#
# all: run all the tests
-# test: run 'all', capture output and compare to expected results
+# test: run 'all', and compare to expected results
# accept: move generated output to expected results
#
# Adding a test case.
# Each feature should get its own set of tests in its own suitably
-# named makefile which should be added to SUBFILES to hook it in.
+# named makefile (*.mk), with its own set of expected results (*.exp),
+# and it should be added to the TESTNAMES list.
#
.MAIN: all
UNIT_TESTS:= ${.PARSEDIR}
-# Simple sub-makefiles - we run them as a black box
-# keep the list sorted.
-SUBFILES= \
+# Each test is in a sub-makefile.
+# Keep the list sorted.
+TESTNAMES= \
comment \
cond1 \
error \
@@ -49,21 +50,16 @@
varcmd \
varshell
-all: ${SUBFILES}
-
+# Override make flags for certain tests
flags.doterror=
flags.order=-j1
-# the tests are actually done with sub-makes.
-.PHONY: ${SUBFILES}
-.PRECIOUS: ${SUBFILES}
-${SUBFILES}:
- -@${.MAKE} ${flags.$@:U-k} -f ${UNIT_TESTS}/$@
+OUTFILES= ${TESTNAMES:S/$/.out/}
+
+all: ${OUTFILES}
clean:
- rm -f *.out *.fail *.core
-
-.-include <bsd.obj.mk>
+ rm -f *.rawout *.out *.status *.tmp *.core *.tmp
TEST_MAKE?= ${.MAKE}
TOOL_SED?= sed
@@ -73,22 +69,48 @@
LANG= C
.export LANG LC_ALL
-# The driver.
+# the tests are actually done with sub-makes.
+.SUFFIXES: .mk .rawout .out
+.mk.rawout:
+ @echo ${TEST_MAKE} ${flags.${.TARGET:R}:U-k} -f ${.IMPSRC}
+ -@cd ${.OBJDIR} && \
+ { ${TEST_MAKE} ${flags.${.TARGET:R}:U-k} -f ${.IMPSRC} \
+ 2>&1 ; echo $$? >${.TARGET:R}.status ; } > ${.TARGET}.tmp
+ @mv ${.TARGET}.tmp ${.TARGET}
+
# We always pretend .MAKE was called 'make'
# and strip ${.CURDIR}/ from the output
# and replace anything after 'stopped in' with unit-tests
# so the results can be compared.
-test:
- @echo "${TEST_MAKE} -f ${MAKEFILE} > ${.TARGET}.out 2>&1"
- cd ${.OBJDIR} && ${TEST_MAKE} -f ${MAKEFILE} 2>&1 | \
- ${TOOL_SED} -e 's,^${TEST_MAKE:T:C/\./\\\./g}[][0-9]*:,make:,' \
- -e 's,${TEST_MAKE:C/\./\\\./g},make,' \
- -e '/stopped/s, /.*, unit-tests,' \
- -e 's,${.CURDIR:C/\./\\\./g}/,,g' \
- -e 's,${UNIT_TESTS:C/\./\\\./g}/,,g' > ${.TARGET}.out || { \
- tail ${.TARGET}.out; mv ${.TARGET}.out ${.TARGET}.fail; exit 1; }
- diff -u ${UNIT_TESTS}/${.TARGET}.exp ${.TARGET}.out
+.rawout.out:
+ @echo postprocess ${.TARGET}
+ @${TOOL_SED} -e 's,^${TEST_MAKE:T:C/\./\\\./g}[][0-9]*:,make:,' \
+ -e 's,${TEST_MAKE:C/\./\\\./g},make,' \
+ -e '/stopped/s, /.*, unit-tests,' \
+ -e 's,${.CURDIR:C/\./\\\./g}/,,g' \
+ -e 's,${UNIT_TESTS:C/\./\\\./g}/,,g' \
+ < ${.IMPSRC} > ${.TARGET}.tmp
+ @echo "exit status `cat ${.TARGET:R}.status`" >> ${.TARGET}.tmp
+ @mv ${.TARGET}.tmp ${.TARGET}
+
+# Compare all output files
+test: ${OUTFILES} .PHONY
+ @failed= ; \
+ for test in ${TESTNAMES}; do \
+ diff -u ${UNIT_TESTS}/$${test}.exp $${test}.out \
+ || failed="$${failed}$${failed:+ }$${test}" ; \
+ done ; \
+ if [ -n "$${failed}" ]; then \
+ echo "Failed tests: $${failed}" ; false ; \
+ else \
+ echo "All tests passed" ; \
+ fi
accept:
- mv test.out ${.CURDIR}/test.exp
+ @for test in ${TESTNAMES}; do \
+ cmp -s ${UNIT_TESTS}/$${test}.exp $${test}.out \
+ || { echo "Replacing $${test}.exp" ; \
+ cp $${test}.out ${UNIT_TESTS}/$${test}.exp ; } \
+ done
+.-include <bsd.obj.mk>
diff -r 790ef114cf7e -r 9bf5e96631ad usr.bin/make/unit-tests/comment
--- a/usr.bin/make/unit-tests/comment Thu Aug 21 13:31:52 2014 +0000
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,31 +0,0 @@
-# This is a comment
-.if ${MACHINE_ARCH} == something
-FOO=bar
-.endif
-
-#\
- Multiline comment
-
-BAR=# defined
-FOOBAR= # defined
-
-# This is an escaped comment \
-that keeps going until the end of this line
-
-# Another escaped comment \
-that \
-goes \
-on
-
-# This is NOT an escaped comment due to the double backslashes \\
-all: hi foo bar
- @echo comment testing done
-
-hi:
- @echo comment testing start
-
-foo:
- @echo this is $@
-
-bar:
- @echo This is how a comment looks: '# comment'
diff -r 790ef114cf7e -r 9bf5e96631ad usr.bin/make/unit-tests/comment.exp
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/usr.bin/make/unit-tests/comment.exp Thu Aug 21 13:44:51 2014 +0000
@@ -0,0 +1,5 @@
+comment testing start
+this is foo
+This is how a comment looks: # comment
+comment testing done
+exit status 0
diff -r 790ef114cf7e -r 9bf5e96631ad usr.bin/make/unit-tests/comment.mk
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/usr.bin/make/unit-tests/comment.mk Thu Aug 21 13:44:51 2014 +0000
@@ -0,0 +1,31 @@
+# This is a comment
+.if ${MACHINE_ARCH} == something
+FOO=bar
+.endif
+
+#\
+ Multiline comment
+
+BAR=# defined
+FOOBAR= # defined
+
+# This is an escaped comment \
+that keeps going until the end of this line
+
+# Another escaped comment \
+that \
+goes \
+on
+
+# This is NOT an escaped comment due to the double backslashes \\
+all: hi foo bar
+ @echo comment testing done
+
+hi:
+ @echo comment testing start
+
+foo:
+ @echo this is $@
+
+bar:
+ @echo This is how a comment looks: '# comment'
diff -r 790ef114cf7e -r 9bf5e96631ad usr.bin/make/unit-tests/cond1
--- a/usr.bin/make/unit-tests/cond1 Thu Aug 21 13:31:52 2014 +0000
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,109 +0,0 @@
-# $Id: cond1,v 1.5 2011/03/06 00:02:14 sjg Exp $
-
-# hard code these!
-TEST_UNAME_S= NetBSD
-TEST_UNAME_M= sparc
-TEST_MACHINE= i386
-
-.if ${TEST_UNAME_S}
-Ok=var,
-.endif
-.if ("${TEST_UNAME_S}")
-Ok+=(\"var\"),
-.endif
-.if (${TEST_UNAME_M} != ${TEST_MACHINE})
-Ok+=(var != var),
-.endif
-.if ${TEST_UNAME_M} != ${TEST_MACHINE}
-Ok+= var != var,
-.endif
-.if !((${TEST_UNAME_M} != ${TEST_MACHINE}) && defined(X))
-Ok+= !((var != var) && defined(name)),
-.endif
-# from bsd.obj.mk
-MKOBJ?=no
-.if ${MKOBJ} == "no"
-o= no
-Ok+= var == "quoted",
-.else
-.if defined(notMAKEOBJDIRPREFIX) || defined(norMAKEOBJDIR)
-.if defined(notMAKEOBJDIRPREFIX)
-o=${MAKEOBJDIRPREFIX}${__curdir}
-.else
-o= ${MAKEOBJDIR}
-.endif
-.endif
-o= o
-.endif
-
-# repeat the above to check we get the same result
-.if ${MKOBJ} == "no"
-o2= no
-.else
-.if defined(notMAKEOBJDIRPREFIX) || defined(norMAKEOBJDIR)
-.if defined(notMAKEOBJDIRPREFIX)
-o2=${MAKEOBJDIRPREFIX}${__curdir}
-.else
-o2= ${MAKEOBJDIR}
-.endif
-.endif
-o2= o
-.endif
-
-PRIMES=2 3 5 7 11
-NUMBERS=1 2 3 4 5
-
-n=2
-.if ${PRIMES:M$n} == ""
-X=not
-.else
-X=
-.endif
-
-.if ${MACHINE_ARCH} == no-such
-A=one
-.else
-.if ${MACHINE_ARCH} == not-this
-.if ${MACHINE_ARCH} == something-else
-A=unlikely
-.else
-A=no
-.endif
-.endif
-A=other
-# We expect an extra else warning - we're not skipping here
-.else
-A=this should be an error
-.endif
-
-.if $X != ""
-.if $X == not
-B=one
-.else
-B=other
-# We expect an extra else warning - we are skipping here
-.else
-B=this should be an error
-.endif
-.else
-B=unknown
-.endif
-
-.if "quoted" == quoted
-C=clever
-.else
Home |
Main Index |
Thread Index |
Old Index