[src/trunk]: src/usr.bin/make/unit-tests tests/make: extend and isolate tests...

[rillig <rillig%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/23f359f828d5
branches:  trunk
changeset: 359867:23f359f828d5
user:      rillig <rillig%NetBSD.org@localhost>
date:      Sat Jan 29 00:52:53 2022 +0000

description:
tests/make: extend and isolate tests for target-local variables

Reusing the target var-scope-local.o for several tests made the test
more difficult to understand than necessary.  The test names '2' and '3'
didn't convey any meaning.

Instead, add more test targets that are named after what they test.  Add
tests for each of the 5 variable assignment operators, to demonstrate an
inconsistency between '+=' and '?='.

Add tests for the built-in target-local variables as well and explain
the general concepts, in particular the exact point where target-local
expressions are expanded.

The lines in the expected output file are not generated in the same
order as they appear in the makefile, so allow the 'expect' lines in
non-linear order, in check-expect.lua.

diffstat:

 usr.bin/make/unit-tests/check-expect.lua    |    5 +-
 usr.bin/make/unit-tests/var-scope-local.exp |   25 ++-
 usr.bin/make/unit-tests/var-scope-local.mk  |  192 +++++++++++++++++++++++----
 3 files changed, 184 insertions(+), 38 deletions(-)

diffs (280 lines):

diff -r df7a87d77990 -r 23f359f828d5 usr.bin/make/unit-tests/check-expect.lua
--- a/usr.bin/make/unit-tests/check-expect.lua  Sat Jan 29 00:03:41 2022 +0000
+++ b/usr.bin/make/unit-tests/check-expect.lua  Sat Jan 29 00:52:53 2022 +0000
@@ -1,5 +1,5 @@
 #!  /usr/bin/lua
--- $NetBSD: check-expect.lua,v 1.1 2022/01/15 12:35:18 rillig Exp $
+-- $NetBSD: check-expect.lua,v 1.2 2022/01/29 00:52:53 rillig Exp $
 
 --[[
 
@@ -82,6 +82,9 @@
           mk_fname, mk_lineno, exp_fname, prev_expect_line + 1, text)
       end
     end
+    if mk_line:match("^#%s*expect%-reset$") then
+      prev_expect_line = 0
+    end
 
     ---@param text string
     for offset, text in mk_line:gmatch("#%s*expect([+%-]%d+):%s*(.*)") do
diff -r df7a87d77990 -r 23f359f828d5 usr.bin/make/unit-tests/var-scope-local.exp
--- a/usr.bin/make/unit-tests/var-scope-local.exp       Sat Jan 29 00:03:41 2022 +0000
+++ b/usr.bin/make/unit-tests/var-scope-local.exp       Sat Jan 29 00:52:53 2022 +0000
@@ -1,10 +1,21 @@
+Global: .ALLTARGETS =  one
+Global: .ALLTARGETS =  one two
+Var_Parse: ${.MAKE.TARGET_LOCAL_VARIABLES} (eval)
+Var_SetExpand: variable name "" expands to empty string, with value "three" - ignored
+Var_SetExpand: variable name "" expands to empty string, with value "three" - ignored
+Global: one two = 
+Global: one two = three
+Global: .MAKEFLAGS =  -r -k -d v -d
+Global: .MAKEFLAGS =  -r -k -d v -d 0
 : Making var-scope-local.c out of nothing.
 : Making var-scope-local.o from var-scope-local.c.
-: Making basename "var-scope-local.o" in "." from "var-scope-local.c" in "." VAR="local".
-: Making var-scope-local2.c out of nothing.
-: Making var-scope-local2.o from var-scope-local2.c.
-: Making basename "var-scope-local2.o" in "." from "var-scope-local2.c" in "." VAR="local to var-scope-local2.o".
-: Making var-scope-local3.c out of nothing.
-: var-scope-local3.o uses .USE VAR="global+local"
-: all overwritten VAR="global"
+: Making basename "var-scope-local.o" in "." from "var-scope-local.c" in ".".
+: Making var-scope-local-assign.o with VAR="local".
+: Making var-scope-local-append.o with VAR="local to var-scope-local-append.o".
+: Making var-scope-local-append-global.o with VAR="global+local".
+: Making var-scope-local-default.o with VAR="global".
+: Making var-scope-local-subst.o with VAR="global+local".
+: Making var-scope-local-shell.o with VAR="output".
+: var-scope-local-use.o uses .USE VAR="global"
+: all overwritten
 exit status 0
diff -r df7a87d77990 -r 23f359f828d5 usr.bin/make/unit-tests/var-scope-local.mk
--- a/usr.bin/make/unit-tests/var-scope-local.mk        Sat Jan 29 00:03:41 2022 +0000
+++ b/usr.bin/make/unit-tests/var-scope-local.mk        Sat Jan 29 00:52:53 2022 +0000
@@ -1,16 +1,67 @@
-# $NetBSD: var-scope-local.mk,v 1.2 2022/01/27 06:56:27 sjg Exp $
+# $NetBSD: var-scope-local.mk,v 1.3 2022/01/29 00:52:53 rillig Exp $
+#
+# Tests for target-local variables, such as ${.TARGET} or $@.  These variables
+# are relatively short-lived as they are created just before making the
+# target.  In contrast, global variables are typically created when the
+# makefiles are read in.
 #
-# Tests for target-local variables, such as ${.TARGET} or $@.
+# The 7 built-in target-local variables are listed in the manual page.  They
+# are defined just before the target is actually made.  Additional
+# target-local variables can be defined in dependency lines like
+# 'target: VAR=value', one at a time.
+
+.MAIN: all
 
-# TODO: Implementation
-
-# Ensure that the name of the variable is exactly the given one.
-# The variable "@" is an alias for ".TARGET", so the implementation might
+# The target-local variables can be used in expressions, just like other
+# variables.  When these expressions are evaluated outside of a target, these
+# expressions are not yet expanded, instead their text is preserved, to allow
+# these expressions to expand right in time when the target-local variables
+# are actually set.
+#
+# Conditions like the ones below are evaluated in the scope of the command
+# line, which means that variables from the command line, from the global
+# scope and from the environment are resolved, in this order (but see the
+# command line option '-e').  In that phase, expressions involving
+# target-local variables need to be preserved, including the exact names of
+# the variables.
+#
+# Each of the built-in target-local variables has two equivalent names, for
+# example '@' is equivalent to '.TARGET'.  The implementation might
 # canonicalize these aliases at some point, and that might be surprising.
 # This aliasing happens for single-character variable names like $@ or $<
 # (see VarFind, CanonicalVarname), but not for braced or parenthesized
 # expressions like ${@}, ${.TARGET} ${VAR:Mpattern} (see Var_Parse,
 # ParseVarname).
+#
+# In the following condition, make does not expand '$@' but instead changes it
+# to the long-format alias '$(.TARGET)'; note that the alias is not written
+# with braces, as would be common in BSD makefiles, but with parentheses.
+# This alternative form behaves equivalently though.
+.if $@ != "\$\(.TARGET)"
+.  error
+.endif
+# In the long form of writing a target-local variable, the expression is
+# preserved exactly as written, no matter whether with '{' or '('.
+.if ${@} != "\$\{@}"
+.  error
+.endif
+.if $(@) != "\$\(@)"
+.  error
+.endif
+# If the variable expression contains modifiers, the behavior depends on the
+# actual modifiers.  The modifier ':M' keeps the expression in the state
+# 'undefined'.  Since the expression is still undefined after evaluating all
+# the modifiers, the value of the expression is discarded and the expression
+# text is used instead.  This preserves the expressions based on target-local
+# variables as long as possible.
+.if ${@:M*} != "\$\{@:M*}"
+.  error
+.endif
+# In the following examples, the expressions are based on target-local
+# variables but use the modifier ':L', which turns an undefined expression
+# into a defined one.  At the end of evaluating the expression, the state of
+# the expression is not 'undefined' anymore, and the value of the expression
+# is the name of the variable, since that's what the modifier ':L' does.
 .if ${@:L} != "@"
 .  error
 .endif
@@ -24,45 +75,126 @@
 .  error
 .endif
 
-all:
+
+# Additional target-local variables may be defined in dependency lines.
+.MAKEFLAGS: -dv
+# In the following line, the ':=' may either be interpreted as an assignment
+# operator or as the dependency operator ':', followed by an empty variable
+# name and the assignment operator '='.  It is the latter since in an
+# assignment, the left-hand side must be at most a single word.  The empty
+# variable name is expanded twice, once for 'one' and once for 'two'.
+# expect: Var_SetExpand: variable name "" expands to empty string, with value "three" - ignored
+# expect: Var_SetExpand: variable name "" expands to empty string, with value "three" - ignored
+one two:=three
+# If the two targets to the left are generated by a variable expression, the
+# line is parsed as a variable assignment since its left-hand side is a single
+# word.
+# expect: Global: one two = three
+${:Uone two}:=three
+.MAKEFLAGS: -d0
+
 
 .SUFFIXES: .c .o
 
-var-scope-local.c var-scope-local2.c var-scope-local3.c:
-       : Making ${.TARGET} out of nothing.
+# One of the dynamic target-local variables is '.TARGET'.  Since this is not
+# a suffix transformation rule, the variable '.IMPSRC' is not defined.
+# expect: : Making var-scope-local.c out of nothing.
+var-scope-local.c:
+       : Making ${.TARGET} ${.IMPSRC:Dfrom ${.IMPSRC}:Uout of nothing}.
 
+# This is a suffix transformation rule, so both '.TARGET' and '.IMPSRC' are
+# defined.
+# expect: : Making var-scope-local.o from var-scope-local.c.
+# expect: : Making basename "var-scope-local.o" in "." from "var-scope-local.c" in ".".
 .c.o:
        : Making ${.TARGET} from ${.IMPSRC}.
 
        # The local variables @F, @D, <F, <D are legacy forms.
        # See the manual page for details.
-       : Making basename "${@F}" in "${@D}" from "${<F}" in "${<D}" VAR="${VAR}".
+       : Making basename "${@F}" in "${@D}" from "${<F}" in "${<D}".
 
-all: var-scope-local.o var-scope-local2.o var-scope-local3.o
+# expect: : all overwritten
+all: var-scope-local.o
        # The ::= modifier overwrites the .TARGET variable in the node
        # 'all', not in the global scope.  This can be seen with the -dv
-       # option, looking for "all:@ = overwritten".
-       : ${.TARGET} ${.TARGET::=overwritten}${.TARGET} VAR="${VAR}"
+       # option, looking for "all: @ = overwritten".
+       : ${.TARGET} ${.TARGET::=overwritten}${.TARGET}
+
 
-# we can set variables per target with some limitations
+# Begin tests for custom target-local variables, for all 5 variable assignment
+# operators.
+all: var-scope-local-assign.o
+all: var-scope-local-append.o
+all: var-scope-local-append-global.o
+all: var-scope-local-default.o
+all: var-scope-local-subst.o
+all: var-scope-local-shell.o
+
+var-scope-local-assign.o \
+var-scope-local-append.o \
+var-scope-local-append-global.o \
+var-scope-local-default.o \
+var-scope-local-subst.o \
+var-scope-local-shell.o:
+       : Making ${.TARGET} with VAR="${VAR}".
+
+# Target-local variables are enabled by default.  Force them to be enabled
+# just in case a test above has disabled them.
 .MAKE.TARGET_LOCAL_VARIABLES= yes
-VAR= global
-# the rest of the line is the value
-var-scope-local.o: VAR= local
-# += will *not* take global value and add to it in local context
-var-scope-local2.o: VAR+= local
-# but once defined in local context += behaves as expected.
-var-scope-local2.o: VAR += to ${.TARGET}
-# we can get the global value though
-# so complex values can always be set via global variable and then
-# assigned to local context
-# Note: that the global ${VAR} is expanded at this point
-# just as with any dependency line.
-var-scope-local3.o: VAR= ${VAR}+local
+
+VAR=   global
+
+# If the sources of a dependency line look like a variable assignment, make
+# treats them as such.  There is only a single variable assignment per
+# dependency line, which makes whitespace around the assignment operator
+# irrelevant.
+#
+# expect-reset
+# expect: : Making var-scope-local-assign.o with VAR="local".
+var-scope-local-assign.o: VAR= local
 
-# while VAR=use will be set for a .USE node, it will never be seen
-# since only the ultimate target's context is searched
+# Assignments using '+=' do *not* look up the global value, instead they only
+# look up the variable in the target's own scope.
+var-scope-local-append.o: VAR+= local
+# Once a variable is defined in the target-local scope, appending using '+='
+# behaves as expected.  Note that the expression '${.TARGET}' is not resolved
+# when parsing the dependency line, its evaluation is deferred until the
+# target is actually made.
+# expect: : Making var-scope-local-append.o with VAR="local to var-scope-local-append.o".
+var-scope-local-append.o: VAR += to ${.TARGET}
+# To access the value of a global variable, use a variable expression.  This
+# expression is expanded before parsing the whole dependency line.  Since the
+# expansion happens to the right of both the dependency operator ':' and also
+# to the right of the assignment operator '=', the expanded text does not
+# affect the dependency or the variable assignment structurally.  The
+# effective variable assignment, after expanding the whole line first, is thus
+# 'VAR= global+local'.
+# expect: : Making var-scope-local-append-global.o with VAR="global+local".
+var-scope-local-append-global.o: VAR= ${VAR}+local
+
+var-scope-local-default.o: VAR ?= first
+var-scope-local-default.o: VAR ?= second
+# XXX: '?=' does look at the global variable.  That's a long-standing
+# inconsistency between the assignment operators '+=' and '?='.  See
+# Var_AppendExpand and VarAssign_Eval.
+# expect: : Making var-scope-local-default.o with VAR="global".
+
+# Using the variable assignment operator ':=' provides another way of
+# accessing a global variable and extending it with local modifications.  The
+# '$' has to be written as '$$' though to survive the expansion of the
+# dependency line as a whole.
+var-scope-local-subst.o: VAR := $${VAR}+local
+
+# The variable assignment operator '!=' assigns the output of the shell
+# command, as everywhere else.
+var-scope-local-shell.o: VAR != echo output
+
+
+# While VAR=use will be set for a .USE node, it will never be seen since only
+# the ultimate target's context is searched; the variable assignments from the
+# .USE target are not copied to the ultimate target's.
 a_use: .USE VAR=use
        : ${.TARGET} uses .USE VAR="${VAR}"
 
-var-scope-local3.o: a_use
+all: var-scope-local-use.o
+var-scope-local-use.o: a_use