Source-Changes-HG archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
[src/trunk]: src Add a new test program to test the "intermediate" shell buil...
details: https://anonhg.NetBSD.org/src/rev/1b63ded8db8f
branches: trunk
changeset: 446431:1b63ded8db8f
user: kre <kre%NetBSD.org@localhost>
date: Wed Dec 05 02:45:05 2018 +0000
description:
Add a new test program to test the "intermediate" shell built in
utilities. That is, not the low level ones that look like syntax,
but aren't: break/continue/return; not those which are really
just external programs that are built in for efficiency (printf,
test, and kill - though kill really needs to be built in) - those
should all have separate test programs (there is a test here for the
built-in echo, as that is an entirely different thing to /bin/echo);
and also not those for which there are other tests because of the
nature of the built-in (like exit, wait, shift, ...). Lastly not
"times" either as that just seems to be too hard to test rationally.
There is a test (well, framework) for ulimit and there's also t_ulimit.sh
one of those should go, but I am not yet sure which is the best way
to reconcile things.
Note: many (in fact) most of the test cases added here are either
entirely empty (no tests at all, beyond testing that the built-in is
in fact a shell built-in) or only very rudimentary tests - assistance
in fleshing those out would be welcome (the boilerplate is all here,
all that is needed is some actual tests to run...)
diffstat:
distrib/sets/lists/tests/mi | 3 +-
tests/bin/sh/Makefile | 3 +-
tests/bin/sh/t_builtins.sh | 793 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 797 insertions(+), 2 deletions(-)
diffs (truncated from 831 to 300 lines):
diff -r a5eb53024175 -r 1b63ded8db8f distrib/sets/lists/tests/mi
--- a/distrib/sets/lists/tests/mi Tue Dec 04 22:06:36 2018 +0000
+++ b/distrib/sets/lists/tests/mi Wed Dec 05 02:45:05 2018 +0000
@@ -1,4 +1,4 @@
-# $NetBSD: mi,v 1.795 2018/09/23 13:36:04 christos Exp $
+# $NetBSD: mi,v 1.796 2018/12/05 02:45:05 kre Exp $
#
# Note: don't delete entries from here - mark them as "obsolete" instead.
#
@@ -1270,6 +1270,7 @@
./usr/tests/bin/sh/dotcmd/while_return_until tests-bin-tests compattestfile,atf
./usr/tests/bin/sh/dotcmd/while_return_while tests-bin-tests compattestfile,atf
./usr/tests/bin/sh/t_arith tests-bin-tests compattestfile,atf
+./usr/tests/bin/sh/t_builtins tests-bin-tests compattestfile,atf
./usr/tests/bin/sh/t_cmdsub tests-bin-tests compattestfile,atf
./usr/tests/bin/sh/t_compexit tests-obsolete obsolete
./usr/tests/bin/sh/t_evaltested tests-bin-tests compattestfile,atf
diff -r a5eb53024175 -r 1b63ded8db8f tests/bin/sh/Makefile
--- a/tests/bin/sh/Makefile Tue Dec 04 22:06:36 2018 +0000
+++ b/tests/bin/sh/Makefile Wed Dec 05 02:45:05 2018 +0000
@@ -1,4 +1,4 @@
-# $NetBSD: Makefile,v 1.13 2018/07/10 06:49:29 kre Exp $
+# $NetBSD: Makefile,v 1.14 2018/12/05 02:45:06 kre Exp $
#
.include <bsd.own.mk>
@@ -8,6 +8,7 @@
TESTS_SUBDIRS += dotcmd
TESTS_SH+= t_arith
+TESTS_SH+= t_builtins
TESTS_SH+= t_cmdsub
TESTS_SH+= t_evaltested
TESTS_SH+= t_exit
diff -r a5eb53024175 -r 1b63ded8db8f tests/bin/sh/t_builtins.sh
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/tests/bin/sh/t_builtins.sh Wed Dec 05 02:45:05 2018 +0000
@@ -0,0 +1,793 @@
+# $NetBSD: t_builtins.sh,v 1.1 2018/12/05 02:45:06 kre Exp $
+#
+# Copyright (c) 2017 The NetBSD Foundation, Inc.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+#
+# the implementation of "sh" to test
+: ${TEST_SH:="/bin/sh"}
+
+#
+# This file tests the various sh builtin utilities.
+#
+# Those utilities that are really external programs, which are builtin in
+# for (mostly) performance (printf, kill, test, ...), are tested elsewhere.
+# We do test the builtin "echo" here as (in NetBSD) it is different than
+# the external one.
+#
+# The (mostly special) builtins which appear to be more syntax than command
+# are tested in other test programs, rather than here (break, continue...)
+#
+# And finally, those which are fundamental to the operation of the shell,
+# like wait, set, shift, ... are also tested in other test programs where
+# all their operations can be more thoroughly verified.
+#
+# This leaves those which need to be built in (cd, umask, ...) but whose
+# purpose is mostly to alter the environment in which the shell operates
+# of that of the commands it runs. These tests act in co-operation with
+# other tests exist here (where thy do) by not duplicating tests run
+# elsewhere (ulimit is one example) but just adding to those.
+# One day these might be unified.
+#
+# We do test both standard use of the builtins (where they are standard)
+# and NetBSD sh extensions (when run on a shell with no support, such tests
+# should be skipped.)
+#
+
+# Utility function able to test whether most of the builtins exist in
+# the shell being tested.
+have_builtin()
+{
+ ${TEST_SH} -c "( $3 $1 $4 ) >/dev/null 2>&1" &&
+ LC_ALL=C ${TEST_SH} -c \
+ 'case "$( (type '"$1"') 2>&1)" in
+ (*built*) exit 0 ;;
+ esac
+ exit 1' ||
+ {
+ test -z "$2" && atf_skip "${TEST_SH} has no '$1$5' built-in"
+ return 1;
+ }
+
+ return 0
+}
+
+### Helper functions
+
+nl='
+'
+reset()
+{
+ TEST_NUM=0
+ TEST_FAILURES=''
+ TEST_FAIL_COUNT=0
+ TEST_ID="$1"
+
+ # These are used in check()
+ atf_require_prog tr
+ atf_require_prog printf
+ atf_require_prog mktemp
+}
+
+# Test run & validate.
+#
+# $1 is the command to run (via sh -c)
+# $2 is the expected output
+# $3 is the expected exit status from sh
+# $4 is optional extra data for the error msg (if there is one)
+#
+# Stderr is exxpected to be empty, unless the expected exit code ($3) is != 0
+# in which case some message there is expected (and nothing is a failure).
+# When non-zero exit is expected, we note a different (non-zero) value
+# observed, but do not fail the test because of that.
+
+check()
+{
+ fail=false
+ TEMP_FILE=$( mktemp OUT.XXXXXX )
+ TEST_NUM=$(( $TEST_NUM + 1 ))
+ MSG=
+
+ # our local shell (ATF_SHELL) better do quoting correctly...
+ # some of the tests expect us to expand $nl internally...
+ CMD="$1"
+
+ # determine what the test generates, preserving trailing \n's
+ result="$( ${TEST_SH} -c "${CMD}" 2>"${TEMP_FILE}" && printf X )"
+ STATUS=$?
+ result="${result%X}"
+
+
+ if [ "${STATUS}" -ne "$3" ]; then
+ MSG="${MSG}${MSG:+${nl}}[$TEST_NUM]"
+ MSG="${MSG} expected exit code $3, got ${STATUS}"
+
+ # don't actually fail just because of wrong exit code
+ # unless we either expected, or received "good"
+ # or something else is detected as incorrect as well.
+ case "$3/${STATUS}" in
+ (*/0|0/*) fail=true;;
+ esac
+ fi
+
+ if [ "$3" -eq 0 ]; then
+ if [ -s "${TEMP_FILE}" ]; then
+ MSG="${MSG}${MSG:+${nl}}[$TEST_NUM]"
+ MSG="${MSG} Messages produced on stderr unexpected..."
+ MSG="${MSG}${nl}$( cat "${TEMP_FILE}" )"
+ fail=true
+ fi
+ else
+ if ! [ -s "${TEMP_FILE}" ]; then
+ MSG="${MSG}${MSG:+${nl}}[$TEST_NUM]"
+ MSG="${MSG} Expected messages on stderr,"
+ MSG="${MSG} nothing produced"
+ fail=true
+ fi
+ fi
+ rm -f "${TEMP_FILE}"
+
+ if [ "$2" != "${result}" ]
+ then
+ MSG="${MSG}${MSG:+${nl}}[$TEST_NUM]"
+ MSG="${MSG} Expected: <<$2>>, received: <<$result>>"
+ fail=true
+ fi
+
+ if $fail
+ then
+ if [ -n "$4" ]; then
+ MSG="${MSG}${MSG:+${nl}}[$TEST_NUM] Note: ${4}"
+ fi
+ MSG="${MSG}${MSG:+${nl}}[$TEST_NUM]"
+ MSG="${MSG} Full command: <<${CMD}>>"
+ fi
+
+ $fail && test -n "$TEST_ID" && {
+ TEST_FAILURES="${TEST_FAILURES}${TEST_FAILURES:+${nl}}"
+ TEST_FAILURES="${TEST_FAILURES}${TEST_ID}[$TEST_NUM]:"
+ TEST_FAILURES="${TEST_FAILURES} Test of <<$1>> failed.";
+ TEST_FAILURES="${TEST_FAILURES}${nl}${MSG}"
+ TEST_FAIL_COUNT=$(( $TEST_FAIL_COUNT + 1 ))
+ return 0
+ }
+ $fail && atf_fail "Test[$TEST_NUM] failed: $(
+ # ATF does not like newlines in messages, so change them...
+ printf '%s' "${MSG}" | tr '\n' ';'
+ )"
+ return 0
+}
+
+results()
+{
+ test -n "$1" && atf_expect_fail "$1"
+
+ test -z "${TEST_ID}" && return 0
+ test -z "${TEST_FAILURES}" && return 0
+
+ echo >&2 "=========================================="
+ echo >&2 "While testing '${TEST_ID}'"
+ echo >&2 " - - - - - - - - - - - - - - - - -"
+ echo >&2 "${TEST_FAILURES}"
+
+ atf_fail \
+ "Test ${TEST_ID}: $TEST_FAIL_COUNT (of $TEST_NUM) subtests failed - see stderr"
+}
+
+####### End helpers
+
+atf_test_case colon
+colon_head() {
+ atf_set "descr" "Tests the shell special builtin ':' command"
+}
+colon_body() {
+ have_builtin : || return 0
+
+ atf_check -s exit:0 -e empty -o empty ${TEST_SH} -c ":"
+
+ # ':' is a special builtin, so we should exit on redirect error
+ # and variable assignments should persist (stupid, but it is the rule)
+
+ atf_check -s not-exit:0 -e not-empty -o empty ${TEST_SH} -c \
+ ": >/foo/bar; printf %s No-exit-BUG"
+ atf_check -s exit:0 -e empty -o inline:OK ${TEST_SH} -c \
+ 'X=BUG; X=OK : ; printf %s "${X}"'
+}
+
+atf_test_case echo
+echo_head() {
+ atf_set "descr" "Tests the shell builtin version of echo"
+}
+echo_body() {
+ have_builtin echo || return 0
+
+ unset NETBSD_SHELL 2>/dev/null
+ if test -z "$( ${TEST_SH} -c 'printf %s "${NETBSD_SHELL}"')"; then
+ atf_skip \
+ "${TEST_SH%% *} is not the NetBSD shell, this test is for it alone"
+ return 0
+ fi
+
+ reset echo
+
+ check 'echo "hello world"' "hello world${nl}" 0
+ check 'echo hello world' "hello world${nl}" 0
+ check 'echo -n "hello world"' "hello world" 0
+ check 'IFS=:; echo hello world' "hello world${nl}" 0
+ check 'IFS=; echo hello world' "hello world${nl}" 0
+
+ check 'echo -e "hello world"' "hello world${nl}" 0
+ check 'echo -e hello world' "hello world${nl}" 0
+ check 'IFS=:; echo -e hello world' "hello world${nl}" 0
+
+ # only one of the options is used
+ check 'echo -e -n "hello world"' "-n hello world${nl}" 0
+ check 'echo -n -e "hello world"' "-e hello world" 0
+ # and only when it is alone
+ check 'echo -en "hello world"' "-en hello world${nl}" 0
+ check 'echo -ne "hello world"' "-ne hello world${nl}" 0
+
+ # echo is specifically required to *not* support --
+ check 'echo -- "hello world"' "-- hello world${nl}" 0
+
+ # similarly any other unknown option is simply part of the output
+ for OPT in a b c v E N Q V 0 1 2 @ , \? \[ \] \( \; . \* -help -version
+ do
+ check "echo '-${OPT}' foo" "-${OPT} foo${nl}" 0
+ done
+
+ # Now test the \\ expansions, with and without -e
+
+ # We rely upon printf %b (tested elsewhere, not only a sh test)
+ # to verify the output when the \\ is supposed to be expanded.
Home |
Main Index |
Thread Index |
Old Index