Re: ATF confusion with improperly terminated script

[Garrett Cooper <yanegomi%gmail.com@localhost>]

On Sun, Feb 6, 2011 at 8:00 AM, Julio Merino 
<jmmv%homeworld.netbsd.org@localhost> wrote:
> On Sun, Feb 06, 2011 at 05:53:54AM -0800, Garrett Cooper wrote:
>> Hi again Julio, et all,
>>     Until one runs set -x in the script below, it isn't apparent why
>> the script was failing. Is this issue going to be fixed in the future
>> with kyua?
>
> Error reporting in atf-run is weak.  I have filed issue 13 in Kyua to
> ensure this is not forgotten:
>
> http://code.google.com/p/kyua/issues/detail?id=13
>
> Is this lacking any detail?

I think that captures the bulk majority of my concerns, but there's
another one I'm currently working through to try and understand why
atf-sh isn't properly running testcases that *should* (in theory) work
as the shell code is sane (just a bit convoluted because I'm trying to
integrate the /bin/sh testcases from FreeBSD without converting or
generating them all beforehand).

The result of the integration effort attempt is below and I can post
the error log up on pastebin if desired (but before that it would be
nice if there's a variable I could pass to atf-run to debug what's
happening exactly from ATF's perspective when executing a testcase).

Thanks!
-Garrett

#!/usr/bin/env atf-sh

# $FreeBSD$

export SH=${SH:=/bin/sh}

do_test() {
        local atf_args expected_exit_status testcase_path testcase_prefix

        testcase_path=$1

        expected_exit_status=${testcase_path##*.}
        testcase_prefix=${testcase_path%.*}

        atf_args="-s exit:${expected_exit_status}"

        for i in e,stderr o,stdout; do
                output="${testcase_prefix}.${i##*,}"
                if [ -f "$output" ]; then
                        atf_args="${atf_args} -${i%,*} '$output'"
                fi
        done

        echo "INFO: atf_args: $atf_args, expected_exit_status:
$expected_exit_status, testcase_path: $testcase_path, testcase_prefix:
$testcase_prefix" >&2

        atf_check ${atf_args} ${SH} ${testcase_path}
}

set -x

atf_init_test_cases() {
        local testcase testcase_path

        for testcase_path in $(find -Es . -regex ".*\.[0-9]+"); do

                testcase=$(echo "$testcase_path" | sed -e 's,^./,,g'
-e 's,/,_,g' -e 's,\.,__,g' -e 's,-,____,g')

                atf_test_case "test__$testcase"
                # XXX: would be nice if this was more descriptive,
i.e. maybe comments
                # in the header, or a custom description file.
                eval "test__${testcase}_head() { atf_set 'descr'
'$testcase testcase'; }"
                eval "test__${testcase}_body() { do_test '$testcase_path'; }"

                atf_add_test_case "test__$testcase"

        done
}