install/52173: C compilations can't find header files, sometimes CWRAPPERS_CONFIG_DIR is not found

[lucio%NetBSD.org@localhost,dere%gmail.com@localhost]

>Number:         52173
>Category:       install
>Synopsis:       C compilations can't find header files, sometimes CWRAPPERS_CONFIG_DIR is not found
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    install-manager
>State:          open
>Class:          support
>Submitter-Id:   net
>Arrival-Date:   Mon Apr 17 14:05:01 +0000 2017
>Originator:     Lucio De Re
>Release:        The most recent stable release as at April 17, 2017
>Organization:
>Environment:
NetBSD treacle.proxima.alt.za 7.1 NetBSD 7.1 (TREACLE) #2: Mon Apr 17 05:50:02 SAST 2017  root%treacle.proxima.alt.za@localhost:/usr/src/sys/arch/amd64/compile/TREACLE amd64
>Description:
This is one example:

===> Building for php-7.1.3nb1
/bin/sh /usr/pkg/obj/lang/php71/work.treacle/php-7.1.3/libtool --silent --preserve-dup-deps --mode=compile gcc -Iext/date/lib -DZEND_ENABLE_STATIC_TSRMLS_CACHE=1 -DHAVE_TIMELIB_CONFIG_H=1 -Iext/date/ -I/usr/pkg/obj/lang/php71/work.treacle/php-7.1.3/ext/date/ -DPHP_ATOM_INC -I/usr/pkg/obj/lang/php71/work.treacle/php-7.1.3/include -I/usr/pkg/obj/lang/php71/work.treacle/php-7.1.3/main -I/usr/pkg/obj/lang/php71/work.treacle/php-7.1.3 -I/usr/pkg/obj/lang/php71/work.treacle/php-7.1.3/ext/date/lib -I/usr/pkg/include/libxml2 -I/usr/pkg/obj/lang/php71/work.treacle/php-7.1.3/TSRM -I/usr/pkg/obj/lang/php71/work.treacle/php-7.1.3/Zend  -I/usr/pkg/include -I/usr/include -I/usr/pkg/include  -O2 -pthread -I/usr/pkg/include -I/usr/include -DZEND_SIGNALS   -c /usr/pkg/obj/lang/php71/work.treacle/php-7.1.3/ext/date/php_date.c -o ext/date/php_date.lo 
/usr/pkg/obj/lang/php71/work.treacle/php-7.1.3/ext/date/php_date.c:21:17: fatal error: php.h: No such file or directory
 #include "php.h"
                 ^
compilation terminated.
Makefile:529: recipe for target 'ext/date/php_date.lo' failed
gmake: *** [ext/date/php_date.lo] Error 1

What is obvious is that "php.h" ought to be visible to the compiler, given that its directory is included in one of the -I command line options, but for some reason I cannot diagnose, it is not.

If I attempt to "make" in the work directory, this problem does not arise. This, however does not completely circumvent the issue, although in a few cases it does. When it doesn't a new show stopper occurs: one or other tool reports:

toolname: CWRAPPPERS_CONFIG_DIR is not found in the environment

(or something along those lines - it is an explicit message, I discovered, in the cwrapper sources).

Now, for some additional data points.

1. I had a working amd64 (64-bits) NetBSD 7.1 installation where this problem just did not occur, at all. I also have an i386 (32-bits) NetBSD 7.1 installation which also built and installed all the selected packages (and it is quite a few) with only minor tweaks required.

2. To my uneducated mind, it seems like the depth of nesting of executions involved in building a package somehow exceeds some parameter in the compilation environment and that is how command line arguments and/or environment variables are not conveyed correctly. I tried to reduce the memory footprint of the kernel to free more memory, in case it is the somewhat limited amount (512MiB - for reasons I don't know, that's the maximum vmware allows me at this point, I could not go off in that direction, not enough time :-(). The smaller kernel made no difference, but it may be important to note that I was using the GENERIC kernel until I got a new kernel built.

3. When building the new kernel (the TREACLE in the uname -a output above), the config facility reported a problem with mk.conf that is still baffling me: the .if ${OPSYS} == IRIX (or similar) line in /etc/mk.conf, edited to my requirements. That was a bit worrying. I could not figure what was bothering the make program about it (it looked contextual, rather than obvious).

4. The most recalcitrant package was devel/glib2, which of course is required by a significant number of useful packages. I resorted to installing it from the binary package, but I'm concerned that my preference of options will not be matched by the released packages and that such a mix-and-match approach may create more problems than it solves.

5. Another obstacle I picked up in the process of trying to work around the primary problem occurred in building devel/cmake. Largely, I found one can execute "make" in the target (object) directory if compilation or similar fails and in general that produces a useful result. Then, "make install" in the pkgsrc directory will complete an installation successfully. Sadly, cmake can be built in this fashion, but the product is actually _installed_ in /usr/local which is of no help to someone in my predicament. Again, digging deeper in this case seemed like a massive endeavour likely to fix a problem that should not have occurred in the first place.

6. I'll try to find the time to create a local QEMU instance of NetBSD 7.1 for the amd64 architecture (the host I had has broken down) and see if I can reproduce the previous successful build of the many packages I need, but that will take some time, I'm really hoping we can deal with this problem sooner than that.

7. I'm not sure I need to include the /etc/mk.conf file I use in this space, but maybe it will save an iteration. I'll include in the next input block (How to repeat the problem:). Oops, I can only cut-n-paste it and that could take some doing, it is a long file. Perhaps I can send it later as an attachment?
>How-To-Repeat:

>Fix: