pkg/55126: lang/python38 fails to build on macOS 10.14.6 (Xcode 11.3)

[mj%mjturner.net@localhost]

>Number:         55126
>Category:       pkg
>Synopsis:       lang/python38 fails to build on macOS 10.14.6 (Xcode 11.3)
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Mar 30 16:45:00 +0000 2020
>Originator:     Michael-John Turner
>Release:        pkgsrc -current
>Organization:
>Environment:
Darwin <redacted> 18.7.0 Darwin Kernel Version 18.7.0: Thu Jan 23 06:52:12 PST 2020; root:xnu-4903.278.25~1/RELEASE_X86_64 x86_64
>Description:
When attempting to build lang/python38 (and lang/python37 and lang/python27 - lang/python36 builds fine) on macOS 10.14.6 with Xcode 11.3, the build fails with the following errors:
---
=> Checking file-check results for python38-3.8.2nb1
ERROR: ************************************************************
ERROR: The following files are in the PLIST but not in /usr/obj/pkg/lang/python38/work/.destdir/usr/pkg:
ERROR:         /usr/obj/pkg/lang/python38/work/.destdir/usr/pkg/lib/python3.8/lib-dynload/_ctypes.so
ERROR:         /usr/obj/pkg/lang/python38/work/.destdir/usr/pkg/lib/python3.8/lib-dynload/_hashlib.so
ERROR:         /usr/obj/pkg/lang/python38/work/.destdir/usr/pkg/lib/python3.8/lib-dynload/_ssl.so
---


The issue appears to be related to the use of Xcode 11.3, which uses the macOS 10.15 SDK, even though the host OS is 11.14 in this case. adam@ tested and was able to build the package on macOS 10.14 with Xcode 10.3 (which uses the macOS 10.14 SDK).

Xcode 11.3 is the default on macOS 10.14 when installing from the App Store so it would be ideal if the build worked on that release (ie it doesn't require the manual installation of Xcode 10.x).



>How-To-Repeat:
1. Update pkgsrc to head.
2. On a machine running macOS 10.14.6 and Xcode 11.3.x (eg, 11.3.1), attempt to build lang/python38.
3. The build will fail with the error above.
>Fix:
There are two fixes that resolve this issue:
1) To ensure that _ctypes.so is built, the following diff can be applied to the package Makefile (thanks to adam@ for the pointer):

--- Makefile.orig	2020-03-12 20:08:20.000000000 +0000
+++ Makefile	2020-03-26 11:30:10.000000000 +0000
@@ -51,6 +51,7 @@
 .if ${OPSYS} == "Darwin"
 PY_PLATNAME=	${LOWER_OPSYS}
 USE_TOOLS+=	gmake
+USE_TOOLS+= pkg-config
 .elif ${OPSYS} == "IRIX"
 PY_PLATNAME=	${LOWER_OPSYS:C/\..*//}
 .elif ${OPSYS} == "SunOS"


2) To ensure that _hashlib.so and _ssl.so are built, a patch to setup.py is required. I'm not 100% certain of the correct fix, but a change to function detect_openssl_hashlib() corrected the issue. Removing the code that adds _ssl and _hashlib to self.missing results in both modules being built. 

In particular, the problematic code is:

---
openssl_includes = split_var('OPENSSL_INCLUDES', '-I')
...
# Find OpenSSL includes
ssl_incs = find_file(
	'openssl/ssl.h', self.inc_dirs, openssl_includes
)
if ssl_incs is None:
	self.missing.extend(['_ssl', '_hashlib'])
	return None, None
---

OPENSSL_INCLUDE has value "-I/usr/pkg/include". Checking the .buildlink
directory I see include/openssl/ssl.h but no usr/pkg. Not sure if it's supposed to be there or not?

Hope the above helps - should hopefully be a minor fix for someone more skilled in buildlink than me!