Add WHEEL_ARGS (for BLAS choices for py-numpy and py-scipy / Re: wip/py-numpy + blas )

["Dr. Thomas Orgis" <thomas.orgis%uni-hamburg.de@localhost>]

Am Thu, 16 Nov 2023 10:53:30 +0100
schrieb "Dr. Thomas Orgis" <thomas.orgis%uni-hamburg.de@localhost>:

> Isn't hat fun? I guess I'll have to take a stab at that. Do we have
> anyone actually using numpy for scientific work, though? I'm only
> building it for unspecific users on my systems.

So here is my current approach to properly link math/py-numpy and
math/py-scipy to BLAS stuff. Problems:

1. The coupling between them is via code duplication. I feel we should
   make sure they use the same setup. It's sortof a common project.

2. The accelerate.framework doesn't provide CBLAS_PC etal. This would need
   a wrapper package that I only sketched out in blas.buildlink3.mk but
   did not implement. Anyone up for testing one? Doesn't make sense for me to
   just hack on that. I have no gain and no way to test. I could do an
   Intel MKL one … maybe … comments on the blaswrap ideas?

Anyhow, a main change I am suggesting is addition of the variable
WHEEL_ARGS to be able to provide argumens to python -m build called
from wheel.mk. Is that OK? Please review and discuss before kindly
giving me commit permission for any derivative of this.

Index: lang/python/wheel.mk
===================================================================
RCS file: /cvsroot/pkgsrc/lang/python/wheel.mk,v
retrieving revision 1.11
diff -u -r1.11 wheel.mk
--- lang/python/wheel.mk	29 Oct 2023 22:50:35 -0000	1.11
+++ lang/python/wheel.mk	5 Dec 2023 23:53:07 -0000
@@ -17,6 +17,7 @@
 WHEEL_NAME?=	${DISTNAME:C/-([^0-9])/_\1/g}
 _WHEEL_INFODIR=	${WHEEL_NAME}.dist-info
 PLIST_SUBST+=	WHEEL_INFODIR=${_WHEEL_INFODIR}
+WHEEL_ARGS?=	# empty
 
 PRINT_PLIST_AWK+=	{ gsub(/${_WHEEL_INFODIR:S,.,\.,g}/, "$${WHEEL_INFODIR}") }
 
@@ -24,7 +25,8 @@
 TOOL_DEPENDS+= ${PYPKGPREFIX}-build>=0:../../devel/py-build
 do-build:
 	${RUN} cd ${WRKSRC} && ${SETENV} ${MAKE_ENV} ${TOOL_PYTHONBIN} \
-		-m build --wheel --skip-dependency-check --no-isolation
+		-m build --wheel --skip-dependency-check --no-isolation \
+		${WHEEL_ARGS}
 .endif
 
 .if !target(do-install)
? math/py-numpy/bmake.log
Index: math/py-numpy/Makefile
===================================================================
RCS file: /cvsroot/pkgsrc/math/py-numpy/Makefile,v
retrieving revision 1.115
diff -u -r1.115 Makefile
--- math/py-numpy/Makefile	23 Nov 2023 08:54:23 -0000	1.115
+++ math/py-numpy/Makefile	5 Dec 2023 23:53:17 -0000
@@ -1,6 +1,7 @@
 # $NetBSD: Makefile,v 1.115 2023/11/23 08:54:23 wiz Exp $
 
 DISTNAME=	numpy-1.26.2
+PKGREVISION=	1
 PKGNAME=	${PYPKGPREFIX}-${DISTNAME}
 CATEGORIES=	math python
 MASTER_SITES=	${MASTER_SITE_PYPI:=n/numpy/}
@@ -59,10 +60,16 @@
 # linking, so ensure that we pass through requisite options.
 LDFLAGS+=		${_COMPILER_ABI_FLAG.${ABI}}
 
+# Check if the numpy/scipy ecosystem is ready for full ILP64 before
+# allowing/setting BLAS_INDEX64=yes. Scipy needs both variants right now,
+# wich does not work without standardized symbol suffix.
+# Will need -Csetup-args=Duse-ilp64, too.
 BLAS_ACCEPTED=		${_BLAS_TYPES} accelerate.framework
 BLAS_C_INTERFACE=	yes
-CPPFLAGS+=		${BLAS_INCLUDES}
-LDFLAGS+=		${CBLAS_LIBS} ${LAPACK_LIBS}
+# Yes, cblas and lapack API (not lapacke) is desired.
+# Meson build uses pkg-config since 1.26.2 (or elaborate own logic).
+WHEEL_ARGS+=		-Csetup-args=-Dblas=${CBLAS_PC}
+WHEEL_ARGS+=		-Csetup-args=-Dlapack=${LAPACK_PC}
 
 # Remove invalid __STDC_VERSION__ setting
 BUILDLINK_TRANSFORM.SunOS+=	rm:-D__STDC_VERSION__=0
Index: math/py-scipy/Makefile
===================================================================
RCS file: /cvsroot/pkgsrc/math/py-scipy/Makefile,v
retrieving revision 1.73
diff -u -r1.73 Makefile
--- math/py-scipy/Makefile	19 Nov 2023 17:06:18 -0000	1.73
+++ math/py-scipy/Makefile	6 Dec 2023 00:15:06 -0000
@@ -1,6 +1,7 @@
 # $NetBSD: Makefile,v 1.73 2023/11/19 17:06:18 adam Exp $
 
 DISTNAME=	scipy-1.11.4
+PKGREVISION=	1
 PKGNAME=	${PYPKGPREFIX}-${DISTNAME}
 CATEGORIES=	math python
 MASTER_SITES=	${MASTER_SITE_PYPI:=s/scipy/}
@@ -52,6 +53,13 @@
 
 LDFLAGS.Darwin+=	-headerpad_max_install_names
 
+# See math/py-numpy. Things should match up.
+# Use common include for this?
+BLAS_ACCEPTED=		${_BLAS_TYPES}
+BLAS_C_INTERFACE=	yes
+WHEEL_ARGS+=		-Csetup-args=-Dblas=${CBLAS_PC}
+WHEEL_ARGS+=		-Csetup-args=-Dlapack=${LAPACK_PC}
+
 PYTHON_VERSIONS_INCOMPATIBLE=	27 38
 
 .include "../../mk/bsd.prefs.mk"
@@ -80,7 +88,8 @@
 .include "../../lang/python/application.mk"
 .include "../../lang/python/wheel.mk"
 .include "../../math/fftw/buildlink3.mk"
-.include "../../math/openblas/buildlink3.mk"
+# BLAS choice should rather match py-numpy.
+.include "../../mk/blas.buildlink3.mk"
 BUILDLINK_API_DEPENDS.py-numpy+=	${PYPKGPREFIX}-numpy>=1.21.6
 .include "../../math/py-numpy/buildlink3.mk"
 .include "../../math/py-pythran/buildlink3.mk"



(scipy really was hardlinked to serial openblas? but then … blas.bl3
got pulled in anyway via py-numpy/bl3 … should it be duplicated or not?)


Alrighty then,

Thomas

-- 
Dr. Thomas Orgis
HPC @ Universität Hamburg