Am Sun, 24 Dec 2023 19:05:41 +0100
schrieb "Dr. Thomas Orgis" <thomas.orgis%uni-hamburg.de@localhost>:
This is on NetBSD. On Darwin I get the _hmmc import error, like you,
but it doesn't reference blas either:
What is your BLAS_TYPE? Did you try with default netlib or also
openblas?
With netlib, I can see your failure (missing dtrsm_). The scipy build
apparently does link with `pkg-config --libs cblas`, which resolves to
-lcblas, but no explict -lblas. Libcblas does require libblas and the
dynamic linker pulls it in. The .pc file correctly includes -lblas when
requesting the line for static linking. I think the .pc is correct here. But
$ readelf -d /data/pkg/lib/libcblas.so|grep blas
0x0000000000000001 (NEEDED) Biblioteca compartida: [libblas.so.3]
0x000000000000000e (SONAME) Nombre-so de la biblioteca: [libcblas.so.3]
Other libs from scipy do reference lapack, and hence pull in libblas,
too:
$ find /data/pkg/lib/python3.11/site-packages/scipy/ -name '*.so' | while read f; do readelf -d $f | grep lapack && echo "^^^^ $f"; done
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/optimize/_lbfgsb.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/optimize/_trlib/_trlib.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/spatial/_qhull.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_propack/_zpropack.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_propack/_dpropack.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_propack/_cpropack.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_propack/_spropack.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/arpack/_arpack.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_isolve/_iterative.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/linalg/_flinalg.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/linalg/_interpolative.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/linalg/cython_blas.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/linalg/cython_lapack.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/linalg/_fblas.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/linalg/_flapack.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/special/_ellip_harm_2.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/special/_ufuncs.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/integrate/_vode.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/integrate/_lsoda.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/integrate/_odepack.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/integrate/_quadpack.so
0x0000000000000001 (NEEDED) Biblioteca compartida: [liblapack.so.3]
^^^^ /data/pkg/lib/python3.11/site-packages/scipy/integrate/_test_odeint_banded.so
But _dsolve/superlu.so needs plain BLAS (just as math/superlu).
The meson code of numpy and scipy is warped … it accepts blas and
lapack options, but handles cblas as a kind of subdependency. It sorta
works when giving it blas=${CBLAS_PC}, as this qualifies as 'BLAS with
CBLAS symbols'. Meson output during scipy setup:
Run-time dependency blas found: YES 3.11.0
Run-time dependency cblas found: YES 3.11.0
Run-time dependency lapack found: YES 3.11.0
I presume -Wl,--as-needed is in effect, where
gcc -o foo foo.o -lcblas
will result in the -lcblas being dropped if no symbols from it are
referenced. With superlu, the dependency is on a dependency of
libcblas, not on this lib itself.
This all is no issue if linking with a combined BLAS library that has
all 4 parts in it. Our approach of separating libblas, libcblas,
liblapack, liblapacke for the netlib variant, as this are separate
upstream components, is not anticipated.
Should I hack the .pc file of libcblas to explcitly include -lblas
although that should not be necessary for the dynamic lib case? Should
py-scipy be hacked not to build superlu itself?
I guess the proper solution is to intervene in what the numpy folks are
doing with their vendored copy of meson. They include it becuase of the
very BLAS machinery that is causing trouble right now. We need to get
the idea in there that BLAS and CBLAS can be two distinct libraries, as
it comes from upstream netlib naturally. Numpy folks seem to assume
that it's just 'a BLAS library that also includes CBLAS symbols'.
In any case, this needs fixing during freeze. Hotfix would be to add
-lblas to math/cblas .pc file, and likewise -lblas to lapack.pc and
-llapack -lblas to lapacke.pc. But this warps pkg-config logic that
already is smart enough to take care of these indirect deps with
--static. The proper solution is that a build links with the correct
library it is using symbols from. It's a question, though, if we can
convince the world to cater for an install of netlib reference
implementation when everyone plans for the usecase of at least
openblas, or some vendor optimized library, which are all downstream of
netlib.
I'll see if I can get a response from the numpy/meson side. This should
not be forgotten before release … scipy is not good right now.
Alrighty then,
Thomas