tech-pkg archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Stumped on pytest issue

On 12/24/23 08:35, Jason Bacon wrote:

On 12/23/23 15:01, Dr. Thomas Orgis wrote:

Am Sat, 23 Dec 2023 08:29:24 -0600
schrieb Jason Bacon <jtocino%gmx.com@localhost>:


/home/bacon/Pkgsrc/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_dsolve/_superlu.so:
Undefined PLT symbol "dnrm2_" (symnum = 2)


Your scipy superlu misses a BLAS symbol from libblas.so / libopenblas.so.

Did you build it with old superlu that provided

$ readelf -d -s /data/pkg/lib/libopenblas_openmp.so |grep -w dnrm2_
   5951: 0000000000138cd0   101 FUNC    GLOBAL DEFAULT   12 dnrm2_
  18042: 0000000000138cd0   101 FUNC    GLOBAL DEFAULT   12 dnrm2_
$ ldd /data/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_dsolve/_superlu.so |grep blas     libopenblas_openmp.so.0 => /data/pkg/lib/libopenblas_openmp.so.0 (0x00007f5dcc5b8000) 

It occurs to me that this probably includes an vendored copy of superlu
and is not linking to math/superlu. Might consider changing that in
scipy. Anyhow, my scipy doesn't have that problem …

Trying to reproduce … is this a step further or one behind?


=================================================================================================================================== ERRORS ====================================================================================================================================
________________________________________________________________________________________________________________________ ERROR collecting test session ________________________________________________________________________________________________________________________
/data/pkg/lib/python3.11/site-packages/_pytest/config/__init__.py:641: in _importconftest 
     mod = import_path(conftestpath, mode=importmode, root=rootpath)
/data/pkg/lib/python3.11/site-packages/_pytest/pathlib.py:567: in import_path 
     importlib.import_module(module_name)
/data/pkg/lib/python3.11/importlib/__init__.py:126: in import_module
     return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1204: in _gcd_import
     ???
<frozen importlib._bootstrap>:1176: in _find_and_load
     ???
<frozen importlib._bootstrap>:1126: in _find_and_load_unlocked
     ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
     ???
<frozen importlib._bootstrap>:1204: in _gcd_import
     ???
<frozen importlib._bootstrap>:1176: in _find_and_load
     ???
<frozen importlib._bootstrap>:1147: in _find_and_load_unlocked
     ???
<frozen importlib._bootstrap>:690: in _load_unlocked
     ???
<frozen importlib._bootstrap_external>:940: in exec_module
     ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
     ???
lib/hmmlearn/tests/__init__.py:9: in <module>
     from hmmlearn.base import DECODER_ALGORITHMS
lib/hmmlearn/base.py:12: in <module>
     from . import _hmmc, _kl_divergence as _kl, _utils
E   ImportError: cannot import name '_hmmc' from 'hmmlearn' (/data/projekte/pkgsrc/work/wip/py-hmmlearn/work/hmmlearn-0.3.0/lib/hmmlearn/__init__.py) 
=========================================================================================================================== short test summary info ===========================================================================================================================
ERROR  - ImportError: cannot import name '_hmmc' from 'hmmlearn' (/data/projekte/pkgsrc/work/wip/py-hmmlearn/work/hmmlearn-0.3.0/lib/hmmlearn/__init__.py) 
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================================================== 1 error in 1.04s ===============================================================================================================================
*** Error code 2

Stop.
Yeah, my scipy superlu doesn't seem to be picking up blas at all.  What scipy are you using, and on what OS?  There's no reference to superlu in the scipy pkgsrc Makefile, . 

NetBSD netbsd9.acadix  bacon ~ 1011: (pkgsrc): pkg_info | grep scipy
py311-scipy-1.11.4nb1 Scientific Algorithms Library for Python
NetBSD netbsd9.acadix  bacon ~ 1012: (pkgsrc): ldd ~/Pkgsrc/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_dsolve/_superlu.so 
/home/bacon/Pkgsrc/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_dsolve/_superlu.so:
         -lm.0 => /usr/lib/libm.so.0
         -lc.12 => /usr/lib/libc.so.12

That's all: No pruning here.

My superlu is the latest:

NetBSD netbsd9.acadix  bacon ~ 1014: (pkgsrc): !pkg_info
pkg_info | grep superlu
superlu-6.0.1nb1    Library for solving large, sparse, nonsymmetric systems of linear equations
NetBSD netbsd9.acadix  bacon ~/ 1015: (pkgsrc): head ~/Pkgsrc/pkgsrc/math/superlu/Makefile 
# $NetBSD: Makefile,v 1.35 2023/12/19 11:53:01 thor Exp $

DISTNAME=               superlu-6.0.1
PKGREVISION=            1
This is on NetBSD.  On Darwin I get the _hmmc import error, like you, but it doesn't reference blas either:
Darwin tarpon.local  bacon ~ 1012: (pkgsrc): otool -L ~/Pkgsrc/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_dsolve/_superlu.so 
/Users/bacon/Pkgsrc/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_dsolve/_superlu.so:
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.0.0) 



And on Alma 8:
Linux alma8.acadix bacon ~/Pkgsrc/pkgsrc/wip/py-hmmlearn 1010: (pkgsrc): bmake test 
=> Bootstrap dependency digest>=20211023: found digest-20220214
===> Checking for vulnerabilities in py311-hmmlearn-0.3.0
=> Test dependency py311-test>=5.4.1: found py311-test-7.4.3
===> Testing for py311-hmmlearn-0.3.0
cd /home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/hmmlearn-0.3.0 && /bin/env USETOOLS=no PTHREAD_CFLAGS=\ -pthread\ PTHREAD_LDFLAGS=\ -pthread PTHREAD_LIBS=-lpthread\ -lrt PTHREADBASE=/usr NPY_BLAS_LIBS=-lblas NPY_LAPACK_LIBS=-llapack\ -lblas NPY_CBLAS_LIBS=-lcblas\ -lblas PYTHON=/home/bacon/Pkgsrc/pkg/bin/python3.11 CC=cc CFLAGS=-O2\ -I/home/bacon/Pkgsrc/pkg/include\ -I/usr/include\ -I/home/bacon/Pkgsrc/pkg/include/python3.11 CPPFLAGS=-I/home/bacon/Pkgsrc/pkg/include/netlib\ -I/home/bacon/Pkgsrc/pkg/include\ -I/usr/include\ -I/home/bacon/Pkgsrc/pkg/include/python3.11 CXX=c++ CXXFLAGS=-O2\ -I/home/bacon/Pkgsrc/pkg/include\ -I/usr/include\ -I/home/bacon/Pkgsrc/pkg/include/python3.11 COMPILER_RPATH_FLAG=-Wl,-R F77=gfortran FC=gfortran FFLAGS=-O LANG=C LC_ALL=C LC_COLLATE=C LC_CTYPE=C LC_MESSAGES=C LC_MONETARY=C LC_NUMERIC=C LC_TIME=C LDFLAGS=-Wl,-zrelro\ -L/home/bacon/Pkgsrc/pkg/lib\ -L/usr/lib64\ -Wl,-R/usr/lib64\ -Wl,-R/home/bacon/Pkgsrc/pkg/lib LINKER_RPATH_FLAG=-R PATH=/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.cwrapper/bin:/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.buildlink/bin:/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.gcc/bin:/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.tools/bin:/home/bacon/Pkgsrc/pkg/bin:/home/bacon/scripts:/home/bacon/Pkgsrc/pkg/bin:/home/bacon/Pkgsrc/pkg/sbin:/usr/local/bin:/home/bacon/scripts:/usr/pkg/bin:/usr/pkg/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/sbin:/usr/sbin:/usr/local/sbin:/usr/pkg/sbin:/home/bacon/scripts:/home/bacon/Pkgsrc/pkg/bin:/home/bacon/Pkgsrc/pkg/bin PREFIX=/home/bacon/Pkgsrc/pkg MAKELEVEL=0 CONFIG_SITE= PKG_SYSCONFDIR=/home/bacon/Pkgsrc/pkg/etc HOME=/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.home CWRAPPERS_CONFIG_DIR=/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.cwrapper/config CPP=cpp LOCALBASE=/home/bacon/Pkgsrc/pkg X11BASE=/home/bacon/Pkgsrc/pkg PKGMANDIR=man PKGINFODIR=info PKGGNUDIR=gnu/ MAKECONF=/dev/null OBJECT_FMT=ELF USETOOLS=no BSD_INSTALL_PROGRAM=/bin/install\ -c\ -s\ -o\ bacon\ -g\ bacon\ -m\ 755 BSD_INSTALL_SCRIPT=/bin/install\ -c\ -o\ bacon\ -g\ bacon\ -m\ 755 BSD_INSTALL_LIB=/bin/install\ -c\ -o\ bacon\ -g\ bacon\ -m\ 755 BSD_INSTALL_DATA=/bin/install\ -c\ -o\ bacon\ -g\ bacon\ -m\ 644 BSD_INSTALL_MAN=/bin/install\ -c\ -o\ bacon\ -g\ bacon\ -m\ 644 BSD_INSTALL=/bin/install BSD_INSTALL_PROGRAM_DIR=/bin/install\ -d\ -o\ bacon\ -g\ bacon\ -m\ 755 BSD_INSTALL_SCRIPT_DIR=/bin/install\ -d\ -o\ bacon\ -g\ bacon\ -m\ 755 BSD_INSTALL_LIB_DIR=/bin/install\ -d\ -o\ bacon\ -g\ bacon\ -m\ 755 BSD_INSTALL_DATA_DIR=/bin/install\ -d\ -o\ bacon\ -g\ bacon\ -m\ 755 BSD_INSTALL_MAN_DIR=/bin/install\ -d\ -o\ bacon\ -g\ bacon\ -m\ 755 BSD_INSTALL_GAME=/bin/install\ -c\ -s\ -o\ bacon\ -g\ bacon\ -m\ 0755 BSD_INSTALL_GAME_DATA=/bin/install\ -c\ -o\ bacon\ -g\ bacon\ -m\ 0644 BSD_INSTALL_GAME_DIR=/bin/install\ -d\ -o\ bacon\ -g\ bacon\ -m\ 0755 INSTALL_INFO= MAKEINFO=/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.tools/bin/makeinfo FLEX= BISON= ITSTOOL=/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.tools/bin/itstool GDBUS_CODEGEN=/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.tools/bin/gdbus-codegen PKG_CONFIG=/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.tools/bin/pkg-config PKG_CONFIG_LIBDIR=/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.buildlink/lib64/pkgconfig:/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.buildlink/lib/pkgconfig:/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.buildlink/share/pkgconfig PKG_CONFIG_LOG=/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.pkg-config.log PKG_CONFIG_PATH= CWRAPPERS_CONFIG_DIR=/home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/.cwrapper/config pytest-3.11 --capture=no ============================= test session starts ============================== 
platform linux -- Python 3.11.7, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn/work/hmmlearn-0.3.0
configfile: setup.cfg
collected 0 items / 1 error
==================================== ERRORS ==================================== ________________________ ERROR collecting test session _________________________ ../../../../../pkg/lib/python3.11/site-packages/_pytest/config/__init__.py:641: in _importconftest 
    mod = import_path(conftestpath, mode=importmode, root=rootpath)
../../../../../pkg/lib/python3.11/site-packages/_pytest/pathlib.py:567: in import_path 
    importlib.import_module(module_name)
../../../../../pkg/lib/python3.11/importlib/__init__.py:126: in import_module 
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1204: in _gcd_import
    ???
<frozen importlib._bootstrap>:1176: in _find_and_load
    ???
<frozen importlib._bootstrap>:1126: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1204: in _gcd_import
    ???
<frozen importlib._bootstrap>:1176: in _find_and_load
    ???
<frozen importlib._bootstrap>:1147: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:690: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:940: in exec_module
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
lib/hmmlearn/tests/__init__.py:5: in <module>
    from sklearn.datasets import make_spd_matrix
../../../../../pkg/lib/python3.11/site-packages/sklearn/__init__.py:83: in <module> 
    from .base import clone
../../../../../pkg/lib/python3.11/site-packages/sklearn/base.py:19: in <module> 
    from .utils import _IS_32BIT
../../../../../pkg/lib/python3.11/site-packages/sklearn/utils/__init__.py:15: in <module> 
    from scipy.sparse import issparse
../../../../../pkg/lib/python3.11/site-packages/scipy/sparse/__init__.py:287: in <module> 
    from . import csgraph
../../../../../pkg/lib/python3.11/site-packages/scipy/sparse/csgraph/__init__.py:185: in <module> 
    from ._laplacian import laplacian
../../../../../pkg/lib/python3.11/site-packages/scipy/sparse/csgraph/_laplacian.py:7: in <module> 
    from scipy.sparse.linalg import LinearOperator
../../../../../pkg/lib/python3.11/site-packages/scipy/sparse/linalg/__init__.py:121: in <module> 
    from ._dsolve import *
../../../../../pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_dsolve/__init__.py:58: in <module> 
    from .linsolve import *
../../../../../pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_dsolve/linsolve.py:11: in <module> 
    from . import _superlu
E ImportError: /home/bacon/Pkgsrc/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_dsolve/_superlu.so: undefined symbol: dtrsm_ =========================== short test summary info ============================ ERROR - ImportError: /home/bacon/Pkgsrc/pkg/lib/python3.11/site-packages/scipy/spar... !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!! =============================== 1 error in 0.53s =============================== 
*** Error code 2

Stop.
bmake[1]: stopped in /home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn
*** Error code 1

Stop.
bmake: stopped in /home/bacon/Pkgsrc/pkgsrc/wip/py-hmmlearn
Linux alma8.acadix bacon ~/Pkgsrc/pkgsrc/wip/py-hmmlearn 1011: (pkgsrc): !ldd ldd ~/Pkgsrc/pkg/lib/python3.11/site-packages/scipy/sparse/linalg/_dsolve/_superlu.so 
        linux-vdso.so.1 (0x00007ffd541c6000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f94f4dde000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f94f4a19000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f94f53ad000)
Note that this issue is not worth risking breakage over right now. I got py-macs2 to build, so if this will involve any significant changes to scipy or other packages, py-macs3 can wait until after the branch.
Home | Main Index | Thread Index | Old Index