Re: Bring BLAS into pkgsrc, pt. II

[Jason Bacon <outpaddling%yahoo.com@localhost>]

On 2019-07-17 12:13, Dr. Thomas Orgis wrote:
> Hello again from the scientific computing corner,
>
> I just pushed a new version of wip/mk/blas.buildlink3.mk (also attached
> for easier review), incorporating changes from our discussion in March
> to May [1].
>
> I really would like to get this from wip into pkgsrc proper. The steps:
>
> 1. Import blas.bl3 into pkgsrc/mk
>
> 2. Import these packages and make them use mk/blas.bl3 instead of
>     wip/mk/blas.bl3, where appropriate:
>
> 	wip/lapack: Netlib reference (libblas.so, liblapack.so)
> 	wip/blas: a dummy that just depends on the above
> 	wip/openblas: single-threaded OpenBLAS (libopenblas.so)
> 	wip/openblas_pthread: threaded OpenBLAS (libopenblas_pthread.so)
> 	wip/openblas_openmp: OpenMP OpenBLAS (libopenblas_openmp.so)
> 	wip/cblas: C wrapper over BLAS
> 	wip/lapacke: C wrapper over LAPACK
>
> 3. Start replacing any of
>
> 	.include "../../math/blas/buildlink3.mk"
> 	.include "../../math/lapack/buildlink3.mk"
>
> with
>
> 	.include "../../mk/blas.buildlink3.mk"
>
> in packages and possibly add/change machinery to actually support
> differing implementations (add BLAS_LIBS to MAKE_ENV, for example).
>
> I have patches ready for math/octave, math/py-numpy, math/R,
> math/R-gstat, and math/R-quantreg.
>
> We can pause after each step without breaking existing packages. The
> upgrade from an installed math/blas might be uncomfortable, as the
> binaries all move to math/lapack. I don't know what I can do about
> that. But apart from that, I expect a gentle process.
>
> So, can we make it happen?
>
>
> Alrighty then,
>
> Thomas
>
> [1] https://mail-index.netbsd.org/tech-pkg/2019/03/26/msg020982.html
Seems to work as advertised.

I tested various scenarios using wip/gemma.

It errors out if there is no commonality between PKGSRC_BLAS_PREFERRED 
and BLAS_ACCEPTED.

Otherwise, it chooses the first match listed in PKGSRC_BLAS_PREFERRED.

A couple of comments:

1. As I mentioned earlier, I think the variable names could be more 
explicit to avoid confusion between what goes in mk.conf and what goes 
in the pkg Makefile.  Also, given that the system errors out if no match 
is found, PKGSRC_BLAS_PREFERRED as-implemented does not really just 
state a preference, but a requirement.  Hence, I would suggest the 
following changes (or something similar):

PKGSRC_BLAS_PREFERRED => PKGSRC_USER_BLAS_ACCEPTED
BLAS_ACCEPTED => PKGSRC_PKG_BLAS_ACCEPTED

I don't feel that strongly about this, but I think it would help future 
developers who have not been part of this discussion avoid mistakes, and 
would be appreciated.

2. I think we need to insert a step 1.5 into the process:

Test/patch all existing dependent packages against the new system before 
committing the new blas and lapack packages.  If any changes are needed, 
duplicate the affected packages in wip with necessary patches.  Then 
we'll then be prepared to commit them immediately after upgrading the 
blas/lapack packages.

Thanks again for all your work on this!

    JB