Re: Bring BLAS into pkgsrc, pt. II

["Dr. Thomas Orgis" <thomas.orgis%uni-hamburg.de@localhost>]

Am Tue, 23 Jul 2019 09:18:44 -0500
schrieb Jason Bacon <outpaddling%yahoo.com@localhost>: 

> For mk.conf, I think PKGSRC_BLAS_TYPES would be OK, as would
> 
> PKGSRC_BLAS (following examples PKGSRC_FORTRAN, PKGSRC_ADA, etc.)
> 
> or
> 
> PKGSRC_ACCEPTABLE_BLAS (following example PKGSRC_ACCEPTABLE_LICENSES)
> 
> I think the latter is better as it implies that this is a list, as does 
> PKGSRC_BLAS_TYPES.

Hm. The practical meaning is a bit different with
PKGSRC_ACCEPTABLE_LICENSES: You usually only add to these. With BLAS,
you want to limit choices, not expand them. To bring home the point of
it being a list, how about PKGSRC_BLAS_LIST or PKGSRC_BLAS_LIBRARIES …
but then, this is what PKGSRC_BLAS_TYPES does, already. So I'd like to
keep it. A native speaker could discuss about TYPES vs. KINDS, VARIANT,
INCARNATION, whatever;-)

> One lesson I've learned multiple times now is that there's no change too 
> small to cause breakage.  There are only 15 committed packages that 
> depend on blas and 12 on lapack, so testing all of them against the new 
> blas/lapack packages while they're still in wip won't take long.  I'll 
> be happy to help with this.

Well, happy testing, then;-) I'll be knocked out for about two weeks.
Got system hardware maintenance.

> For the upgrade path, I tink what we need here is a recursive revbump 
> on all dependent packages.

Sounds about right. This means increasing revision on all dependent
packages, right?

> What's the reasoning again for installing lib/libblas.* via math/lapack 
> instead of math/blas?  You may have explained this a while back, but I 
> don't recall.

It's hinted at in blas.buildlink3.mk, even:

# Since we always ship BLAS and LAPACK together (as upstream variants
# do), this sets both BLAS_LIBS to the linker flags needed to pull in
# the BLAS library and LAPACK_LIBS to the flags for the LAPACK library.
# Often, they will be identical or at least redundant. It is a matter
# of style to stay consistent in their use.

Even Netlib BLAS and LAPACK comes in one package, together with the C
interfaces. You need to intentionally pull things apart. This used to
be of use to build LAPACK against an external optimized BLAS. Nowadays,
the optimized BLAS implementations (OpenBLAS, MKL) already integrate a
LAPACK, likely pulled from Netlib with possibly some specific
optimizations. If you want to use the libraries like upstream intended,
you use the BLAS+LAPACK from one package. 

While OpenBLAS also will happily give you cblas.h, I opted to keep the
C interfaces separate as wrappers installed from the Netlib code.
They're supposed to contain rather generic glue code and not everyone
ships them. We might try to build BLAS-specific libcblas and liblapacke
in future with each BLAS package (so, libcblas_openblas_pthread.so, for
example), with all offering the interface of $prefix/include/cblas.h and
$prefix/include/lapacke.h. But I am not confident in all interface
headers being identical with differing BLAS underneath. 

Could of course build OpenBLAS with it's C interface and make sure that
any cblas user pulls in the correct cblas.h and no extra lib. The
question is also how much work we really want to put into offering many
BLAS variants at once, along with C interfaces that are not used that
often anyway.

The short answer is: It would be more work separating out LAPACK from
the likes of OpenBLAS.


Alrighty then,

Thomas

-- 
Dr. Thomas Orgis
HPC @ Universität Hamburg