Re: Deciding on wich variant(s) of OpenBLAS library to install

[Jason Bacon <outpaddling%yahoo.com@localhost>]

On 03/12/18 14:16, Dr. Thomas Orgis wrote:
> Am Mon, 12 Mar 2018 12:48:55 -0500
> schrieb Jason Bacon <outpaddling%yahoo.com@localhost>:
>
> > Trying to make them interchangeable would be somewhat like
> > trying to make all POSIX operating systems interchangeable, but on a
> > smaller scale.
> This is a bit harsh. Sure, POSIX systems _are_ interchangeable, to some
> degree, but the complexities of differing BLAS implementations only
> really start once you include parallelization which changes the game in
> case you link to an application that also does some kind of
> parallelization.
>
> The job of getting serial BLAS interchangeable needs some work, but it
> has been done already: Look at Debian.
>
> 	https://packages.debian.org/stretch/libblas.so.3
>
> When you look closer at their libopenblas:
>
> /usr/lib/libopenblas.so.0
> /usr/lib/libopenblasp-r0.2.19.so
> /usr/lib/openblas-base/libblas.so.3
> /usr/lib/openblas-base/liblapack.so.3
> /usr/share/doc/libopenblas-base/README.Debian
> /usr/share/doc/libopenblas-base/TODO.Debian
> /usr/share/doc/libopenblas-base/changelog.Debian.gz
> /usr/share/doc/libopenblas-base/changelog.gz
> /usr/share/doc/libopenblas-base/copyright
>
> You notice that they have namespaced wrapper libs with the generic name
> libblas.so.3 and liblapack.so.3 (they come together nowadays) and
> switch among the variants via the alternatives system.
>
> What you also should notice is the libopenblasp file. This is a
> parallel build and the parallel variants are not included in the
> generic BLAS abstraction at all right now. They did have issues with
> libopenblasp switching between OpenMP and pthreads … different things
> broke with each of them. The situation is better nowadays, but the
> potential for conflict of the paralellization with a host application
> or other libs is of course still present. That is why I would not make
> a parallel BLAS the default setting. Only the local admin may choose to
> do so in the installation, or just choose it in a certain package like
> Jason is suggesting. I am stressing that the local admin shall make that
> choice, not the packager.
>
> I am not proposing (anymore) that we really go the interchangeable
> libblas route in pkgsrc. We don't have to, as pkgsrc can (re)build things
> from source. I'd just like my global switch and am fine with offering
> all variants of BLAS to the users at the same time for building their
> own software.
Had I not said "on a smaller scale", I think it would have been harsh, yes.
But the issues are analogous, as they are with Linux/NetBSD, clang/gcc, 
libc++/libstdc++,
and any other independent attempts to implement the same complex 
specification.
My perspective is based on years of experience rather than the details 
of the
implementations.  There have always been differences in how dependent 
software
functions with different BLAS/LAPACK implementations and my world view 
will be
altered if that ever changes.
> > I agree that it would be nice to have a tunable default for
> > BLAS-agnostic dependent packages, but some dependent packages will
> > always rely on extensions or performance-enhancing features (e.g.
> > openmp) of a particular implementation.
> Does the pkgsrc framework make this easy? When I introduce the global
> variable for the BLAS, would you introduce PKG_OPTIONS.somepacke to
> override the default (not including mk/blas.b3.mk but
> math/openblas-openmp.b3.mk directly when the non-default option is
> set)? Or just call bmake once with an overridden setting?
What I envision is a mk/blas interface for agnostic dependents
and the ability for packages to bypass it entirely and go
straight to, say, openblas/buildlink3.mk.
> I just wanted to give a little reference of the relative performance we
> are talking about here. Computer science folks sometimes can have a
> hard time believing that the same basic algorithm can have such huge
> performance differences. They're busy counting and sorting the leaves
> on their pretty trees while physicists are crunching numbers;-) Getting
> some optimized choice of BLAS into pkgsrc (or the option to use an
> external one) is an absolute must to be of any use in HPC besides for
> simply providing things like Gtk+ or HDF5.
We are in perfect agreement here.

One of the issues this raises is that defaulting to something other than
Netlib BLAS will be necessary to achieve good performance.  But some
applications have issues with OpenBLAS while others have issues with ATLAS.
The packager will become aware of this and should have the ability to select
certain high-performance implementations while blacklisting others. For now
this could be done by just using an implementation directly instead of 
the mk/blas
interface.  We can add features to mk/blas incrementally as time passes 
- let it
evolve with the available blas packages and need.
> > Allowing only one BLAS implementation to be installed at a time would
> > effectively eliminate pkgsrc as a viable deployment tool for
> > BLAS-dependent software.
> Well, unless you bootstrap different prefixes. But you convinced me
> that it makes sense and does not have to hurt to install the differing
> variants next to each other.
I can tell you with certainty that requiring a whole separate tree just
to get access to another BLAS implementation would be unacceptable to
a lot of HPC users.  Again, flexibility is one of the major concerns.  
They want the ability
to easily install and access a wide variety of software.  I've had success
selling the idea of separate trees for different versions of the same 
package,
but that's about as far as I think we can go.
> Alrighty then,
>
> Thomas
>
> * Actually, I just observed that Netlib LAPACK (which also ships a copy
>    of BLAS) by default strips -O3 down to -O2 in a release build. So
>    auto-vectorization is not even present. But then, compilers easily
>    fail optimizing Netlib stuff and produce broken code. This is a human's
>    job:-(
Do you have any data on the performance gains of O3 over O2?
For most programs it's around zero, but if it actually makes a difference
for BLAS, we should deal with this.
> PS: We might want to have a separate package for lapack-manpages. They
> come in a separate tarball and I didn't see them in openblas.
That may be a good idea.  Is there only one source for open source 
lapack manpages as far

as you know?

Cheers,

    JB