Re: I want to merge some BLAS updates to get things in a future-proof shape before the branch. Please review.

["Dr. Thomas Orgis" <thomas.orgis%uni-hamburg.de@localhost>]

So … 

I did it now. The reworked BLAS is in. This has been months in the
making for me and hence it would be quite a personal burden to keep
carrying this in limbo. After all, I also need to focus my attention
elsewhere.

The main change:

- Each BLAS_TYPE now installs with CBLAS and LAPACKE
  (except accelerate.framework, which might need a LAPACKE graft).

I hope I'm not messing up any schedules and intend to quickly be on top
of any upcoming issues during the freeze. I got math/py-numpy and
math/py-numpy16 being happy. I built math/R with openblas … all the
users that need plain BLAS or LAPACK should not care anyway.

One big point is that I really need testers on Darwin to settle the
situation with accelerate.framework. It's being used in the NumPy
packages, but as I am really uneasy about the level of compatibility of
this with the other choices (g77 vs gfortran calling convention clang
vs. gcc toolchain … what are people using?).

Right now, I excluded it from the defaults. Look at math/py-numpy on
how to include it. Our policy is in general to default to netlib so far.

There are existing packages that use -framework Accelerate explicitly
or maybe implicitly as the build just detects it, without consulting
our choice.

Mentions:

./audio/chromaprint/Makefile:# On Darwin, chromaprint uses Accelerate.framework, but fpcalc still needs FFmpeg
./math/py-Numeric/patches/patch-setup.py:+ACCELERATE_PATH = '/System/Library/Frameworks/Accelerate.framework'
./math/py-Numeric/patches/patch-setup.py:+    lapack_link_args.extend(['-framework', 'Accelerate'])
./math/py-Numeric/patches/patch-setup.py:+                            extra_link_args=['-framework', 'Accelerate'],
./math/py-numarray/patches/patch-cfg__packages.py:use the Accelerate framework instead
./math/py-numarray/patches/patch-cfg__packages.py:+    if os.path.exists('/System/Library/Frameworks/Accelerate.framework'):
./math/py-numarray/patches/patch-cfg__packages.py:+        lapack_link_args = ['-framework', 'Accelerate']
./math/py-numarray/patches/patch-cfg__packages.py:+        lapack_compile_args = ['-framework', 'Accelerate']
./math/py-numarray/Makefile:.if exists(/System/Library/Frameworks/Accelerate.framework)

These I hope could just work right now also on Darwin:

./math/py-numpy
./math/py-numpy16
./math/R

WIP stuff that needs more work anyway:

./wip/suitesparse/files/SuiteSparse_config.mk:# BLAS = -framework Accelerate
./wip/suitesparse/files/SuiteSparse_config.mk:# LAPACK = -framework Accelerate
./wip/gemma/Makefile:BLAS_ACCEPTED=	openblas_pthread openblas_openmp accelerate.framework
./wip/armadillo/patches/patch-CMakeLists.txt:   set(ARMA_LIBS ${ARMA_LIBS} "-framework Accelerate")  # or "-framework accelerate" ?
./wip/armadillo/patches/patch-CMakeLists.txt:   message(STATUS "MacOS X detected. Added '-framework Accelerate' to compiler flags")

Those should be unified. And the basic mess is something that I did not
create right now, but just made more visible: We have a mix of BLAS usage
from Accelerate/VecLib behind the scenes and explicitly, by default or
even without choice, and the usage of the managed dependencies in
pkgsrc for that.

A clear goal for the release would be to have checked them all at least
to the point where we know that only leaf packages or diverging
branches have differing BLAS choices. I must admit that the -framework
Accelerate thing was not clearly on my radar before … but it is nasty
in that it is 'just there' or not. Or could we force things to use the
compiler wrappers and catch usage if not configured via BLAS_TYPE?
There are other legitimate uses of the framework, though. BLAS is only
one aspect of it.


Back to the reference: Considering that we have very few CBLAS users
yet, I am open to removing the 

include/cblas.h

symlink for the netlib case. this would mean less chance to mix up
headers from two implementations. As long as you use ${BLAS_INCLUDES},
you don't need the symlink after all.

I won't hide that I see cblas header conflict in wip/gemma. That's an
interesting one. But I do not know how well the package worked before.
Jason?


Alrighty then,

Thomas


PS: Again, sorry for this somewhat scary-looking changeset right before
the freeze. I'll make sure it works out in the end, to the best of my
abilities.

-- 
Dr. Thomas Orgis
HPC @ Universität Hamburg