Re: Bring BLAS into pkgsrc, pt. II

[Jason Bacon <outpaddling%yahoo.com@localhost>]

On 2019-07-23 02:03, Dr. Thomas Orgis wrote:
> Am Sun, 21 Jul 2019 10:09:04 -0500
> schrieb Jason Bacon <outpaddling%yahoo.com@localhost>:
>
> > 1. As I mentioned earlier, I think the variable names could be more
> > explicit to avoid confusion between what goes in mk.conf and what goes
> > in the pkg Makefile.
> The variable with PKGSRC_ prefix goes into mk.conf. The others are for
> the Makefile. That's what I understood from the earlier discussion (no
> tradition of actually prefixing variables in package Makefiles with
> PKGSRC_).
Got it.  I hadn't recalled that conversation well, so thanks for clarifying.
> I agree that 'preferred' does not convey the actual meaning.
>
> > PKGSRC_BLAS_PREFERRED => PKGSRC_USER_BLAS_ACCEPTED
> > BLAS_ACCEPTED => PKGSRC_PKG_BLAS_ACCEPTED
> I don't like four words in the variable name. Too verbose.
>
> Right now we have those similar variables in the Makefiles:
> $ find . -name Makefile | xargs grep _ACCEPTED | grep -v '#' | cut -f 2 -d : | cut -f 1 -d = | sort -u
> BDB_ACCEPTED
> EMACS_VERSIONS_ACCEPTED
> EMACS_VERSIONS_ACCEPTED+
> KRB5_ACCEPTED
> LUA_VERSIONS_ACCEPTED
> MYSQL_VERSIONS_ACCEPTED
> NODE_VERSIONS_ACCEPTED
> PGSQL_VERSIONS_ACCEPTED
> PHP_VERSIONS_ACCEPTED
> PKG_APACHE_ACCEPTED
> PKG_JVMS_ACCEPTED
> PYTHON_VERSIONS_ACCEPTED
> PYTHON_VERSIONS_REALLY_ACCEPTED
> RUBY_RAILS_ACCEPTED
> RUBY_VERSIONS_ACCEPTED
>
> So, there is the example of two-word BDB_ACCEPTED and KRB5_ACCEPTED
> that match BLAS_ACCEPTED. But the majority is three-word names, mostly
> with VERSION in the middle. That's not the right word here, but we
> could do
>
> BLAS_ACCEPTED => BLAS_TYPES_ACCEPTED
>
> Then the resulting BLAS_TYPE matches that wording. And actually, we can
> use that for the user variable, too:
>
> PKGSRC_BLAS_PREFERRED => PKGSRC_BLAS_TYPES
>
> That may make more clear that it's about an exhaustive list of BLAS
> implementations, not just a preference. So we'd have:
>
> PKGSRC_BLAS_TYPES in mk.conf
> BLAS_TYPES_ACCEPTED in Makefile
> BLAS_TYPE, BLAS_LIBS, LAPACK_LIBS to be used by Makefile
>
> OK?
Yeah, I think following established conventions is ultimately the best 
way to minimize confusion.

So I think either BLAS_ACCEPTED or BLAS_TYPES_ACCEPTED would be fine for 
the pkg Makefile.  I don't see any significant difference for someone 
who knows what BLAS is, i.e. I think it's obvious enough that 
BLAS_ACCEPTED refers to BLAS implementations.

For mk.conf, I think PKGSRC_BLAS_TYPES would be OK, as would

PKGSRC_BLAS (following examples PKGSRC_FORTRAN, PKGSRC_ADA, etc.)

or

PKGSRC_ACCEPTABLE_BLAS (following example PKGSRC_ACCEPTABLE_LICENSES)

I think the latter is better as it implies that this is a list, as does 
PKGSRC_BLAS_TYPES.
> > 2. I think we need to insert a step 1.5 into the process:
> >
> > Test/patch all existing dependent packages against the new system before
> > committing the new blas and lapack packages.
> Well, of course you need testing and fixing, but the import of
> wip/lapack basically means a version update to 3.8.0 for dependent
> packages. There is no fundamental change in what is provided. Checking
> for one math/blas user that the change of now depending indirectly on
> math/lapack causes no trouble makes sense.
>
> What we need to check is the upgrade path. Will it cause trouble that
> we now got liblapack.so.3 instead of liblapack.so.4? Things need a
> rebuild. Or should be include a hacky symlink after all?

One lesson I've learned multiple times now is that there's no change too 
small to cause breakage.  There are only 15 committed packages that 
depend on blas and 12 on lapack, so testing all of them against the new 
blas/lapack packages while they're still in wip won't take long.  I'll 
be happy to help with this.

For the upgrade path, I think what we need here is a recursive revbump 
on all dependent packages.

Found this note from Jonathan back in January 2018:

Hi all,

I'm getting an increasing number of reports from my OSX trunk binary
package users that library dependencies are incorrect, and after
installing a new package or upgrading an existing one it will fail to
work until a particular library dependency is also upgraded.

This is caused by failing to perform a recursive revision bump after
you have committed an update to an existing library.  This should
always be performed if the version number of a shared library is
changed.

Recent examples include:

  https://github.com/joyent/pkgsrc/issues/62
  https://github.com/joyent/pkgsrc/issues/60
  https://github.com/joyent/pkgsrc/issues/22

and another report today on IRC of nghttp2 not being revbumped after
the last curl update.

If you are in any doubt, please perform a revision bump.  Bulk builds
will be unaffected as they will rebuild all packages after a
dependency is bumped anyway, and these failures give a terrible user
experience of pkgsrc that we should avoid at all costs.

> Hm, I am not at my workstation where my access key for pkgsrc-wip is
> stored. I just noticed that wip/blas should have an empty PLIST. It's
> not installing anything. I guess this causes no trouble as the check is
> not done for the meta package, but it looks unclean.
What's the reasoning again for installing lib/libblas.* via math/lapack 
instead of math/blas?  You may have explained this a while back, but I 
don't recall.

Cheers,

    JB