tech-pkg archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: Bring BLAS into pkgsrc, pt. II



On 2019-07-23 02:03, Dr. Thomas Orgis wrote:
Am Sun, 21 Jul 2019 10:09:04 -0500
schrieb Jason Bacon <outpaddling%yahoo.com@localhost>:

1. As I mentioned earlier, I think the variable names could be more
explicit to avoid confusion between what goes in mk.conf and what goes
in the pkg Makefile.
The variable with PKGSRC_ prefix goes into mk.conf. The others are for
the Makefile. That's what I understood from the earlier discussion (no
tradition of actually prefixing variables in package Makefiles with
PKGSRC_).
Got it.  I hadn't recalled that conversation well, so thanks for clarifying.

I agree that 'preferred' does not convey the actual meaning.

PKGSRC_BLAS_PREFERRED => PKGSRC_USER_BLAS_ACCEPTED
BLAS_ACCEPTED => PKGSRC_PKG_BLAS_ACCEPTED
I don't like four words in the variable name. Too verbose.

Right now we have those similar variables in the Makefiles:
$ find . -name Makefile | xargs grep _ACCEPTED | grep -v '#' | cut -f 2 -d : | cut -f 1 -d = | sort -u
BDB_ACCEPTED
EMACS_VERSIONS_ACCEPTED
EMACS_VERSIONS_ACCEPTED+
KRB5_ACCEPTED
LUA_VERSIONS_ACCEPTED
MYSQL_VERSIONS_ACCEPTED
NODE_VERSIONS_ACCEPTED
PGSQL_VERSIONS_ACCEPTED
PHP_VERSIONS_ACCEPTED
PKG_APACHE_ACCEPTED
PKG_JVMS_ACCEPTED
PYTHON_VERSIONS_ACCEPTED
PYTHON_VERSIONS_REALLY_ACCEPTED
RUBY_RAILS_ACCEPTED
RUBY_VERSIONS_ACCEPTED

So, there is the example of two-word BDB_ACCEPTED and KRB5_ACCEPTED
that match BLAS_ACCEPTED. But the majority is three-word names, mostly
with VERSION in the middle. That's not the right word here, but we
could do

BLAS_ACCEPTED => BLAS_TYPES_ACCEPTED

Then the resulting BLAS_TYPE matches that wording. And actually, we can
use that for the user variable, too:

PKGSRC_BLAS_PREFERRED => PKGSRC_BLAS_TYPES

That may make more clear that it's about an exhaustive list of BLAS
implementations, not just a preference. So we'd have:

PKGSRC_BLAS_TYPES in mk.conf
BLAS_TYPES_ACCEPTED in Makefile
BLAS_TYPE, BLAS_LIBS, LAPACK_LIBS to be used by Makefile

OK?
Yeah, I think following established conventions is ultimately the best way to minimize confusion.

So I think either BLAS_ACCEPTED or BLAS_TYPES_ACCEPTED would be fine for the pkg Makefile.  I don't see any significant difference for someone who knows what BLAS is, i.e. I think it's obvious enough that BLAS_ACCEPTED refers to BLAS implementations.

For mk.conf, I think PKGSRC_BLAS_TYPES would be OK, as would

PKGSRC_BLAS (following examples PKGSRC_FORTRAN, PKGSRC_ADA, etc.)

or

PKGSRC_ACCEPTABLE_BLAS (following example PKGSRC_ACCEPTABLE_LICENSES)

I think the latter is better as it implies that this is a list, as does PKGSRC_BLAS_TYPES.

2. I think we need to insert a step 1.5 into the process:

Test/patch all existing dependent packages against the new system before
committing the new blas and lapack packages.
Well, of course you need testing and fixing, but the import of
wip/lapack basically means a version update to 3.8.0 for dependent
packages. There is no fundamental change in what is provided. Checking
for one math/blas user that the change of now depending indirectly on
math/lapack causes no trouble makes sense.

What we need to check is the upgrade path. Will it cause trouble that
we now got liblapack.so.3 instead of liblapack.so.4? Things need a
rebuild. Or should be include a hacky symlink after all?

One lesson I've learned multiple times now is that there's no change too small to cause breakage.  There are only 15 committed packages that depend on blas and 12 on lapack, so testing all of them against the new blas/lapack packages while they're still in wip won't take long.  I'll be happy to help with this.

For the upgrade path, I think what we need here is a recursive revbump on all dependent packages.

Found this note from Jonathan back in January 2018:

Hi all,

I'm getting an increasing number of reports from my OSX trunk binary
package users that library dependencies are incorrect, and after
installing a new package or upgrading an existing one it will fail to
work until a particular library dependency is also upgraded.

This is caused by failing to perform a recursive revision bump after
you have committed an update to an existing library.  This should
always be performed if the version number of a shared library is
changed.

Recent examples include:

  https://github.com/joyent/pkgsrc/issues/62
  https://github.com/joyent/pkgsrc/issues/60
  https://github.com/joyent/pkgsrc/issues/22

and another report today on IRC of nghttp2 not being revbumped after
the last curl update.

If you are in any doubt, please perform a revision bump.  Bulk builds
will be unaffected as they will rebuild all packages after a
dependency is bumped anyway, and these failures give a terrible user
experience of pkgsrc that we should avoid at all costs.


Hm, I am not at my workstation where my access key for pkgsrc-wip is
stored. I just noticed that wip/blas should have an empty PLIST. It's
not installing anything. I guess this causes no trouble as the check is
not done for the meta package, but it looks unclean.

What's the reasoning again for installing lib/libblas.* via math/lapack instead of math/blas?  You may have explained this a while back, but I don't recall.

Cheers,

    JB



Home | Main Index | Thread Index | Old Index