Re: py-scipy and py-pandas with python[39-310]

["David H. Gutteridge" <david%gutteridge.ca@localhost>]

On Wed, 27 Sep 2023 at 08:08:52 +0200, Paolo Vincenzo Olivo wrote:
> Sorry for not replying earlier; I somehow missed this message in the
> multitude of mails hitting my inbox.
> What follows is just for future reference.
>
> On 23/09/21 02:59PM, Greg Troxel wrote:
> > Paolo Vincenzo Olivo <vins%NetBSD.org@localhost> writes:
> >
> > > I see math/py[39-310]-pandas and math/py[39-310]-scipy failing on all
> > > NetBSD bulk builds for pkgsrc-trunk.
> >
> > Does this mean that for 311 (default) it's ok?  To me that is the most
> > important thing, but it of course should work elsewhere.
>
> Yes, that was the case. Cython is python versioned while meson is not,
> so si packages depending on cython packages as tool were failing due to
> the fact that meson would always look for cython-3.11 regardless of the
> python version in use.
>
> > > Looking at the build log, it seems that meson fails to find
> > > devel/py-cython even if it's pulled as TOOL dependency. Apparently meson
> > > only looks for binaries named `cython' and `cython-3.11' (probably
> > > reflecting PYTHON_VERSION_DEFAULT ?).
> > > devel/py-cython will install bin/cython-${PYVERSSUFFIX}, so the build
> > > should be adjusted to look for the correct binary depending on the
> > > python version in use.
> >
> > It strikes me that meson should be told what the python version is and
> > handle multiple versions, but perhaps the linux world does not allow
> > multiple versions and thus people don't bother to make that work.   It
> > sounds like you are suggesting that we patch in python-version-aware
> > support and that makes sense to me.
>
> And this is indeed how @gutteridge managed to work around the problem,
> at least for 2023Q3. See 
> https://marc.info/?l=pkgsrc-changes&m=169558149012184

Some further details about this. The change I made shouldn't introduce
a regression from the prior state. It will try the current Python
default (3.11) first as did the previous version of the package. It
will then try each prior default in turn. If someone is building
outside of sandboxed pbulk and wants multiple Python 3 versions (surely
an edge case), it may still mismatch the Cython version invoked vs. the
Python version being built, but that was the case with the previous
state of the package as well.

I tried to also patch Scipy and Pandas so they would cleanly force the
currently selected Python version specifically when invoking Meson via
meson.build, but this turned out to be more involved than I expected
because of how Meson expects to treat Cython as a language. (Plus,
this approach doesn't scale to patch a bunch of packages, with more
than one relevant meson.build in tree, plus this is an edge case
consideration.) It would be better to get Meson to accept a "preferred"
version of a compiler when more than one is present. It does have the
ability to read in such selections, but it sources those from
additional "native" or "cross" input files, rather than accepting them
on the command line (unless I've missed something, which is possible).
We already have a facility to handle the "cross" input files in
meson/build.mk, but not the equivalent for "native", and I didn't want
to tinker with that make infrastructure during a freeze. (Then, once
again, there's the "not optimal to patch multiple packages" thing.)

I'm not sure how concerned anyone is about this? (Meson was originally
py-meson in pkgsrc, then was changed to be unversioned so it could "use
Python 3 to build Python 2 packages". A simple question being, if we
can't easily have both, which is more important now, Python 3 multi-
version packages, or Python 2 support?)

Regards,

Dave