Re: infrastructure change needed for python modules

[Taylor R Campbell <riastradh%NetBSD.org@localhost>]

> Date: Wed, 15 Dec 2021 11:45:32 +0100
> From: Niclas Rosenvik <nros%netbsd.org@localhost>
> 
> I don't really know how long you have come with this.
> But I have an idea for wheel.mk . Allow pkgsrc to download
> and extract platform independent wheels. These have filenames
> following the pattern ${project}-${version}-py3-none-any.whl .
> Like build[1] for example the has build-0.7.0-py3-none-any.whl .

Are any of these source distributions?  I assumed they were all
pre-compiled, not designed to be human-readable or -auditable and as
such generally unfit for pkgsrc (except as a last resort, with scary
warnings, for special exceptions that absolutely cannot be done
another way).

> They don't need building and can just be extracted to
> ${DESTDIR}${PREFIX}${sitepackages} and then packaged from what I
> understand. I think this would also remove the circular dependecy
> on tomli that Gorny mentions in his blog post since tomli 
> also provides such files.

I don't know anything about the bootstrapping story for tomli and
whatnot, but...

> I plan to also provide pyproject.mk to use builds based on 
> pyproject.toml and have wheels.mk install the wheels these create.
> The files will be in wip/mk (or should it be wip/python?) and
> moved to lang/python in pkgsrc-current after the freeze.
> How does that sound for a plan?

In case this is helpful to avoid duplicating work: A Python package I
(semi-)recently packaged, security/py-solo, uses pyproject.toml
exclusively, with no setup.py.

I had to figure out how to

(a) bootstrap flit (which py-solo explicitly requests instead of
    `build'),

(b) build a wheel with it, and then

(c) install the wheel using pip with a lot of options to disable
    network access, use of an index, use of a cache, its own
    dependency resolution, &c.

You can see the results in:

devel/py-flit_core/Makefile
devel/py-flit/Makefile
security/py-solo/Makefile

That said, someone from the Python steering committee indicated that
there might be a simpler path with `build', for those packages that
can use it instead of flit or whatever in their pyproject.toml, if we
can deal with the bootstrapping story -- it might not require the
intermediate wheel at all:

python -m build --outdir ${DESTDIR} .

https://discuss.python.org/t/best-practice-suggestions-including-bootstrap/12202/17

(Not sure if this can be separated into a build stage and an install
stage, but maybe it doesn't really matter if it's going into a destdir
anyway.)