Re: devel/py-joblib patch

[Taylor R Campbell <campbell+netbsd-tech-pkg%mumble.net@localhost>]

> Date: Wed, 10 Jan 2024 08:29:32 -0600
> From: Jason Bacon <jtocino%gmx.com@localhost>
> 
> On 12/29/23 11:14, Jason Bacon wrote:
> > On 12/29/23 08:38, Martin Husemann wrote:
> >> This sounds pretty expensive, can't it use os.sysconf(NPROCESSORS_CONF) ?
> > 
> > Probably.  I was just following the other examples in the code, as I'm
> > no pythonista, but your suggestion sounds like a better approach.
> > 
> > https://github.com/joblib/joblib/issues/1535
> 
> As per the github discussion, we'll have to stick with subprocesses for 
> now.  sysconf does not return the desired core count and there is no 
> portable sysctl interface for python at this time.  The pypi sysctl 
> package is FreeBSD-specific, while completely different implementations 
> exist for NetBSD and Linux.

I'm confused, what number is wrong and why do you need sysctl at the
moment?

I understand we might add sysctls in the future to express the full
topology, including the gory details of hyperthreading and big.LITTLE
and newer finer-grained variants thereof or whatever -- but for now
you're just going for the number of configured or online `cores'
(i.e., threads), right?

$ sysctl hw.ncpu
hw.ncpu = 8
$ python3
Python 3.11.5 (main, Sep 27 2023, 11:02:36) [GCC 7.4.0] on netbsd9
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.sysconf('SC_NPROCESSORS_CONF')
8
>>> os.sysconf('SC_NPROCESSORS_ONLN')
8

These are baked into CPython itself, no pypi packages needed.