Re: Various py-* packages missing atomic64.mk

[Havard Eidnes <he%NetBSD.org@localhost>]

>> a review of the state of my bulk build host for NetBSD/macppc
>> 10.0 reveals that all the following packages have failed to build
>> because -latomic could not be found:
>>
>> py310-rapidfuzz-3.10.1/build.log
>> py311-rapidfuzz-3.10.1/build.log
>> py312-rapidfuzz-3.10.1/build.log
>> py313-Pillow-11.0.0/build.log
>> py313-brotli-1.1.0/build.log
>> py313-cffi-1.17.1/build.log
>> py313-cython-3.0.11/build.log
>> py313-genshi-0.7.9/build.log
>> py313-lxml-5.3.0nb3/build.log
>> py313-markupsafe-3.0.2/build.log
>> py313-psycopg2-2.9.10/build.log
>> py313-rapidfuzz-3.10.1/build.log
>> py313-wrapt-1.17.0/build.log
>> py313-zope.interface-7.2/build.log
>> py313-zopfli-0.2.3/build.log
>>
>> This is fixable by adding an include of ../../mk/atomic64.mk.
>
> It seems that this is a small subset of python modules, so anything
> about 'all python' is wrong.

OK.

> Do you believe that these have modules implemented in C that uses
> atomics?

Yep.  Typical errors:

py310-rapidfuzz-3.10.1/build.log:

-- Performing Test HAVE_CXX_LIBATOMICS64_2ARGS
-- Performing Test HAVE_CXX_LIBATOMICS64_2ARGS - Failed
CMake Error at extern/taskflow/cmake/Atomic/CheckAtomic.cmake:150 (message):
  Host compiler appears to require libatomic, but cannot find it.

py313-Pillow-11.0.0/build.log:

ld: cannot find -latomic
ld: cannot find -latomic

ipy313-brotli-1.1.0/build.log:

ld: cannot find -latomic
ld: cannot find -latomic
error: command '/usr/pkgsrc/archivers/py-brotli/work/.cwrapper/bin/gcc' failed with exit code 1

py313-cffi-1.17.1/build.log:

gcc -shared -Wl,-zrelro -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -latomic -L/usr/X11R7/lib -Wl,-R/usr/X11R7/lib -Wl,-zrelro -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -latomic -L/usr/X11R7/lib -Wl,-R/usr/X11R7/lib -Wl,-zrelro -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -L/usr/lib -Wl,-R/usr/lib -O2 -I/usr/pkg/include -I/usr/include -I/usr/pkg/include/python3.13 -I/usr/pkg/include -I/usr/include -I/usr/pkg/include/python3.13 build/temp.netbsd-10.0-macppc-cpython-313/src/c/_cffi_backend.o -L/usr/pkg/lib -L/usr/pkg/lib -lffi -o build/lib.netbsd-10.0-macppc-cpython-313/_cffi_backend.so -Wl,-R/usr/pkg/lib
ld: cannot find -latomic
ld: cannot find -latomic
error: command '/usr/pkgsrc/devel/py-cffi/work/.cwrapper/bin/gcc' failed with exit code 1
Exception ignored in: <function _DeleteDummyThreadOnDel.__del__ at 0xfca91478>
Traceback (most recent call last):
  File "/usr/pkg/lib/python3.13/threading.py", line 1383, in __del__
TypeError: 'NoneType' object does not support the context manager protocol

ERROR Backend subprocess exited when trying to invoke build_wheel

py313-cython-3.0.11/build.log:

gcc -shared -Wl,-zrelro -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -latomic -L/usr/X11R7/lib -Wl,-R/usr/X11R7/lib -Wl,-zrelro -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -latomic -L/usr/X11R7/lib -Wl,-R/usr/X11R7/lib -Wl,-zrelro -L/usr/lib -Wl,-R/usr/lib -Wl,-R/usr/pkg/lib -O2 -I/usr/include -I/usr/pkg/include/python3.13 -I/usr/include -I/usr/pkg/include/python3.13 build/temp.netbsd-10.0-macppc-cpython-313/usr/pkgsrc/devel/py-cython/work/cython-3.0.11/Cython/Compiler/Parsing.o -L/usr/pkg/lib -o build/lib.netbsd-10.0-macppc-cpython-313/Cython/Compiler/Parsing.so
ld: cannot find -latomic
ld: cannot find -latomic
error: command '/usr/pkgsrc/devel/py-cython/work/.cwrapper/bin/gcc' failed with exit code 1
Exception ignored in: <function _DeleteDummyThreadOnDel.__del__ at 0xfca91608>
Traceback (most recent call last):
  File "/usr/pkg/lib/python3.13/threading.py", line 1383, in __del__
TypeError: 'NoneType' object does not support the context manager protocol

ERROR Backend subprocess exited when trying to invoke build_wheel

etc. etc. along those lines.

> Do you understand why rapidfuzz is ok on 313 (or is it)?

I just think it's not tried yet; I can't find a binary package
for it.

> Do you understand why everything else fails on 313, and is, presumably,
> ok on 310/311/312?

Nope.  Binary results:

malus: {44} ls -l py*Pillow*
-rw-r--r--  1 root  wheel  1169527 Jan  9 19:27 py310-Pillow-11.0.0.tgz
-rw-r--r--  1 root  wheel  1555141 Jan  9 19:24 py311-Pillow-11.0.0.tgz
-rw-r--r--  1 root  wheel  1504888 Jan  2 19:01 py312-Pillow-11.0.0.tgz
malus: {45} ls -l py*brotli-*
-rw-r--r--  1 root  wheel  440837 Jan  1 07:07 py310-brotli-1.1.0.tgz
-rw-r--r--  1 root  wheel  441000 Jan  1 08:05 py311-brotli-1.1.0.tgz
-rw-r--r--  1 root  wheel  441309 Jan  1 00:44 py312-brotli-1.1.0.tgz
-rw-r--r--  1 root  wheel  440857 Jun  4  2024 py38-brotli-1.1.0.tgz
-rw-r--r--  1 root  wheel  440799 Jun  3  2024 py39-brotli-1.1.0.tgz
malus: {46} ls -l py*-cffi-1.17*
-rw-r--r--  1 root  wheel  378735 Jan  1 00:46 py310-cffi-1.17.1.tgz
-rw-r--r--  1 root  wheel  544659 Jan  1 00:53 py311-cffi-1.17.1.tgz
-rw-r--r--  1 root  wheel  512563 Dec 31 12:44 py312-cffi-1.17.1.tgz
malus: {47} ls -l py*cython-3*
-rw-r--r--  1 root  wheel  5007145 Jan  1 02:31 py310-cython-3.0.11.tgz
-rw-r--r--  1 root  wheel  6446982 Jan  1 03:13 py311-cython-3.0.11.tgz
-rw-r--r--  1 root  wheel  6356849 Jan  1 00:07 py312-cython-3.0.11.tgz
malus: {48} ls -l py*genshi-0*
-rw-r--r--  1 root  wheel  483266 Jan  2 17:44 py310-genshi-0.7.9.tgz
-rw-r--r--  1 root  wheel  758108 Jan  2 17:45 py311-genshi-0.7.9.tgz
-rw-r--r--  1 root  wheel  702399 Jan  2 17:29 py312-genshi-0.7.9.tgz
malus: {49} ls -l py*lxml-5*
-rw-r--r--  1 root  wheel  1870879 Jan  1 17:26 py310-lxml-5.3.0nb3.tgz
-rw-r--r--  1 root  wheel  1946411 Jan  1 17:39 py311-lxml-5.3.0nb3.tgz
-rw-r--r--  1 root  wheel  1993128 Dec 26 17:21 py312-lxml-5.3.0nb3.tgz
malus: {50} ls -l py*markupsafe-3*
-rw-r--r--  1 root  wheel  20812 Jan  1 04:46 py310-markupsafe-3.0.2.tgz
-rw-r--r--  1 root  wheel  23806 Jan  1 06:22 py311-markupsafe-3.0.2.tgz
-rw-r--r--  1 root  wheel  22824 Dec 28 19:50 py312-markupsafe-3.0.2.tgz
malus: {51} ls -l py*psycopg2-2*
-rw-r--r--  1 root  wheel  243213 Jan  2 21:48 py310-psycopg2-2.9.10.tgz
-rw-r--r--  1 root  wheel  272032 Jan  2 21:46 py311-psycopg2-2.9.10.tgz
-rw-r--r--  1 root  wheel  269660 Jan  2 21:29 py312-psycopg2-2.9.10.tgz
malus: {52} ls -l py*rapidfuzz*
ls: No match.
malus: {53} ls -l py*wrapt-1*
-rw-r--r--  1 root  wheel  64071 Jan  2 18:38 py310-wrapt-1.17.0.tgz
-rw-r--r--  1 root  wheel  83252 Jan  2 18:43 py311-wrapt-1.17.0.tgz
-rw-r--r--  1 root  wheel  80889 Jan  2 17:41 py312-wrapt-1.17.0.tgz
malus: {54} ls -l py*zope.inter*
-rw-r--r--  1 root  wheel  526650 Jan 11 15:51 py310-zope.interface-7.2.tgz
-rw-r--r--  1 root  wheel  760803 Jan 11 15:50 py311-zope.interface-7.2.tgz
-rw-r--r--  1 root  wheel  718178 Jan 11 09:08 py312-zope.interface-7.2.tgz
malus: {55} ls -l py*zopfli-*
-rw-r--r--  1 root  wheel  146994 Jan 15 19:32 py310-zopfli-0.2.3.tgz
-rw-r--r--  1 root  wheel  148199 Jan 15 17:39 py311-zopfli-0.2.3.tgz
-rw-r--r--  1 root  wheel  148042 Jan 11 11:30 py312-zopfli-0.2.3.tgz
-rw-r--r--  1 root  wheel  146820 Jun 23  2024 py38-zopfli-0.2.3.tgz
-rw-r--r--  1 root  wheel  146847 Jun 11  2024 py39-zopfli-0.2.3.tgz
malus: {56} 

> Do you expect that these build problems would occur on all 32-bit CPUs,
> or do you think this is somehow special to macppc?
> (Not arguing against a fix -- just trying to understand more clearly.)

I think these are generic, and would be seen also on i386.

Regards,

- Håvard