Re: Python 3.6 retirement

Hi,

Well, I got good news and bad news.

Good news :

bootstrapping pkgsrc2021Q3 on Solaris worked. Yeah! Guess the issue was on my side or something changed between 2020Q3 and 2021Q3. Whatever, it works!

Bad news :

As I previously said on 2020Q3, the same issue happens in 2021Q3. This means: Python3.9(tested), Python3.8(tested), and Python3.7(tested) do not build. Pretty much with the same error :

/var/tmp/pkgbuild/lang/python39/work/Python-3.9.7/Modules/socketmodule.c: In function 'socket_sethostname':
/var/tmp/pkgbuild/lang/python39/work/Python-3.9.7/Modules/socketmodule.c:5514:15: error: implicit declaration of function 'sethostname'; did you mean 'gethostname'? [-Werror=implicit-function-declaration]
res = sethostname(buf.buf, buf.len);
^~~~~~~~~~~
gethostname
cc1: some warnings being treated as errors
...
building '_ssl' extension
...
*** WARNING: renaming "_asyncio" since importing it failed: No module named '_socket'
*** WARNING: renaming "_ssl" since importing it failed: No module named '_socket'
Python build finished successfully!
The necessary bits to build these optional modules were not found:
_gdbm _tkinter ossaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc atexit pwd
time
Failed to build these modules:
_socket
Following modules built successfully but were removed because they could not be imported:
_asyncio _ssl
Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381
...
=> Checking file-check results for python39-3.9.7
ERROR: ************************************************************
ERROR: The following files are in the PLIST but not in /var/tmp/pkgbuild/lang/python39/work/.destdir/usr/pkg:
ERROR: /var/tmp/pkgbuild/lang/python39/work/.destdir/usr/pkg/lib/python3.9/lib-dynload/_asyncio.so
ERROR: /var/tmp/pkgbuild/lang/python39/work/.destdir/usr/pkg/lib/python3.9/lib-dynload/_socket.so
ERROR: /var/tmp/pkgbuild/lang/python39/work/.destdir/usr/pkg/lib/python3.9/lib-dynload/_ssl.so
ERROR: ************************************************************
ERROR: The following files are in /var/tmp/pkgbuild/lang/python39/work/.destdir/usr/pkg but not in the PLIST:
ERROR: /var/tmp/pkgbuild/lang/python39/work/.destdir/usr/pkg/lib/python3.9/lib-dynload/_asyncio_failed.so
ERROR: /var/tmp/pkgbuild/lang/python39/work/.destdir/usr/pkg/lib/python3.9/lib-dynload/_ssl_failed.so
*** Error code 1

Python 3.6 remains the last working version working on Solaris 10 (tested).

On Sun, Oct 10, 2021 at 2:28 PM Julien Savard <juliensavard17%gmail.com@localhost> wrote:

I confirm python3.7 and 3.8 not building on Solaris 10 (with pkgsrc2020Q3) :

Python 3.7 :
...
*** WARNING: renaming "_asyncio" since importing it failed: No module named '_socket'
*** WARNING: renaming "_ssl" since importing it failed: No module named '_socket'

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_gdbm _tkinter ossaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc atexit pwd
time

Failed to build these modules:
_socket

Following modules built successfully but were removed because they could not be imported:
_asyncio _ssl

Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381
...
=> Checking file-check results for python37-3.7.9nb1
ERROR: ************************************************************
ERROR: The following files are in the PLIST but not in /var/tmp/pkgbuild/lang/python37/work/.destdir/usr/pkg:
ERROR: /var/tmp/pkgbuild/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/_asyncio.so
ERROR: /var/tmp/pkgbuild/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/_socket.so
ERROR: /var/tmp/pkgbuild/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/_ssl.so
ERROR: ************************************************************
ERROR: The following files are in /var/tmp/pkgbuild/lang/python37/work/.destdir/usr/pkg but not in the PLIST:
ERROR: /var/tmp/pkgbuild/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/_asyncio_failed.so
ERROR: /var/tmp/pkgbuild/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/_ssl_failed.so
*** Error code 1

Stop.
bmake[1]: stopped in /srv/pkgsrc/lang/python37
*** Error code 1

Python 3.8 :
*** WARNING: renaming "_asyncio" since importing it failed: No module named '_socket'
*** WARNING: renaming "_ssl" since importing it failed: No module named '_socket'

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_gdbm _tkinter ossaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.

The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc atexit pwd
time

Failed to build these modules:
_socket

Following modules built successfully but were removed because they could not be imported:
_asyncio _ssl

Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

running build_scripts
copying and adjusting /var/tmp/pkgbuild/lang/python38/work/Python-3.8.5/Tools/scripts/pydoc3 -> build/scripts-3.8
copying and adjusting /var/tmp/pkgbuild/lang/python38/work/Python-3.8.5/Tools/scripts/2to3 -> build/scripts-3.8
...
=> Generating post-install file lists
=> Checking file-check results for python38-3.8.5
ERROR: ************************************************************
ERROR: The following files are in the PLIST but not in /var/tmp/pkgbuild/lang/python38/work/.destdir/usr/pkg:
ERROR: /var/tmp/pkgbuild/lang/python38/work/.destdir/usr/pkg/lib/python3.8/lib-dynload/_asyncio.so
ERROR: /var/tmp/pkgbuild/lang/python38/work/.destdir/usr/pkg/lib/python3.8/lib-dynload/_socket.so
ERROR: /var/tmp/pkgbuild/lang/python38/work/.destdir/usr/pkg/lib/python3.8/lib-dynload/_ssl.so
ERROR: ************************************************************
ERROR: The following files are in /var/tmp/pkgbuild/lang/python38/work/.destdir/usr/pkg but not in the PLIST:
ERROR: /var/tmp/pkgbuild/lang/python38/work/.destdir/usr/pkg/lib/python3.8/lib-dynload/_asyncio_failed.so
ERROR: /var/tmp/pkgbuild/lang/python38/work/.destdir/usr/pkg/lib/python3.8/lib-dynload/_ssl_failed.so
*** Error code 1

I will try to bootstrap the last pkgsrc version on Solaris 10 ASAP and start a new thread about it.

On Sun, Oct 10, 2021 at 9:40 AM Greg Troxel <gdt%lexort.com@localhost> wrote:

Peter Lai <cowbert%gmail.com@localhost> writes:

> On Sat, Oct 9, 2021 at 7:40 PM Greg Troxel <gdt%lexort.com@localhost> wrote:

>> Regarding nia@'s point about Solaris 10, it would be nice for Solaris
>> people to comment on why newer python can't reasonably be made to work
>> by fixing either Python or Solaris. Maybe this really is intractable,
>> but that would seem somewhat surprising. Agree that the presence of 3.6
>> is not causing that much, if any, trouble, and that accomodating people
>> is nice if it doesn't hurt.
>
> There is a reason why we support PYTHON_VERSIONS_ACCEPTED and
> PYTHON_VERSIONS_INCOMPATIBLE

If you mean that having 3.6 in tree doesn't hurt because if some
package doesn't work with 3.6 we can just mark it, agreed. If you mean
something else, I didn't follow.

For Solaris 10, I looked in python37 to find BROKEN_ON and didn't. So
if people care about pkgsrc on Solaris 10, it would be good for them to
address newer python.

> Note that upstream will support 3.6 until 2021-12. Perhaps target 22Q1
> release for removal to align with this.

Good to know, so that argues for not now. After the EOL date, the
Solaris 10 issue is the only reason I'm aware of why anyone would want
to run 3.6. Thereefore dropping 3.6 in January seems reasonable, absent
an argument from Solaris 10 people or others that there is a good reason
ii should stay.

> Upstream EOL for PY37 is 2023-06.

That's a ways out. I hadn't realized there was a 2-year gap between
3.6 and 3.7.

>> It would be interesting to a bulk build or a few, identifying the count
>> of packages that do and don't build with 3.8. Other than 3.8 not
>> running on some platform, I wonder how much there really are good
>> reasons to run 3.6 or 3.7 instead of 3.8, other than not having updated
>> yet.
>
> Mostly change control prioritization (coordination/testing/validation
> requirements) for folks in production environments that have a bunch
> of abi-dependent pkgsrc python packages (numpy, psycopg2, etc.).

Sure, but pkgsrc switched away from 3.6 as default long ago, and when
2021Q4 comes out, it will have been 2 years and 8 months. That's IMHO
more than long enough, and of course anybody who wants longer is welcome
to use the old pkgsrc bits or do whatever.

3.7 2019-04-24
3.8 2020-12-04
3.9 2021-10-01

But, this makes me disinclined to drop 3.7 soon, as it's still in
support and has not even been default for a full year. It does feel
like we are carrying a lot of versions, but python is releasing minor
versions every year, and it seems they take a year to get to the point
where they can be made default, a year of default, a year of old, and
year of plus-old.