CVS commit: pkgsrc/www/py-scrapy

To: pkgsrc-changes%NetBSD.org@localhost
Subject: CVS commit: pkgsrc/www/py-scrapy
From: "Adam Ciarcinski" <adam%netbsd.org@localhost>
Date: Thu, 3 Sep 2026 08:35:38 +0000

Module Name:    pkgsrc
Committed By:   adam
Date:           Thu Sep  3 08:35:38 UTC 2026

Modified Files:
        pkgsrc/www/py-scrapy: Makefile PLIST distinfo

Log Message:
py-scrapy: updated to 2.18.0

Scrapy 2.18.0 (2026-08-20)

Highlights:

-   ``HttpxDownloadHandler`` now uses `httpx2 <https://httpx2.pydantic.dev/>`__

-   The Twisted-based HTTP/2 download handler is no longer experimental

-   ``brotli`` and Zstandard support are now always available, and :ref:`optional
    extras <extras>` cover the rest of the optional features

-   Late :class:`~scrapy.crawler.Crawler` attributes, such as
    :attr:`~scrapy.crawler.Crawler.stats`, now raise :exc:`RuntimeError`
    instead of being ``None`` before the crawl starts

-   Item exporters now export fields in declaration order

-   New :class:`~scrapy.spidermiddlewares.metacopy.MetaCopyDetectionMiddleware`

-   New :ref:`optimization <optimize>` page and :ref:`built-in stats reference
    <topics-stats-reference>`


To generate a diff of this commit:
cvs rdiff -u -r1.42 -r1.43 pkgsrc/www/py-scrapy/Makefile
cvs rdiff -u -r1.18 -r1.19 pkgsrc/www/py-scrapy/PLIST
cvs rdiff -u -r1.31 -r1.32 pkgsrc/www/py-scrapy/distinfo

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: pkgsrc/www/py-scrapy/Makefile
diff -u pkgsrc/www/py-scrapy/Makefile:1.42 pkgsrc/www/py-scrapy/Makefile:1.43
--- pkgsrc/www/py-scrapy/Makefile:1.42  Thu Jul 23 07:40:43 2026
+++ pkgsrc/www/py-scrapy/Makefile       Thu Sep  3 08:35:37 2026
@@ -1,6 +1,6 @@
-# $NetBSD: Makefile,v 1.42 2026/07/23 07:40:43 adam Exp $
+# $NetBSD: Makefile,v 1.43 2026/09/03 08:35:37 adam Exp $
 
-DISTNAME=      scrapy-2.17.0
+DISTNAME=      scrapy-2.18.0
 PKGNAME=       ${PYPKGPREFIX}-${DISTNAME}
 CATEGORIES=    www python
 MASTER_SITES=  ${MASTER_SITE_PYPI:=S/Scrapy/}
@@ -11,24 +11,27 @@ COMMENT=    High-level Web Crawling and Web
 LICENSE=       modified-bsd
 
 TOOL_DEPENDS+= ${PYPKGPREFIX}-hatchling>=1.27.0:../../devel/py-hatchling
+DEPENDS+=      ${PYPKGPREFIX}-OpenSSL>=24.3.0:../../security/py-OpenSSL
+DEPENDS+=      ${PYPKGPREFIX}-brotli>=1.2.0:../../archivers/py-brotli
+DEPENDS+=      ${PYPKGPREFIX}-cryptography>=41.0.5:../../security/py-cryptography
 DEPENDS+=      ${PYPKGPREFIX}-cssselect>=0.9.1:../../textproc/py-cssselect
 DEPENDS+=      ${PYPKGPREFIX}-defusedxml>=0.7.1:../../textproc/py-defusedxml
 DEPENDS+=      ${PYPKGPREFIX}-itemadapter>=0.1.0:../../textproc/py-itemadapter
 DEPENDS+=      ${PYPKGPREFIX}-itemloaders>=1.0.1:../../textproc/py-itemloaders
 DEPENDS+=      ${PYPKGPREFIX}-lxml>=4.6.4:../../textproc/py-lxml
-DEPENDS+=      ${PYPKGPREFIX}-packaging-[0-9]*:../../devel/py-packaging
+DEPENDS+=      ${PYPKGPREFIX}-packaging>=0:../../devel/py-packaging
 DEPENDS+=      ${PYPKGPREFIX}-parsel>=1.5.0:../../www/py-parsel
 DEPENDS+=      ${PYPKGPREFIX}-protego>=0.1.15:../../www/py-protego
 DEPENDS+=      ${PYPKGPREFIX}-pydispatcher>=2.0.5:../../devel/py-pydispatcher
-DEPENDS+=      ${PYPKGPREFIX}-queuelib>=1.4.2:../../devel/py-queuelib
-DEPENDS+=      ${PYPKGPREFIX}-service_identity>=23.1.0:../../security/py-service_identity
-DEPENDS+=      ${PYPKGPREFIX}-tldextract-[0-9]*:../../net/py-tldextract
+DEPENDS+=      ${PYPKGPREFIX}-queuelib>=1.6.1:../../devel/py-queuelib
+DEPENDS+=      ${PYPKGPREFIX}-service_identity>=24.2.0:../../security/py-service_identity
+DEPENDS+=      ${PYPKGPREFIX}-tldextract>=0:../../net/py-tldextract
 DEPENDS+=      ${PYPKGPREFIX}-twisted>=21.7.0:../../net/py-twisted
-DEPENDS+=      ${PYPKGPREFIX}-w3lib>=1.17.0:../../www/py-w3lib
+DEPENDS+=      ${PYPKGPREFIX}-w3lib>=2.1.1:../../www/py-w3lib
 DEPENDS+=      ${PYPKGPREFIX}-zope.interface>=5.1.0:../../devel/py-zope.interface
-TEST_DEPENDS+= ${PYPKGPREFIX}-pexpect-[0-9]*:../../devel/py-pexpect
-TEST_DEPENDS+= ${PYPKGPREFIX}-sybil-[0-9]*:../../devel/py-sybil
-TEST_DEPENDS+= ${PYPKGPREFIX}-testfixtures-[0-9]*:../../devel/py-testfixtures
+TEST_DEPENDS+= ${PYPKGPREFIX}-pexpect>=0:../../devel/py-pexpect
+TEST_DEPENDS+= ${PYPKGPREFIX}-sybil>=0:../../devel/py-sybil
+TEST_DEPENDS+= ${PYPKGPREFIX}-testfixtures>=0:../../devel/py-testfixtures
 
 USE_LANGUAGES= # none
 
@@ -36,7 +39,10 @@ PY_RENAME_BINARIES=  scrapy
 
 PYTHON_VERSIONS_INCOMPATIBLE=  310 311
 
+.include "../../lang/python/pyversion.mk"
+.if ${PYTHON_VERSION} < 314
+DEPENDS+=      ${PYPKGPREFIX}-backports.zstd>=1.3.0:../../archivers/py-backports.zstd
+.endif
+
 .include "../../lang/python/wheel.mk"
-PYTHON_VERSIONED_DEPENDENCIES+=        OpenSSL cryptography
-.include "../../lang/python/versioned_dependencies.mk"
 .include "../../mk/bsd.pkg.mk"

Index: pkgsrc/www/py-scrapy/PLIST
diff -u pkgsrc/www/py-scrapy/PLIST:1.18 pkgsrc/www/py-scrapy/PLIST:1.19
--- pkgsrc/www/py-scrapy/PLIST:1.18     Wed May 20 12:47:09 2026
+++ pkgsrc/www/py-scrapy/PLIST  Thu Sep  3 08:35:37 2026
@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.18 2026/05/20 12:47:09 adam Exp $
+@comment $NetBSD: PLIST,v 1.19 2026/09/03 08:35:37 adam Exp $
 bin/scrapy-${PYVERSSUFFIX}
 ${PYSITELIB}/${WHEEL_INFODIR}/METADATA
 ${PYSITELIB}/${WHEEL_INFODIR}/RECORD
@@ -73,9 +73,24 @@ ${PYSITELIB}/scrapy/contracts/default.py
 ${PYSITELIB}/scrapy/core/__init__.py
 ${PYSITELIB}/scrapy/core/__init__.pyc
 ${PYSITELIB}/scrapy/core/__init__.pyo
+${PYSITELIB}/scrapy/core/_http2/__init__.py
+${PYSITELIB}/scrapy/core/_http2/__init__.pyc
+${PYSITELIB}/scrapy/core/_http2/__init__.pyo
+${PYSITELIB}/scrapy/core/_http2/agent.py
+${PYSITELIB}/scrapy/core/_http2/agent.pyc
+${PYSITELIB}/scrapy/core/_http2/agent.pyo
+${PYSITELIB}/scrapy/core/_http2/protocol.py
+${PYSITELIB}/scrapy/core/_http2/protocol.pyc
+${PYSITELIB}/scrapy/core/_http2/protocol.pyo
+${PYSITELIB}/scrapy/core/_http2/stream.py
+${PYSITELIB}/scrapy/core/_http2/stream.pyc
+${PYSITELIB}/scrapy/core/_http2/stream.pyo
 ${PYSITELIB}/scrapy/core/downloader/__init__.py
 ${PYSITELIB}/scrapy/core/downloader/__init__.pyc
 ${PYSITELIB}/scrapy/core/downloader/__init__.pyo
+${PYSITELIB}/scrapy/core/downloader/_idna_patch.py
+${PYSITELIB}/scrapy/core/downloader/_idna_patch.pyc
+${PYSITELIB}/scrapy/core/downloader/_idna_patch.pyo
 ${PYSITELIB}/scrapy/core/downloader/contextfactory.py
 ${PYSITELIB}/scrapy/core/downloader/contextfactory.pyc
 ${PYSITELIB}/scrapy/core/downloader/contextfactory.pyo
@@ -124,18 +139,6 @@ ${PYSITELIB}/scrapy/core/downloader/tls.
 ${PYSITELIB}/scrapy/core/engine.py
 ${PYSITELIB}/scrapy/core/engine.pyc
 ${PYSITELIB}/scrapy/core/engine.pyo
-${PYSITELIB}/scrapy/core/http2/__init__.py
-${PYSITELIB}/scrapy/core/http2/__init__.pyc
-${PYSITELIB}/scrapy/core/http2/__init__.pyo
-${PYSITELIB}/scrapy/core/http2/agent.py
-${PYSITELIB}/scrapy/core/http2/agent.pyc
-${PYSITELIB}/scrapy/core/http2/agent.pyo
-${PYSITELIB}/scrapy/core/http2/protocol.py
-${PYSITELIB}/scrapy/core/http2/protocol.pyc
-${PYSITELIB}/scrapy/core/http2/protocol.pyo
-${PYSITELIB}/scrapy/core/http2/stream.py
-${PYSITELIB}/scrapy/core/http2/stream.pyc
-${PYSITELIB}/scrapy/core/http2/stream.pyo
 ${PYSITELIB}/scrapy/core/scheduler.py
 ${PYSITELIB}/scrapy/core/scheduler.pyc
 ${PYSITELIB}/scrapy/core/scheduler.pyo
@@ -375,6 +378,9 @@ ${PYSITELIB}/scrapy/spidermiddlewares/de
 ${PYSITELIB}/scrapy/spidermiddlewares/httperror.py
 ${PYSITELIB}/scrapy/spidermiddlewares/httperror.pyc
 ${PYSITELIB}/scrapy/spidermiddlewares/httperror.pyo
+${PYSITELIB}/scrapy/spidermiddlewares/metacopy.py
+${PYSITELIB}/scrapy/spidermiddlewares/metacopy.pyc
+${PYSITELIB}/scrapy/spidermiddlewares/metacopy.pyo
 ${PYSITELIB}/scrapy/spidermiddlewares/referer.py
 ${PYSITELIB}/scrapy/spidermiddlewares/referer.pyc
 ${PYSITELIB}/scrapy/spidermiddlewares/referer.pyo

Index: pkgsrc/www/py-scrapy/distinfo
diff -u pkgsrc/www/py-scrapy/distinfo:1.31 pkgsrc/www/py-scrapy/distinfo:1.32
--- pkgsrc/www/py-scrapy/distinfo:1.31  Thu Jul 23 07:40:43 2026
+++ pkgsrc/www/py-scrapy/distinfo       Thu Sep  3 08:35:37 2026
@@ -1,5 +1,5 @@
-$NetBSD: distinfo,v 1.31 2026/07/23 07:40:43 adam Exp $
+$NetBSD: distinfo,v 1.32 2026/09/03 08:35:37 adam Exp $
 
-BLAKE2s (scrapy-2.17.0.tar.gz) = 3866c1c08618b848614800f313d516fab563bb5a15913f9f74485cd18442a2f2
-SHA512 (scrapy-2.17.0.tar.gz) = 97482910f75bdf2812fa8a13b9d31dea579454d2235d761fd655e154d4096cc86483ca88150153d7cb0a14449fcb550515a4a23493c55e4ec0a435adaf0be80b
-Size (scrapy-2.17.0.tar.gz) = 1312077 bytes
+BLAKE2s (scrapy-2.18.0.tar.gz) = c8cab4d08ccf4fe404cc787a780887c2833a8d89e32fd4a47d1967476dc9ba8c
+SHA512 (scrapy-2.18.0.tar.gz) = 237a0ebc0937f871d4d0ec6e29622993f211337b3ff16e2087139b39f9753697526f9145c8c88feee7760516bf8249f7b9393e3df80c6d792e098210424c7e34
+Size (scrapy-2.18.0.tar.gz) = 1407695 bytes

Prev by Date: CVS commit: pkgsrc/devel/py-xopen
Next by Date: CVS commit: pkgsrc/doc
Previous by Thread: CVS commit: pkgsrc/devel/py-xopen
Next by Thread: CVS commit: pkgsrc/doc
Indexes:

Home | Main Index | Thread Index | Old Index