CVS commit: pkgsrc/www/py-scrapy

To: pkgsrc-changes%NetBSD.org@localhost
Subject: CVS commit: pkgsrc/www/py-scrapy
From: "Adam Ciarcinski" <adam%netbsd.org@localhost>
Date: Fri, 16 Feb 2024 19:02:45 +0000

Module Name:    pkgsrc
Committed By:   adam
Date:           Fri Feb 16 19:02:45 UTC 2024

Modified Files:
        pkgsrc/www/py-scrapy: Makefile PLIST distinfo

Log Message:
py-scrapy: updated to 2.11.1

Scrapy 2.11.1 (2024-02-14)
--------------------------

Highlights:

-   Security bug fixes.

-   Support for Twisted >= 23.8.0.

-   Documentation improvements.

Security bug fixes
~~~~~~~~~~~~~~~~~~

-   Addressed `ReDoS vulnerabilities`_:

    -   ``scrapy.utils.iterators.xmliter`` is now deprecated in favor of
        :func:`~scrapy.utils.iterators.xmliter_lxml`, which
        :class:`~scrapy.spiders.XMLFeedSpider` now uses.

        To minimize the impact of this change on existing code,
        :func:`~scrapy.utils.iterators.xmliter_lxml` now supports indicating
        the node namespace with a prefix in the node name, and big files with
        highly nested trees when using libxml2 2.7+.

    -   Fixed regular expressions in the implementation of the
        :func:`~scrapy.utils.response.open_in_browser` function.

    Please, see the `cc65-xxvf-f7r9 security advisory`_ for more information.

    .. _ReDoS vulnerabilities: https://owasp.org/www-community/attacks/Regular_expression_Denial_of_Service_-_ReDoS
    .. _cc65-xxvf-f7r9 security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-cc65-xxvf-f7r9

-   :setting:`DOWNLOAD_MAXSIZE` and :setting:`DOWNLOAD_WARNSIZE` now also apply
    to the decompressed response body. Please, see the `7j7m-v7m3-jqm7 security
    advisory`_ for more information.

    .. _7j7m-v7m3-jqm7 security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-7j7m-v7m3-jqm7

-   Also in relation with the `7j7m-v7m3-jqm7 security advisory`_, the
    deprecated ``scrapy.downloadermiddlewares.decompression`` module has been
    removed.

-   The ``Authorization`` header is now dropped on redirects to a different
    domain. Please, see the `cw9j-q3vf-hrrv security advisory`_ for more
    information.

    .. _cw9j-q3vf-hrrv security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-cw9j-q3vf-hrrv

Modified requirements
~~~~~~~~~~~~~~~~~~~~~

-   The Twisted dependency is no longer restricted to < 23.8.0. (:issue:`6024`,
    :issue:`6064`, :issue:`6142`)

Bug fixes
~~~~~~~~~

-   The OS signal handling code was refactored to no longer use private Twisted
    functions. (:issue:`6024`, :issue:`6064`, :issue:`6112`)

Documentation
~~~~~~~~~~~~~

-   Improved documentation for :class:`~scrapy.crawler.Crawler` initialization
    changes made in the 2.11.0 release. (:issue:`6057`, :issue:`6147`)

-   Extended documentation for :attr:`Request.meta <scrapy.http.Request.meta>`.
    (:issue:`5565`)

-   Fixed the :reqmeta:`dont_merge_cookies` documentation. (:issue:`5936`,
    :issue:`6077`)

-   Added a link to Zyte's export guides to the :ref:`feed exports
    <topics-feed-exports>` documentation. (:issue:`6183`)

-   Added a missing note about backward-incompatible changes in
    :class:`~scrapy.exporters.PythonItemExporter` to the 2.11.0 release notes.
    (:issue:`6060`, :issue:`6081`)

-   Added a missing note about removing the deprecated
    ``scrapy.utils.boto.is_botocore()`` function to the 2.8.0 release notes.
    (:issue:`6056`, :issue:`6061`)

-   Other documentation improvements. (:issue:`6128`, :issue:`6144`,
    :issue:`6163`, :issue:`6190`, :issue:`6192`)

Quality assurance
~~~~~~~~~~~~~~~~~

-   Added Python 3.12 to the CI configuration, re-enabled tests that were
    disabled when the pre-release support was added. (:issue:`5985`,
    :issue:`6083`, :issue:`6098`)

-   Fixed a test issue on PyPy 7.3.14. (:issue:`6204`, :issue:`6205`)


To generate a diff of this commit:
cvs rdiff -u -r1.20 -r1.21 pkgsrc/www/py-scrapy/Makefile
cvs rdiff -u -r1.10 -r1.11 pkgsrc/www/py-scrapy/PLIST
cvs rdiff -u -r1.15 -r1.16 pkgsrc/www/py-scrapy/distinfo

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: pkgsrc/www/py-scrapy/Makefile
diff -u pkgsrc/www/py-scrapy/Makefile:1.20 pkgsrc/www/py-scrapy/Makefile:1.21
--- pkgsrc/www/py-scrapy/Makefile:1.20  Tue Oct 10 17:18:23 2023
+++ pkgsrc/www/py-scrapy/Makefile       Fri Feb 16 19:02:45 2024
@@ -1,6 +1,6 @@
-# $NetBSD: Makefile,v 1.20 2023/10/10 17:18:23 triaxx Exp $
+# $NetBSD: Makefile,v 1.21 2024/02/16 19:02:45 adam Exp $
 
-DISTNAME=      Scrapy-2.11.0
+DISTNAME=      Scrapy-2.11.1
 PKGNAME=       ${PYPKGPREFIX}-${DISTNAME:tl}
 CATEGORIES=    www python
 MASTER_SITES=  ${MASTER_SITE_PYPI:=S/Scrapy/}
@@ -10,7 +10,7 @@ HOMEPAGE=     https://scrapy.org/
 COMMENT=       High-level Web Crawling and Web Scraping framework
 LICENSE=       modified-bsd
 
-DEPENDS+=      ${PYPKGPREFIX}-zope.interface>=5.1.0:../../devel/py-zope.interface
+TOOL_DEPENDS+= ${PYPKGPREFIX}-wheel-[0-9]*:../../devel/py-wheel
 DEPENDS+=      ${PYPKGPREFIX}-cssselect>=0.9.1:../../textproc/py-cssselect
 DEPENDS+=      ${PYPKGPREFIX}-itemadapter>=0.1.0:../../textproc/py-itemadapter
 DEPENDS+=      ${PYPKGPREFIX}-itemloaders>=1.0.1:../../textproc/py-itemloaders
@@ -21,21 +21,21 @@ DEPENDS+=   ${PYPKGPREFIX}-protego>=0.1.15
 DEPENDS+=      ${PYPKGPREFIX}-pydispatcher>=2.0.5:../../devel/py-pydispatcher
 DEPENDS+=      ${PYPKGPREFIX}-queuelib>=1.4.2:../../devel/py-queuelib
 DEPENDS+=      ${PYPKGPREFIX}-service_identity>=18.1.0:../../security/py-service_identity
+DEPENDS+=      ${PYPKGPREFIX}-setuptools-[0-9]*:../../devel/py-setuptools
 DEPENDS+=      ${PYPKGPREFIX}-tldextract-[0-9]*:../../net/py-tldextract
 DEPENDS+=      ${PYPKGPREFIX}-twisted>=18.9.0:../../net/py-twisted
 DEPENDS+=      ${PYPKGPREFIX}-w3lib>=1.17.0:../../www/py-w3lib
+DEPENDS+=      ${PYPKGPREFIX}-zope.interface>=5.1.0:../../devel/py-zope.interface
 
 USE_LANGUAGES= # none
 
-USE_PKG_RESOURCES=     yes
-
 PYTHON_VERSIONS_INCOMPATIBLE=  27
 
 post-install:
        cd ${DESTDIR}${PREFIX}/bin && \
        ${MV} scrapy scrapy-${PYVERSSUFFIX} || ${TRUE}
 
-.include "../../lang/python/egg.mk"
+.include "../../lang/python/wheel.mk"
 PYTHON_VERSIONED_DEPENDENCIES+=        cryptography
 PYTHON_VERSIONED_DEPENDENCIES+=        OpenSSL
 .include "../../lang/python/versioned_dependencies.mk"

Index: pkgsrc/www/py-scrapy/PLIST
diff -u pkgsrc/www/py-scrapy/PLIST:1.10 pkgsrc/www/py-scrapy/PLIST:1.11
--- pkgsrc/www/py-scrapy/PLIST:1.10     Tue Oct 10 17:18:23 2023
+++ pkgsrc/www/py-scrapy/PLIST  Fri Feb 16 19:02:45 2024
@@ -1,12 +1,12 @@
-@comment $NetBSD: PLIST,v 1.10 2023/10/10 17:18:23 triaxx Exp $
+@comment $NetBSD: PLIST,v 1.11 2024/02/16 19:02:45 adam Exp $
 bin/scrapy-${PYVERSSUFFIX}
-${PYSITELIB}/${EGG_INFODIR}/PKG-INFO
-${PYSITELIB}/${EGG_INFODIR}/SOURCES.txt
-${PYSITELIB}/${EGG_INFODIR}/dependency_links.txt
-${PYSITELIB}/${EGG_INFODIR}/entry_points.txt
-${PYSITELIB}/${EGG_INFODIR}/not-zip-safe
-${PYSITELIB}/${EGG_INFODIR}/requires.txt
-${PYSITELIB}/${EGG_INFODIR}/top_level.txt
+${PYSITELIB}/${WHEEL_INFODIR}/AUTHORS
+${PYSITELIB}/${WHEEL_INFODIR}/LICENSE
+${PYSITELIB}/${WHEEL_INFODIR}/METADATA
+${PYSITELIB}/${WHEEL_INFODIR}/RECORD
+${PYSITELIB}/${WHEEL_INFODIR}/WHEEL
+${PYSITELIB}/${WHEEL_INFODIR}/entry_points.txt
+${PYSITELIB}/${WHEEL_INFODIR}/top_level.txt
 ${PYSITELIB}/scrapy/VERSION
 ${PYSITELIB}/scrapy/__init__.py
 ${PYSITELIB}/scrapy/__init__.pyc
@@ -152,9 +152,6 @@ ${PYSITELIB}/scrapy/downloadermiddleware
 ${PYSITELIB}/scrapy/downloadermiddlewares/cookies.py
 ${PYSITELIB}/scrapy/downloadermiddlewares/cookies.pyc
 ${PYSITELIB}/scrapy/downloadermiddlewares/cookies.pyo
-${PYSITELIB}/scrapy/downloadermiddlewares/decompression.py
-${PYSITELIB}/scrapy/downloadermiddlewares/decompression.pyc
-${PYSITELIB}/scrapy/downloadermiddlewares/decompression.pyo
 ${PYSITELIB}/scrapy/downloadermiddlewares/defaultheaders.py
 ${PYSITELIB}/scrapy/downloadermiddlewares/defaultheaders.pyc
 ${PYSITELIB}/scrapy/downloadermiddlewares/defaultheaders.pyo
@@ -420,6 +417,9 @@ ${PYSITELIB}/scrapy/templates/spiders/xm
 ${PYSITELIB}/scrapy/utils/__init__.py
 ${PYSITELIB}/scrapy/utils/__init__.pyc
 ${PYSITELIB}/scrapy/utils/__init__.pyo
+${PYSITELIB}/scrapy/utils/_compression.py
+${PYSITELIB}/scrapy/utils/_compression.pyc
+${PYSITELIB}/scrapy/utils/_compression.pyo
 ${PYSITELIB}/scrapy/utils/asyncgen.py
 ${PYSITELIB}/scrapy/utils/asyncgen.pyc
 ${PYSITELIB}/scrapy/utils/asyncgen.pyo

Index: pkgsrc/www/py-scrapy/distinfo
diff -u pkgsrc/www/py-scrapy/distinfo:1.15 pkgsrc/www/py-scrapy/distinfo:1.16
--- pkgsrc/www/py-scrapy/distinfo:1.15  Tue Oct 10 17:18:23 2023
+++ pkgsrc/www/py-scrapy/distinfo       Fri Feb 16 19:02:45 2024
@@ -1,5 +1,5 @@
-$NetBSD: distinfo,v 1.15 2023/10/10 17:18:23 triaxx Exp $
+$NetBSD: distinfo,v 1.16 2024/02/16 19:02:45 adam Exp $
 
-BLAKE2s (Scrapy-2.11.0.tar.gz) = c4bfc4779599de5e70dab45d023b9e97ff3457e9f6c21c31bf5b77401f101a2d
-SHA512 (Scrapy-2.11.0.tar.gz) = bbebea94329ffacfb2b867884b3800986f4013bbbe34eb2d299c09a0a653ac2793e581d92509dabaa0f8b74a0b4fbeebedbad8fb1074b18ee522e73fad039d2b
-Size (Scrapy-2.11.0.tar.gz) = 1171092 bytes
+BLAKE2s (Scrapy-2.11.1.tar.gz) = ec247564bb7f25be4bca8e966e593c7c6c222b9644cf05686d6d9a0a4a436b07
+SHA512 (Scrapy-2.11.1.tar.gz) = c33bf8fe45c96865483398920e823bd169d7d7e5d67dcfd5e57e4546f1016cfdcb404ebcbf67a6710a4597d5970f55481226fee25c27291dfaedfc00322327d9
+Size (Scrapy-2.11.1.tar.gz) = 1176726 bytes

Prev by Date: CVS commit: pkgsrc/devel/py-zope.interface
Next by Date: CVS commit: pkgsrc/devel/py-limits
Previous by Thread: CVS commit: pkgsrc/devel/py-zope.interface
Next by Thread: CVS commit: pkgsrc/devel/py-limits
Indexes:

Home | Main Index | Thread Index | Old Index