CVS commit: pkgsrc/www/py-scrapy

To: pkgsrc-changes%NetBSD.org@localhost
Subject: CVS commit: pkgsrc/www/py-scrapy
From: "Adam Ciarcinski" <adam%netbsd.org@localhost>
Date: Tue, 14 May 2024 19:15:59 +0000

Module Name:    pkgsrc
Committed By:   adam
Date:           Tue May 14 19:15:59 UTC 2024

Modified Files:
        pkgsrc/www/py-scrapy: Makefile PLIST distinfo

Log Message:
py-scrapy: updated to 2.11.2

Scrapy 2.11.2 (2024-05-14)
--------------------------

Security bug fixes
~~~~~~~~~~~~~~~~~~

-   Redirects to non-HTTP protocols are no longer followed. Please, see the
    `23j4-mw76-5v7h security advisory`_ for more information. (:issue:`457`)

    .. _23j4-mw76-5v7h security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-23j4-mw76-5v7h

-   The ``Authorization`` header is now dropped on redirects to a different
    scheme (``http://`` or ``https://``) or port, even if the domain is the
    same. Please, see the `4qqq-9vqf-3h3f security advisory`_ for more
    information.

    .. _4qqq-9vqf-3h3f security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-4qqq-9vqf-3h3f

-   When using system proxy settings that are different for ``http://`` and
    ``https://``, redirects to a different URL scheme will now also trigger the
    corresponding change in proxy settings for the redirected request. Please,
    see the `jm3v-qxmh-hxwv security advisory`_ for more information.
    (:issue:`767`)

    .. _jm3v-qxmh-hxwv security advisory: https://github.com/scrapy/scrapy/security/advisories/GHSA-jm3v-qxmh-hxwv

-   :attr:`Spider.allowed_domains <scrapy.Spider.allowed_domains>` is now
    enforced for all requests, and not only requests from spider callbacks.
    (:issue:`1042`, :issue:`2241`, :issue:`6358`)

-   :func:`~scrapy.utils.iterators.xmliter_lxml` no longer resolves XML
    entities. (:issue:`6265`)

-   defusedxml_ is now used to make
    :class:`scrapy.http.request.rpc.XmlRpcRequest` more secure.
    (:issue:`6250`, :issue:`6251`)

    .. _defusedxml: https://github.com/tiran/defusedxml

Bug fixes
~~~~~~~~~

-   Restored support for brotlipy_, which had been dropped in Scrapy 2.11.1 in
    favor of brotli_. (:issue:`6261`)

    .. _brotli: https://github.com/google/brotli

    .. note:: brotlipy is deprecated, both in Scrapy and upstream. Use brotli
        instead if you can.

-   Make :setting:`METAREFRESH_IGNORE_TAGS` ``["noscript"]`` by default. This
    prevents
    :class:`~scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware` from
    following redirects that would not be followed by web browsers with
    JavaScript enabled. (:issue:`6342`, :issue:`6347`)

-   During :ref:`feed export <topics-feed-exports>`, do not close the
    underlying file from :ref:`built-in post-processing plugins
    <builtin-plugins>`.
    (:issue:`5932`, :issue:`6178`, :issue:`6239`)

-   :class:`LinkExtractor <scrapy.linkextractors.lxmlhtml.LxmlLinkExtractor>`
    now properly applies the ``unique`` and ``canonicalize`` parameters.
    (:issue:`3273`, :issue:`6221`)

-   Do not initialize the scheduler disk queue if :setting:`JOBDIR` is an empty
    string. (:issue:`6121`, :issue:`6124`)

-   Fix :attr:`Spider.logger <scrapy.Spider.logger>` not logging custom extra
    information. (:issue:`6323`, :issue:`6324`)

-   ``robots.txt`` files with a non-UTF-8 encoding no longer prevent parsing
    the UTF-8-compatible (e.g. ASCII) parts of the document.
    (:issue:`6292`, :issue:`6298`)

-   :meth:`scrapy.http.cookies.WrappedRequest.get_header` no longer raises an
    exception if ``default`` is ``None``.
    (:issue:`6308`, :issue:`6310`)

-   :class:`~scrapy.selector.Selector` now uses
    :func:`scrapy.utils.response.get_base_url` to determine the base URL of a
    given :class:`~scrapy.http.Response`. (:issue:`6265`)

-   The :meth:`media_to_download` method of :ref:`media pipelines
    <topics-media-pipeline>` now logs exceptions before stripping them.
    (:issue:`5067`, :issue:`5068`)

-   When passing a callback to the :command:`parse` command, build the callback
    callable with the right signature.
    (:issue:`6182`)

Documentation
~~~~~~~~~~~~~

-   Add a FAQ entry about :ref:`creating blank requests <faq-blank-request>`.
    (:issue:`6203`, :issue:`6208`)

-   Document that :attr:`scrapy.selector.Selector.type` can be ``"json"``.
    (:issue:`6328`, :issue:`6334`)

Quality assurance
~~~~~~~~~~~~~~~~~

-   Make builds reproducible. (:issue:`5019`, :issue:`6322`)

-   Packaging and test fixes.


To generate a diff of this commit:
cvs rdiff -u -r1.21 -r1.22 pkgsrc/www/py-scrapy/Makefile
cvs rdiff -u -r1.11 -r1.12 pkgsrc/www/py-scrapy/PLIST
cvs rdiff -u -r1.16 -r1.17 pkgsrc/www/py-scrapy/distinfo

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: pkgsrc/www/py-scrapy/Makefile
diff -u pkgsrc/www/py-scrapy/Makefile:1.21 pkgsrc/www/py-scrapy/Makefile:1.22
--- pkgsrc/www/py-scrapy/Makefile:1.21  Fri Feb 16 19:02:45 2024
+++ pkgsrc/www/py-scrapy/Makefile       Tue May 14 19:15:59 2024
@@ -1,7 +1,7 @@
-# $NetBSD: Makefile,v 1.21 2024/02/16 19:02:45 adam Exp $
+# $NetBSD: Makefile,v 1.22 2024/05/14 19:15:59 adam Exp $
 
-DISTNAME=      Scrapy-2.11.1
-PKGNAME=       ${PYPKGPREFIX}-${DISTNAME:tl}
+DISTNAME=      scrapy-2.11.2
+PKGNAME=       ${PYPKGPREFIX}-${DISTNAME}
 CATEGORIES=    www python
 MASTER_SITES=  ${MASTER_SITE_PYPI:=S/Scrapy/}
 
@@ -10,11 +10,15 @@ HOMEPAGE=   https://scrapy.org/
 COMMENT=       High-level Web Crawling and Web Scraping framework
 LICENSE=       modified-bsd
 
+WHEEL_NAME=    Scrapy-${PKGVERSION_NOREV}
+
 TOOL_DEPENDS+= ${PYPKGPREFIX}-wheel-[0-9]*:../../devel/py-wheel
 DEPENDS+=      ${PYPKGPREFIX}-cssselect>=0.9.1:../../textproc/py-cssselect
+DEPENDS+=      ${PYPKGPREFIX}-defusedxml>=0.7.1:../../textproc/py-defusedxml
 DEPENDS+=      ${PYPKGPREFIX}-itemadapter>=0.1.0:../../textproc/py-itemadapter
 DEPENDS+=      ${PYPKGPREFIX}-itemloaders>=1.0.1:../../textproc/py-itemloaders
-DEPENDS+=      ${PYPKGPREFIX}-lxml>=4.3.0:../../textproc/py-lxml
+DEPENDS+=      ${PYPKGPREFIX}-lxml>=4.4.1:../../textproc/py-lxml
+DEPENDS+=      ${PYPKGPREFIX}-cssselect>=0.9.1:../../textproc/py-cssselect
 DEPENDS+=      ${PYPKGPREFIX}-packaging-[0-9]*:../../devel/py-packaging
 DEPENDS+=      ${PYPKGPREFIX}-parsel>=1.5.0:../../www/py-parsel
 DEPENDS+=      ${PYPKGPREFIX}-protego>=0.1.15:../../www/py-protego

Index: pkgsrc/www/py-scrapy/PLIST
diff -u pkgsrc/www/py-scrapy/PLIST:1.11 pkgsrc/www/py-scrapy/PLIST:1.12
--- pkgsrc/www/py-scrapy/PLIST:1.11     Fri Feb 16 19:02:45 2024
+++ pkgsrc/www/py-scrapy/PLIST  Tue May 14 19:15:59 2024
@@ -1,4 +1,4 @@
-@comment $NetBSD: PLIST,v 1.11 2024/02/16 19:02:45 adam Exp $
+@comment $NetBSD: PLIST,v 1.12 2024/05/14 19:15:59 adam Exp $
 bin/scrapy-${PYVERSSUFFIX}
 ${PYSITELIB}/${WHEEL_INFODIR}/AUTHORS
 ${PYSITELIB}/${WHEEL_INFODIR}/LICENSE
@@ -170,6 +170,9 @@ ${PYSITELIB}/scrapy/downloadermiddleware
 ${PYSITELIB}/scrapy/downloadermiddlewares/httpproxy.py
 ${PYSITELIB}/scrapy/downloadermiddlewares/httpproxy.pyc
 ${PYSITELIB}/scrapy/downloadermiddlewares/httpproxy.pyo
+${PYSITELIB}/scrapy/downloadermiddlewares/offsite.py
+${PYSITELIB}/scrapy/downloadermiddlewares/offsite.pyc
+${PYSITELIB}/scrapy/downloadermiddlewares/offsite.pyo
 ${PYSITELIB}/scrapy/downloadermiddlewares/redirect.py
 ${PYSITELIB}/scrapy/downloadermiddlewares/redirect.pyc
 ${PYSITELIB}/scrapy/downloadermiddlewares/redirect.pyo

Index: pkgsrc/www/py-scrapy/distinfo
diff -u pkgsrc/www/py-scrapy/distinfo:1.16 pkgsrc/www/py-scrapy/distinfo:1.17
--- pkgsrc/www/py-scrapy/distinfo:1.16  Fri Feb 16 19:02:45 2024
+++ pkgsrc/www/py-scrapy/distinfo       Tue May 14 19:15:59 2024
@@ -1,5 +1,5 @@
-$NetBSD: distinfo,v 1.16 2024/02/16 19:02:45 adam Exp $
+$NetBSD: distinfo,v 1.17 2024/05/14 19:15:59 adam Exp $
 
-BLAKE2s (Scrapy-2.11.1.tar.gz) = ec247564bb7f25be4bca8e966e593c7c6c222b9644cf05686d6d9a0a4a436b07
-SHA512 (Scrapy-2.11.1.tar.gz) = c33bf8fe45c96865483398920e823bd169d7d7e5d67dcfd5e57e4546f1016cfdcb404ebcbf67a6710a4597d5970f55481226fee25c27291dfaedfc00322327d9
-Size (Scrapy-2.11.1.tar.gz) = 1176726 bytes
+BLAKE2s (scrapy-2.11.2.tar.gz) = 43f17692a3ceb60435ed9e6a8f1921ac029b1e2f36689c992978c9cee3d47e26
+SHA512 (scrapy-2.11.2.tar.gz) = 488e3d1aff4f76ff795da7dbcaed6459d39ad45dc5ba5482f628b2d52dafbe740ee2f0c2caa4ffb6a4fe70b4268114877be833b6066a61f8d1fab8a95a74badf
+Size (scrapy-2.11.2.tar.gz) = 1187710 bytes

Prev by Date: CVS commit: pkgsrc/doc
Next by Date: CVS commit: pkgsrc/textproc/py-mdit-py-plugins
Previous by Thread: CVS commit: pkgsrc/doc
Next by Thread: CVS commit: pkgsrc/textproc/py-mdit-py-plugins
Indexes:

Home | Main Index | Thread Index | Old Index