CVS commit: pkgsrc/www/py-scrapy

["Adam Ciarcinski" <adam%netbsd.org@localhost>]

Module Name:    pkgsrc
Committed By:   adam
Date:           Wed May 10 12:40:45 UTC 2023

Modified Files:
        pkgsrc/www/py-scrapy: Makefile distinfo

Log Message:
py-scrapy: updated to 2.9.0

Scrapy 2.9.0 (2023-05-08)
-------------------------

Highlights:

-   Per-domain download settings.
-   Compatibility with new cryptography_ and new parsel_.
-   JMESPath selectors from the new parsel_.
-   Bug fixes.

Deprecations
~~~~~~~~~~~~

-   :class:`scrapy.extensions.feedexport._FeedSlot` is renamed to
    :class:`scrapy.extensions.feedexport.FeedSlot` and the old name is
    deprecated. (:issue:`5876`)

New features
~~~~~~~~~~~~

-   Settings correponding to :setting:`DOWNLOAD_DELAY`,
    :setting:`CONCURRENT_REQUESTS_PER_DOMAIN` and
    :setting:`RANDOMIZE_DOWNLOAD_DELAY` can now be set on a per-domain basis
    via the new :setting:`DOWNLOAD_SLOTS` setting. (:issue:`5328`)

-   Added :meth:`TextResponse.jmespath`, a shortcut for JMESPath selectors
    available since parsel_ 1.8.1. (:issue:`5894`, :issue:`5915`)

-   Added :signal:`feed_slot_closed` and :signal:`feed_exporter_closed`
    signals. (:issue:`5876`)

-   Added :func:`scrapy.utils.request.request_to_curl`, a function to produce a
    curl command from a :class:`~scrapy.Request` object. (:issue:`5892`)

-   Values of :setting:`FILES_STORE` and :setting:`IMAGES_STORE` can now be
    :class:`pathlib.Path` instances. (:issue:`5801`)

Bug fixes
~~~~~~~~~

-   Fixed a warning with Parsel 1.8.1+. (:issue:`5903`, :issue:`5918`)

-   Fixed an error when using feed postprocessing with S3 storage.
    (:issue:`5500`, :issue:`5581`)

-   Added the missing :meth:`scrapy.settings.BaseSettings.setdefault` method.
    (:issue:`5811`, :issue:`5821`)

-   Fixed an error when using cryptography_ 40.0.0+ and
    :setting:`DOWNLOADER_CLIENT_TLS_VERBOSE_LOGGING` is enabled.
    (:issue:`5857`, :issue:`5858`)

-   The checksums returned by :class:`~scrapy.pipelines.files.FilesPipeline`
    for files on Google Cloud Storage are no longer Base64-encoded.
    (:issue:`5874`, :issue:`5891`)

-   :func:`scrapy.utils.request.request_from_curl` now supports $-prefixed
    string values for the curl ``--data-raw`` argument, which are produced by
    browsers for data that includes certain symbols. (:issue:`5899`,
    :issue:`5901`)

-   The :command:`parse` command now also works with async generator callbacks.
    (:issue:`5819`, :issue:`5824`)

-   The :command:`genspider` command now properly works with HTTPS URLs.
    (:issue:`3553`, :issue:`5808`)

-   Improved handling of asyncio loops. (:issue:`5831`, :issue:`5832`)

-   :class:`LinkExtractor <scrapy.linkextractors.lxmlhtml.LxmlLinkExtractor>`
    now skips certain malformed URLs instead of raising an exception.
    (:issue:`5881`)

-   :func:`scrapy.utils.python.get_func_args` now supports more types of
    callables. (:issue:`5872`, :issue:`5885`)

-   Fixed an error when processing non-UTF8 values of ``Content-Type`` headers.
    (:issue:`5914`, :issue:`5917`)

-   Fixed an error breaking user handling of send failures in
    :meth:`scrapy.mail.MailSender.send()`. (:issue:`1611`, :issue:`5880`)

Documentation
~~~~~~~~~~~~~

-   Expanded contributing docs. (:issue:`5109`, :issue:`5851`)

-   Added blacken-docs_ to pre-commit and reformatted the docs with it.
    (:issue:`5813`, :issue:`5816`)

-   Fixed a JS issue. (:issue:`5875`, :issue:`5877`)

-   Fixed ``make htmlview``. (:issue:`5878`, :issue:`5879`)

-   Fixed typos and other small errors. (:issue:`5827`, :issue:`5839`,
    :issue:`5883`, :issue:`5890`, :issue:`5895`, :issue:`5904`)

Quality assurance
~~~~~~~~~~~~~~~~~

-   Extended typing hints. (:issue:`5805`, :issue:`5889`, :issue:`5896`)

-   Tests for most of the examples in the docs are now run as a part of CI,
    found problems were fixed. (:issue:`5816`, :issue:`5826`, :issue:`5919`)

-   Removed usage of deprecated Python classes. (:issue:`5849`)

-   Silenced ``include-ignored`` warnings from coverage. (:issue:`5820`)

-   Fixed a random failure of the ``test_feedexport.test_batch_path_differ``
    test. (:issue:`5855`, :issue:`5898`)

-   Updated docstrings to match output produced by parsel_ 1.8.1 so that they
    don't cause test failures. (:issue:`5902`, :issue:`5919`)

-   Other CI and pre-commit improvements. (:issue:`5802`, :issue:`5823`,
    :issue:`5908`)


To generate a diff of this commit:
cvs rdiff -u -r1.17 -r1.18 pkgsrc/www/py-scrapy/Makefile
cvs rdiff -u -r1.13 -r1.14 pkgsrc/www/py-scrapy/distinfo

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.

Modified files:

Index: pkgsrc/www/py-scrapy/Makefile
diff -u pkgsrc/www/py-scrapy/Makefile:1.17 pkgsrc/www/py-scrapy/Makefile:1.18
--- pkgsrc/www/py-scrapy/Makefile:1.17  Thu Apr 27 09:33:44 2023
+++ pkgsrc/www/py-scrapy/Makefile       Wed May 10 12:40:44 2023
@@ -1,6 +1,6 @@
-# $NetBSD: Makefile,v 1.17 2023/04/27 09:33:44 adam Exp $
+# $NetBSD: Makefile,v 1.18 2023/05/10 12:40:44 adam Exp $
 
-DISTNAME=      Scrapy-2.8.0
+DISTNAME=      Scrapy-2.9.0
 PKGNAME=       ${PYPKGPREFIX}-${DISTNAME:tl}
 CATEGORIES=    www python
 MASTER_SITES=  ${MASTER_SITE_PYPI:=S/Scrapy/}

Index: pkgsrc/www/py-scrapy/distinfo
diff -u pkgsrc/www/py-scrapy/distinfo:1.13 pkgsrc/www/py-scrapy/distinfo:1.14
--- pkgsrc/www/py-scrapy/distinfo:1.13  Thu Apr 27 09:33:44 2023
+++ pkgsrc/www/py-scrapy/distinfo       Wed May 10 12:40:44 2023
@@ -1,5 +1,5 @@
-$NetBSD: distinfo,v 1.13 2023/04/27 09:33:44 adam Exp $
+$NetBSD: distinfo,v 1.14 2023/05/10 12:40:44 adam Exp $
 
-BLAKE2s (Scrapy-2.8.0.tar.gz) = e3bd0d640992aab05d1a5c9108b5b741cea4ad9dc902afc5d385eb9ba14c6a61
-SHA512 (Scrapy-2.8.0.tar.gz) = 6e631f84e27aeab5aeae672047869deb783b3c2f6de66c9bd6df768598a638d7e76a3f38945bfdf82f5ca0eb69491c262960f1645fe2f4947f72c0829a0eefaf
-Size (Scrapy-2.8.0.tar.gz) = 1140185 bytes
+BLAKE2s (Scrapy-2.9.0.tar.gz) = 6e844d42e3fe57c2ada806619cf63374243167f386b2ba97e661e381eae92bf7
+SHA512 (Scrapy-2.9.0.tar.gz) = 92f78d785e9770d5c38e72daeeb2a72d4f410692063c35d4997aa00afb8c9174b2ecb904eae551e20a0ea5a2585dece8e467909caf8f82411c58f04bdd9a83f0
+Size (Scrapy-2.9.0.tar.gz) = 1150623 bytes