pkgsrc-WIP-changes archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
py-dask: Update to 2022.6.1
Module Name:	pkgsrc-wip
Committed By:	Matthew Danielson <matthewd%fastmail.us@localhost>
Pushed By:	matthewd
Date:		Sun Jul 3 19:18:21 2022 -0700
Changeset:	f73bb1f475c18a70d84cd5c8e2fb03a9b9997ba9
Modified Files:
	py-dask/Makefile
	py-dask/PLIST
	py-dask/distinfo
	py-dask/options.mk
Log Message:
py-dask: Update to 2022.6.1
2022.6.1
Released on June 24, 2022
Enhancements
    Dask in pyodide (GH#9053) Ian Rose
    Create dask.utils.show_versions (GH#9144) Sultan Orazbayev
    Better error message for unsupported numpy operations on dask.dataframe objects. (GH#9201) Julia Signell
    Add allow_rechunk kwarg to dask.array.overlap function (GH#7776) Genevieve Buckley
    Add minutes and hours to dask.utils.format_time (GH#9116) Matthew Rocklin
    More retries when writing parquet to remote filesystem (GH#9175) Ian Rose
Bug Fixes
    Timedelta deterministic hashing (GH#9213) Fabien Aulaire
    Enum deterministic hashing (GH#9212) Fabien Aulaire
    shuffle_group(): avoid converting to arrays (GH#9157) Mads R. B. Kristensen
Deprecations
    Deprecate extra format_time utility (GH#9184) James Bourbeau
Documentation
    Better SEO for 10 Minutes to Dask (GH#9182) Sarah Charlotte Johnson
    Better SEO for Delayed and Best Practices (GH#9194) Sarah Charlotte Johnson
    Include known inconsistency in DataFrame str.split accessor docstring (GH#9177) Richard Pelgrim
    Add inconsistencies keyword to derived_from (GH#9192) Richard Pelgrim
    Add missing append in delayed best practices example (GH#9202) Ben
    Fix indentation in Best Practices (GH#9196) Sarah Charlotte Johnson
    Add link to Genevieve Buckley’s blog on chunk sizes (GH#9199) Pavithra Eswaramoorthy
    Update to_csv docstring (GH#9094) Sarah Charlotte Johnson
Maintenance
    Update versioneer: change from using SafeConfigParser to ConfigParser (GH#9205) Thomas A Caswell
    Remove ipython hack in CI(GH#9200) crusaderky
2022.6.0
Released on June 10, 2022
Enhancements
    Add feature to show names of layer dependencies in HLG JupyterLab repr (GH#9081) Angelos Omirolis
    Add arrow schema extraction dispatch (GH#9169) GALI PREM SAGAR
    Add sort_results argument to assert_eq (GH#9130) Pavithra Eswaramoorthy
    Add weeks to parse_timedelta (GH#9168) Matthew Rocklin
    Warn that cloudpickle is not always deterministic (GH#9148) Pavithra Eswaramoorthy
    Switch parquet default engine (GH#9140) Jim Crist-Harif
    Use deterministic hashing with _iLocIndexer / _LocIndexer (GH#9108) Fabien Aulaire
    Enfore consistent schema in to_parquet pyarrow (GH#9131) Jim Crist-Harif
Bug Fixes
    Fix pyarrow.StringArray pickle (GH#9170) Jim Crist-Harif
    Fix parallel metadata collection in pyarrow engine (GH#9165) Richard (Rick) Zamora
    Improve pyarrow partitioning logic (GH#9147) James Bourbeau
    pyarrow 8.0 partitioning fix (GH#9143) James Bourbeau
Documentation
    Better SEO for Installing Dask and Dask DataFrame Best Practices (GH#9178) Sarah Charlotte Johnson
    Update logos page in docs (GH#9167) Sarah Charlotte Johnson
    Add example using pandas Series to map_partition doctring (GH#9161) Alex-JG3
    Update docs theme for rebranding (GH#9160) Sarah Charlotte Johnson
    Better SEO for docs on Dask DataFrames (GH#9128) Sarah Charlotte Johnson
Maintenance
    Remove ensure_file from recommended practice for downstream libraries (GH#9171) Matthew Rocklin
    Test round-tripping DataFrame parquet I/O including pyspark (GH#9156) Ian Rose
    Try disabling HDF5 locking (GH#9154) Ian Rose
    Link best practices to DataFrame-parquet (GH#9150) Tom Augspurger
    Fix typo in map_partitions func parameter description (GH#9149) Christopher Akiki
    Un-xfail test_groupby_grouper_dispatch (GH#9139) GALI PREM SAGAR
    Temporarily import cleanup fixture from distributed (GH#9138) James Bourbeau
    Simplify partitioning logic in pyarrow parquet engine (GH#9041) Richard (Rick) Zamora
2022.05.2
Released on May 26, 2022
Enhancements
    Add a dispatch for non-pandas Grouper objects and use it in GroupBy (GH#9074) brandon-b-miller
    Error if read_parquet & to_parquet files intersect (GH#9124) Jim Crist-Harif
    Visualize task graphs using ipycytoscape (GH#9091) Ian Rose
Documentation
    Fix various typos (GH#9126) Ryan Russell
Maintenance
    Fix flaky test_filter_nonpartition_columns (GH#9127) Pavithra Eswaramoorthy
    Update gpuCI RAPIDS_VER to 22.08 (GH#9120)
    Include conftest.py` in sdists (GH#9115) Ben Beasley
2022.05.1
Released on May 24, 2022
New Features
    Add DataFrame.from_dict classmethod (GH#9017) Matthew Powers
    Add from_map function to Dask DataFrame (GH#8911) Richard (Rick) Zamora
Enhancements
    Improve to_parquet error for appended divisions overlap (GH#9102) Jim Crist-Harif
    Enabled user-defined process-initializer functions (GH#9087) ParticularMiner
    Mention align_dataframes=False option in map_partitions error (GH#9075) Gabe Joseph
    Add kwarg enforce_ndim to dask.array.map_blocks() (GH#8865) ParticularMiner
    Implement Series.GroupBy.fillna / DataFrame.GroupBy.fillna methods (GH#8869) Pavithra Eswaramoorthy
    Allow fillna with Dask DataFrame (GH#8950) Pavithra Eswaramoorthy
    Update error message for assignment with 1-d dask array (GH#9036) Pavithra Eswaramoorthy
    Collection Protocol (GH#8674) Doug Davis
    Patch around pandas ArrowStringArray pickling (GH#9024) Jim Crist-Harif
    Band-aid for compute_as_if_collection (GH#8998) Ian Rose
    Add p2p shuffle option (GH#8836) Matthew Rocklin
Bug Fixes
    Fixup column projection with no columns (GH#9106) Jim Crist-Harif
    Blockwise cull NumPy dtype (GH#9100) Ian Rose
    Fix column-projection bug in from_map (GH#9078) Richard (Rick) Zamora
    Prevent nulls in index for non-numeric dtypes (GH#8963) Jorge López
    Fix is_monotonic methods for more than 8 partitions (GH#9019) Julia Signell
    Handle enumerate and generator inputs to from_map (GH#9066) Richard (Rick) Zamora
    Revert is_dask_collection; back to previous implementation (GH#9062) Doug Davis
    Fix Blockwise.clone does not handle iterable literal arguments correctly (GH#8979) JSKenyon
    Array setitem hardmask (GH#9027) David Hassell
    Fix overlapping divisions error on append (GH#8997) Ian Rose
Deprecations
    Add pre-deprecation warnings for read_parquet kwargs chunksize and aggregate_files (GH#9052) Richard (Rick) Zamora
Documentation
    Document map_partitions handling of args vs kwargs, usage of partition_info (GH#9084) Charles Blackmon-Luca
    Update custom collection documentation (leverage new collection protocol) (GH#9097) Doug Davis
    Better SEO for docs on creating and storing Dask DataFrames (GH#9098) Sarah Charlotte Johnson
    Clarify chunking in imread docstring (GH#9082) Genevieve Buckley
    Rearrange docs TOC (GH#9001) Matthew Rocklin
    Corrected map_blocks() docstring for kwarg enforce_ndim (GH#9071) ParticularMiner
    Update DataFrame SQL docs references to other libraries (GH#9077) Charles Blackmon-Luca
    Update page on creating and storing Dask DataFrames (GH#9025) Sarah Charlotte Johnson
Maintenance
    Include NUMPY_LICENSE.txt in license files (GH#9113) Ben Beasley
    Increase retries when installing nightly pandas (GH#9103) James Bourbeau
    Force nightly pyarrow in the upstream build (GH#9095) Joris Van den Bossche
    Improve object handling & testing of ensure_unicode (GH#9059) John A Kirkham
    Force nightly pyarrow in the upstream build (GH#8993) Joris Van den Bossche
    Additional check on is_dask_collection (GH#9054) Doug Davis
    Update ensure_bytes (GH#9050) John A Kirkham
    Add end of file pre-commit hook (GH#9045) James Bourbeau
    Add codespell pre-commit hook (GH#9040) James Bourbeau
    Remove the HDFS tests (GH#9039) Jim Crist-Harif
    Fix flaky test_reductions_2D (GH#9037) Jim Crist-Harif
    Prevent codecov from notifying of failure too soon (GH#9031) Jim Crist-Harif
    Only test on Python 3.9 on macos (GH#9029) Jim Crist-Harif
    Update to_timedelta default unit (GH#9010) Pavithra Eswaramoorthy
To see a diff of this commit:
https://wip.pkgsrc.org/cgi-bin/gitweb.cgi?p=pkgsrc-wip.git;a=commitdiff;h=f73bb1f475c18a70d84cd5c8e2fb03a9b9997ba9
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
diffstat:
 py-dask/Makefile   |  2 +-
 py-dask/PLIST      | 30 ++++++++++++++++++++----------
 py-dask/distinfo   |  6 +++---
 py-dask/options.mk |  2 +-
 4 files changed, 25 insertions(+), 15 deletions(-)
diffs:
diff --git a/py-dask/Makefile b/py-dask/Makefile
index a3d0324756..aac92f136b 100644
--- a/py-dask/Makefile
+++ b/py-dask/Makefile
@@ -1,6 +1,6 @@
 # $NetBSD$
 
-GITHUB_TAG=	2022.05.0
+GITHUB_TAG=	2022.6.1
 DISTNAME=	dask-${GITHUB_TAG}
 PKGNAME=	${PYPKGPREFIX}-${DISTNAME}
 GITHUB_PROJECT=	dask
diff --git a/py-dask/PLIST b/py-dask/PLIST
index 473b638cc9..668bb7465b 100644
--- a/py-dask/PLIST
+++ b/py-dask/PLIST
@@ -1,16 +1,18 @@
 @comment $NetBSD$
-${PYSITELIB}/dask-2022.5.0.dist-info/INSTALLER
-${PYSITELIB}/dask-2022.5.0.dist-info/LICENSE.txt
-${PYSITELIB}/dask-2022.5.0.dist-info/METADATA
-${PYSITELIB}/dask-2022.5.0.dist-info/RECORD
-${PYSITELIB}/dask-2022.5.0.dist-info/REQUESTED
-${PYSITELIB}/dask-2022.5.0.dist-info/WHEEL
-${PYSITELIB}/dask-2022.5.0.dist-info/direct_url.json
-${PYSITELIB}/dask-2022.5.0.dist-info/top_level.txt
+${PYSITELIB}/${WHEEL_INFODIR}/INSTALLER
+${PYSITELIB}/${WHEEL_INFODIR}/LICENSE.txt
+${PYSITELIB}/${WHEEL_INFODIR}/METADATA
+${PYSITELIB}/${WHEEL_INFODIR}/NUMPY_LICENSE.txt
+${PYSITELIB}/${WHEEL_INFODIR}/RECORD
+${PYSITELIB}/${WHEEL_INFODIR}/REQUESTED
+${PYSITELIB}/${WHEEL_INFODIR}/WHEEL
+${PYSITELIB}/${WHEEL_INFODIR}/direct_url.json
+${PYSITELIB}/${WHEEL_INFODIR}/top_level.txt
 ${PYSITELIB}/dask/__init__.py
 ${PYSITELIB}/dask/__init__.pyc
 ${PYSITELIB}/dask/_version.py
 ${PYSITELIB}/dask/_version.pyc
+${PYSITELIB}/dask/array/NUMPY_LICENSE.txt
 ${PYSITELIB}/dask/array/__init__.py
 ${PYSITELIB}/dask/array/__init__.pyc
 ${PYSITELIB}/dask/array/backends.py
@@ -195,8 +197,6 @@ ${PYSITELIB}/dask/bytes/tests/test_bytes_utils.py
 ${PYSITELIB}/dask/bytes/tests/test_bytes_utils.pyc
 ${PYSITELIB}/dask/bytes/tests/test_compression.py
 ${PYSITELIB}/dask/bytes/tests/test_compression.pyc
-${PYSITELIB}/dask/bytes/tests/test_hdfs.py
-${PYSITELIB}/dask/bytes/tests/test_hdfs.pyc
 ${PYSITELIB}/dask/bytes/tests/test_http.py
 ${PYSITELIB}/dask/bytes/tests/test_http.pyc
 ${PYSITELIB}/dask/bytes/tests/test_local.py
@@ -225,6 +225,8 @@ ${PYSITELIB}/dask/dataframe/_compat.py
 ${PYSITELIB}/dask/dataframe/_compat.pyc
 ${PYSITELIB}/dask/dataframe/_dtypes.py
 ${PYSITELIB}/dask/dataframe/_dtypes.pyc
+${PYSITELIB}/dask/dataframe/_pyarrow_compat.py
+${PYSITELIB}/dask/dataframe/_pyarrow_compat.pyc
 ${PYSITELIB}/dask/dataframe/accessor.py
 ${PYSITELIB}/dask/dataframe/accessor.pyc
 ${PYSITELIB}/dask/dataframe/backends.py
@@ -343,6 +345,8 @@ ${PYSITELIB}/dask/dataframe/tests/test_numeric.py
 ${PYSITELIB}/dask/dataframe/tests/test_numeric.pyc
 ${PYSITELIB}/dask/dataframe/tests/test_optimize_dataframe.py
 ${PYSITELIB}/dask/dataframe/tests/test_optimize_dataframe.pyc
+${PYSITELIB}/dask/dataframe/tests/test_pyarrow_compat.py
+${PYSITELIB}/dask/dataframe/tests/test_pyarrow_compat.pyc
 ${PYSITELIB}/dask/dataframe/tests/test_reshape.py
 ${PYSITELIB}/dask/dataframe/tests/test_reshape.pyc
 ${PYSITELIB}/dask/dataframe/tests/test_rolling.py
@@ -456,16 +460,22 @@ ${PYSITELIB}/dask/tests/test_rewrite.py
 ${PYSITELIB}/dask/tests/test_rewrite.pyc
 ${PYSITELIB}/dask/tests/test_sizeof.py
 ${PYSITELIB}/dask/tests/test_sizeof.pyc
+${PYSITELIB}/dask/tests/test_spark_compat.py
+${PYSITELIB}/dask/tests/test_spark_compat.pyc
 ${PYSITELIB}/dask/tests/test_system.py
 ${PYSITELIB}/dask/tests/test_system.pyc
 ${PYSITELIB}/dask/tests/test_threaded.py
 ${PYSITELIB}/dask/tests/test_threaded.pyc
+${PYSITELIB}/dask/tests/test_typing.py
+${PYSITELIB}/dask/tests/test_typing.pyc
 ${PYSITELIB}/dask/tests/test_utils.py
 ${PYSITELIB}/dask/tests/test_utils.pyc
 ${PYSITELIB}/dask/tests/test_utils_test.py
 ${PYSITELIB}/dask/tests/test_utils_test.pyc
 ${PYSITELIB}/dask/threaded.py
 ${PYSITELIB}/dask/threaded.pyc
+${PYSITELIB}/dask/typing.py
+${PYSITELIB}/dask/typing.pyc
 ${PYSITELIB}/dask/utils.py
 ${PYSITELIB}/dask/utils.pyc
 ${PYSITELIB}/dask/utils_test.py
diff --git a/py-dask/distinfo b/py-dask/distinfo
index 6c10819315..c8b6277a76 100644
--- a/py-dask/distinfo
+++ b/py-dask/distinfo
@@ -1,5 +1,5 @@
 $NetBSD$
 
-BLAKE2s (dask-2022.05.0.tar.gz) = 1e85168e81b9727daed8b85e05a394a8b4ba81ff2c100a9af46f8d2191731483
-SHA512 (dask-2022.05.0.tar.gz) = 15605321143f354ffb9c890823f5751131d942cc16c523b2d00bdcc752ddc5243525f7cda138091604c8f2f9c5b4a309b1efc95f483b753673df98cc8a81ce81
-Size (dask-2022.05.0.tar.gz) = 8658529 bytes
+BLAKE2s (dask-2022.6.1.tar.gz) = ec1bfaff04ff050935344127bf00818ebd9f8905b6cc5be4ee94004507f543a9
+SHA512 (dask-2022.6.1.tar.gz) = 6e06b63dd5c7f1d7c6c7dce867ffa42b956023b313b25245a339f698b06bea17a03cc4ead9bd5aef8ddf081750cad4df4b6b2972038315c89585221382c9e95c
+Size (dask-2022.6.1.tar.gz) = 8729249 bytes
diff --git a/py-dask/options.mk b/py-dask/options.mk
index 97a755aa17..859f4dbe6a 100644
--- a/py-dask/options.mk
+++ b/py-dask/options.mk
@@ -20,7 +20,7 @@ DEPENDS+=	${PYPKGPREFIX}-cytoolz-[0-9]*:../../devel/py-cytoolz
 .if !empty(PKG_OPTIONS:Mparquet)
 # TODO Also needs pyarrow
 DEPENDS+=	${PYPKGPREFIX}-fastparquet-[0-9]*:../../wip/py-fastparquet
-DEPENDS+=	${PYPKGPREFIX}-apache-arrow-[0-9]*:../../wip/py-apache-arrow
+DEPENDS+=	${PYPKGPREFIX}-pyarrow-[0-9]*:../../wip/py-apache-arrow
 .endif
 
 # graphviz is used to display the task dependencies
- Prev by Date:
py-threadpoolctl: Hook up make test correctly
 
- Next by Date:
py-distributed: Updated to 2022.6.2 2022.6.1 Released on June 24, 2022 Highlights This release includes the Worker State Machine refactor. The expectation should be that the worker state is its own synchronous subclass. Pulling all the state out into its own class allows us to write targeted unit tests without invoking any concurrent or asynchronous code. See GH#5736 for more information. Enhancements     Make worker state machine methods private (GH#6564) crusaderky     Yank state machine out of Worker class (GH#6566) crusaderky     Track worker_state_machine.TaskState instances (GH#6525) Hendrik Makait     Trivial tweaks to the Worker State Machine (GH#6586) crusaderky     Replace loop.call_later and loop.add_callback with background tasks added to Server. (GH#6603) Thomas Grainger     Support for neater WorkerState tests (GH#6609) crusaderky     Limit TCP writes with Tornado to 2GB (GH#6557) hhuuggoo     Enable no_implicit_optional for scheduler (GH#6622) Thomas Grainger Bug Fixes     Partial revert of compute-task message format (GH#6626) Florian Jetter     Restore log message about received signals in CLI (GH#6618) Florian Jetter     Handle empty memoryviews of bytearrays when (de)serializing (GH#6576) Benjamin Zaitlen     Ensure steal requests from same-IP but distinct workers are rejected (GH#6585) Florian Jetter     Fix tls_(min|max)_ version having no effect on openssl 1.1.0g or lower (GH#6562) Thomas Grainger     Fix idle_timeout and unxfail test (GH#6563) Matthew Rocklin     Fix crashing debug statement in _purge_state (GH#6589) crusaderky     Abort connections on CancelledError (GH#6574) Thomas Grainger     Fix Active Memory Manager ignores nbytes thresholds (GH#6583) crusaderky Deprecations     Deprecate WorkerState accessors (GH#6579) crusaderky Maintenance     Remove ipython hack (GH#6599) crusaderky     Mypy enforce --no-implicit-optional (GH#6606) crusaderky     Update versioneer: change from using SafeConfigParser to ConfigParser (GH#6605) Thomas A Caswell     Warn unreachable for scheduler.py (GH#6611) Florian Jetter     Refactor wait_for_state() (GH#6581) crusaderky     Hardcode wait_for_signals signal list (GH#6619) Thomas Grainger     Always pick an open port when running tests (GH#6591) Florian Jetter     Log popen stdout/err when subprocess times out (GH#6567) Gabe Joseph     Fix test_error_during_startup (GH#6608) Florian Jetter     Make test_idle_timeout_no_workers more robust (GH#6602) Florian Jetter     Mypy enforce --disallow-incomplete-defs (GH#6601) crusaderky     Do not log during signal handler (GH#6590) Florian Jetter     Don’t initialize mp_context on import (GH#6580) Lawrence Mitchell     Test retire workers deadlock (GH#6240) Gabe Joseph     Rework some tests related to gather_dep (GH#6472) crusaderky     Minor cosmetic review of scheduler_story and worker_story (GH#6442) crusaderky     Force __future__.annotations with isort (GH#6621) Thomas Grainger 2022.6.0 Released on June 10, 2022 Enhancements     Make disk access in system monitor configurable (GH#6537) Matthew Rocklin     Log and except errors on preload start (GH#6553) Matthew Rocklin     Fix Scheduler.restart logic (GH#6504) Gabe Joseph     Don’t heartbeat while Worker is closing (GH#6543) Gabe Joseph     No longer retry LocalCluster on errno.EADDRINUSE (GH#6369) Thomas Grainger     Don’t invoke log_event from state machine (GH#6512) crusaderky     Add config option to disable profiling and disable it in many tests per default (GH#6490) Hendrik Makait     Encapsulate Worker.batched_stream.send() (GH#6475) crusaderky Bug Fixes     refresh-who-has can break the worker state machine (GH#6529) crusaderky     Restart worker if it’s unrecognized by scheduler (GH#6505) Gabe Joseph     Fix import error when distributed.rmm.pool-size is set (GH#6482) KoyamaSohei Deprecations     Restore signature compatibility for dask-gateway (GH#6561) Tom Augspurger     Deprecate the io_loop and loop kwarg to Server, Worker, and Nanny (GH#6473) Thomas Grainger     Deprecate the loop kwarg to Scheduler (GH#6443) Thomas Grainger Documentation     Fix typo in .nthreads() docstring example (GH#6545) Pavithra Eswaramoorthy     Update docs theme for rebranding (GH#6495) Sarah Charlotte Johnson Maintenance     Refactor gather_dep (GH#6388) crusaderky     Fix flaky test_gather_dep_one_worker_always_busy (GH#6554) crusaderky     Remove missing-data message (GH#6546) crusaderky     Port test_local.LocalTest to pytest tests to allow use of loop fixture (GH#6523) Thomas Grainger     Fix test_quiet_client_close (GH#6541) Gabe Joseph     Use default_initializer in WorkerProcess (GH#6534) jakirkham     Avoid deadlocks in tests that use popen (GH#6483) Gabe Joseph     Revert “Fix CLI Scheduler Tests (GH#6502)” (GH#6547) Gabe Joseph     Update test report URL in summary message (GH#6532) Gabe Joseph     Update test report url (GH#6531) Ian Rose     Assert AsyncProcess.set_exit_callback is not called with a coroutine function (GH#6526) Thomas Grainger     Typing and docstring for Worker.close (GH#6518) Hendrik Makait     Fix CLI Scheduler Tests (GH#6502) Benjamin Zaitlen     Collect assertions in test_as_current_is_thread_local (GH#6520) Thomas Grainger     Link test report from test results comment (GH#6524) Hendrik Makait     Ignore the return value of signal.signal (GH#6519) Thomas Grainger     Refactor all event handlers (GH#6410) crusaderky     Fix dashboard favicon background (GH#6514) Jacob Tomlinson     Update dashboard logo (GH#6513) Jacob Tomlinson     Fix test_stress_scatter_death (GH#6404) Florian Jetter     Remove CrossFilter widget (GH#6484) crusaderky     data_needed exclusively contains tasks in fetch state (GH#6481) crusaderky     Assert possible previous states (GH#6488) Florian Jetter     @fail_hard can kill the whole test suite; hide errors (GH#6474) crusaderky     Assert that a fetch->cancelled->resumed->fetch cycle is impossible (GH#6460) crusaderky     Refactor busy workers reinsertion (GH#6379) crusaderky     Refactor find_missing and refresh_who_has (GH#6348) crusaderky     Rename test_collections.py to test_dask_collections.py (GH#6486) crusaderky     update_who_has can remove workers (GH#6342) crusaderky     Restructure test_watch_requires_lock_to_run to avoid flakes (GH#6469) Hendrik Makait     Fix intermittent test_profile_plot failure (GH#6456) Matthew Rocklin     Use asyncio.run to run gen_cluster, gen_test and cluster (GH#6231) Thomas Grainger     Improve tests that watch for subprocess logs (GH#6461) Gabe Joseph 2022.05.2 Released on May 26, 2022 Enhancements     Add a lock to distributed.profile for better concurrency control (GH#6421) Hendrik Makait     Send SIGKILL after SIGTERM when passing 95% memory (GH#6419) crusaderky Bug Fixes     Log rather than raise exceptions in preload.teardown() (GH#6458) Matthew Rocklin     Handle failing plugin.close() calls during scheduler shutdown (GH#6450) Matthew Rocklin     Fix slicing bug in ensure_memoryview (GH#6449) jakirkham     Generalize UCX errors on connect() and correct pytest fixtures (GH#6434) Peter Andreas Entschev     Run cluster widget periodic callbacks on the correct event loop (GH#6444) Thomas Grainger Maintenance     Disable pytest-asyncio if installed (GH#6436) Jacob Tomlinson     Close client in sync test_actor tests (GH#6459) Thomas Grainger     Ignore ServerSession.with_document_locked unawaited (GH#6447) Thomas Grainger     Remove coverage pin from Python 3.10 environment (GH#6439) Thomas Grainger     Annotate remove_worker (GH#6441) crusaderky     Update gpuCI RAPIDS_VER to 22.08 (GH#6428) 2022.05.1 Released on May 24, 2022 New Features     Add HTTP API to scheduler (GH#6270) Matthew Murray     Shuffle Service with Scheduler Logic (GH#6007) Matthew Rocklin Enhancements     Follow-up on removing report and safe from Worker.close (GH#6423) Gabe Joseph     Server close faster (GH#6415) Florian Jetter     Disable HTTP API by default (GH#6420) Jacob Tomlinson     Remove report and safe from Worker.close (GH#6363) Florian Jetter     Allow deserialized plugins in register_scheduler_plugin (GH#6401) Matthew Rocklin     WorkerState are different for different addresses (GH#6398) Florian Jetter     Do not filter tasks before gathering data (GH#6371) crusaderky     Remove worker reconnect (GH#6361) Gabe Joseph     Add SchedulerPlugin.log_event handler (GH#6381) Matthew Rocklin     Ensure occupancy tracking works as expected for long running tasks (GH#6351) Florian Jetter     stimulus_id for all Instructions (GH#6347) crusaderky     Refactor missing-data command (GH#6332) crusaderky     Add idempotent to register_scheduler_plugin client (GH#6328) Alex Ford     Add option to specify a scheduler address for workers to use (GH#5944) Enric Tejedor Bug Fixes     Remove stray breakpoint (GH#6417) Thomas Grainger     Fix API JSON MIME type (GH#6397) Jacob Tomlinson     Remove wrong assert in handle compute (GH#6370) Florian Jetter     Ensure multiple clients can cancel their key without interference (GH#6016) Florian Jetter     Fix Nanny shutdown assertion (GH#6357) Gabe Joseph     Fix fail_hard for sync functions (GH#6269) Gabe Joseph     Prevent infinite transition loops; more aggressive validate_state() (GH#6318) crusaderky     Ensure cleanup of many GBs of spilled data on terminate (GH#6280) crusaderky     Fix WORKER_ANY_RUNNING regression (GH#6297) Florian Jetter     Race conditions from fetch to compute while AMM requests replica (GH#6248) Florian Jetter     Ensure resumed tasks are not accidentally forgotten (GH#6217) Florian Jetter     Do not allow closing workers to be awaited again (GH#5910) Florian Jetter Deprecations     Move wait_for_signals to private module and deprecate distributed.cli.utils (GH#6367) Hendrik Makait Documentation     Fix typos and whitespace in worker.py (GH#6326) Hendrik Makait     Fix link to memory trimming documentation (GH#6317) Marco Wolsza Maintenance     Make gen_test show up in VSCode test discovery (GH#6424) Gabe Joseph     WSMR / deserialize_task (GH#6411) crusaderky     Restore signal handlers after wait for signals is done (GH#6400) Thomas Grainger     fail_hard should reraise (GH#6399) crusaderky     Revisit tests mocking gather_dep (GH#6385) crusaderky     Fix flaky test_in_flight_lost_after_resumed (GH#6372) Florian Jetter     Restore install_signal_handlers due to downstream dependencies (GH#6366) Hendrik Makait     Improve catch_unhandled_exceptions (GH#6358) Gabe Joseph     Remove all invocations of IOLoop.run_sync from CLI (GH#6205) Hendrik Makait     Remove transition-counter-max from config (GH#6349) crusaderky     Use list comprehension in pickle_loads (GH#6343) jakirkham     Improve ensure_memoryview test coverage & make minor fixes (GH#6333) jakirkham     Remove leaking reference to workers from gen_cluster (GH#6337) Hendrik Makait     Partial annotations for stealing.py (GH#6338) crusaderky     Validate and debug state machine on handle_compute_task (GH#6327) crusaderky     Bump pyupgrade and clean up # type: ignore (GH#6293) crusaderky     gen_cluster to write to /tmp (GH#6335) crusaderky     Transition table as a ClassVar (GH#6331) crusaderky     Simplify ensure_memoryview test with array (GH#6322) jakirkham     Refactor ensure_communicating (GH#6165) crusaderky     Review scheduler annotations, part 2 (GH#6253) crusaderky     Use w for writeable branch in pickle_loads (GH#6314) jakirkham     Simplify frame handling in ws (GH#6294) jakirkham     Use ensure_bytes from dask.utils (GH#6295) jakirkham     Use ensure_memoryview in array deserialization (GH#6300) jakirkham     Escape < > when generating Junit report (GH#6306) crusaderky     Use codecs.decode to deserialize errors (GH#6274) jakirkham     Minimize copying in maybe_compress & byte_sample (GH#6273) jakirkham     Skip test_release_evloop_while_spilling on OSX (GH#6291) Florian Jetter     Simplify logic in get_default_compression (GH#6260) jakirkham     Cleanup old compression workarounds (GH#6259) jakirkham     Re-enable NVML monitoring for WSL (GH#6119) Charles Blackmon-Luca
 
- Previous by Thread:
py-threadpoolctl: Hook up make test correctly
 
- Next by Thread:
py-distributed: Updated to 2022.6.2 2022.6.1 Released on June 24, 2022 Highlights This release includes the Worker State Machine refactor. The expectation should be that the worker state is its own synchronous subclass. Pulling all the state out into its own class allows us to write targeted unit tests without invoking any concurrent or asynchronous code. See GH#5736 for more information. Enhancements     Make worker state machine methods private (GH#6564) crusaderky     Yank state machine out of Worker class (GH#6566) crusaderky     Track worker_state_machine.TaskState instances (GH#6525) Hendrik Makait     Trivial tweaks to the Worker State Machine (GH#6586) crusaderky     Replace loop.call_later and loop.add_callback with background tasks added to Server. (GH#6603) Thomas Grainger     Support for neater WorkerState tests (GH#6609) crusaderky     Limit TCP writes with Tornado to 2GB (GH#6557) hhuuggoo     Enable no_implicit_optional for scheduler (GH#6622) Thomas Grainger Bug Fixes     Partial revert of compute-task message format (GH#6626) Florian Jetter     Restore log message about received signals in CLI (GH#6618) Florian Jetter     Handle empty memoryviews of bytearrays when (de)serializing (GH#6576) Benjamin Zaitlen     Ensure steal requests from same-IP but distinct workers are rejected (GH#6585) Florian Jetter     Fix tls_(min|max)_ version having no effect on openssl 1.1.0g or lower (GH#6562) Thomas Grainger     Fix idle_timeout and unxfail test (GH#6563) Matthew Rocklin     Fix crashing debug statement in _purge_state (GH#6589) crusaderky     Abort connections on CancelledError (GH#6574) Thomas Grainger     Fix Active Memory Manager ignores nbytes thresholds (GH#6583) crusaderky Deprecations     Deprecate WorkerState accessors (GH#6579) crusaderky Maintenance     Remove ipython hack (GH#6599) crusaderky     Mypy enforce --no-implicit-optional (GH#6606) crusaderky     Update versioneer: change from using SafeConfigParser to ConfigParser (GH#6605) Thomas A Caswell     Warn unreachable for scheduler.py (GH#6611) Florian Jetter     Refactor wait_for_state() (GH#6581) crusaderky     Hardcode wait_for_signals signal list (GH#6619) Thomas Grainger     Always pick an open port when running tests (GH#6591) Florian Jetter     Log popen stdout/err when subprocess times out (GH#6567) Gabe Joseph     Fix test_error_during_startup (GH#6608) Florian Jetter     Make test_idle_timeout_no_workers more robust (GH#6602) Florian Jetter     Mypy enforce --disallow-incomplete-defs (GH#6601) crusaderky     Do not log during signal handler (GH#6590) Florian Jetter     Don’t initialize mp_context on import (GH#6580) Lawrence Mitchell     Test retire workers deadlock (GH#6240) Gabe Joseph     Rework some tests related to gather_dep (GH#6472) crusaderky     Minor cosmetic review of scheduler_story and worker_story (GH#6442) crusaderky     Force __future__.annotations with isort (GH#6621) Thomas Grainger 2022.6.0 Released on June 10, 2022 Enhancements     Make disk access in system monitor configurable (GH#6537) Matthew Rocklin     Log and except errors on preload start (GH#6553) Matthew Rocklin     Fix Scheduler.restart logic (GH#6504) Gabe Joseph     Don’t heartbeat while Worker is closing (GH#6543) Gabe Joseph     No longer retry LocalCluster on errno.EADDRINUSE (GH#6369) Thomas Grainger     Don’t invoke log_event from state machine (GH#6512) crusaderky     Add config option to disable profiling and disable it in many tests per default (GH#6490) Hendrik Makait     Encapsulate Worker.batched_stream.send() (GH#6475) crusaderky Bug Fixes     refresh-who-has can break the worker state machine (GH#6529) crusaderky     Restart worker if it’s unrecognized by scheduler (GH#6505) Gabe Joseph     Fix import error when distributed.rmm.pool-size is set (GH#6482) KoyamaSohei Deprecations     Restore signature compatibility for dask-gateway (GH#6561) Tom Augspurger     Deprecate the io_loop and loop kwarg to Server, Worker, and Nanny (GH#6473) Thomas Grainger     Deprecate the loop kwarg to Scheduler (GH#6443) Thomas Grainger Documentation     Fix typo in .nthreads() docstring example (GH#6545) Pavithra Eswaramoorthy     Update docs theme for rebranding (GH#6495) Sarah Charlotte Johnson Maintenance     Refactor gather_dep (GH#6388) crusaderky     Fix flaky test_gather_dep_one_worker_always_busy (GH#6554) crusaderky     Remove missing-data message (GH#6546) crusaderky     Port test_local.LocalTest to pytest tests to allow use of loop fixture (GH#6523) Thomas Grainger     Fix test_quiet_client_close (GH#6541) Gabe Joseph     Use default_initializer in WorkerProcess (GH#6534) jakirkham     Avoid deadlocks in tests that use popen (GH#6483) Gabe Joseph     Revert “Fix CLI Scheduler Tests (GH#6502)” (GH#6547) Gabe Joseph     Update test report URL in summary message (GH#6532) Gabe Joseph     Update test report url (GH#6531) Ian Rose     Assert AsyncProcess.set_exit_callback is not called with a coroutine function (GH#6526) Thomas Grainger     Typing and docstring for Worker.close (GH#6518) Hendrik Makait     Fix CLI Scheduler Tests (GH#6502) Benjamin Zaitlen     Collect assertions in test_as_current_is_thread_local (GH#6520) Thomas Grainger     Link test report from test results comment (GH#6524) Hendrik Makait     Ignore the return value of signal.signal (GH#6519) Thomas Grainger     Refactor all event handlers (GH#6410) crusaderky     Fix dashboard favicon background (GH#6514) Jacob Tomlinson     Update dashboard logo (GH#6513) Jacob Tomlinson     Fix test_stress_scatter_death (GH#6404) Florian Jetter     Remove CrossFilter widget (GH#6484) crusaderky     data_needed exclusively contains tasks in fetch state (GH#6481) crusaderky     Assert possible previous states (GH#6488) Florian Jetter     @fail_hard can kill the whole test suite; hide errors (GH#6474) crusaderky     Assert that a fetch->cancelled->resumed->fetch cycle is impossible (GH#6460) crusaderky     Refactor busy workers reinsertion (GH#6379) crusaderky     Refactor find_missing and refresh_who_has (GH#6348) crusaderky     Rename test_collections.py to test_dask_collections.py (GH#6486) crusaderky     update_who_has can remove workers (GH#6342) crusaderky     Restructure test_watch_requires_lock_to_run to avoid flakes (GH#6469) Hendrik Makait     Fix intermittent test_profile_plot failure (GH#6456) Matthew Rocklin     Use asyncio.run to run gen_cluster, gen_test and cluster (GH#6231) Thomas Grainger     Improve tests that watch for subprocess logs (GH#6461) Gabe Joseph 2022.05.2 Released on May 26, 2022 Enhancements     Add a lock to distributed.profile for better concurrency control (GH#6421) Hendrik Makait     Send SIGKILL after SIGTERM when passing 95% memory (GH#6419) crusaderky Bug Fixes     Log rather than raise exceptions in preload.teardown() (GH#6458) Matthew Rocklin     Handle failing plugin.close() calls during scheduler shutdown (GH#6450) Matthew Rocklin     Fix slicing bug in ensure_memoryview (GH#6449) jakirkham     Generalize UCX errors on connect() and correct pytest fixtures (GH#6434) Peter Andreas Entschev     Run cluster widget periodic callbacks on the correct event loop (GH#6444) Thomas Grainger Maintenance     Disable pytest-asyncio if installed (GH#6436) Jacob Tomlinson     Close client in sync test_actor tests (GH#6459) Thomas Grainger     Ignore ServerSession.with_document_locked unawaited (GH#6447) Thomas Grainger     Remove coverage pin from Python 3.10 environment (GH#6439) Thomas Grainger     Annotate remove_worker (GH#6441) crusaderky     Update gpuCI RAPIDS_VER to 22.08 (GH#6428) 2022.05.1 Released on May 24, 2022 New Features     Add HTTP API to scheduler (GH#6270) Matthew Murray     Shuffle Service with Scheduler Logic (GH#6007) Matthew Rocklin Enhancements     Follow-up on removing report and safe from Worker.close (GH#6423) Gabe Joseph     Server close faster (GH#6415) Florian Jetter     Disable HTTP API by default (GH#6420) Jacob Tomlinson     Remove report and safe from Worker.close (GH#6363) Florian Jetter     Allow deserialized plugins in register_scheduler_plugin (GH#6401) Matthew Rocklin     WorkerState are different for different addresses (GH#6398) Florian Jetter     Do not filter tasks before gathering data (GH#6371) crusaderky     Remove worker reconnect (GH#6361) Gabe Joseph     Add SchedulerPlugin.log_event handler (GH#6381) Matthew Rocklin     Ensure occupancy tracking works as expected for long running tasks (GH#6351) Florian Jetter     stimulus_id for all Instructions (GH#6347) crusaderky     Refactor missing-data command (GH#6332) crusaderky     Add idempotent to register_scheduler_plugin client (GH#6328) Alex Ford     Add option to specify a scheduler address for workers to use (GH#5944) Enric Tejedor Bug Fixes     Remove stray breakpoint (GH#6417) Thomas Grainger     Fix API JSON MIME type (GH#6397) Jacob Tomlinson     Remove wrong assert in handle compute (GH#6370) Florian Jetter     Ensure multiple clients can cancel their key without interference (GH#6016) Florian Jetter     Fix Nanny shutdown assertion (GH#6357) Gabe Joseph     Fix fail_hard for sync functions (GH#6269) Gabe Joseph     Prevent infinite transition loops; more aggressive validate_state() (GH#6318) crusaderky     Ensure cleanup of many GBs of spilled data on terminate (GH#6280) crusaderky     Fix WORKER_ANY_RUNNING regression (GH#6297) Florian Jetter     Race conditions from fetch to compute while AMM requests replica (GH#6248) Florian Jetter     Ensure resumed tasks are not accidentally forgotten (GH#6217) Florian Jetter     Do not allow closing workers to be awaited again (GH#5910) Florian Jetter Deprecations     Move wait_for_signals to private module and deprecate distributed.cli.utils (GH#6367) Hendrik Makait Documentation     Fix typos and whitespace in worker.py (GH#6326) Hendrik Makait     Fix link to memory trimming documentation (GH#6317) Marco Wolsza Maintenance     Make gen_test show up in VSCode test discovery (GH#6424) Gabe Joseph     WSMR / deserialize_task (GH#6411) crusaderky     Restore signal handlers after wait for signals is done (GH#6400) Thomas Grainger     fail_hard should reraise (GH#6399) crusaderky     Revisit tests mocking gather_dep (GH#6385) crusaderky     Fix flaky test_in_flight_lost_after_resumed (GH#6372) Florian Jetter     Restore install_signal_handlers due to downstream dependencies (GH#6366) Hendrik Makait     Improve catch_unhandled_exceptions (GH#6358) Gabe Joseph     Remove all invocations of IOLoop.run_sync from CLI (GH#6205) Hendrik Makait     Remove transition-counter-max from config (GH#6349) crusaderky     Use list comprehension in pickle_loads (GH#6343) jakirkham     Improve ensure_memoryview test coverage & make minor fixes (GH#6333) jakirkham     Remove leaking reference to workers from gen_cluster (GH#6337) Hendrik Makait     Partial annotations for stealing.py (GH#6338) crusaderky     Validate and debug state machine on handle_compute_task (GH#6327) crusaderky     Bump pyupgrade and clean up # type: ignore (GH#6293) crusaderky     gen_cluster to write to /tmp (GH#6335) crusaderky     Transition table as a ClassVar (GH#6331) crusaderky     Simplify ensure_memoryview test with array (GH#6322) jakirkham     Refactor ensure_communicating (GH#6165) crusaderky     Review scheduler annotations, part 2 (GH#6253) crusaderky     Use w for writeable branch in pickle_loads (GH#6314) jakirkham     Simplify frame handling in ws (GH#6294) jakirkham     Use ensure_bytes from dask.utils (GH#6295) jakirkham     Use ensure_memoryview in array deserialization (GH#6300) jakirkham     Escape < > when generating Junit report (GH#6306) crusaderky     Use codecs.decode to deserialize errors (GH#6274) jakirkham     Minimize copying in maybe_compress & byte_sample (GH#6273) jakirkham     Skip test_release_evloop_while_spilling on OSX (GH#6291) Florian Jetter     Simplify logic in get_default_compression (GH#6260) jakirkham     Cleanup old compression workarounds (GH#6259) jakirkham     Re-enable NVML monitoring for WSL (GH#6119) Charles Blackmon-Luca
 
- Indexes:
 
Home |
Main Index |
Thread Index |
Old Index