pkgsrc-WIP-changes archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
py-distributed: Updated to 2022.8.1
Module Name: pkgsrc-wip
Committed By: Matthew Danielson <matthewd%fastmail.us@localhost>
Pushed By: matthewd
Date: Fri Aug 26 06:05:19 2022 -0700
Changeset: d9677099fbe5a788707a5be5e6b65f9b0ae6f15d
Modified Files:
py-distributed/Makefile
py-distributed/PLIST
py-distributed/distinfo
Added Files:
py-distributed/TODO
Log Message:
py-distributed: Updated to 2022.8.1
2022.8.1
Released on August 19, 2022
New Features
Enhancements
Drop comparison of versions against all clients (GH#6861) Hendrik Makait
Log the worker name if set (GH#6866) Johannes Lange
Skip getaddrinfo thread if host is already resolved, using socket.AI_NUMERIC* (GH#6847) Thomas Grainger
Display unexpected state in Worker.execute validation (GH#6856) James Bourbeau
pre-spawn-environ (GH#6841) crusaderky
Dump has_what, missing_dep_flight (GH#6830) Gabe Joseph
Bug Fixes
cancelled / resumed -> rescheduled transition (GH#6913) crusaderky
Fix resource deallocation for resumed tasks (GH#6914) crusaderky
Only close scheduler in SpecCluster if it exists (GH#6888) Matthew Rocklin
Fix issue if exc.reason is None (GH#6881) Hendrik Makait
Always close BatchedSend write coroutines (GH#6865) Gabe Joseph
Harden preamble of Worker.execute against race conditions (GH#6878) crusaderky
Documentation
Fix typo (GH#6870) Pieter Gijsbers
Maintenance
Use retries for the test report (GH#6926) Florian Jetter
Duplicated code: in_flight_tasks validation (GH#6917) crusaderky
ipywidgets 8 compatibility (GH#6912) James Bourbeau
Overhaul transitions for the resumed state (GH#6699) crusaderky
Don’t upgrade to ipywidgets 8 (GH#6910) crusaderky
Clean up cluster process reaping (GH#6840) Gabe Joseph
Don’t use bokeh Figure in tests (GH#6721) Bryan Van de Ven
Work around incompatibility of crick with setuptools 65 (GH#6887) crusaderky
Add max version constraint for dask-core in nightlies (GH#6862) Charles Blackmon-Luca
Replace port = random.randint( with port = d.utils.open_port (GH#6883) Thomas Grainger
Fix flaky test_wall_clock (GH#6879) crusaderky
Add descriptive error message to assert (GH#6871) Hendrik Makait
Increase timeout in test_quiet_process (GH#6857) Florian Jetter
Descriptive title for test report (GH#6849) Hendrik Makait
Add flake8-bugbear as plugin to pre-commit (GH#6809) Hendrik Makait
Remove redundant use of with clean(): (GH#6852) Thomas Grainger
Show actual Job URL on test report (GH#6837) Florian Jetter
Update pre-commit dependencies (GH#6851) Hendrik Makait
Call exit callback even if AsyncProcess is reaped elsewhere (GH#6684) Thomas Grainger
Avoid function calls in argument defaults (GH#6812) Hendrik Makait
Ignore warning for unclose SSHCluster in tests (GH#6827) Florian Jetter
2022.8.0
Released on August 5, 2022
New Features
Add Jupyter Server to Dask Scheduler (GH#6737) Matthew Rocklin
Enhancements
Human-readable formatting for disk I/O and renaming to diff net and disk (GH#6835) Hendrik Makait
Add Cluster.get_client() method (GH#6745) Julia Signell
Start bokeh app to activate bokeh’s clean session callbacks (GH#6728) Martí Zamora
Bug Fixes
Ensure Nanny doesn’t restart workers that fail to start, and joins subprocess (GH#6427) Gabe Joseph
Don’t connect to cluster subprocesses at shutdown (GH#6829) Gabe Joseph
Fix restart wait for workers edge case (GH#6823) Gabe Joseph
Fix spilled size calculation in Slow (GH#6789) Hendrik Makait
Deprecations
Deprecate passing stopped loops to LoopRunner (and therefore Client/Cluster) (GH#6680) Thomas Grainger
Documentation
Add text to top of API docs to make sure that users are exposed to LocalCluster early (GH#6793) Julia Signell
Change title for plugins documentation (GH#6733) Sarah Charlotte Johnson
Maintenance
Only set 5s connect timeout in gen_cluster tests (GH#6822) Gabe Joseph
Fix flaky test_worker_who_has_clears_after_failed_connection (GH#6832) Gabe Joseph
Add missing skips for pyarrow (GH#6787) Elliott Sales de Andrade
Miscellaneous flake8-bugbear issues (GH#6814) Hendrik Makait
Assert otherwise pointless comparisons (B015) (GH#6811) Hendrik Makait
Remove unused functions from utils_test.py (GH#6807) Hendrik Makait
Fix Jupyter security note (GH#6818) Jacob Tomlinson
Improve check_thread_leak output (GH#6797) Gabe Joseph
Use contextmanager to ensure clients are closed and do not leak (GH#6817) Hendrik Makait
Robust thread termination in test_watch and test_watch_requires_lock_to_run (GH#6788) Hendrik Makait
Avoid unused loop control variable or name them _ (GH#6813) Hendrik Makait
Avoid mutable argument defaults in tests (GH#6810) Hendrik Makait
Avoid mutable argument defaults outside of tests (GH#6665) Hendrik Makait
Update gpuCI RAPIDS_VER to 22.10 (GH#6798)
Use same Python for dask worker tests (GH#6786) Elliott Sales de Andrade
2022.7.1
Released on July 22, 2022
New Features
Dashboard for failed tasks (GH#6595) Ian Rose
Enhancements
Wait for workers to return in Client.restart (GH#6714) Gabe Joseph
Remove global mutable Cluster._cluster_info (GH#6487) Thomas Grainger
Bug Fixes
Fix: nvml no early init (GH#6678) Lawrence Mitchell
Fix bug when restarting client (GH#6654) Iain Dorrington
Failure to spill breaks available resources (GH#6703) crusaderky
Fix resource allocation for tasks with dependencies (GH#6676) Hendrik Makait
Maintenance
Revert “Set MALLOC_TRIM_THRESHOLD_ before interpreter start” (GH#6777) Gabe Joseph
Fix mypy lint in CI (GH#6779) jakirkham
Remove test_restart_fast_sync, test_fast_kill (GH#6750) Gabe Joseph
Fix flaky test_async_task_group_call_later_executes_delayed_task_in_background (GH#6744) Hendrik Makait
Drop redundant geninc (GH#6740) Hendrik Makait
Remove unused worker_coroutines (GH#6739) Gabe Joseph
Store ready and constrained tasks in heapsets (GH#6711) crusaderky
Improve tests for cancelled state (GH#6717) crusaderky
Future-proof Bokeh value import (GH#6707) Bryan Van de Ven
Revert temporary stress test (GH#6712) crusaderky
Validate constrained tasks (GH#6698) crusaderky
Minor quality-of-life tweaks to cancelled state (GH#6701) crusaderky
Pickle worker state machine exceptions (GH#6702) crusaderky
Partial matches for worker state machine instructions (GH#6704) crusaderky
Automatically mark all WorkerState tests (GH#6706) crusaderky
2022.7.0
Released on July 8, 2022
Enhancements
Use a tempdir path by default instead of cwd for the worker scratch dir (GH#6658) Florian Jetter
Add WorkerState.all_running_tasks (GH#6690) Hendrik Makait
Scheduler.reschedule() works only by accident (GH#6339) crusaderky
Remove spurious UnpauseEvent at worker start (GH#6652) crusaderky
Log if closing an executor is not possible in thread (GH#6644) Florian Jetter
Cloudpickle register by value (GH#6466) Ian Rose
Adding replicas to a task in fetch now sends it to flight immediately (GH#6594) crusaderky
Bug Fixes
Fix dump output of parameter-less events (GH#6695) crusaderky
Set MALLOC_TRIM_THRESHOLD_ before interpreter start (GH#6681) crusaderky
Fix deadlocks around rescheduled and resumed states (GH#6673) crusaderky
has_arg returns True for keyword-only arguments (GH#6648) Lawrence Mitchell
Transitions caused by worker death use old ‘worker-connect’ stimulus_id (GH#6657) crusaderky
A key is forgotten while acquire-replicas is running (GH#6638) crusaderky
Maintenance
Revisit WorkerState.long_running set (GH#6697) crusaderky
WorkerState unit tests for resumed state (GH#6688) crusaderky
Bump version of pandas-stubs (GH#6691) crusaderky
Add dummy factory methods for ExecuteSuccessEvent and ExecuteFailureEvent (GH#6687) Hendrik Makait
Pin tornado<6.2 in nightly conda recipes (GH#6675) Peter Andreas Entschev
Refactor resource restriction handling in WorkerState (GH#6672) Hendrik Makait
test_signal and test_terminate occasionally return SIGKILL on MacOS (GH#6671) crusaderky
Use the loop fixture in even more tests (GH#6674) Thomas Grainger
Inconditionally import ssl (GH#6670) Thomas Grainger
Use the loop fixture in more tests (GH#6642) Thomas Grainger
Pin tornado to <6.2 (GH#6668) Florian Jetter
Handle AsyncTaskGroupClosedError (GH#6664) Hendrik Makait
Replace occurences of large delay slowinc with locks (GH#6656) Florian Jetter
Merge extend-ignore and ignore values for flake8 (GH#6660) Hendrik Makait
Remove server close background task grace period (GH#6633) Thomas Grainger
Do not use tempfile in utils_test (GH#6651) Florian Jetter
close_worker cleanup (GH#6650) crusaderky
Rewrite test_cancelled_resumed_after_flight_with_dependencies using WorkerState (GH#6645) crusaderky
Log the actual duration to create a directory (GH#6647) Florian Jetter
pandas type stubs (GH#6635) crusaderky
Remove unused __started Event in Server (GH#6615) Florian Jetter
Use safe temp directory in gen_cluster (GH#6628) Florian Jetter
Print CI host info (GH#6629) crusaderky
Deduplicate data_needed (GH#6587) crusaderky
Remove EnsureCommunicatingAfterTransitions (GH#6462) crusaderky
Pickle WorkerState (GH#6623) crusaderky
Harden vs. TaskState collisions (GH#6593) crusaderky
Do not interact with the event loop when the cluster is garbage collected (GH#6627) Thomas Grainger
To see a diff of this commit:
https://wip.pkgsrc.org/cgi-bin/gitweb.cgi?p=pkgsrc-wip.git;a=commitdiff;h=d9677099fbe5a788707a5be5e6b65f9b0ae6f15d
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
diffstat:
py-distributed/Makefile | 17 ++++++++++++++---
py-distributed/PLIST | 4 ++++
py-distributed/TODO | 2 ++
py-distributed/distinfo | 6 +++---
4 files changed, 23 insertions(+), 6 deletions(-)
diffs:
diff --git a/py-distributed/Makefile b/py-distributed/Makefile
index 3453ee1459..1196f4be0b 100644
--- a/py-distributed/Makefile
+++ b/py-distributed/Makefile
@@ -1,9 +1,10 @@
# $NetBSD$
-DISTNAME= distributed-2022.6.1
+DISTNAME= distributed-2022.8.1
PKGNAME= ${PYPKGPREFIX}-${DISTNAME}
CATEGORIES= devel net
-MASTER_SITES= ${MASTER_SITE_PYPI:=d/distributed/}
+GITHUB_PROJECT= distributed
+MASTER_SITES= ${MASTER_SITE_GITHUB:=dask/}
MAINTAINER= matthewd%fastmail.us@localhost
@@ -11,8 +12,15 @@ HOMEPAGE= https://github.com/dask/distributed/
COMMENT= Distributed is the parallel scheduler for dask
LICENSE= modified-bsd
+TEST_DEPENDS+= ${PYPKGPREFIX}-lz4>=3.1.10:../../archivers/py-lz4
+TEST_DEPENDS+= ${PYPKGPREFIX}-zstandard>=0.18.0:../../archivers/py-zstandard
+TEST_DEPENDS+= ${PYPKGPREFIX}-requests>=2.28.1:../../devel/py-requests
TEST_DEPENDS+= ${PYPKGPREFIX}-test-[0-9]*:../../devel/py-test
-TEST_DEPENDS+= ${PYPKGPREFIX}-dask-[0-9]*:../../wip/py-dask
+TEST_DEPENDS+= ${PYPKGPREFIX}-test-timeout-[0-9]*:../../devel/py-test-timeout
+# TEST_DEPENDS+= ${PYPKGPREFIX}-paramiko>=2.11.0:../../security/py-paramiko
+TEST_DEPENDS+= ${PYPKGPREFIX}-dask-2022.8.1:../../wip/py-dask
+TEST_DEPENDS+= ${PYPKGPREFIX}-ipywidgets>=7.7.0:../../www/py-ipywidgets
+DEPENDS+= ${PYPKGPREFIX}-click>=8.1.3:../../devel/py-click
DEPENDS+= ${PYPKGPREFIX}-sortedcontainers>=2.0.1:../../devel/py-sortedcontainers
DEPENDS+= ${PYPKGPREFIX}-cloudpickle>=1.5.0:../../wip/py-cloudpickle
DEPENDS+= ${PYPKGPREFIX}-jinja2-[0-9]*:../../textproc/py-jinja2
@@ -24,5 +32,8 @@ DEPENDS+= ${PYPKGPREFIX}-tblib>=1.7.0:../../wip/py-tblib
USE_LANGUAGES= # none
+do-test:
+ cd ${WRKSRC} && ${SETENV} ${TEST_ENV} ${PYTHONBIN} -m pytest -s
+
.include "../../lang/python/egg.mk"
.include "../../mk/bsd.pkg.mk"
diff --git a/py-distributed/PLIST b/py-distributed/PLIST
index 88123b8007..b042394c42 100644
--- a/py-distributed/PLIST
+++ b/py-distributed/PLIST
@@ -276,6 +276,7 @@ ${PYSITELIB}/distributed/http/statics.pyc
${PYSITELIB}/distributed/http/statics.pyo
${PYSITELIB}/distributed/http/templates/base.html
${PYSITELIB}/distributed/http/templates/call-stack.html
+${PYSITELIB}/distributed/http/templates/exceptions.html
${PYSITELIB}/distributed/http/templates/gpu.html
${PYSITELIB}/distributed/http/templates/json-index.html
${PYSITELIB}/distributed/http/templates/logs.html
@@ -389,6 +390,9 @@ ${PYSITELIB}/distributed/pubsub.py
${PYSITELIB}/distributed/pubsub.pyc
${PYSITELIB}/distributed/pubsub.pyo
${PYSITELIB}/distributed/py.typed
+${PYSITELIB}/distributed/pytest_resourceleaks.py
+${PYSITELIB}/distributed/pytest_resourceleaks.pyc
+${PYSITELIB}/distributed/pytest_resourceleaks.pyo
${PYSITELIB}/distributed/queues.py
${PYSITELIB}/distributed/queues.pyc
${PYSITELIB}/distributed/queues.pyo
diff --git a/py-distributed/TODO b/py-distributed/TODO
new file mode 100644
index 0000000000..99ed20b781
--- /dev/null
+++ b/py-distributed/TODO
@@ -0,0 +1,2 @@
+* Add more TEST_DEPENDS (like paramako)
+* Add --runslow to test args
diff --git a/py-distributed/distinfo b/py-distributed/distinfo
index fac0ca7f23..fc7daf2a20 100644
--- a/py-distributed/distinfo
+++ b/py-distributed/distinfo
@@ -1,5 +1,5 @@
$NetBSD$
-BLAKE2s (distributed-2022.6.1.tar.gz) = cc1c32d90fb17186076beb9e33f6002f167559b452255a287cbab0bb1d2b7c50
-SHA512 (distributed-2022.6.1.tar.gz) = 84e3c8136caccde71639f64405663a7a44632fbd76a63ff0372f192b4b23cbe3e21b3ab7fa87e17918eb6542b96c24c047dc8849f74f16c2da805cd8e3aeb233
-Size (distributed-2022.6.1.tar.gz) = 959007 bytes
+BLAKE2s (distributed-2022.8.1.tar.gz) = a8aa2037f69bb7612c2606af4db54f05c54528e319be1e70c0aa948c2f7b1b64
+SHA512 (distributed-2022.8.1.tar.gz) = f92d6b260fe09b2817284ae2584820c7dd3d106240e44dbdb3493f0fcd0f5a07418798942cd37c1d75deb18454ae05f82c60ec682f755ef5e04f225322556c2a
+Size (distributed-2022.8.1.tar.gz) = 1712217 bytes
Home |
Main Index |
Thread Index |
Old Index