pkgsrc-WIP-changes archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
py-dask: Update to 2022.6.1
Module Name: pkgsrc-wip
Committed By: Matthew Danielson <matthewd%fastmail.us@localhost>
Pushed By: matthewd
Date: Sun Jul 3 19:18:21 2022 -0700
Changeset: f73bb1f475c18a70d84cd5c8e2fb03a9b9997ba9
Modified Files:
py-dask/Makefile
py-dask/PLIST
py-dask/distinfo
py-dask/options.mk
Log Message:
py-dask: Update to 2022.6.1
2022.6.1
Released on June 24, 2022
Enhancements
Dask in pyodide (GH#9053) Ian Rose
Create dask.utils.show_versions (GH#9144) Sultan Orazbayev
Better error message for unsupported numpy operations on dask.dataframe objects. (GH#9201) Julia Signell
Add allow_rechunk kwarg to dask.array.overlap function (GH#7776) Genevieve Buckley
Add minutes and hours to dask.utils.format_time (GH#9116) Matthew Rocklin
More retries when writing parquet to remote filesystem (GH#9175) Ian Rose
Bug Fixes
Timedelta deterministic hashing (GH#9213) Fabien Aulaire
Enum deterministic hashing (GH#9212) Fabien Aulaire
shuffle_group(): avoid converting to arrays (GH#9157) Mads R. B. Kristensen
Deprecations
Deprecate extra format_time utility (GH#9184) James Bourbeau
Documentation
Better SEO for 10 Minutes to Dask (GH#9182) Sarah Charlotte Johnson
Better SEO for Delayed and Best Practices (GH#9194) Sarah Charlotte Johnson
Include known inconsistency in DataFrame str.split accessor docstring (GH#9177) Richard Pelgrim
Add inconsistencies keyword to derived_from (GH#9192) Richard Pelgrim
Add missing append in delayed best practices example (GH#9202) Ben
Fix indentation in Best Practices (GH#9196) Sarah Charlotte Johnson
Add link to Genevieve Buckley’s blog on chunk sizes (GH#9199) Pavithra Eswaramoorthy
Update to_csv docstring (GH#9094) Sarah Charlotte Johnson
Maintenance
Update versioneer: change from using SafeConfigParser to ConfigParser (GH#9205) Thomas A Caswell
Remove ipython hack in CI(GH#9200) crusaderky
2022.6.0
Released on June 10, 2022
Enhancements
Add feature to show names of layer dependencies in HLG JupyterLab repr (GH#9081) Angelos Omirolis
Add arrow schema extraction dispatch (GH#9169) GALI PREM SAGAR
Add sort_results argument to assert_eq (GH#9130) Pavithra Eswaramoorthy
Add weeks to parse_timedelta (GH#9168) Matthew Rocklin
Warn that cloudpickle is not always deterministic (GH#9148) Pavithra Eswaramoorthy
Switch parquet default engine (GH#9140) Jim Crist-Harif
Use deterministic hashing with _iLocIndexer / _LocIndexer (GH#9108) Fabien Aulaire
Enfore consistent schema in to_parquet pyarrow (GH#9131) Jim Crist-Harif
Bug Fixes
Fix pyarrow.StringArray pickle (GH#9170) Jim Crist-Harif
Fix parallel metadata collection in pyarrow engine (GH#9165) Richard (Rick) Zamora
Improve pyarrow partitioning logic (GH#9147) James Bourbeau
pyarrow 8.0 partitioning fix (GH#9143) James Bourbeau
Documentation
Better SEO for Installing Dask and Dask DataFrame Best Practices (GH#9178) Sarah Charlotte Johnson
Update logos page in docs (GH#9167) Sarah Charlotte Johnson
Add example using pandas Series to map_partition doctring (GH#9161) Alex-JG3
Update docs theme for rebranding (GH#9160) Sarah Charlotte Johnson
Better SEO for docs on Dask DataFrames (GH#9128) Sarah Charlotte Johnson
Maintenance
Remove ensure_file from recommended practice for downstream libraries (GH#9171) Matthew Rocklin
Test round-tripping DataFrame parquet I/O including pyspark (GH#9156) Ian Rose
Try disabling HDF5 locking (GH#9154) Ian Rose
Link best practices to DataFrame-parquet (GH#9150) Tom Augspurger
Fix typo in map_partitions func parameter description (GH#9149) Christopher Akiki
Un-xfail test_groupby_grouper_dispatch (GH#9139) GALI PREM SAGAR
Temporarily import cleanup fixture from distributed (GH#9138) James Bourbeau
Simplify partitioning logic in pyarrow parquet engine (GH#9041) Richard (Rick) Zamora
2022.05.2
Released on May 26, 2022
Enhancements
Add a dispatch for non-pandas Grouper objects and use it in GroupBy (GH#9074) brandon-b-miller
Error if read_parquet & to_parquet files intersect (GH#9124) Jim Crist-Harif
Visualize task graphs using ipycytoscape (GH#9091) Ian Rose
Documentation
Fix various typos (GH#9126) Ryan Russell
Maintenance
Fix flaky test_filter_nonpartition_columns (GH#9127) Pavithra Eswaramoorthy
Update gpuCI RAPIDS_VER to 22.08 (GH#9120)
Include conftest.py` in sdists (GH#9115) Ben Beasley
2022.05.1
Released on May 24, 2022
New Features
Add DataFrame.from_dict classmethod (GH#9017) Matthew Powers
Add from_map function to Dask DataFrame (GH#8911) Richard (Rick) Zamora
Enhancements
Improve to_parquet error for appended divisions overlap (GH#9102) Jim Crist-Harif
Enabled user-defined process-initializer functions (GH#9087) ParticularMiner
Mention align_dataframes=False option in map_partitions error (GH#9075) Gabe Joseph
Add kwarg enforce_ndim to dask.array.map_blocks() (GH#8865) ParticularMiner
Implement Series.GroupBy.fillna / DataFrame.GroupBy.fillna methods (GH#8869) Pavithra Eswaramoorthy
Allow fillna with Dask DataFrame (GH#8950) Pavithra Eswaramoorthy
Update error message for assignment with 1-d dask array (GH#9036) Pavithra Eswaramoorthy
Collection Protocol (GH#8674) Doug Davis
Patch around pandas ArrowStringArray pickling (GH#9024) Jim Crist-Harif
Band-aid for compute_as_if_collection (GH#8998) Ian Rose
Add p2p shuffle option (GH#8836) Matthew Rocklin
Bug Fixes
Fixup column projection with no columns (GH#9106) Jim Crist-Harif
Blockwise cull NumPy dtype (GH#9100) Ian Rose
Fix column-projection bug in from_map (GH#9078) Richard (Rick) Zamora
Prevent nulls in index for non-numeric dtypes (GH#8963) Jorge López
Fix is_monotonic methods for more than 8 partitions (GH#9019) Julia Signell
Handle enumerate and generator inputs to from_map (GH#9066) Richard (Rick) Zamora
Revert is_dask_collection; back to previous implementation (GH#9062) Doug Davis
Fix Blockwise.clone does not handle iterable literal arguments correctly (GH#8979) JSKenyon
Array setitem hardmask (GH#9027) David Hassell
Fix overlapping divisions error on append (GH#8997) Ian Rose
Deprecations
Add pre-deprecation warnings for read_parquet kwargs chunksize and aggregate_files (GH#9052) Richard (Rick) Zamora
Documentation
Document map_partitions handling of args vs kwargs, usage of partition_info (GH#9084) Charles Blackmon-Luca
Update custom collection documentation (leverage new collection protocol) (GH#9097) Doug Davis
Better SEO for docs on creating and storing Dask DataFrames (GH#9098) Sarah Charlotte Johnson
Clarify chunking in imread docstring (GH#9082) Genevieve Buckley
Rearrange docs TOC (GH#9001) Matthew Rocklin
Corrected map_blocks() docstring for kwarg enforce_ndim (GH#9071) ParticularMiner
Update DataFrame SQL docs references to other libraries (GH#9077) Charles Blackmon-Luca
Update page on creating and storing Dask DataFrames (GH#9025) Sarah Charlotte Johnson
Maintenance
Include NUMPY_LICENSE.txt in license files (GH#9113) Ben Beasley
Increase retries when installing nightly pandas (GH#9103) James Bourbeau
Force nightly pyarrow in the upstream build (GH#9095) Joris Van den Bossche
Improve object handling & testing of ensure_unicode (GH#9059) John A Kirkham
Force nightly pyarrow in the upstream build (GH#8993) Joris Van den Bossche
Additional check on is_dask_collection (GH#9054) Doug Davis
Update ensure_bytes (GH#9050) John A Kirkham
Add end of file pre-commit hook (GH#9045) James Bourbeau
Add codespell pre-commit hook (GH#9040) James Bourbeau
Remove the HDFS tests (GH#9039) Jim Crist-Harif
Fix flaky test_reductions_2D (GH#9037) Jim Crist-Harif
Prevent codecov from notifying of failure too soon (GH#9031) Jim Crist-Harif
Only test on Python 3.9 on macos (GH#9029) Jim Crist-Harif
Update to_timedelta default unit (GH#9010) Pavithra Eswaramoorthy
To see a diff of this commit:
https://wip.pkgsrc.org/cgi-bin/gitweb.cgi?p=pkgsrc-wip.git;a=commitdiff;h=f73bb1f475c18a70d84cd5c8e2fb03a9b9997ba9
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
diffstat:
py-dask/Makefile | 2 +-
py-dask/PLIST | 30 ++++++++++++++++++++----------
py-dask/distinfo | 6 +++---
py-dask/options.mk | 2 +-
4 files changed, 25 insertions(+), 15 deletions(-)
diffs:
diff --git a/py-dask/Makefile b/py-dask/Makefile
index a3d0324756..aac92f136b 100644
--- a/py-dask/Makefile
+++ b/py-dask/Makefile
@@ -1,6 +1,6 @@
# $NetBSD$
-GITHUB_TAG= 2022.05.0
+GITHUB_TAG= 2022.6.1
DISTNAME= dask-${GITHUB_TAG}
PKGNAME= ${PYPKGPREFIX}-${DISTNAME}
GITHUB_PROJECT= dask
diff --git a/py-dask/PLIST b/py-dask/PLIST
index 473b638cc9..668bb7465b 100644
--- a/py-dask/PLIST
+++ b/py-dask/PLIST
@@ -1,16 +1,18 @@
@comment $NetBSD$
-${PYSITELIB}/dask-2022.5.0.dist-info/INSTALLER
-${PYSITELIB}/dask-2022.5.0.dist-info/LICENSE.txt
-${PYSITELIB}/dask-2022.5.0.dist-info/METADATA
-${PYSITELIB}/dask-2022.5.0.dist-info/RECORD
-${PYSITELIB}/dask-2022.5.0.dist-info/REQUESTED
-${PYSITELIB}/dask-2022.5.0.dist-info/WHEEL
-${PYSITELIB}/dask-2022.5.0.dist-info/direct_url.json
-${PYSITELIB}/dask-2022.5.0.dist-info/top_level.txt
+${PYSITELIB}/${WHEEL_INFODIR}/INSTALLER
+${PYSITELIB}/${WHEEL_INFODIR}/LICENSE.txt
+${PYSITELIB}/${WHEEL_INFODIR}/METADATA
+${PYSITELIB}/${WHEEL_INFODIR}/NUMPY_LICENSE.txt
+${PYSITELIB}/${WHEEL_INFODIR}/RECORD
+${PYSITELIB}/${WHEEL_INFODIR}/REQUESTED
+${PYSITELIB}/${WHEEL_INFODIR}/WHEEL
+${PYSITELIB}/${WHEEL_INFODIR}/direct_url.json
+${PYSITELIB}/${WHEEL_INFODIR}/top_level.txt
${PYSITELIB}/dask/__init__.py
${PYSITELIB}/dask/__init__.pyc
${PYSITELIB}/dask/_version.py
${PYSITELIB}/dask/_version.pyc
+${PYSITELIB}/dask/array/NUMPY_LICENSE.txt
${PYSITELIB}/dask/array/__init__.py
${PYSITELIB}/dask/array/__init__.pyc
${PYSITELIB}/dask/array/backends.py
@@ -195,8 +197,6 @@ ${PYSITELIB}/dask/bytes/tests/test_bytes_utils.py
${PYSITELIB}/dask/bytes/tests/test_bytes_utils.pyc
${PYSITELIB}/dask/bytes/tests/test_compression.py
${PYSITELIB}/dask/bytes/tests/test_compression.pyc
-${PYSITELIB}/dask/bytes/tests/test_hdfs.py
-${PYSITELIB}/dask/bytes/tests/test_hdfs.pyc
${PYSITELIB}/dask/bytes/tests/test_http.py
${PYSITELIB}/dask/bytes/tests/test_http.pyc
${PYSITELIB}/dask/bytes/tests/test_local.py
@@ -225,6 +225,8 @@ ${PYSITELIB}/dask/dataframe/_compat.py
${PYSITELIB}/dask/dataframe/_compat.pyc
${PYSITELIB}/dask/dataframe/_dtypes.py
${PYSITELIB}/dask/dataframe/_dtypes.pyc
+${PYSITELIB}/dask/dataframe/_pyarrow_compat.py
+${PYSITELIB}/dask/dataframe/_pyarrow_compat.pyc
${PYSITELIB}/dask/dataframe/accessor.py
${PYSITELIB}/dask/dataframe/accessor.pyc
${PYSITELIB}/dask/dataframe/backends.py
@@ -343,6 +345,8 @@ ${PYSITELIB}/dask/dataframe/tests/test_numeric.py
${PYSITELIB}/dask/dataframe/tests/test_numeric.pyc
${PYSITELIB}/dask/dataframe/tests/test_optimize_dataframe.py
${PYSITELIB}/dask/dataframe/tests/test_optimize_dataframe.pyc
+${PYSITELIB}/dask/dataframe/tests/test_pyarrow_compat.py
+${PYSITELIB}/dask/dataframe/tests/test_pyarrow_compat.pyc
${PYSITELIB}/dask/dataframe/tests/test_reshape.py
${PYSITELIB}/dask/dataframe/tests/test_reshape.pyc
${PYSITELIB}/dask/dataframe/tests/test_rolling.py
@@ -456,16 +460,22 @@ ${PYSITELIB}/dask/tests/test_rewrite.py
${PYSITELIB}/dask/tests/test_rewrite.pyc
${PYSITELIB}/dask/tests/test_sizeof.py
${PYSITELIB}/dask/tests/test_sizeof.pyc
+${PYSITELIB}/dask/tests/test_spark_compat.py
+${PYSITELIB}/dask/tests/test_spark_compat.pyc
${PYSITELIB}/dask/tests/test_system.py
${PYSITELIB}/dask/tests/test_system.pyc
${PYSITELIB}/dask/tests/test_threaded.py
${PYSITELIB}/dask/tests/test_threaded.pyc
+${PYSITELIB}/dask/tests/test_typing.py
+${PYSITELIB}/dask/tests/test_typing.pyc
${PYSITELIB}/dask/tests/test_utils.py
${PYSITELIB}/dask/tests/test_utils.pyc
${PYSITELIB}/dask/tests/test_utils_test.py
${PYSITELIB}/dask/tests/test_utils_test.pyc
${PYSITELIB}/dask/threaded.py
${PYSITELIB}/dask/threaded.pyc
+${PYSITELIB}/dask/typing.py
+${PYSITELIB}/dask/typing.pyc
${PYSITELIB}/dask/utils.py
${PYSITELIB}/dask/utils.pyc
${PYSITELIB}/dask/utils_test.py
diff --git a/py-dask/distinfo b/py-dask/distinfo
index 6c10819315..c8b6277a76 100644
--- a/py-dask/distinfo
+++ b/py-dask/distinfo
@@ -1,5 +1,5 @@
$NetBSD$
-BLAKE2s (dask-2022.05.0.tar.gz) = 1e85168e81b9727daed8b85e05a394a8b4ba81ff2c100a9af46f8d2191731483
-SHA512 (dask-2022.05.0.tar.gz) = 15605321143f354ffb9c890823f5751131d942cc16c523b2d00bdcc752ddc5243525f7cda138091604c8f2f9c5b4a309b1efc95f483b753673df98cc8a81ce81
-Size (dask-2022.05.0.tar.gz) = 8658529 bytes
+BLAKE2s (dask-2022.6.1.tar.gz) = ec1bfaff04ff050935344127bf00818ebd9f8905b6cc5be4ee94004507f543a9
+SHA512 (dask-2022.6.1.tar.gz) = 6e06b63dd5c7f1d7c6c7dce867ffa42b956023b313b25245a339f698b06bea17a03cc4ead9bd5aef8ddf081750cad4df4b6b2972038315c89585221382c9e95c
+Size (dask-2022.6.1.tar.gz) = 8729249 bytes
diff --git a/py-dask/options.mk b/py-dask/options.mk
index 97a755aa17..859f4dbe6a 100644
--- a/py-dask/options.mk
+++ b/py-dask/options.mk
@@ -20,7 +20,7 @@ DEPENDS+= ${PYPKGPREFIX}-cytoolz-[0-9]*:../../devel/py-cytoolz
.if !empty(PKG_OPTIONS:Mparquet)
# TODO Also needs pyarrow
DEPENDS+= ${PYPKGPREFIX}-fastparquet-[0-9]*:../../wip/py-fastparquet
-DEPENDS+= ${PYPKGPREFIX}-apache-arrow-[0-9]*:../../wip/py-apache-arrow
+DEPENDS+= ${PYPKGPREFIX}-pyarrow-[0-9]*:../../wip/py-apache-arrow
.endif
# graphviz is used to display the task dependencies
- Prev by Date:
py-threadpoolctl: Hook up make test correctly
- Next by Date:
py-distributed: Updated to 2022.6.2 2022.6.1 Released on June 24, 2022 Highlights This release includes the Worker State Machine refactor. The expectation should be that the worker state is its own synchronous subclass. Pulling all the state out into its own class allows us to write targeted unit tests without invoking any concurrent or asynchronous code. See GH#5736 for more information. Enhancements Make worker state machine methods private (GH#6564) crusaderky Yank state machine out of Worker class (GH#6566) crusaderky Track worker_state_machine.TaskState instances (GH#6525) Hendrik Makait Trivial tweaks to the Worker State Machine (GH#6586) crusaderky Replace loop.call_later and loop.add_callback with background tasks added to Server. (GH#6603) Thomas Grainger Support for neater WorkerState tests (GH#6609) crusaderky Limit TCP writes with Tornado to 2GB (GH#6557) hhuuggoo Enable no_implicit_optional for scheduler (GH#6622) Thomas Grainger Bug Fixes Partial revert of compute-task message format (GH#6626) Florian Jetter Restore log message about received signals in CLI (GH#6618) Florian Jetter Handle empty memoryviews of bytearrays when (de)serializing (GH#6576) Benjamin Zaitlen Ensure steal requests from same-IP but distinct workers are rejected (GH#6585) Florian Jetter Fix tls_(min|max)_ version having no effect on openssl 1.1.0g or lower (GH#6562) Thomas Grainger Fix idle_timeout and unxfail test (GH#6563) Matthew Rocklin Fix crashing debug statement in _purge_state (GH#6589) crusaderky Abort connections on CancelledError (GH#6574) Thomas Grainger Fix Active Memory Manager ignores nbytes thresholds (GH#6583) crusaderky Deprecations Deprecate WorkerState accessors (GH#6579) crusaderky Maintenance Remove ipython hack (GH#6599) crusaderky Mypy enforce --no-implicit-optional (GH#6606) crusaderky Update versioneer: change from using SafeConfigParser to ConfigParser (GH#6605) Thomas A Caswell Warn unreachable for scheduler.py (GH#6611) Florian Jetter Refactor wait_for_state() (GH#6581) crusaderky Hardcode wait_for_signals signal list (GH#6619) Thomas Grainger Always pick an open port when running tests (GH#6591) Florian Jetter Log popen stdout/err when subprocess times out (GH#6567) Gabe Joseph Fix test_error_during_startup (GH#6608) Florian Jetter Make test_idle_timeout_no_workers more robust (GH#6602) Florian Jetter Mypy enforce --disallow-incomplete-defs (GH#6601) crusaderky Do not log during signal handler (GH#6590) Florian Jetter Don’t initialize mp_context on import (GH#6580) Lawrence Mitchell Test retire workers deadlock (GH#6240) Gabe Joseph Rework some tests related to gather_dep (GH#6472) crusaderky Minor cosmetic review of scheduler_story and worker_story (GH#6442) crusaderky Force __future__.annotations with isort (GH#6621) Thomas Grainger 2022.6.0 Released on June 10, 2022 Enhancements Make disk access in system monitor configurable (GH#6537) Matthew Rocklin Log and except errors on preload start (GH#6553) Matthew Rocklin Fix Scheduler.restart logic (GH#6504) Gabe Joseph Don’t heartbeat while Worker is closing (GH#6543) Gabe Joseph No longer retry LocalCluster on errno.EADDRINUSE (GH#6369) Thomas Grainger Don’t invoke log_event from state machine (GH#6512) crusaderky Add config option to disable profiling and disable it in many tests per default (GH#6490) Hendrik Makait Encapsulate Worker.batched_stream.send() (GH#6475) crusaderky Bug Fixes refresh-who-has can break the worker state machine (GH#6529) crusaderky Restart worker if it’s unrecognized by scheduler (GH#6505) Gabe Joseph Fix import error when distributed.rmm.pool-size is set (GH#6482) KoyamaSohei Deprecations Restore signature compatibility for dask-gateway (GH#6561) Tom Augspurger Deprecate the io_loop and loop kwarg to Server, Worker, and Nanny (GH#6473) Thomas Grainger Deprecate the loop kwarg to Scheduler (GH#6443) Thomas Grainger Documentation Fix typo in .nthreads() docstring example (GH#6545) Pavithra Eswaramoorthy Update docs theme for rebranding (GH#6495) Sarah Charlotte Johnson Maintenance Refactor gather_dep (GH#6388) crusaderky Fix flaky test_gather_dep_one_worker_always_busy (GH#6554) crusaderky Remove missing-data message (GH#6546) crusaderky Port test_local.LocalTest to pytest tests to allow use of loop fixture (GH#6523) Thomas Grainger Fix test_quiet_client_close (GH#6541) Gabe Joseph Use default_initializer in WorkerProcess (GH#6534) jakirkham Avoid deadlocks in tests that use popen (GH#6483) Gabe Joseph Revert “Fix CLI Scheduler Tests (GH#6502)” (GH#6547) Gabe Joseph Update test report URL in summary message (GH#6532) Gabe Joseph Update test report url (GH#6531) Ian Rose Assert AsyncProcess.set_exit_callback is not called with a coroutine function (GH#6526) Thomas Grainger Typing and docstring for Worker.close (GH#6518) Hendrik Makait Fix CLI Scheduler Tests (GH#6502) Benjamin Zaitlen Collect assertions in test_as_current_is_thread_local (GH#6520) Thomas Grainger Link test report from test results comment (GH#6524) Hendrik Makait Ignore the return value of signal.signal (GH#6519) Thomas Grainger Refactor all event handlers (GH#6410) crusaderky Fix dashboard favicon background (GH#6514) Jacob Tomlinson Update dashboard logo (GH#6513) Jacob Tomlinson Fix test_stress_scatter_death (GH#6404) Florian Jetter Remove CrossFilter widget (GH#6484) crusaderky data_needed exclusively contains tasks in fetch state (GH#6481) crusaderky Assert possible previous states (GH#6488) Florian Jetter @fail_hard can kill the whole test suite; hide errors (GH#6474) crusaderky Assert that a fetch->cancelled->resumed->fetch cycle is impossible (GH#6460) crusaderky Refactor busy workers reinsertion (GH#6379) crusaderky Refactor find_missing and refresh_who_has (GH#6348) crusaderky Rename test_collections.py to test_dask_collections.py (GH#6486) crusaderky update_who_has can remove workers (GH#6342) crusaderky Restructure test_watch_requires_lock_to_run to avoid flakes (GH#6469) Hendrik Makait Fix intermittent test_profile_plot failure (GH#6456) Matthew Rocklin Use asyncio.run to run gen_cluster, gen_test and cluster (GH#6231) Thomas Grainger Improve tests that watch for subprocess logs (GH#6461) Gabe Joseph 2022.05.2 Released on May 26, 2022 Enhancements Add a lock to distributed.profile for better concurrency control (GH#6421) Hendrik Makait Send SIGKILL after SIGTERM when passing 95% memory (GH#6419) crusaderky Bug Fixes Log rather than raise exceptions in preload.teardown() (GH#6458) Matthew Rocklin Handle failing plugin.close() calls during scheduler shutdown (GH#6450) Matthew Rocklin Fix slicing bug in ensure_memoryview (GH#6449) jakirkham Generalize UCX errors on connect() and correct pytest fixtures (GH#6434) Peter Andreas Entschev Run cluster widget periodic callbacks on the correct event loop (GH#6444) Thomas Grainger Maintenance Disable pytest-asyncio if installed (GH#6436) Jacob Tomlinson Close client in sync test_actor tests (GH#6459) Thomas Grainger Ignore ServerSession.with_document_locked unawaited (GH#6447) Thomas Grainger Remove coverage pin from Python 3.10 environment (GH#6439) Thomas Grainger Annotate remove_worker (GH#6441) crusaderky Update gpuCI RAPIDS_VER to 22.08 (GH#6428) 2022.05.1 Released on May 24, 2022 New Features Add HTTP API to scheduler (GH#6270) Matthew Murray Shuffle Service with Scheduler Logic (GH#6007) Matthew Rocklin Enhancements Follow-up on removing report and safe from Worker.close (GH#6423) Gabe Joseph Server close faster (GH#6415) Florian Jetter Disable HTTP API by default (GH#6420) Jacob Tomlinson Remove report and safe from Worker.close (GH#6363) Florian Jetter Allow deserialized plugins in register_scheduler_plugin (GH#6401) Matthew Rocklin WorkerState are different for different addresses (GH#6398) Florian Jetter Do not filter tasks before gathering data (GH#6371) crusaderky Remove worker reconnect (GH#6361) Gabe Joseph Add SchedulerPlugin.log_event handler (GH#6381) Matthew Rocklin Ensure occupancy tracking works as expected for long running tasks (GH#6351) Florian Jetter stimulus_id for all Instructions (GH#6347) crusaderky Refactor missing-data command (GH#6332) crusaderky Add idempotent to register_scheduler_plugin client (GH#6328) Alex Ford Add option to specify a scheduler address for workers to use (GH#5944) Enric Tejedor Bug Fixes Remove stray breakpoint (GH#6417) Thomas Grainger Fix API JSON MIME type (GH#6397) Jacob Tomlinson Remove wrong assert in handle compute (GH#6370) Florian Jetter Ensure multiple clients can cancel their key without interference (GH#6016) Florian Jetter Fix Nanny shutdown assertion (GH#6357) Gabe Joseph Fix fail_hard for sync functions (GH#6269) Gabe Joseph Prevent infinite transition loops; more aggressive validate_state() (GH#6318) crusaderky Ensure cleanup of many GBs of spilled data on terminate (GH#6280) crusaderky Fix WORKER_ANY_RUNNING regression (GH#6297) Florian Jetter Race conditions from fetch to compute while AMM requests replica (GH#6248) Florian Jetter Ensure resumed tasks are not accidentally forgotten (GH#6217) Florian Jetter Do not allow closing workers to be awaited again (GH#5910) Florian Jetter Deprecations Move wait_for_signals to private module and deprecate distributed.cli.utils (GH#6367) Hendrik Makait Documentation Fix typos and whitespace in worker.py (GH#6326) Hendrik Makait Fix link to memory trimming documentation (GH#6317) Marco Wolsza Maintenance Make gen_test show up in VSCode test discovery (GH#6424) Gabe Joseph WSMR / deserialize_task (GH#6411) crusaderky Restore signal handlers after wait for signals is done (GH#6400) Thomas Grainger fail_hard should reraise (GH#6399) crusaderky Revisit tests mocking gather_dep (GH#6385) crusaderky Fix flaky test_in_flight_lost_after_resumed (GH#6372) Florian Jetter Restore install_signal_handlers due to downstream dependencies (GH#6366) Hendrik Makait Improve catch_unhandled_exceptions (GH#6358) Gabe Joseph Remove all invocations of IOLoop.run_sync from CLI (GH#6205) Hendrik Makait Remove transition-counter-max from config (GH#6349) crusaderky Use list comprehension in pickle_loads (GH#6343) jakirkham Improve ensure_memoryview test coverage & make minor fixes (GH#6333) jakirkham Remove leaking reference to workers from gen_cluster (GH#6337) Hendrik Makait Partial annotations for stealing.py (GH#6338) crusaderky Validate and debug state machine on handle_compute_task (GH#6327) crusaderky Bump pyupgrade and clean up # type: ignore (GH#6293) crusaderky gen_cluster to write to /tmp (GH#6335) crusaderky Transition table as a ClassVar (GH#6331) crusaderky Simplify ensure_memoryview test with array (GH#6322) jakirkham Refactor ensure_communicating (GH#6165) crusaderky Review scheduler annotations, part 2 (GH#6253) crusaderky Use w for writeable branch in pickle_loads (GH#6314) jakirkham Simplify frame handling in ws (GH#6294) jakirkham Use ensure_bytes from dask.utils (GH#6295) jakirkham Use ensure_memoryview in array deserialization (GH#6300) jakirkham Escape < > when generating Junit report (GH#6306) crusaderky Use codecs.decode to deserialize errors (GH#6274) jakirkham Minimize copying in maybe_compress & byte_sample (GH#6273) jakirkham Skip test_release_evloop_while_spilling on OSX (GH#6291) Florian Jetter Simplify logic in get_default_compression (GH#6260) jakirkham Cleanup old compression workarounds (GH#6259) jakirkham Re-enable NVML monitoring for WSL (GH#6119) Charles Blackmon-Luca
- Previous by Thread:
py-threadpoolctl: Hook up make test correctly
- Next by Thread:
py-distributed: Updated to 2022.6.2 2022.6.1 Released on June 24, 2022 Highlights This release includes the Worker State Machine refactor. The expectation should be that the worker state is its own synchronous subclass. Pulling all the state out into its own class allows us to write targeted unit tests without invoking any concurrent or asynchronous code. See GH#5736 for more information. Enhancements Make worker state machine methods private (GH#6564) crusaderky Yank state machine out of Worker class (GH#6566) crusaderky Track worker_state_machine.TaskState instances (GH#6525) Hendrik Makait Trivial tweaks to the Worker State Machine (GH#6586) crusaderky Replace loop.call_later and loop.add_callback with background tasks added to Server. (GH#6603) Thomas Grainger Support for neater WorkerState tests (GH#6609) crusaderky Limit TCP writes with Tornado to 2GB (GH#6557) hhuuggoo Enable no_implicit_optional for scheduler (GH#6622) Thomas Grainger Bug Fixes Partial revert of compute-task message format (GH#6626) Florian Jetter Restore log message about received signals in CLI (GH#6618) Florian Jetter Handle empty memoryviews of bytearrays when (de)serializing (GH#6576) Benjamin Zaitlen Ensure steal requests from same-IP but distinct workers are rejected (GH#6585) Florian Jetter Fix tls_(min|max)_ version having no effect on openssl 1.1.0g or lower (GH#6562) Thomas Grainger Fix idle_timeout and unxfail test (GH#6563) Matthew Rocklin Fix crashing debug statement in _purge_state (GH#6589) crusaderky Abort connections on CancelledError (GH#6574) Thomas Grainger Fix Active Memory Manager ignores nbytes thresholds (GH#6583) crusaderky Deprecations Deprecate WorkerState accessors (GH#6579) crusaderky Maintenance Remove ipython hack (GH#6599) crusaderky Mypy enforce --no-implicit-optional (GH#6606) crusaderky Update versioneer: change from using SafeConfigParser to ConfigParser (GH#6605) Thomas A Caswell Warn unreachable for scheduler.py (GH#6611) Florian Jetter Refactor wait_for_state() (GH#6581) crusaderky Hardcode wait_for_signals signal list (GH#6619) Thomas Grainger Always pick an open port when running tests (GH#6591) Florian Jetter Log popen stdout/err when subprocess times out (GH#6567) Gabe Joseph Fix test_error_during_startup (GH#6608) Florian Jetter Make test_idle_timeout_no_workers more robust (GH#6602) Florian Jetter Mypy enforce --disallow-incomplete-defs (GH#6601) crusaderky Do not log during signal handler (GH#6590) Florian Jetter Don’t initialize mp_context on import (GH#6580) Lawrence Mitchell Test retire workers deadlock (GH#6240) Gabe Joseph Rework some tests related to gather_dep (GH#6472) crusaderky Minor cosmetic review of scheduler_story and worker_story (GH#6442) crusaderky Force __future__.annotations with isort (GH#6621) Thomas Grainger 2022.6.0 Released on June 10, 2022 Enhancements Make disk access in system monitor configurable (GH#6537) Matthew Rocklin Log and except errors on preload start (GH#6553) Matthew Rocklin Fix Scheduler.restart logic (GH#6504) Gabe Joseph Don’t heartbeat while Worker is closing (GH#6543) Gabe Joseph No longer retry LocalCluster on errno.EADDRINUSE (GH#6369) Thomas Grainger Don’t invoke log_event from state machine (GH#6512) crusaderky Add config option to disable profiling and disable it in many tests per default (GH#6490) Hendrik Makait Encapsulate Worker.batched_stream.send() (GH#6475) crusaderky Bug Fixes refresh-who-has can break the worker state machine (GH#6529) crusaderky Restart worker if it’s unrecognized by scheduler (GH#6505) Gabe Joseph Fix import error when distributed.rmm.pool-size is set (GH#6482) KoyamaSohei Deprecations Restore signature compatibility for dask-gateway (GH#6561) Tom Augspurger Deprecate the io_loop and loop kwarg to Server, Worker, and Nanny (GH#6473) Thomas Grainger Deprecate the loop kwarg to Scheduler (GH#6443) Thomas Grainger Documentation Fix typo in .nthreads() docstring example (GH#6545) Pavithra Eswaramoorthy Update docs theme for rebranding (GH#6495) Sarah Charlotte Johnson Maintenance Refactor gather_dep (GH#6388) crusaderky Fix flaky test_gather_dep_one_worker_always_busy (GH#6554) crusaderky Remove missing-data message (GH#6546) crusaderky Port test_local.LocalTest to pytest tests to allow use of loop fixture (GH#6523) Thomas Grainger Fix test_quiet_client_close (GH#6541) Gabe Joseph Use default_initializer in WorkerProcess (GH#6534) jakirkham Avoid deadlocks in tests that use popen (GH#6483) Gabe Joseph Revert “Fix CLI Scheduler Tests (GH#6502)” (GH#6547) Gabe Joseph Update test report URL in summary message (GH#6532) Gabe Joseph Update test report url (GH#6531) Ian Rose Assert AsyncProcess.set_exit_callback is not called with a coroutine function (GH#6526) Thomas Grainger Typing and docstring for Worker.close (GH#6518) Hendrik Makait Fix CLI Scheduler Tests (GH#6502) Benjamin Zaitlen Collect assertions in test_as_current_is_thread_local (GH#6520) Thomas Grainger Link test report from test results comment (GH#6524) Hendrik Makait Ignore the return value of signal.signal (GH#6519) Thomas Grainger Refactor all event handlers (GH#6410) crusaderky Fix dashboard favicon background (GH#6514) Jacob Tomlinson Update dashboard logo (GH#6513) Jacob Tomlinson Fix test_stress_scatter_death (GH#6404) Florian Jetter Remove CrossFilter widget (GH#6484) crusaderky data_needed exclusively contains tasks in fetch state (GH#6481) crusaderky Assert possible previous states (GH#6488) Florian Jetter @fail_hard can kill the whole test suite; hide errors (GH#6474) crusaderky Assert that a fetch->cancelled->resumed->fetch cycle is impossible (GH#6460) crusaderky Refactor busy workers reinsertion (GH#6379) crusaderky Refactor find_missing and refresh_who_has (GH#6348) crusaderky Rename test_collections.py to test_dask_collections.py (GH#6486) crusaderky update_who_has can remove workers (GH#6342) crusaderky Restructure test_watch_requires_lock_to_run to avoid flakes (GH#6469) Hendrik Makait Fix intermittent test_profile_plot failure (GH#6456) Matthew Rocklin Use asyncio.run to run gen_cluster, gen_test and cluster (GH#6231) Thomas Grainger Improve tests that watch for subprocess logs (GH#6461) Gabe Joseph 2022.05.2 Released on May 26, 2022 Enhancements Add a lock to distributed.profile for better concurrency control (GH#6421) Hendrik Makait Send SIGKILL after SIGTERM when passing 95% memory (GH#6419) crusaderky Bug Fixes Log rather than raise exceptions in preload.teardown() (GH#6458) Matthew Rocklin Handle failing plugin.close() calls during scheduler shutdown (GH#6450) Matthew Rocklin Fix slicing bug in ensure_memoryview (GH#6449) jakirkham Generalize UCX errors on connect() and correct pytest fixtures (GH#6434) Peter Andreas Entschev Run cluster widget periodic callbacks on the correct event loop (GH#6444) Thomas Grainger Maintenance Disable pytest-asyncio if installed (GH#6436) Jacob Tomlinson Close client in sync test_actor tests (GH#6459) Thomas Grainger Ignore ServerSession.with_document_locked unawaited (GH#6447) Thomas Grainger Remove coverage pin from Python 3.10 environment (GH#6439) Thomas Grainger Annotate remove_worker (GH#6441) crusaderky Update gpuCI RAPIDS_VER to 22.08 (GH#6428) 2022.05.1 Released on May 24, 2022 New Features Add HTTP API to scheduler (GH#6270) Matthew Murray Shuffle Service with Scheduler Logic (GH#6007) Matthew Rocklin Enhancements Follow-up on removing report and safe from Worker.close (GH#6423) Gabe Joseph Server close faster (GH#6415) Florian Jetter Disable HTTP API by default (GH#6420) Jacob Tomlinson Remove report and safe from Worker.close (GH#6363) Florian Jetter Allow deserialized plugins in register_scheduler_plugin (GH#6401) Matthew Rocklin WorkerState are different for different addresses (GH#6398) Florian Jetter Do not filter tasks before gathering data (GH#6371) crusaderky Remove worker reconnect (GH#6361) Gabe Joseph Add SchedulerPlugin.log_event handler (GH#6381) Matthew Rocklin Ensure occupancy tracking works as expected for long running tasks (GH#6351) Florian Jetter stimulus_id for all Instructions (GH#6347) crusaderky Refactor missing-data command (GH#6332) crusaderky Add idempotent to register_scheduler_plugin client (GH#6328) Alex Ford Add option to specify a scheduler address for workers to use (GH#5944) Enric Tejedor Bug Fixes Remove stray breakpoint (GH#6417) Thomas Grainger Fix API JSON MIME type (GH#6397) Jacob Tomlinson Remove wrong assert in handle compute (GH#6370) Florian Jetter Ensure multiple clients can cancel their key without interference (GH#6016) Florian Jetter Fix Nanny shutdown assertion (GH#6357) Gabe Joseph Fix fail_hard for sync functions (GH#6269) Gabe Joseph Prevent infinite transition loops; more aggressive validate_state() (GH#6318) crusaderky Ensure cleanup of many GBs of spilled data on terminate (GH#6280) crusaderky Fix WORKER_ANY_RUNNING regression (GH#6297) Florian Jetter Race conditions from fetch to compute while AMM requests replica (GH#6248) Florian Jetter Ensure resumed tasks are not accidentally forgotten (GH#6217) Florian Jetter Do not allow closing workers to be awaited again (GH#5910) Florian Jetter Deprecations Move wait_for_signals to private module and deprecate distributed.cli.utils (GH#6367) Hendrik Makait Documentation Fix typos and whitespace in worker.py (GH#6326) Hendrik Makait Fix link to memory trimming documentation (GH#6317) Marco Wolsza Maintenance Make gen_test show up in VSCode test discovery (GH#6424) Gabe Joseph WSMR / deserialize_task (GH#6411) crusaderky Restore signal handlers after wait for signals is done (GH#6400) Thomas Grainger fail_hard should reraise (GH#6399) crusaderky Revisit tests mocking gather_dep (GH#6385) crusaderky Fix flaky test_in_flight_lost_after_resumed (GH#6372) Florian Jetter Restore install_signal_handlers due to downstream dependencies (GH#6366) Hendrik Makait Improve catch_unhandled_exceptions (GH#6358) Gabe Joseph Remove all invocations of IOLoop.run_sync from CLI (GH#6205) Hendrik Makait Remove transition-counter-max from config (GH#6349) crusaderky Use list comprehension in pickle_loads (GH#6343) jakirkham Improve ensure_memoryview test coverage & make minor fixes (GH#6333) jakirkham Remove leaking reference to workers from gen_cluster (GH#6337) Hendrik Makait Partial annotations for stealing.py (GH#6338) crusaderky Validate and debug state machine on handle_compute_task (GH#6327) crusaderky Bump pyupgrade and clean up # type: ignore (GH#6293) crusaderky gen_cluster to write to /tmp (GH#6335) crusaderky Transition table as a ClassVar (GH#6331) crusaderky Simplify ensure_memoryview test with array (GH#6322) jakirkham Refactor ensure_communicating (GH#6165) crusaderky Review scheduler annotations, part 2 (GH#6253) crusaderky Use w for writeable branch in pickle_loads (GH#6314) jakirkham Simplify frame handling in ws (GH#6294) jakirkham Use ensure_bytes from dask.utils (GH#6295) jakirkham Use ensure_memoryview in array deserialization (GH#6300) jakirkham Escape < > when generating Junit report (GH#6306) crusaderky Use codecs.decode to deserialize errors (GH#6274) jakirkham Minimize copying in maybe_compress & byte_sample (GH#6273) jakirkham Skip test_release_evloop_while_spilling on OSX (GH#6291) Florian Jetter Simplify logic in get_default_compression (GH#6260) jakirkham Cleanup old compression workarounds (GH#6259) jakirkham Re-enable NVML monitoring for WSL (GH#6119) Charles Blackmon-Luca
- Indexes:
Home |
Main Index |
Thread Index |
Old Index