mirror of
https://github.com/vale981/ray
synced 2025-03-11 05:46:37 -04:00
1f9488724a
Adds support for Python generators instead of just normal return functions when a task has multiple return values. This will allow developers to cut down on total memory usage for tasks, as they can free previous return values before allocating the next one on the heap. The semantics for num_returns are about the same as usual tasks - the function will throw an error if the number of values returned by the generator does not match the number of return values specified by the user. The one difference is that if num_returns=1, the task will throw the usual Python exception that the generator cannot be pickled. As an example, this feature will allow us to reduce memory usage in Datasets shuffle operations (see #25200 for a prototype).
27 lines
692 B
ReStructuredText
27 lines
692 B
ReStructuredText
.. _task-patterns:
|
|
|
|
Task Design Patterns
|
|
====================
|
|
|
|
This section is a collection of common design patterns (and anti-patterns) for Ray tasks. It is meant as a handbook for both:
|
|
|
|
- New users trying to understand how to get started with Ray, and
|
|
- Advanced users trying to optimize their use of Ray tasks
|
|
|
|
You may also be interested in visiting the design patterns section for :ref:`actors <actor-patterns>`.
|
|
|
|
.. toctree::
|
|
:maxdepth: -1
|
|
|
|
tree-of-tasks
|
|
map-reduce
|
|
limit-tasks
|
|
generators
|
|
closure-capture
|
|
fine-grained-tasks
|
|
global-variables
|
|
ray-get-loop
|
|
submission-order
|
|
too-many-results
|
|
redefine-task-actor-loop
|
|
unnecessary-ray-get
|