Siyuan (Ryans) Zhuang
1bcd3e41d1
[Workflow] Cleanup workflow docs ( #27197 )
...
* cleanup workflow docs
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
2022-07-29 13:03:50 -07:00
Siyuan (Ryans) Zhuang
e1db8fb382
[Workflow] Workflow client integration ( #26702 )
...
## Why are these changes needed?
This PR ensures that workflow can work properly with Ray client.
Regular workflow tests will (also) be running under client mode (as a pytest parameter). Some tests are moved and reorganized, because the Ray client tests requires starting the cluster, so some tests requires isolation or related changes.
Tests that literally take down the cluster are not tested with Ray client, since Ray client would fail in this scenario.
Limitations of Ray Workflow under Ray client are noted in the doc.
## Related issue number
Closes #21595
2022-07-26 11:15:47 -07:00
Siyuan (Ryans) Zhuang
4a1ad3e87a
[Workflow] Support "retry_exceptions" of Ray tasks ( #26913 )
...
* support 'retry_exceptions'
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
* add test
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
* add doc
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
* fix
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
* typo
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
2022-07-24 20:50:11 -07:00
Siyuan (Ryans) Zhuang
4b50ef6a28
[Workflow] Rename the argument of "workflow.get_output" ( #26876 )
...
* rename get_output
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
* update doc
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
2022-07-22 12:06:19 -07:00
Siyuan (Ryans) Zhuang
5b937167d3
[Workflow] Fix typo in workflow event doc ( #26686 )
...
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
2022-07-18 23:26:50 -07:00
Siyuan (Ryans) Zhuang
eb4ed49c1f
[Workflow] Unify the semantics of max_retries of workflow task and Ray task ( #26350 )
...
* workflow task retry
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
* move and enhance tests
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
* use "max_retries" of Ray task
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
* add test for disabling lineage reconstruction in workflow
Signed-off-by: Siyuan Zhuang <suquark@gmail.com>
2022-07-18 23:25:44 -07:00
Siyuan (Ryans) Zhuang
b0e913fd07
[workflow] Workflow queue ( #24697 )
...
* implement workflow queue
2022-07-08 17:24:45 -07:00
Siyuan (Ryans) Zhuang
b803792b58
[workflow] Standardize workflow blocking and nonblocking APIs ( #26318 )
...
This PR unified the semantics of some workflow APIs.
Those workflow APIs acts on workflow tasks so they could be blocked for a long time. So we have both the blocking and non-blocking versions for them: xxx for blocking and xxx_async for non-blocking APIs.
2022-07-06 13:35:36 -07:00
Siyuan (Ryans) Zhuang
5a094f1d18
[workflow] Deprecate workflow.create ( #26106 )
2022-07-02 21:24:05 -07:00
Siyuan (Ryans) Zhuang
ddd63aba77
[workflow] Major refactoring - new async workflow executor ( #25618 )
...
* major workflow refactoring
2022-06-29 20:31:40 -07:00
Rina Ueno
a29eeaa1f6
[Workflows] Explain workflow_id and task_name in the docs ( #25800 )
2022-06-21 15:24:16 -07:00
Siyuan (Ryans) Zhuang
b5e71fde23
[workflow] Remove workflow virtual actor ( #25394 )
...
* remove workflow virtual actor
2022-06-02 18:17:25 -07:00
Siyuan (Ryans) Zhuang
2766284b14
[workflow] Update workflow doc and examples ( #24804 )
...
* update doc of workflow options
* update examples and make sure they are working
2022-05-16 15:41:14 -07:00
Siyuan (Ryans) Zhuang
417b72efdc
[workflow] Update workflow docs ( #24249 )
...
* update workflow docs
* rename "step" to "task"
2022-05-05 22:22:51 -07:00
Siyuan (Ryans) Zhuang
6b1b25168f
[workflow][doc] Doc for workflow checkpointing ( #23510 )
2022-03-27 12:18:14 -07:00
Jiaxin Shan
8823ca48b4
[Workflow] Improve workflow docs ( #23114 )
...
* [Workflow] Improve workflow docs
* Update doc/source/workflows/concepts.rst
Co-authored-by: Siyuan (Ryans) Zhuang <suquark@gmail.com>
2022-03-13 18:55:45 -07:00
Max Pumperla
b34099e764
[docs] landing page ( fixes #21750 ) ( #21859 )
...
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2022-01-26 17:14:25 -08:00
xwjiang2010
9af8f11191
Revert "[docs] Clean up doc structure (first part) ( #21667 )" ( #21763 )
...
This reverts commit 38e46c9fb3
2022-01-20 15:30:56 -08:00
Max Pumperla
38e46c9fb3
[docs] Clean up doc structure (first part) ( #21667 )
2022-01-20 16:19:04 +01:00
Richard Liaw
169e422937
[docs] Make Jobs more prominent in documentation ( #21575 )
2022-01-13 23:49:34 -08:00
Linsong Chu
61bbecdb7d
[Workflow]add doc for metadata ( #20156 )
...
This PR adds documentation for Workflow Metadata, which we recently added support in https://github.com/ray-project/ray/pull/19372 .
Co-authored-by: Yi Cheng <74173148+iycheng@users.noreply.github.com>
2021-12-20 17:24:07 -08:00
Siyuan (Ryans) Zhuang
3eb76466a0
[workflow] workflow.wait() feature ( #20163 )
...
This PR implements `workflow.wait()`. When combined with checkpointing, it allows skipping sync & checkpointing of unfinished workflows.
2021-11-30 12:30:28 -08:00
architkulkarni
42085fd3d5
[runtime env] [Doc] Add concepts and basic workflows ( #20222 )
...
Address followup comments from https://github.com/ray-project/ray/pull/19863
- Add short "Concepts" section
- Add more section headings to break up the text
- Add "Workflow: Local Files" example
- Add "Workflow: Library development" example
2021-11-19 13:58:50 -08:00
Yi Cheng
cbf5826040
[workflow] Fix workflow event doc typo ( #20465 )
...
In the example, it says `after_checkpoint`, but this should be `event_checkpointed`
2021-11-17 16:18:20 -08:00
Siyuan (Ryans) Zhuang
3b62388a9a
[Workflow] Workflow tail recursion optimization ( #19928 )
...
* tail recursion optimization
2021-11-12 09:13:40 -08:00
Alex Wu
81194f5660
[workflow][docs] Fix api comparison formatting ( #20069 )
...
## Why are these changes needed?
The API comparison formatting uses \`code\` which is rendered as italicization not code. This PR puts the code in code blocks instead of italics.
## Related issue number
## Checks
2021-11-05 17:05:35 -07:00
Alex Wu
31d89be926
[Workflow] Basic event support ( #19239 )
...
* basics
* .
* .
* a test
* a test
* tests
* cleanup
* concepts page
* docs
* polish
* fix sleep
* fix yi things
* lint
* fix
* .
* .
* .
* fix?
* .
Co-authored-by: Alex Wu <alex@anyscale.com>
2021-10-22 15:27:33 -07:00
Alex Wu
a819e417ac
Revert "[Hotfix] Revert "[Workflow] workflow.delete"" ( #19248 )
...
* Revert "Revert "[Workflow] workflow.delete (#19178 )" (#19247 )"
This reverts commit b59317520d
2021-10-19 09:47:56 -07:00
matthewdeng
4674c78050
[Train] Rename Ray SGD v2 to Ray Train ( #19436 )
2021-10-18 22:27:46 -07:00
Eric Liang
b59317520d
Revert "[Workflow] workflow.delete ( #19178 )" ( #19247 )
...
This reverts commit 7ea512f343
2021-10-08 19:12:55 -07:00
Alex Wu
7ea512f343
[Workflow] workflow.delete ( #19178 )
...
Why are these changes needed?
This PR implements workflow.delete which allows users to delete the information in storage related to a workflow. (This assumes the workflow isn't currently running).
Related issue number
Closes #18848
2021-10-08 16:11:59 -07:00
Yi Cheng
7d1f408de9
[workflow] Move experimental/workflow to workflow ( #18521 )
2021-09-13 17:45:18 -07:00
Eric Liang
53a2a47655
Polish workflows doc, add semantics and best practices for sub-workflows ( #18525 )
2021-09-12 11:08:06 -07:00
Yi Cheng
370473fc5f
[workflow] Update documentation ( #18522 )
2021-09-11 13:40:09 -07:00
Yi Cheng
965c55fe1b
[workflow] set max retry to 3 ( #18477 )
2021-09-09 23:03:24 -07:00
Eric Liang
cbdafa0b63
[doc] Fix various workflow doc bugs ( #18357 )
2021-09-06 01:39:08 -07:00
Eric Liang
c4199a8054
Add more workflow comparisons ( #18347 )
2021-09-03 19:26:33 -07:00
Eric Liang
95b5ad12ba
Initial version of workflow documentation ( #18138 )
2021-08-27 16:20:48 -07:00
