mirror of
https://github.com/vale981/ray
synced 2025-03-08 19:41:38 -05:00
f376dd8902
Ray Tune currently gracefully stops training on SIGINT. However, the Ray core worker prevents SIGINT (and SIGTERM) to be processed by child tasks, which means that Ray Tune runs that are started in remote tasks (e.g. via Ray client) cannot be gracefully interrupted. In k8s-based cloud tests that used the Ray client to kick off a Ray Tune run, this lead to test flakiness, as final experiment state could not be gracefully persisted to cloud storage. This PR adds support for SIGUSR1 in addition to SIGINT to interrupt training gracefully. |
||
|---|---|---|
| .. | ||
| api_docs | ||
| doc_code | ||
| examples | ||
| images | ||
| tutorials | ||
| faq.rst | ||
| getting-started.rst | ||
| index.rst | ||
| key-concepts.rst | ||