- Move the URI reference logic from raylet to agent.
- Redefine the runtime env agent RPC to `CreateRuntimeEnvOrGet` and `DeleteRuntimeEnvIfPossible`
- More details https://github.com/ray-project/ray/issues/21695#issuecomment-1032161528
Future works
- We don't remove the `RuntimeEnvUris` from `RuntimeEnv` protobuf in current PR because gcs also uses those URIs to do GC by runtime_env_manager. We should also clear this.
- Ray client server shouldn't interact with agent directly. Or Ray client server should also decrease the reference count.
- Currently, `WorkerPool::HandleJobStarted` will be called multiple times for one job. So we should make sure this function is idempotent. Can we change this logic and make this function be called only once?
This is part of gcs ha project. This PR try to bootstrap dashboard with gcs address instead of redis.
Co-authored-by: mwtian <81660174+mwtian@users.noreply.github.com>
* Dashboard select port; Fix dashboard may hangs when exit
* Add test case
* Fix
* Fix test_stats_collector.py::test_get_all_node_details
* Refine dashboard error messages
* Refine code
* Refine code
* Show last 10 lines of dashboard log if start dashboard failed
* Fix ValueError: too many values to unpack (expected 2) when getsockname
* Fix test_multi_node_3.py::test_calling_start_ray_head may fail
* Fix Windows CI
* Disable dashboard in C++ test
* Refine code
* Fix issue 7084
Co-authored-by: 刘宝 <po.lb@antfin.com>
* In Progress.
* Done.
* Fix the issue.
* Add wait for condition because logs are not written right away now.
* debug string.
* lint.
* Fix flaky test.
* Fix issues.
* Fix test.
* lint.
* Use new dashboard if environment var RAY_USE_NEW_DASHBOARD exists; new dashboard startup
* Make fake client/build/static directory for dashboard
* Add test_dashboard.py for new dashboard
* Travis CI enable new dashboard test
* Update new dashboard
* Agent manager service
* Add agent manager
* Register agent to agent manager
* Add a new line to the end of agent_manager.cc
* Fix merge; Fix lint
* Update dashboard/agent.py
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
* Update dashboard/head.py
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
* Fix bug
* Add tests for dashboard
* Fix
* Remove const from Process::Kill() & Fix bugs
* Revert error check of execute_after
* Raise exception from DashboardAgent.run
* Add more tests.
* Fix compile on Linux
* Use dict comprehension instead of dict(generator)
* Fix lint
* Fix windows compile
* Fix lint
* Test Windows CI
* Revert "Test Windows CI"
This reverts commit 945e01051ec95cff5fcc1c0bc37045b46e7ad9a6.
* Fix ParseWindowsCommandLine bug
* Update src/ray/util/util.cc
Co-authored-by: Robert Nishihara <robertnishihara@gmail.com>
Co-authored-by: 刘宝 <po.lb@antfin.com>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Robert Nishihara <robertnishihara@gmail.com>
