Guyang Song
f65971756d
[dashboard agent] Catch agent port conflict ( #23024 )
2022-03-15 16:09:15 +08:00
Yi Cheng
11bbf00338
[dashboard] Remove redis in dashboard ( #22788 )
...
As we are turning redisless ray by default, dashboard doesn't need to talk with redis anymore. Instead it should talk with gcs and gcs can talk with redis.
2022-03-04 12:32:17 -08:00
Guyang Song
902243fb03
[runtime env] support raylet sharing fate with agent ( #22382 )
...
- Remove the agent restart feature.
- Raylet shares fate with agent to make the failover logic easier.
Refer to issue https://github.com/ray-project/ray/issues/21695#issuecomment-1032161528
2022-02-21 18:16:21 +08:00
SangBin Cho
20ab9188c6
[Ray Usage Stats] Record cluster metadata + Refactoring. ( #22170 )
...
This is the first PR to implement usage stats on Ray. Please refer to the file `usage_lib.py` for more details.
The full specification is here https://docs.google.com/document/d/1ZT-l9YbGHh-iWRUC91jS-ssQ5Qe2UQ43Lsoc1edCalc/edit#heading=h.17dss3b9evbj .
You can see the full PR for phase 1 from here; https://github.com/rkooo567/ray/pull/108/files .
The PR is doing some basic refactoring + adding cluster metadata to GCS instead of the version numbers.
After this PR, we will add code to enable usage report "off by default".
2022-02-08 22:12:36 -08:00
SangBin Cho
3566cfd279
[Dashboard] Enable dashboard in the minimal ray installation ( #21896 )
...
This is the last PR to enable dashboard in the minimal ray installation.
Look https://docs.google.com/document/d/12qP3x5uaqZSKS-A_kK0ylPOp0E02_l-deAbmm8YtdFw/edit# for more details;
2022-01-31 22:34:40 -08:00
Balaji Veeramani
7f1bacc7dc
[CI] Format Python code with Black ( #21975 )
...
See #21316 and #21311 for the motivation behind these changes.
2022-01-29 18:41:57 -08:00
Alex Wu
7a45f60dbc
[autoscaler] Fix ray.autoscaler.sdk import issue ( #21795 )
...
This PR moves the sdk to its own folder, then includes everything in `import ray.autoscaler.sdk` in ray's import path.
Note: that there were circular dependencies in naively doing this because the ray core now uses constants that were defined in the autoscaler for internal kv operations (and the autoscaler similarly calls into the ray core). The solution was to move those internal kv keys into ray core constants so the imports flow (more) one way.
Co-authored-by: Alex Wu <alex@anyscale.com>
2022-01-25 14:43:24 -08:00
SangBin Cho
2010f13175
Fix dashboard test bug ( #21742 )
...
Currently `wait_until_succeeded_without_exception` is used in the dashboard, and it returns True/False. Unfortunately, there are lots of code that doesn't assert on this method (which means things are not actually tested).
2022-01-24 11:38:51 -06:00
SangBin Cho
1ae14ec513
[Dashboard] Make dashboard / agent work in minimal ray installation 1/3. ( #21774 )
...
This is the doc that explains how to achieve this: https://docs.google.com/document/d/12qP3x5uaqZSKS-A_kK0ylPOp0E02_l-deAbmm8YtdFw/edit?usp=sharing
The fully working e2e prototype is here (it passes all tests): cdad913883https://user-images.githubusercontent.com/18510752/150528494-c3cdedf4-3a66-4557-b540-61436b1dbab6.png ">
2022-01-23 21:11:32 -08:00
Yi Cheng
6dccfbffa9
Revert "Revert "[gcs] turn on grpc pubsub by default"" ( #21585 )
...
Reverts ray-project/ray#21584 and turn the flag off
2022-01-13 16:12:03 -08:00
Yi Cheng
bc696212d2
Revert "[gcs] turn on grpc pubsub by default" ( #21584 )
...
test-reconnect seems flaky.
Reverts ray-project/ray#21513
2022-01-13 12:34:02 -08:00
Yi Cheng
6194783312
[gcs] turn on grpc pubsub by default ( #21513 )
...
Turn on grpc pubsub by default. This PR also fixed several tests which are failed before.
Co-authored-by: Mingwei Tian <mwtian@anyscale.com>
2022-01-12 22:13:03 -08:00
Yi Cheng
09421a4ca6
[2/gcs] Bootstrap dashboard for gcs ha ( #21179 )
...
This is part of gcs ha project. This PR try to bootstrap dashboard with gcs address instead of redis.
Co-authored-by: mwtian <81660174+mwtian@users.noreply.github.com>
2021-12-21 16:58:03 -08:00
Antoni Baum
20fc9f907d
[CI] Fix tune dashboard, increase timeout for test_commands ( #20453 )
2021-11-16 17:52:17 -08:00
Simon Mo
32a4f48aa2
[CI] Don't test tune dashboard ( #20452 )
2021-11-16 15:07:56 -08:00
Yi Cheng
e54d3117a4
[gcs] Update all redis kv usage in python except function table ( #20014 )
...
## Why are these changes needed?
This is part of redis removal project. In this PR all direct usage of redis got removed except function table.
Function table will be migrated in the next PR
## Related issue number
#19443
2021-11-10 20:24:53 -08:00
Guyang Song
119318932a
remove the env config 'RAY_DASHBOARD_MODULE_EVENT' ( #19629 )
2021-10-28 16:51:59 +09:00
Matti Picus
f372bb07aa
Enable dashboard on Windows ( #19319 )
2021-10-14 14:42:22 -07:00
SangBin Cho
7fcf1bf57e
[Dashboard] Refine the dashboard restart logic. ( #18973 )
...
* in progress
* Refine the dashboard agent retry logic
* refine
* done
* lint
2021-10-04 05:01:51 -07:00
Eric Liang
11a2dfcaab
Improve unschedulable task warning messages by integrating with the autoscaler ( #18724 )
2021-09-24 12:19:58 -07:00
Edward Oakes
7736cdd91d
[dashboard] Rename "new_dashboard" -> "dashboard" ( #18214 )
2021-09-15 11:17:15 -05:00
Simon Mo
e61160d514
[Dashboard] Move gcs health check to a separate thread to avoid crashing due to excessive CPU usage. ( #18236 )
2021-09-03 14:23:56 -07:00
Clark Zinzow
d958457d07
[Core] Second pass at privatizing APIs. ( #17885 )
...
* gcs_utils
* resource_spec
* profiling
* ray_perf and ray_cluster_perf
* test_utils
2021-08-18 20:56:33 -07:00
fyrestone
57b9b1bb0f
[Dashboard] Use a dedicated RPC to check the GCS is alive ( #16330 )
...
* Dashboard check gcs is alive
* Fix dashboard hangs at exit
* ray health-check call GCS CheckAlive
* Minor fixes
Co-authored-by: 刘宝 <po.lb@antfin.com>
2021-07-27 14:05:44 +08:00
Amog Kamsetty
8dfd471823
Revert "Revert "[Dashboard][event] Basic event module ( #16985 )" ( #17068 )" ( #17107 )
...
This reverts commit c17e171f92
2021-07-18 12:59:04 +08:00
Amog Kamsetty
c17e171f92
Revert "[Dashboard][event] Basic event module ( #16985 )" ( #17068 )
...
This reverts commit f1faa79a04
2021-07-13 23:18:43 -07:00
fyrestone
f1faa79a04
[Dashboard][event] Basic event module ( #16985 )
...
* Basic event module
* Fix comments
* Set the SCAN_EVENT_DIR_INTERVAL_SECONDS defaults to 2
* Fix lint
* Fix lint
* Clean code
* Try to fix flaky
* Fix test
* Disable event module by default
* Make monitor events task cancellable
* Fix error
Co-authored-by: 刘宝 <po.lb@antfin.com>
2021-07-13 19:08:39 -07:00
Amog Kamsetty
a14342ce6f
Revert "[Dashboard][event] Basic event module ( #16698 )" ( #17004 )
...
This reverts commit 66ea099897
2021-07-12 11:22:46 -07:00
fyrestone
66ea099897
[Dashboard][event] Basic event module ( #16698 )
...
* Basic event module
* Fix comments
* Set the SCAN_EVENT_DIR_INTERVAL_SECONDS defaults to 2
* Fix lint
* Fix lint
* Clean code
* Try to fix flaky
* Fix test
* Disable event module by default
Co-authored-by: 刘宝 <po.lb@antfin.com>
2021-07-09 10:25:30 -07:00
fyrestone
4ca316a0f4
Move test_snapshot from test_dashboard.py to modules/snapshot/tests/test_snapshot.py ( #16306 )
...
Co-authored-by: 刘宝 <po.lb@antfin.com>
2021-06-08 10:26:03 -07:00
fyrestone
dfadf33a94
[Dashboard] Reorganize dashboard modules - node ( #16217 )
2021-06-07 19:50:46 -07:00
Alex Wu
cd2fc7792f
[dashboard] Snapshot of cluster state ( #15868 )
2021-05-20 08:10:32 -07:00
Amog Kamsetty
ebc44c3d76
[CI] Upgrade flake8 to 3.9.1 ( #15527 )
...
* formatting
* format util
* format release
* format rllib/agents
* format rllib/env
* format rllib/execution
* format rllib/evaluation
* format rllib/examples
* format rllib/policy
* format rllib utils and tests
* format streaming
* more formatting
* update requirements files
* fix rllib type checking
* updates
* update
* fix circular import
* Update python/ray/tests/test_runtime_env.py
* noqa
2021-05-03 14:23:28 -07:00
fyrestone
43de7f48a7
Fix reported dashboard ip when using 0.0.0.0 ( #15506 )
2021-04-27 23:48:22 +08:00
Kathryn Zhou
456d9aab47
Add Cypress test for Ray Dashboard ( #14253 )
2021-02-24 20:41:52 -08:00
fyrestone
5e76a51d56
[Dashboard] Select port in dashboard ( #13763 )
...
* Dashboard select port; Fix dashboard may hangs when exit
* Add test case
* Fix
* Fix test_stats_collector.py::test_get_all_node_details
* Refine dashboard error messages
* Refine code
* Refine code
* Show last 10 lines of dashboard log if start dashboard failed
* Fix ValueError: too many values to unpack (expected 2) when getsockname
* Fix test_multi_node_3.py::test_calling_start_ray_head may fail
* Fix Windows CI
* Disable dashboard in C++ test
* Refine code
* Fix issue 7084
Co-authored-by: 刘宝 <po.lb@antfin.com>
2021-02-23 16:27:48 -08:00
Xianyang Liu
4ecd29ea2b
[dashboard] Fixes dashboard issues when environments have set http_proxy ( #12598 )
...
* fixes ray start with http_proxy
* format
* fixes
* fixes
* increase timeout
* address comments
2021-01-21 20:10:01 -08:00
Simon Mo
dac8b3d58a
[CI] Enable Dashboard tests for master ( #13425 )
2021-01-15 09:43:34 -08:00
Alex Wu
8df94e33e0
[Autoscaler] New output log format ( #12772 )
2020-12-23 12:02:55 -08:00
Eric Liang
03a5b90ed6
Revert "Revert "Increase the number of unique bits for actors to avoi… ( #12990 )
2020-12-21 15:16:42 -08:00
Eric Liang
5d987f5988
Revert "Increase the number of unique bits for actors to avoid handle collisions ( #12894 )" ( #12988 )
...
This reverts commit 3e492a79ec
2020-12-18 23:51:44 -08:00
Eric Liang
3e492a79ec
Increase the number of unique bits for actors to avoid handle collisions ( #12894 )
2020-12-18 15:59:03 -08:00
Edward Oakes
261b2f9053
Check for raylet PID as ppid in dashboard agent fate-sharing ( #12867 )
2020-12-15 12:13:11 -06:00
Max Fitton
d0813c1c58
[Dashboard] Add dashboard multi-node churn test ( #11768 )
2020-12-14 17:03:33 -06:00
Stephanie Wang
a776209aec
Revert "Fix dashboard agent check ppid is raylet pid ( #12256 )" ( #12729 )
...
This reverts commit 3ce9286977
2020-12-09 17:20:38 -05:00
fyrestone
3ce9286977
Fix dashboard agent check ppid is raylet pid ( #12256 )
...
* Dashboard agent check ppid is raylet pid
* Improve implementation
* Refine code
* Make the RAY_NODE_PID environment required for dashboard agent
Co-authored-by: 刘宝 <po.lb@antfin.com>
2020-12-09 09:12:34 -05:00
Max Fitton
2708b3abbc
[Dashboard][Bug] Fix duplicate node total rows in dashboard ( #12410 )
...
* Fix duplicate node total rows in dashboard by changing the react key of the NodeTotalRow component from the node IP to the node ID (node IP can be duplicated in the case of docker).
* simplify a piece of test code and fix a flaky time out
* lint
2020-11-30 18:43:09 -08:00
SangBin Cho
753cda2f28
[Dashboard] Delete old dashboard ( #12144 )
...
* Delete old dashboard from repo.
* Delete old dashboard from repo. 2
2020-11-25 11:31:02 -08:00
fyrestone
05ad4c7499
[Dashboard] Optimize dashboard datacenter ( #11391 )
...
* Optimize dashboard datacenter
* Fix tests
* Fix tests
* Fix
* Fix CI
* python/build-wheel-macos.sh
Co-authored-by: 刘宝 <po.lb@antfin.com>
Co-authored-by: Max Fitton <maxfitton@anyscale.com>
2020-10-27 23:49:31 -07:00
Max Fitton
caf3b04b27
[Dashboard] Turn on new dashboard by default pt 2 ( #11510 )
2020-10-23 15:52:14 -05:00
