Kai Fricke
c77cfaa5ad
[tune] use dated experiment dir per default ( #11104 )
2020-09-30 14:43:59 -07:00
Siyuan (Ryans) Zhuang
f0dba6bd2b
Code cleanup about python3 asyncio compat ( #11134 )
...
* cleanup python3 compat and others
2020-09-30 14:22:25 -07:00
Tao Wang
0dcfa9ed6c
Add light heartbeat flag in python and use it in load metrics ( #11032 )
2020-09-30 11:39:28 -07:00
SangBin Cho
e44828d928
[Test] Make the test size bigger for tune to fix timeout. ( #11138 )
2020-09-30 11:35:31 -07:00
SangBin Cho
474af36591
[Core] Fix random bugs found. ( #11128 )
2020-09-30 00:11:57 -07:00
Kai Yang
3504391fd2
[Core] Multi-tenancy: enable multi-tenancy by default ( #10570 )
...
* Add new job in Travis to enable multi-tenancy
* fix
* Update .bazelrc
* Update .travis.yml
* fix test_job_gc_with_detached_actor
* fix test_multiple_downstream_tasks
* fix lint
* Enable multi-tenancy by default
* Kill idle workers in FIFO order
* Update test
* minor update
* Address comments
* fix some cases
* fix test_remote_cancel
* Address comments
* fix after merge
* remove kill
* fix worker_pool_test
* fix java test timeout
* fix test_two_custom_resources
* Add a delay when killing idle workers
* fix test_worker_failure
* fix test_worker_failed again
* fix DisconnectWorker
* update test_worker_failed
* Revert some python tests
* lint
* address comments
2020-09-29 23:54:53 -07:00
SangBin Cho
f54f7b23f6
[Object Spilling] Name IO workers properly. ( #11097 )
...
* Fix.
* Addressed code review.
2020-09-29 23:48:31 -07:00
Tao Wang
15ae8816f7
[GCS]Remove useless / heavy heartbeat pub ( #11132 )
2020-09-29 23:38:17 -07:00
Maksim Smolin
94494c43d6
[cli] Updates ( #10777 )
2020-09-29 16:55:07 -07:00
Ameer Haj Ali
f17ad5d060
[autoscaler] resource demand scheduler get_nodes_to_launch should return Dict not List ( #11118 )
2020-09-29 15:32:56 -07:00
Kai Fricke
10015e60fb
[tune] fix flaky callback test ( #11102 )
2020-09-29 09:44:32 -07:00
Tao Wang
1db83764bf
[GCS]Use new getting all available resources interface instead of pub-sub … ( #10914 )
...
* Use new all available resources getting interface instead of pub-sub in state.py
* add missing server handler and test cases, fix comments
* add fine grained test assert
* per comments
* involve new added function _available_resources_per_node
* change ClientID to NodeID
* fix compile
* fix client id and lint
* robust tests check
* robust tests
2020-09-29 09:41:10 -07:00
Ameer Haj Ali
0d36e4c025
[autoscaler] Support min_workers for multi node type ( #11041 )
...
* prepare for head node
* move command runner interface outside _private
* remove space
* Eric
* flake
* min_workers in multi node type
* fixing edge cases
* eric not idle
* fix target_workers to consider min_workers of node types
* idle timeout
* minor
* minor fix
* test
* lint
* eric v2
* eric 3
* min_workers constraint before bin packing
* Update resource_demand_scheduler.py
* Revert "Update resource_demand_scheduler.py"
This reverts commit 818a63a2c86d8437b3ef21c5035d701c1d1127b5.
* reducing diff
Co-authored-by: Ameer Haj Ali <ameerhajali@ameers-mbp.lan>
Co-authored-by: Alex Wu <alex@anyscale.io>
Co-authored-by: Alex Wu <itswu.alex@gmail.com>
2020-09-28 22:02:01 -07:00
Simon Mo
6cfc1a16a0
[Serve] Fix resource checking ( #11084 )
2020-09-28 18:10:43 -07:00
SangBin Cho
cef263a333
[Custom Metrics] Improve ray runtime context. ( #10981 )
...
* done.
* Addressed code review.
* Addressed code review.
2020-09-28 16:18:19 -07:00
fangfengbin
872219940b
[GCS]Fix miss PollOwnerForActorOutOfScope after gcs restarts bug ( #11054 )
...
* fix_RemoveActorFromOwner_crash_bug
* fix review comment
* fix review comment
* rm unused ut
* add testcase
* fix review comment
* rm unused import
* fix code style
* fix ut bug
Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-09-28 10:06:40 -07:00
Lingxuan Zuo
27e1f513e3
[Log] make glog flush and RAY_LOG thread-safe ( #11002 )
...
* make glog flush and RAY_LOG thread-safe
* dump error log to console
* mapping all levels to destination
* hack glog for exporting message to stdout if no base name given
* patch lint
* use stdout logger by default
* add raylet std/err pytest checker
* add worker logs file check
* fix asan check
* loop in glog enums
* fix python lint
* lint for autoindent
* fix indent lint
* make raylet.err is not empty
2020-09-28 22:15:15 +08:00
SangBin Cho
1e39c40370
[Placement Group] Capture child tasks by default. ( #11025 )
...
* In progress.
* Finished up.
* Improve comment.
* Addressed code review.
* Fix test failure.
* Fix ci failures.
* Fix CI issues.
2020-09-27 19:33:00 -07:00
Kai Fricke
e7315b0856
[tune] Callbacks for tune runs ( #11001 )
2020-09-27 16:50:07 -07:00
DK.Pino
db7097fb1f
[Refactor] Rename ClientId to NodeId ( #10992 )
...
* rename ClientId to NodeId
* format lint
* format lint
* fix conflicts
* rename new ClientId to NodeId
* update lint
* make same version of clang-format with travis ci
2020-09-27 10:24:21 -07:00
Eric Liang
b5ecdd088b
Deflake object manager test ( #11052 )
2020-09-26 18:33:02 -07:00
Richard Liaw
51038f2197
confirm ( #11049 )
...
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
2020-09-26 13:48:13 -07:00
SangBin Cho
29663d89f1
[Placement Group] Remove warning msg for placement groups. ( #11034 )
...
* Done.
* Addressed code review.
* Fixed typo.
* Addressed code review.
2020-09-25 20:53:42 -07:00
SangBin Cho
109481afd9
[Metric] custom metrics refinement ( #10861 )
...
* In progress
* In Progress.
* Addressed code review.
* Add unit tests.
* Add a simple doc.
* Fixed test failure.
* Fix all test failures from serve.
* Addressed code review.
2020-09-25 09:10:28 -07:00
Eric Liang
609c1b8acd
Start moving ray internal files to _private module ( #10994 )
2020-09-24 22:46:35 -07:00
Ameer Haj Ali
3b6fe72029
Make the command runner interface public ( #11022 )
2020-09-24 22:45:17 -07:00
Amog Kamsetty
ee85cb31a5
[Tune] Fix Memory Leak ( #10989 )
...
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-09-24 20:26:55 -07:00
Eric Liang
a26394d184
[tests] Blacklist failing win32 tests ( #10935 )
2020-09-24 20:15:28 -07:00
Simon Mo
4f6e218a3d
Add a new _available_resources_per_node for state API ( #11014 )
2020-09-24 17:25:15 -07:00
Amog Kamsetty
3716fb1ca9
[Ray SGD] Fix hotfix test ( #11015 )
2020-09-24 14:14:03 -07:00
Amog Kamsetty
07bdf062b9
[Ray SGD] [Hotfix] Worker group hotfix ( #11008 )
2020-09-24 12:21:30 -07:00
SangBin Cho
8c241d5f1d
[Core] Use node ip address properly in ray.init ( #10829 )
...
* Fix.
* Addressed code review.
* Addressed code review.
2020-09-24 11:44:52 -07:00
Ameer Haj Ali
4ac58d54d6
prepare for head node ( #10997 )
...
Co-authored-by: Ameer Haj Ali <ameerhajali@ameers-mbp.lan>
2020-09-24 11:18:38 -07:00
Kai Fricke
d9c4dea7cf
[tune] strict metric checking ( #10972 )
2020-09-24 10:00:48 -07:00
SangBin Cho
5e6b887f2d
[Placement Group] Capture Child Task Part 1 ( #10968 )
...
* In progress.
* In progers.
* Done.
* Addressed code review.
* Increase timeout to make a test less flaky.
* Addressed code review.
* Addressed code review.
2020-09-24 09:02:03 -07:00
Keqiu Hu
46a560e876
[cli][ray] ray start should error by default if there's already an instance running ( #10826 )
2020-09-24 10:29:59 -05:00
SongGuyang
f9b040db52
add log-dir to new dashboard ( #10885 )
2020-09-24 13:40:37 +08:00
Amog Kamsetty
52e1495e30
[Ray SGD] TorchTrainable pre 0.8.7 deprecation warning ( #10984 )
...
* torch trainable add pre 0.8.7 backwards compat
* raise instead
* Update python/ray/util/sgd/torch/torch_trainer.py
2020-09-23 18:19:43 -07:00
Ian Rodney
4c3f09094a
[docs] redis-port -> port ( #10937 )
2020-09-23 17:04:13 -07:00
Alex Wu
295782d411
[New Scheduler] Refactor cluster resource scheduler ( #10938 )
2020-09-23 15:46:31 -07:00
Allen
567009d5fd
[Autoscaler] Fix k8s command runner when command fails ( #10966 )
...
Co-authored-by: Allen Yin <allenyin@anyscale.io>
2020-09-23 13:25:49 -07:00
Kai Fricke
5921e87ecd
[tune] Only add new trial when there is no pending trial ( #10979 )
2020-09-23 11:08:12 -07:00
Amog Kamsetty
7dbd0ff824
fix example ( #10964 )
2020-09-23 10:33:19 -07:00
SangBin Cho
390107b6cb
[Core] Allow to pass node ip address to gcs server. ( #10946 )
...
* Allow to pass node ip address to gcs server.
* Fix.
* Addressed code review.
* Fixed an error.
* Addressed code review.
2020-09-23 01:52:26 -07:00
Lee moon soo
df4c3abe30
[autoscaler] Staroid node provider ( #10956 )
2020-09-22 21:25:29 -07:00
Simon Mo
7fbe076813
[Serve] Unwrap Flask Request from ServeHandle ( #10845 )
2020-09-22 12:44:23 -07:00
Kai Yang
864d1d2b59
[Core] Multi-tenancy: Kill idle workers in FIFO order ( #10597 )
...
* Kill idle workers in FIFO order
* Update test
* minor update
* Address comments
* fix after merge
* fix worker_pool_test
2020-09-22 10:59:11 -07:00
architkulkarni
67c653c053
[Serve] Only install dataclasses on Python 3.6 ( #10936 )
2020-09-22 09:39:39 -07:00
Kai Fricke
2d16ab2e16
[tune] Remove unnecessary wandb group parameter ( #10950 )
2020-09-22 09:36:51 -07:00
Sumanth Ratna
770c3633f0
Update max_failures kwarg docstring ( #10953 )
2020-09-22 09:02:15 -07:00
