Commit graph

6176 commits

Author SHA1 Message Date
Philsik Chang
2b26d2ca1b
[rllib] Fix for Torch checkpoint taken on GPU fails to deserialize on CPU (#11071) (#11208) 2020-10-05 22:01:55 -07:00
Alex Wu
dc7c2a70b8
[Core] Report worker backlog in GCS heartbeat (#11039) 2020-10-05 22:00:44 -07:00
Robert Nishihara
0d02aa10b2
Add Slack badge to README (#11215) 2020-10-05 19:39:23 -07:00
Ameer Haj Ali
518876ef20
Remove test_autoscaler_yaml from windows tests (#11197)
Co-authored-by: Ameer Haj Ali <ameerhajali@ameers-mbp.lan>
Co-authored-by: Alex Wu <alex@anyscale.io>
Co-authored-by: Alex Wu <itswu.alex@gmail.com>
Co-authored-by: Eric Liang <ekhliang@gmail.com>
2020-10-05 17:07:00 -07:00
Ian Rodney
ab30940e50
Use 'mkdir -p' in manylinux (#11219) 2020-10-05 16:18:18 -07:00
Sumanth Ratna
0630242522
[Metrics] Use isinstance for type-checking keys of tags for metrics (#11202)
* Use isinstance instead of type for type-checking keys of tags for metrics

* Fix lint error
2020-10-05 14:24:22 -07:00
Gekho457
aaa8147acb
[CI] Add mypy to ci (#11117)
* Add mypy check to format.sh

* format_all() runs mypy check only on specified files
2020-10-05 11:20:45 -07:00
desktable
56b56cf7a1
[RLlib] Add RecSim environment wrapper (#11205) 2020-10-05 09:05:02 +02:00
SangBin Cho
80cc161f3e
[Placement Group] Report placement group load through heartbeat. (#11129)
* In progress.

* Fix a minor issue.

* Removed unnecessary comments.

* Addressed code review.

* Fix build failure.

* remove stray logs.

* Move global state to a med size test to avoid windows CI breakage.
2020-10-04 16:47:22 -07:00
Ian Rodney
035a4ad0a2
[autoscaler] Update Autoscaler SDK (#11185)
* small sdk addition

* create RAY_AUTOSCALER_CONSTANTS

* better name

* move to private

* better naming
2020-10-03 13:18:01 -07:00
fangfengbin
1244dafad3
[GCS]Optimization: Clear task_spec of destroyed actors (#11149)
* Clear task_spec of destroyed actors

* fix commnet

* disable ut

* fix windows compile bug

* fix ut bug

Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-10-03 00:00:41 -07:00
Amog Kamsetty
6325a973a2
[Ray SGD] Support num_steps continue training (#11142) 2020-10-02 23:43:26 -07:00
Richard Liaw
90e0054b60
[Doc] Disable tune-sklearn test until tests are fixed. (#11187)
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
2020-10-02 21:52:26 -07:00
Ameer Haj Ali
6b86d4d280
Automatically detect CPU, GPU, accelerator_type for AWS (#11147) 2020-10-02 21:16:43 -07:00
SangBin Cho
6974cea0cd
[Core] Use optional return instead of nullptr for the GetNode method. 2020-10-02 20:54:26 -07:00
Sumanth Ratna
6288d954b5
[tune] Use isinstance instead of type in PBT (#11175) 2020-10-02 18:32:14 -07:00
Max Fitton
ff6d412ad9
[Dashboard] Add API support for the logical view and machine view in new backend (#11012)
* Add API support for the logical view and machine view, which lean on datacenter in common.

* Update dashboard/datacenter.py

Co-authored-by: fyrestone <fyrestone@outlook.com>

* Update dashboard/modules/logical_view/logical_view_head.py

Co-authored-by: fyrestone <fyrestone@outlook.com>

* Address PR comments

* lint

* Add dashboard tests to CI build

* Fix integration issues

* lint

Co-authored-by: Max Fitton <max@semprehealth.com>
Co-authored-by: fyrestone <fyrestone@outlook.com>
2020-10-02 17:58:44 -07:00
SangBin Cho
ee71fecd9e
[Test] Make test_serialization a medium size test. (#11183) 2020-10-02 17:53:46 -07:00
Eric Liang
546af2bd81
Fix windows build (#11184) 2020-10-02 17:44:24 -07:00
Stephanie Wang
ada58abcd9
[Object spilling] Update object directory and reload spilled objects automatically (#11021)
* Fix pytest...

* Release objects that have been spilled

* GCS object table interface refactor

* Add spilled URL to object location info

* refactor to include spilled URL in notifications

* improve tests

* Add spilled URL to object directory results

* Remove force restore call

* Merge spilled URL and location

* fix

* CI

* build

* osx

* Fix multitenancy issues

* Skip windows tests
2020-10-02 15:52:42 -07:00
Sven Mika
c17169dc11
[RLlib] Fix all example scripts to run on GPUs. (#11105) 2020-10-02 23:07:44 +02:00
Max Fitton
5a42ed1848
[Dashboard] Add support for new backend to existing front-end (#11013)
* Trying to commit on top of old code again

* address comment

Co-authored-by: Max Fitton <max@semprehealth.com>
2020-10-02 12:46:47 -07:00
Max Fitton
6ed8459f25
[Dashboard] Add tune API to support tune tab in new backend (#11009)
* Add tune API to support tune tab in new backend

* Make requested changes

Co-authored-by: Max Fitton <max@semprehealth.com>
2020-10-02 11:48:48 -07:00
Michael Luo
47b499d899
Cartpole MAML + Discrete (#11028) 2020-10-02 12:56:34 +02:00
fangfengbin
180c259702
[GCS]Remove unused api(ServiceBasedActorInfoAccessor::AsyncRegister/ServiceBasedActorInfoAccessor::AsyncUpdate) (#11099)
* remove unused gcs api

* fix ut bug

Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-10-02 00:54:28 -07:00
Max Fitton
9a6d01ebf9
[Dashboard] Add utility functions for actor and memory APIs (#11011)
* Add actor and memory utility functions needed by upcoming logical view and memory view APIs

* Add a method to allow printing Dict custom class and add support for hot-reloading local dev environment.

* Address PR comments

* Add unit tests from test metrics to branch for new memory_utils module

* Add note about sorting / grouping

* lint

Co-authored-by: Max Fitton <max@semprehealth.com>
2020-10-01 23:48:03 -07:00
Amog Kamsetty
874da9a25f
[Ray SGD] LightningModule integration + MNIST Example (#11042)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-10-01 20:07:32 -07:00
Richard Liaw
9dc7b7b11d
[tune] enable ax test (#11165)
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
2020-10-01 17:25:29 -07:00
Richard Liaw
f60d77d78e
[tune] Fix GPU thread leakage 2020-10-01 17:07:13 -07:00
Ian Rodney
0d5b09f426
[Docker] Automagically add "runtime=nvidia" (#11125) 2020-10-01 17:04:19 -07:00
Kai Fricke
681c24754a
[tune] Example for using huggingface hyperparamer_search API (#11158) 2020-10-01 16:00:57 -07:00
Alex Wu
a866be381c
[New Scheduler] Heartbeat (#11024)
* .

* refactor

* .

* .

* done?

* .

* .

* .

* lint

* no light heartbeat, no tests, fields 2,3

* .

* manually clang format :(

* .

* .

* test

* .

* .

* task manager heartbeat

* lint

* .

* add reminder

* CR

* CR

* cleanup

* CR

* comment

* lint

* .
2020-10-01 15:54:53 -07:00
Siyuan (Ryans) Zhuang
1ad52bdfbc
Remove the stale directory (pyarrow_files) (#11163) 2020-10-01 14:52:04 -07:00
Kai Fricke
bdf647c4ec
[tune] docker syncer (#11035)
* Add DockerSyncer

* Add docs

* Update python/ray/tune/integration/docker.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Updated docs

* fix dir

* Added docker integration test

* added docker integration test to bazel build

* Use sdk.rsync API

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-10-01 11:59:23 -07:00
Sumanth Ratna
98ebf8e2d8
[tune][docs] fix typo in Tune FAQ (#11161)
* Fix typo in tune FAQ (used to use)

* Update doc/source/tune/_tutorials/_faq.rst

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-10-01 11:20:41 -07:00
Richard Liaw
9070ed3e44
[hotfix] revert ax (#11159)
This reverts commit de3c606b20.
2020-10-01 10:05:03 -07:00
Richard Liaw
b969e8cf59
[tune][hotfix] comment out test (#11140) 2020-10-01 10:04:40 -07:00
Kai Fricke
de3c606b20
[tune] Use latest Ax release (#11155)
* Use latest Ax release

* specify egg

* Install latest Ax only for py3.7
2020-10-01 08:51:15 -07:00
Sven Mika
36bda8432b
[RLlib] Trajectory view API: Simple List Collector (on by default for PPO); LSTM-agnostic (#11056) 2020-10-01 16:57:10 +02:00
Richard Liaw
0d93b1de93
[autoscaler] rsync to node (#11122) 2020-10-01 01:42:49 -07:00
Eric Liang
8f73014291
[autoscaler] Make it more clear NodeProvider and CommandRunner are private APIs (#11086) 2020-09-30 16:24:37 -07:00
Simon Mo
a518fb1031
[Doc] Use different pygments_style (#11143) 2020-09-30 16:16:55 -07:00
Eric Liang
090002b0ce
Blocking asyncio get warning should be logger.warn not logger.debug (#11141) 2020-09-30 16:16:35 -07:00
Benjamin Black
4445f32798
[rllib] Fixed pettingzoo wrapper (#11060)
* fixed PettingZooEnv, relevant docs, examples

* fixed linting error

* pettingzoo wrapper fix

* fixed linting issue
2020-09-30 15:34:47 -07:00
Kai Fricke
c77cfaa5ad
[tune] use dated experiment dir per default (#11104) 2020-09-30 14:43:59 -07:00
Siyuan (Ryans) Zhuang
f0dba6bd2b
Code cleanup about python3 asyncio compat (#11134)
* cleanup python3 compat and others
2020-09-30 14:22:25 -07:00
Tao Wang
0dcfa9ed6c
Add light heartbeat flag in python and use it in load metrics (#11032) 2020-09-30 11:39:28 -07:00
SangBin Cho
e44828d928
[Test] Make the test size bigger for tune to fix timeout. (#11138) 2020-09-30 11:35:31 -07:00
fangfengbin
138d6cced9
[GCS]Optimizing actor info query interface (#11067)
* add part code

* add part code

* fix review comment

* fix review comment

* fix review comment

* fix crash bug

* fix ut bug

* fix bug

* fix ut bug

Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-09-30 11:34:42 -07:00
SangBin Cho
474af36591
[Core] Fix random bugs found. (#11128) 2020-09-30 00:11:57 -07:00