Commit graph

5055 commits

Author SHA1 Message Date
Lingxuan Zuo
0562236533
[Stats] Make aggeration window and report interval configurable. (#9235) 2020-07-09 12:44:38 +08:00
Kai Yang
fa1eadd61f
fix java worker log dir (#9352) 2020-07-09 10:11:03 +08:00
SangBin Cho
9f8ff2e3b1
[Core] GCS Actor Management Race Condition (#9215)
* GCS Actor management on by default.

* Fix travis config.

* Change condition.

* Finish the initial race condition fix.

* Lint.

* Refine the codebase.

* Finish the initial version

* Improve logic.

* Remove unnecessary log messages.

* Address code review.

* Add tests

* Revert the second race condition that doesn't happen anymore, handle some edge cases. add tests.

* Address the second race condition found.

* Addressed code review.

* Addressed code review.

* Run a new unit test only when gcs actor management is on.
2020-07-08 10:56:52 -07:00
SangBin Cho
c5aa1eb042
[GCS Actor Management] Fix a dashboard doesn't show the correct actor state. (#9275)
* Fix a bug.

* Address code review.

* Remove an unnecessary comment.
2020-07-08 10:55:47 -07:00
mehrdadn
05d4cced4e
[CI] GHA path with spaces (#9356)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-08 10:44:37 -07:00
Ian Rodney
9172f8c3a6
[core] Store Internal Config in GCS (#8921) 2020-07-08 11:22:08 -05:00
Sven Mika
4da0e542d5
[RLlib] DDPG and SAC eager support (preparation for tf2.x) (#9204) 2020-07-08 16:12:20 +02:00
ZhuSenlin
42f8f16c04
[GCS] Fix detached actor with empty name (#9283) 2020-07-08 12:21:24 +08:00
Tao Wang
f395e48031
[GCS] Add integration test for actor info cleaning (#8900) 2020-07-08 10:25:08 +08:00
Max Fitton
ca54da66b6
Worker <> GPU Mapping Information in Dashboard (#9163) 2020-07-07 18:58:22 -05:00
yncxcw
4ba4110dec
[Core] Make worker_register_timeout_seconds configurable (#9221) 2020-07-07 18:27:15 -05:00
SangBin Cho
8f19f1eafb
[Core] Actor handle refactoring (#8895)
* Marking needed changes.

* Resolve basic dependencies.

* In progress.

* linting.

* In progress 2.

* Linting.

* Refactor done. Cleanup needed.

* Linting.

* Recover kill actor in core worker because it is used inside raylet

* Cleanup.

* Use unique pointer instead. Unit tests are broken now.

* Fix the upstream change.

* Addressed code review 1.

* Lint.

* Addressed code review 2.

* Fix weird github history.

* Lint.

* Linting using clang 7.0.

* Use a better check message.

* Revert cpp stuff.

* Fix weird linting errors.

* Manuall fix all lint issues.

* Update a newline.

* Refactor some interface.

* Addressed all code review.

* Addressed code review
2020-07-07 11:11:41 -07:00
Ian Rodney
f69d8b951e
[core] Disable flaky cancellation test (#9325) 2020-07-07 09:52:35 -07:00
Ian Rodney
a1e14380ce
[core] Switch Async Callback to C++ [WIP] (#9228)
Co-authored-by: simon-mo <simon.mo@hey.com>
2020-07-07 09:47:25 -07:00
fangfengbin
8391f66086
Fix gcs actor manager destroy actor crash bug (#9329) 2020-07-07 21:12:30 +08:00
Ian Rodney
079c1eaa5c
[autoscaler] Require keyword arguments (#9256) 2020-07-06 22:12:59 -07:00
Benjamin Black
1425cdf834
Pettingzoo environment support (#9271)
* added pettingzoo wrapper env and example

* added docs, examples for pettingzoo env support

* fixed pettingzoo env flake8, added test

* fixed pettingzoo env import

* fixed pettingzoo env import

* fixed pettingzoo import issue

* fixed pettingzoo test

* fixed linting problem

* fixed bad quotes

* future proofed pettingzoo dependency

* fixed ray init in pettingzoo env

* lint

* manual lint

Co-authored-by: Eric Liang <ekhliang@gmail.com>
2020-07-06 21:32:26 -07:00
Stephanie Wang
b42d6a1ddc
[core] Refactor task arguments and attach owner address (#9152)
* Add intended worker ID to GetObjectStatus, tests

* Remove TaskID owner_id

* lint

* Add owner address to task args

* Make TaskArg a virtual class, remove multi args

* Set owner address for task args

* merge

* Fix tests

* Fix

* build

* update

* build

* java

* Move code

* build

* Revert "Fix Google log directory again (#9063)"

This reverts commit 275da2e400.

* Fix free

* x

* build

* Fix java

* Revert "Revert "Fix Google log directory again (#9063)""

This reverts commit 4a326fcb148ca09a35bc7de11d89df10edbb56e7.

* lint
2020-07-06 21:25:14 -07:00
Ian Rodney
6fecd3cfce
[autoscaler] Run initialization_commands without a persistent connection (#9020)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
2020-07-06 16:34:59 -07:00
Richard Liaw
139d21e068
[tune] Docs for tune-sklearn (#9129)
Co-authored-by: krfricke <krfricke@users.noreply.github.com>
2020-07-06 15:35:10 -07:00
SangBin Cho
557da7044f
Fix flaky test that says ray.init is called twice. (#9234) 2020-07-06 15:19:00 -07:00
Ian Rodney
c79a49488c
[Doc] Fixing Wheel Build Command (#8789) 2020-07-06 11:17:59 -07:00
Piotr Januszewski
155cc81e40
Clarify training intensity configuration docstring (#9244) (#9306) 2020-07-05 20:07:27 -07:00
ChenZhilei
6f3d993681
GCS server use worker table to handle RegisterWorker instead of redis accessor (#9168) 2020-07-06 10:37:25 +08:00
Ian Rodney
dcf989292e
[docker] Update base image (#9281) 2020-07-05 17:31:59 -07:00
Sven Mika
f43d934817
[RLlib] Type annotations for policy. (#9248) 2020-07-05 13:09:51 +02:00
Richard Liaw
b71c912da7
[tune] Fix up examples (#9201) 2020-07-05 01:16:20 -07:00
Tao Wang
f7ac495a68
[Core] Use map instead of list to represent resources in heartbeat message (#9294) 2020-07-05 10:59:25 +08:00
mehrdadn
28d18ec836
[windows] Improve GPU detection (#9300)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-04 13:20:23 -07:00
David Fidalgo
c0ba337fe0
[tune] Add np.bool8 and np.int to allowed HPARAMS types (#9297) 2020-07-03 18:34:45 -07:00
Michael Luo
851d02463b
[Doc] RLlib Algorithms Documentation: MAML + PyTorch MAML (#9189) 2020-07-03 11:05:15 -07:00
Lingxuan Zuo
7a2d7964d8
Bool config string stream must be in true or 1 (#9292) 2020-07-03 20:37:19 +08:00
ZhuSenlin
726c4cd6d4
fix actor accessor initialize inside redis gcs client (#9284)
Co-authored-by: senlin.zsl <senlin.zsl@antfin.com>
2020-07-03 15:48:32 +08:00
mehrdadn
65f89753b0
Move full build to install_ray() (#9276)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-02 21:06:31 -07:00
Lixin Wei
aea3d53545
[Streaming] Supports multiple downstream collector (#9240) 2020-07-03 11:05:07 +08:00
Eric Liang
4b62a888cc
[rllib] Remove deprecated policy optimizer package. (#9262) 2020-07-02 14:39:40 -07:00
mehrdadn
7135cb2aec
Fix .exe file extensions (#9197)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-02 15:29:34 -05:00
Simon Mo
a25472c657
[Serve] Fix prometheus exporter label override (#9227) 2020-07-02 11:40:14 -07:00
Ian Rodney
f1173d55e0
[autoscaler] Fix cluster_shutdown using wrong args (#9252) 2020-07-02 09:34:50 -07:00
mehrdadn
29acf272b7
Build with Visual C++ (#9190)
Co-authored-by: Mehrdad <noreply@github.com>
Co-authored-by: Simon Mo <xmo@berkeley.edu>
2020-07-02 09:34:24 -07:00
SangBin Cho
bfb8a28a3c
Revert "Support accessing underlying attributes in RayTaskErrors (#9210)" (#9263)
This reverts commit 21cad5250a.
2020-07-02 08:35:10 -05:00
Sven Mika
5b2a97597b
[RLlib] Retire try_import_tree (should be installed along with other requirements). (#9211)
- Retire try_import_tree.
- Stabilize test_supported_multi_agent.py.
2020-07-02 13:06:34 +02:00
Sven Mika
c4ccbfdfa9
[RLlib] tf-eager support for ES and ARS (tf2.x preparation). (#9207) 2020-07-02 13:03:10 +02:00
Lingxuan Zuo
8a1cc7f8f9
Reset bool config by checking type first (#9247) 2020-07-02 18:32:41 +08:00
fangfengbin
8fcfcc4100
GCS server error handling for actor creation (#8899) 2020-07-02 16:27:32 +08:00
fangfengbin
a7a7bef622
Fix Java test crash due to incorrect destruction order (#9236) 2020-07-02 15:44:08 +08:00
Tao Wang
d35776e92d
[Core] Do not heartbeat resources if no update (#9182) 2020-07-02 13:56:49 +08:00
Eric Liang
82a5ed3fb5
[autoscaler] Instance type assert fix for non-AWS node providers (#9223) 2020-07-01 17:54:20 -07:00
Andrija Djurisic
e0d8b58969
Fix Azure example-gpu.yml (#9242)
`imageSku` should be string ... otherwise this is the error it shows up:
```
msrestazure.azure_exceptions.CloudError: Azure Error: InvalidTemplate
Message: Deployment template validation failed: 'Template parameter JToken type is not valid. Expected 'String, Uri'. Actual 'Integer'. Please see https://aka.ms/resource-manager-parameter-files for usage details.'.
Additional Information:
	Type: TemplateViolation
	Info: {
	    "lineNumber": 1,
	    "linePosition": 673,
	    "path": "properties.template.parameters.imageSku"
	}

```
2020-07-01 12:02:34 -07:00
Richard Liaw
d35f0e40d0
[tune] Use public methods for trainable (#9184) 2020-07-01 11:00:00 -07:00