Commit graph

5101 commits

Author SHA1 Message Date
SangBin Cho
22b2e51152
Fix test-multi-node (#9453) 2020-07-13 20:44:27 -05:00
Richard Liaw
a567f7977c
[tune] Put examples under proper version control (#9427)
Co-authored-by: krfricke <krfricke@users.noreply.github.com>
2020-07-13 18:01:10 -07:00
Richard Liaw
7abf7a0109
[docs] Render ActorPool documentation, etc (#9433) 2020-07-13 17:59:22 -07:00
Vasily Litvinov
6ad13e0da8
Add ability to specify SOCKS proxy for SSH connections (#8833) 2020-07-13 16:10:07 -07:00
Richard Liaw
dfe3ebe4a2
[tune] sklearn comment out (#9454) 2020-07-13 16:06:44 -07:00
Siyuan (Ryans) Zhuang
4da97a7c99
[Core] Build raylet client as an independent component (#9434) 2020-07-13 16:00:32 -07:00
mehrdadn
3d65682e62
Bazel selects compiler flags based on compiler (#9313)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-13 15:31:46 -07:00
Henk Tillman
c7714ca575
GCP authentication using oauth tokens (#9279) 2020-07-13 14:36:40 -07:00
Ian Rodney
0085cf75d0
Allow --lru-evict to be passed into ray start (#8959) 2020-07-13 14:09:39 -07:00
Amog Kamsetty
4454d05bcf
[Tune] Trainable documentation fix (#9448) 2020-07-13 13:15:01 -07:00
Hao Chen
e6225bdfa1
[GCS] Fix the bug about raylet receiving duplicate actor creation tasks (#9422) 2020-07-13 11:34:02 -07:00
mehrdadn
5291bf235b
TRAVIS_PULL_REQUEST is false for non-PRs, not empty (#9439)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-13 14:52:40 +02:00
Nicolaus93
b5a6c57295
[tune] handling nan values (#9381) 2020-07-12 17:08:36 -07:00
Tanay Wakhare
15aa08a3d1
[RLLib] WindowStat bug fix (#9213)
* WindowStat error catching, which processes NaNs properly instead of erroring. This ought to resolve issue #7910.
https://github.com/ray-project/ray/issues/7910
2020-07-12 23:01:32 +02:00
Tanay Wakhare
3536d8e4b3
Masking error. With t*valid_mask, we get the error np.inf*0 = np.inf (#9407) 2020-07-12 22:59:35 +02:00
Siyuan (Ryans) Zhuang
381c242f6b
[Core] Simplify Raylet Client (#9420) 2020-07-12 12:42:54 -07:00
Henk Tillman
8c985dc797
Update conda and ray wheel on GCP images (#9388) 2020-07-12 12:12:27 -07:00
Sven Mika
fcdf410ae1
[RLlib] Tf2.x native. (#8752) 2020-07-11 22:06:35 +02:00
mehrdadn
5c853eaa6a
Fix copy to workspace (#9400) 2020-07-11 14:27:56 +02:00
Ian Rodney
26fcda50e7
Pass run args to DockerCommandRunner (#9411) 2020-07-10 18:09:01 -07:00
Simon Mo
d4a5d09dab
[Serve] Merge router with HTTPProxy (#9225) 2020-07-10 13:52:48 -07:00
Siyuan (Ryans) Zhuang
1798deae94
[Core] Plasma RAII support (#9370) 2020-07-10 09:22:29 -07:00
SangBin Cho
d8a0d76d02
Fix macos compliation bug (#9391)
* Fix.
2020-07-10 09:18:09 -07:00
Sven Mika
14160ca58c
[RLlib] Issue #9366 (DQN w/o dueling produces invalid actions). (#9386) 2020-07-10 12:43:03 +02:00
Kai Yang
c89b59cf48
Remove the RAY_CHECK in Worker::Port() (#9348) 2020-07-10 18:06:25 +08:00
Kai Yang
a98cd0670e
[Java] Improve JNI performance when submitting and executing tasks (#9032) 2020-07-10 17:51:07 +08:00
Hao Chen
d49dadf891
Change Python's ObjectID to ObjectRef (#9353) 2020-07-10 17:49:04 +08:00
Tao Wang
6311e5a947
[HOTFIX] Fix compile direct_actor_transport_test on mac (#9403) 2020-07-10 17:19:34 +08:00
fangfengbin
35861f17a3
Fix gcs_table_storage testcase bug (#9393)
Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-07-10 16:16:28 +08:00
Hao Chen
bed1be611e
Fix flaky test_dynres.py (#9310) 2020-07-10 10:34:23 +08:00
mehrdadn
dd2cc6eb48
Update hiredis and remove Windows patches (#9289)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-09 18:45:44 -07:00
Patrick Ames
dc51b08c36
[autoscaler] Allow users to disable the cluster config cache (#8117)
* [autoscaler] Remove autoscaler config cache.

* [autoscaler] Add flag allowing users to explicitly disable the config cache.
2020-07-09 15:47:58 -07:00
Stefan Schneider
6db55ca8db
[docs][rllib] Recommended workflow for training, saving, and testing (#9319) 2020-07-09 15:47:10 -07:00
Eric Liang
09b9b81ea4
[autoscaler] Move command runners into separate file and clean up interface. (#9340)
* cleanup

* wip

* fix imports

* fix lint
2020-07-09 15:40:56 -07:00
Zhuohan Li
8a76f4cbb5
[Core] put small objects in memory store (#8972)
* remove the put in memory store

* put small objects directly in memory store

* cast data type

* fix another place that uses Put to spill to plasma store

* fix multiple tests related to memory limits

* partially fix test_metrics

* remove not functioning codes

* fix core_worker_test

* refactor put to plasma codes

* add a flag for the new feature

* add flag to more places

* do a warmup round for the plasma store

* lint

* lint again

* fix warmup store

* Update _raylet.pyx

Co-authored-by: Eric Liang <ekhliang@gmail.com>
2020-07-09 15:39:40 -07:00
Alex Wu
34b85659d4
[Core] New scheduler fixes (#9186)
* .

* test_args passes

* .

* test_basic.py::test_many_fractional_resources causes ray to hang

* test_basic.py::test_many_fractional_resources causes ray to hang

* .

* .

* useful

* test_many_fractional_resources fails instead of hanging now :)

* Passes test_fractional_resources

* .

* .

* Some cleanup

* git is hard

* cleanup

* Fixed scheduling tests

* .

* .
2020-07-09 15:37:51 -07:00
Alisa
f0a72ad985
[Core] Add placement group scheduler and some api in resource scheduler (#9039)
* Add placement group scheduler and some api of resource scheduler.
Merge fix cv hang in multithread variables race (#8984).

* change the bundle id and delete unit count in bundle

change vector<bundle_spec> to vector<shared_ptr<bundle_spec>>

Add placement group scheduler and some api of resource scheduler.
Merge fix cv hang in multithread variables race (#8984).

change the bundle id and delete unit count in bundle

remove CheckIfSchedulable()

add comments and fix the bug in resource

* fix placement group schedule

* add placement group scheduler and change some api in resource scheduler

* fix by the comments

* fix conflict

* fix lint

* fix lint

* fix bug in merge

* fix lint

Co-authored-by: Lingxuan Zuo <skyzlxuan@gmail.com>
2020-07-09 15:37:18 -07:00
Stephanie Wang
0389735d7a
[core] Pass owner address from the workers to the raylet (#9299)
* Add intended worker ID to GetObjectStatus, tests

* Remove TaskID owner_id

* lint

* Add owner address to task args

* Make TaskArg a virtual class, remove multi args

* Set owner address for task args

* merge

* Fix tests

* Add ObjectRefs to task dependency manager, pass from task spec args

* tmp

* tmp

* Fix

* Add ownership info for task arguments

* Convert WaitForDirectActorCallArgs

* lint

* build

* update

* build

* java

* Move code

* build

* Revert "Fix Google log directory again (#9063)"

This reverts commit 275da2e400.

* Fix free

* fix tests

* Fix tests

* build

* build

* fix

* Change assertion to warning to fix java
2020-07-09 14:35:54 -07:00
mehrdadn
4687b807c4
Combine different severities into the same log files (#9230)
* Combine different severities into the same log files

Co-authored-by: Mehrdad <noreply@github.com>
2020-07-09 14:14:28 -07:00
Richard Liaw
b5103bacd1
[tune] Fix github readme (#9365)
Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>
2020-07-09 12:37:24 -07:00
mehrdadn
b6c11f3dd7
Bazel move more shell to native rules (#9314)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-09 09:02:35 -07:00
Siyuan (Ryans) Zhuang
7e1326c0f6
[Core] Enhance common client connection (#9367)
* enhance client connection

* add write buffer async

* read message

* add test
2020-07-09 08:59:37 -07:00
fangfengbin
b97b474ae9
Fix GcsActorManagerTest multi-thread bug (#9358) 2020-07-09 20:03:46 +08:00
fangfengbin
62fe403a57
Fix the bug that PollOwnerForActorOutOfScope's reply may be missing (#9330) 2020-07-09 17:33:52 +08:00
Sven Mika
01125b8fcf
[RLlib] DQN rainbow eager-mode (keras style NoisyLayer) (preparation for native tf2.x support). (#9304) 2020-07-09 10:44:10 +02:00
Ruifang Chen
c37d30a079
Remove unnecessary asan config (#9350)
Co-authored-by: ruifang.crf <ruifang.crf@alibaba-inc.com>
2020-07-09 14:50:54 +08:00
Lingxuan Zuo
0562236533
[Stats] Make aggeration window and report interval configurable. (#9235) 2020-07-09 12:44:38 +08:00
Kai Yang
fa1eadd61f
fix java worker log dir (#9352) 2020-07-09 10:11:03 +08:00
SangBin Cho
9f8ff2e3b1
[Core] GCS Actor Management Race Condition (#9215)
* GCS Actor management on by default.

* Fix travis config.

* Change condition.

* Finish the initial race condition fix.

* Lint.

* Refine the codebase.

* Finish the initial version

* Improve logic.

* Remove unnecessary log messages.

* Address code review.

* Add tests

* Revert the second race condition that doesn't happen anymore, handle some edge cases. add tests.

* Address the second race condition found.

* Addressed code review.

* Addressed code review.

* Run a new unit test only when gcs actor management is on.
2020-07-08 10:56:52 -07:00
SangBin Cho
c5aa1eb042
[GCS Actor Management] Fix a dashboard doesn't show the correct actor state. (#9275)
* Fix a bug.

* Address code review.

* Remove an unnecessary comment.
2020-07-08 10:55:47 -07:00