Commit graph

7453 commits

Author SHA1 Message Date
Sven Mika
732197e23a
[RLlib] Multi-GPU for tf-DQN/PG/A2C. (#13393) 2021-03-08 15:41:27 +01:00
Kai Fricke
b0bf44b154
[tune/docs] Add high level trial runner flow to documentation (#14468)
* [tune/docs] Add high level trial runner flow to documentation

* Apply suggestions from code review
2021-03-08 10:35:54 +01:00
Kai Yang
7977474899
[Core] Filter out dead nodes when getting address info from redis (#14440) 2021-03-08 15:48:26 +08:00
Edward Oakes
8e139046b9
[metrics] Remove unused unit field from cython classes (#14497) 2021-03-07 20:06:02 -06:00
Richard Liaw
dec3aa3453
Split tests for timeout (#14516) 2021-03-07 16:46:52 -08:00
Eric Liang
3fab5e2ada
Switch memory units to bytes (#14433) 2021-03-06 19:32:35 -08:00
Richard Liaw
5fc761c562
Fix test_advanced_3 timeout (#14509)
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
2021-03-06 10:59:06 -08:00
EscapeReality846089495
33b271aa97
[tune] Fixed save_to_dir w/ os.replace (#14510)
The method save_to_dir of the class Searcher in ray.tune.suggest.suggestion.py uses the os.rename method to replace tmp_search_ckpt to current ckpt. os.rename method will raise the [WinError 183] or file exists error of other operating system. os.replace is the currect way.
2021-03-06 01:14:56 -08:00
Alex Wu
2395e25fc0
[hotfix][core] Load balancing spillback feature flag (#14457) 2021-03-05 16:45:33 -08:00
Antoni Baum
2002cff42e
[Tune] HEBO concurrency fix after discussion with authors (#14504) 2021-03-05 14:05:37 -08:00
Sven Mika
ef944bc5f0
[RLlib] Re-enable placement group support for RLlib. (#14384) 2021-03-05 08:16:24 +01:00
Qstar
6f151ad510
reset memory for tasks and actors to 5% when cached memory added (#14345) 2021-03-05 10:36:29 +08:00
DK.Pino
26907b7708
Support placement group for normal task in Java API (#14342)
* support pg for normal task

* fix lint

* fix comment

* fix comment

* update comment

* fix java typo
2021-03-05 10:21:37 +08:00
Dmitri Gekhtman
736c99fadb
[kubernetes][test][minor] Operator test modification (#14488) 2021-03-04 14:38:58 -08:00
Edward Oakes
be974a6596
[metrics] Only put live nodes in prometheus service discovery file (#14495) 2021-03-04 16:17:00 -06:00
Eric Liang
2cf4c7253c [ray client] Fix ctrl-c for ray.get() by setting a short-server side timeout (#14425) 2021-03-04 10:36:42 -08:00
SangBin Cho
190ab40645
[Core] Display ip address when node dies (#14489)
* done.

* Addressed code review.
2021-03-04 10:27:00 -08:00
Kai Yang
1d7bd990b6
[Java] Update System.gc() log to debug level (#14490) 2021-03-04 18:54:10 +08:00
qicosmos
d77e25b4b1
[C++ worker] Avoid recursive inclusion of API header files (#14414) 2021-03-04 18:51:02 +08:00
Kai Yang
5d79821e69
[Core] Initialize system config in CoreWorkerProcess constructor (#14439) 2021-03-04 16:34:54 +08:00
Qing Wang
07e619f404
Remove unsed script. (#14462) 2021-03-04 11:24:00 +08:00
Ian Rodney
759892740a
[Autoscaler] chown Ray_bootstrap Files in DockerCommandRunner (#14380) 2021-03-03 19:13:20 -08:00
Antoine Galataud
460c2757a3
Allow assigning weight to var with close name (#14109)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2021-03-03 19:11:34 -08:00
Eric Liang
99a63b3dd1
Remove old scheduler and friends (#14184) 2021-03-03 18:29:15 -08:00
Dmitri Gekhtman
3f6c23e3cc
[doc][autoscaler][minor] Fix quickstart guide: ray.init(address='auto') (#14459) 2021-03-03 17:58:52 -08:00
Richard Liaw
dba533dd84
Disable more torch (#14480)
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
2021-03-03 15:46:32 -08:00
tchordia
e40dc3a3e9
[serve] Better validation for arguments to client.start() (#14327) 2021-03-03 14:33:36 -08:00
Richard Liaw
60a8b67488
Disable mnist tests (#14474) 2021-03-03 13:25:01 -08:00
Hao Zhang
4135b0eb4a
[Collective] Supporting multistream, stream pool, and CUDA events. (#14127)
Co-authored-by: fustinose <fustinosej@gmail.com>
2021-03-03 09:53:45 -08:00
ZhuSenlin
dcff25aed6
remove invalid code inside NodeManager::NodeAdded (#14273)
Co-authored-by: senlin.zsl <senlin.zsl@antgroup.com>
2021-03-03 09:20:21 -08:00
SangBin Cho
a04ab9b472
[Core] Fix ray memory bug (#14452)
* ray memory bug

* Fix ray memory issue.

* done.
2021-03-03 09:20:00 -08:00
SangBin Cho
1d2136959f
[Core] Fix port issue (#14435)
* Initial impl.

* Update.

* fixed a bug.

* Fix all the issues.

* Addressed code review.

* Addressed code review.

* Fix a test failure.
2021-03-03 09:16:00 -08:00
Xianyang Liu
fc9182e63c
Fixes autoscaling monitor when environment has set http_proxy or https_proxy (#14351) 2021-03-03 18:22:53 +02:00
Sven Mika
5637d89ecc
[RLlib] Serve + RLlib example script. (#14416) 2021-03-03 14:33:03 +01:00
Sven Mika
7718ec70fb
[RLlib] Remove old SegmentTree from tests dir and unflake respective segment tree test. (#14450) 2021-03-03 14:31:30 +01:00
Kai Yang
d653394c7f
[Java] Some bug fixes about Java UT workflow (#14444) 2021-03-03 19:32:14 +08:00
Kai Yang
c53c909130
[Java] Quit worker process after RunTaskExecutionLoop to avoid orphan Java worker processses (#14442) 2021-03-03 16:47:17 +08:00
Antoni Baum
85a092c3d7
[Tune] Fix HEBO evaluated rewards for max mode & save/restore (#14427)
* Fix HEBO evaluated rewards for max mode

* Lint

* Make sure everything necessary is saved
2021-03-03 09:44:43 +01:00
Richard Liaw
63c2b7356e
Disable windows tests for test_iter and test_reference_counting (#14455)
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
2021-03-03 00:39:59 -08:00
fangfengbin
1054613da1
[Core]Fix ray.kill doesn't cancel pending actor bug (#14154)
Co-authored-by: 灵洵 <fengbin.ffb@antgroup.com>
2021-03-03 16:12:32 +08:00
Stephanie Wang
5c6c9d5b91
[core] Spill tasks from waiting queue (#14288)
* Spill back waiting tasks

* test

* test

* todo

* Avoid iterating over args

* update

* lint

* Fix test

* test

* Test force spillback

* Unit test resource scheduler

* test

* travis?

* rename

* debug

* revert flaky test

* lint

* fix test

* fix
2021-03-02 22:30:02 -08:00
Dmitri Gekhtman
1675156a8b
[autoscaler][interface] Use multi node types in defaults.yaml and example-full.yaml (#14239)
* random doc typo

* example-full-multi

* left off max workers

* wip

* address comments, modify defaults, wip

* fix

* wip

* reformat more things

* undo useless diff

* space

* max workers

* space

* copy-paste mishaps

* space

* More copy-paste mishaps

* copy-paste issues, space, max_workers

* head_node_type

* legacy yamls

* line undeleted

* correct-gpu

* Remove redundant GPU example.

* Extraneous comment

* whitespace

* example-java.yaml

* Revert "example-java.yaml"

This reverts commit 1e9c0124b9d97e651aaeeb6ec5bf7a4ef2a2df17.

* tests and other things

* doc

* doc

* revert max worker default

* Kubernetes comment

* wip

* wip

* tweak

* Address comments

* test_resource_demand_scheduler fixes

* Head type min/max workers, aws resources

* fix example_cluster2.yaml

* Fix external node type test (compatibility with legacy-style external node types)

* fix test_autoscaler_aws

* gcp-images

* gcp node type names

* fix gcp defaults

* doc format

* typo

* Skip failed Windows tests

* doc string and comment

* assert

* remove contents of default external head and worker

* legacy external failed validation test

* Readability -- define the minimal external config at the top of the file.

* Remove default worker type min worker

* Remove extraneous global min_workers comment.

* per-node-type docker in aws/example-gpu-docker

* ray.worker.small -> ray.worker.default

* fix-docker

* fix gpu docker again

* undo kubernetes experiment

* fix doc

* remove worker max_worker from kubernetes

* remove max_worker from local worker node type

* fix doc again

* py38

* eric-comment

* fix cluster name

* fix-test-autoscaler

* legacy config logic

* pop resources

* Remove min_workers AFTER merge

* comment, warning message

* warning, comment
2021-03-03 06:16:19 +02:00
Eric Liang
ef873be9e8
Require opt-in to switching plasma to /tmp instead of /dev/shm (#14451) 2021-03-02 16:44:33 -08:00
Richard Liaw
d92c00e233
Pin autogluon.core for builds (#14448) 2021-03-02 15:55:03 -08:00
Kai Fricke
47603045f9
[tune] Move Optuna to ask/tell interface (#14387) 2021-03-02 15:35:11 -08:00
SangBin Cho
bacbdd297b
[Core] Do not unregister workers that own objects by worker capping mechanism. (#14408)
* Almost done.

* Initial implementation done.

* Fix issue.

* Addressed the initial code review.

* improve comments.

* Addressed code review.

* Adding unit tests.

* Complete unit tests.

* Resolve all issues.

* Fix issues.
2021-03-02 12:24:22 -08:00
Edward Oakes
b7516ef667
hide CLI option for redis shard ports (#14434) 2021-03-02 11:06:34 -08:00
Alex Wu
4572c6cf0f
[autoscaler] Fix tag cache bug, don't kill workers on error (#14424) 2021-03-02 11:06:06 -08:00
Yi Cheng
d921dca075
[core] Fixing bug when dispatching tasks to deleted placement group (#14300) 2021-03-02 10:24:53 -08:00
Richard Liaw
aa24f8db9d
[tune] fix tune builds (#14447) 2021-03-02 10:20:20 -08:00