hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-06 10:31:39 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
6022 commits 227 branches 67 tags 234 MiB
Commit graph

3076 commits

Author SHA1 Message Date
Ian Rodney
ab30940e50
Use 'mkdir -p' in manylinux (#11219) 2020-10-05 16:18:18 -07:00
Sumanth Ratna
0630242522
[Metrics] Use isinstance for type-checking keys of tags for metrics (#11202) 
* Use isinstance instead of type for type-checking keys of tags for metrics

* Fix lint error
 2020-10-05 14:24:22 -07:00
Gekho457
aaa8147acb
[CI] Add mypy to ci (#11117) 
* Add mypy check to format.sh

* format_all() runs mypy check only on specified files
 2020-10-05 11:20:45 -07:00
SangBin Cho
80cc161f3e
[Placement Group] Report placement group load through heartbeat. (#11129) 
* In progress.

* Fix a minor issue.

* Removed unnecessary comments.

* Addressed code review.

* Fix build failure.

* remove stray logs.

* Move global state to a med size test to avoid windows CI breakage.
 2020-10-04 16:47:22 -07:00
Ian Rodney
035a4ad0a2
[autoscaler] Update Autoscaler SDK (#11185) 
* small sdk addition

* create RAY_AUTOSCALER_CONSTANTS

* better name

* move to private

* better naming
 2020-10-03 13:18:01 -07:00
Amog Kamsetty
6325a973a2
[Ray SGD] Support num_steps continue training (#11142) 2020-10-02 23:43:26 -07:00
Ameer Haj Ali
6b86d4d280
Automatically detect CPU, GPU, accelerator_type for AWS (#11147) 2020-10-02 21:16:43 -07:00
Sumanth Ratna
6288d954b5
[tune] Use isinstance instead of type in PBT (#11175) 2020-10-02 18:32:14 -07:00
SangBin Cho
ee71fecd9e
[Test] Make test_serialization a medium size test. (#11183) 2020-10-02 17:53:46 -07:00
Stephanie Wang
ada58abcd9
[Object spilling] Update object directory and reload spilled objects automatically (#11021) 
* Fix pytest...

* Release objects that have been spilled

* GCS object table interface refactor

* Add spilled URL to object location info

* refactor to include spilled URL in notifications

* improve tests

* Add spilled URL to object directory results

* Remove force restore call

* Merge spilled URL and location

* fix

* CI

* build

* osx

* Fix multitenancy issues

* Skip windows tests
 2020-10-02 15:52:42 -07:00
Amog Kamsetty
874da9a25f
[Ray SGD] LightningModule integration + MNIST Example (#11042) 
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
 2020-10-01 20:07:32 -07:00
Richard Liaw
9dc7b7b11d
[tune] enable ax test (#11165) 
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
 2020-10-01 17:25:29 -07:00
Richard Liaw
f60d77d78e
[tune] Fix GPU thread leakage 2020-10-01 17:07:13 -07:00
Ian Rodney
0d5b09f426
[Docker] Automagically add "runtime=nvidia" (#11125) 2020-10-01 17:04:19 -07:00
Kai Fricke
681c24754a
[tune] Example for using huggingface hyperparamer_search API (#11158) 2020-10-01 16:00:57 -07:00
Siyuan (Ryans) Zhuang
1ad52bdfbc
Remove the stale directory (pyarrow_files) (#11163) 2020-10-01 14:52:04 -07:00
Kai Fricke
bdf647c4ec
[tune] docker syncer (#11035) 
* Add DockerSyncer

* Add docs

* Update python/ray/tune/integration/docker.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Updated docs

* fix dir

* Added docker integration test

* added docker integration test to bazel build

* Use sdk.rsync API

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
 2020-10-01 11:59:23 -07:00
Richard Liaw
b969e8cf59
[tune][hotfix] comment out test (#11140) 2020-10-01 10:04:40 -07:00
Richard Liaw
0d93b1de93
[autoscaler] rsync to node (#11122) 2020-10-01 01:42:49 -07:00
Eric Liang
8f73014291
[autoscaler] Make it more clear NodeProvider and CommandRunner are private APIs (#11086) 2020-09-30 16:24:37 -07:00
Eric Liang
090002b0ce
Blocking asyncio get warning should be logger.warn not logger.debug (#11141) 2020-09-30 16:16:35 -07:00
Benjamin Black
4445f32798
[rllib] Fixed pettingzoo wrapper (#11060) 
* fixed PettingZooEnv, relevant docs, examples

* fixed linting error

* pettingzoo wrapper fix

* fixed linting issue
 2020-09-30 15:34:47 -07:00
Kai Fricke
c77cfaa5ad
[tune] use dated experiment dir per default (#11104) 2020-09-30 14:43:59 -07:00
Siyuan (Ryans) Zhuang
f0dba6bd2b
Code cleanup about python3 asyncio compat (#11134) 
* cleanup python3 compat and others
 2020-09-30 14:22:25 -07:00
Tao Wang
0dcfa9ed6c
Add light heartbeat flag in python and use it in load metrics (#11032) 2020-09-30 11:39:28 -07:00
SangBin Cho
e44828d928
[Test] Make the test size bigger for tune to fix timeout. (#11138) 2020-09-30 11:35:31 -07:00
SangBin Cho
474af36591
[Core] Fix random bugs found. (#11128) 2020-09-30 00:11:57 -07:00
Kai Yang
3504391fd2
[Core] Multi-tenancy: enable multi-tenancy by default (#10570) 
* Add new job in Travis to enable multi-tenancy

* fix

* Update .bazelrc

* Update .travis.yml

* fix test_job_gc_with_detached_actor

* fix test_multiple_downstream_tasks

* fix lint

* Enable multi-tenancy by default

* Kill idle workers in FIFO order

* Update test

* minor update

* Address comments

* fix some cases

* fix test_remote_cancel

* Address comments

* fix after merge

* remove kill

* fix worker_pool_test

* fix java test timeout

* fix test_two_custom_resources

* Add a delay when killing idle workers

* fix test_worker_failure

* fix test_worker_failed again

* fix DisconnectWorker

* update test_worker_failed

* Revert some python tests

* lint

* address comments
 2020-09-29 23:54:53 -07:00
SangBin Cho
f54f7b23f6
[Object Spilling] Name IO workers properly. (#11097) 
* Fix.

* Addressed code review.
 2020-09-29 23:48:31 -07:00
Tao Wang
15ae8816f7
[GCS]Remove useless / heavy heartbeat pub (#11132) 2020-09-29 23:38:17 -07:00
Maksim Smolin
94494c43d6
[cli] Updates (#10777) 2020-09-29 16:55:07 -07:00
Ameer Haj Ali
f17ad5d060
[autoscaler] resource demand scheduler get_nodes_to_launch should return Dict not List (#11118) 2020-09-29 15:32:56 -07:00
Kai Fricke
10015e60fb
[tune] fix flaky callback test (#11102) 2020-09-29 09:44:32 -07:00
Tao Wang
1db83764bf
[GCS]Use new getting all available resources interface instead of pub-sub … (#10914) 
* Use new all available resources getting interface instead of pub-sub in state.py

* add missing server handler and test cases, fix comments

* add fine grained test assert

* per comments

* involve new added function _available_resources_per_node

* change  ClientID to NodeID

* fix compile

* fix client id and lint

* robust tests check

* robust tests
 2020-09-29 09:41:10 -07:00
Ameer Haj Ali
0d36e4c025
[autoscaler] Support min_workers for multi node type (#11041) 
* prepare for head node

* move command runner interface outside _private

* remove space

* Eric

* flake

* min_workers in multi node type

* fixing edge cases

* eric not idle

* fix target_workers to consider min_workers of node types

* idle timeout

* minor

* minor fix

* test

* lint

* eric v2

* eric 3

* min_workers constraint before bin packing

* Update resource_demand_scheduler.py

* Revert "Update resource_demand_scheduler.py"

This reverts commit 818a63a2c86d8437b3ef21c5035d701c1d1127b5.

* reducing diff

Co-authored-by: Ameer Haj Ali <ameerhajali@ameers-mbp.lan>
Co-authored-by: Alex Wu <alex@anyscale.io>
Co-authored-by: Alex Wu <itswu.alex@gmail.com>
 2020-09-28 22:02:01 -07:00
Simon Mo
6cfc1a16a0
[Serve] Fix resource checking (#11084) 2020-09-28 18:10:43 -07:00
SangBin Cho
cef263a333
[Custom Metrics] Improve ray runtime context. (#10981) 
* done.

* Addressed code review.

* Addressed code review.
 2020-09-28 16:18:19 -07:00
fangfengbin
872219940b
[GCS]Fix miss PollOwnerForActorOutOfScope after gcs restarts bug (#11054) 
* fix_RemoveActorFromOwner_crash_bug

* fix review comment

* fix review comment

* rm unused ut

* add testcase

* fix review comment

* rm unused import

* fix code style

* fix ut bug

Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
 2020-09-28 10:06:40 -07:00
Lingxuan Zuo
27e1f513e3
[Log] make glog flush and RAY_LOG thread-safe (#11002) 
* make glog flush and RAY_LOG thread-safe

* dump error log to console

* mapping all levels to destination

* hack glog for exporting message to stdout if no base name given

* patch lint

* use stdout logger by default

* add raylet std/err pytest checker

* add worker logs file check

* fix asan check

* loop in glog enums

* fix python lint

* lint for autoindent

* fix indent lint

* make raylet.err is not empty
 2020-09-28 22:15:15 +08:00
SangBin Cho
1e39c40370
[Placement Group] Capture child tasks by default. (#11025) 
* In progress.

* Finished up.

* Improve comment.

* Addressed code review.

* Fix test failure.

* Fix ci failures.

* Fix CI issues.
 2020-09-27 19:33:00 -07:00
Kai Fricke
e7315b0856
[tune] Callbacks for tune runs (#11001) 2020-09-27 16:50:07 -07:00
DK.Pino
db7097fb1f
[Refactor] Rename ClientId to NodeId (#10992) 
* rename ClientId to NodeId

* format lint

* format lint

* fix conflicts

* rename new ClientId to NodeId

* update lint

* make same version of clang-format with travis ci
 2020-09-27 10:24:21 -07:00
Eric Liang
b5ecdd088b
Deflake object manager test (#11052) 2020-09-26 18:33:02 -07:00
Richard Liaw
51038f2197
confirm (#11049) 
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
 2020-09-26 13:48:13 -07:00
SangBin Cho
29663d89f1
[Placement Group] Remove warning msg for placement groups. (#11034) 
* Done.

* Addressed code review.

* Fixed typo.

* Addressed code review.
 2020-09-25 20:53:42 -07:00
SangBin Cho
109481afd9
[Metric] custom metrics refinement (#10861) 
* In progress

* In Progress.

* Addressed code review.

* Add unit tests.

* Add a simple doc.

* Fixed test failure.

* Fix all test failures from serve.

* Addressed code review.
 2020-09-25 09:10:28 -07:00
Eric Liang
609c1b8acd
Start moving ray internal files to _private module (#10994) 2020-09-24 22:46:35 -07:00
Ameer Haj Ali
3b6fe72029
Make the command runner interface public (#11022) 2020-09-24 22:45:17 -07:00
Amog Kamsetty
ee85cb31a5
[Tune] Fix Memory Leak (#10989) 
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
 2020-09-24 20:26:55 -07:00
Eric Liang
a26394d184
[tests] Blacklist failing win32 tests (#10935) 2020-09-24 20:15:28 -07:00