Commit graph

8850 commits

Author SHA1 Message Date
Amog Kamsetty
6ff4d1ddb1
[Datasets] to_torch implementation (#17113) 2021-07-15 13:02:07 -07:00
Stephanie Wang
bdaa96bf43
[core] Fix bugs in worker cleanup on driver exit (#17049)
* unit test

* cleanup test

* Don't kill workers when job finishes

* better test

* lint

* lint

* comment

* check
2021-07-15 12:53:51 -07:00
Chen Shen
02f58a5c6b
[nightly-test] increase timeout to 1 hour (#17125) 2021-07-15 12:30:08 -07:00
Qing Wang
d4635836ba
Port python API on get_current_actor_handle. (#17110)
* Port python API on get current actor handle.

* Address comment.
2021-07-15 11:22:46 -07:00
Eric Liang
3d764d7b4b
[data] Fix the ObjectRef type in the dataset docs (#17111)
* fix reft

* remove exp

* fix
2021-07-15 09:50:37 -07:00
architkulkarni
95a7c28ed5
[Core] [runtime env] Use global lock for conda install instead of per-env lock (#17101) 2021-07-15 11:33:30 -05:00
architkulkarni
8ece30246f
[Core] [runtime env] [Test] Partially deflake test_runtime_env_complicated by bumping 0.1s timeout to 0.5s (#17109) 2021-07-15 11:26:37 -05:00
Edward Oakes
58f62dbc52
[flaky test] Skip some flaky list_named_actors tests on windows (#17087) 2021-07-15 08:40:54 -07:00
Antoni Baum
f20311f194
[tune] ResourceChangingScheduler improvements (#17082) 2021-07-15 15:03:27 +01:00
Sven Mika
649580d735
[RLlib] Redo simplify multi agent config dict: Reverted b/c seemed to break test_typing (non RLlib test). (#17046) 2021-07-15 05:51:24 -04:00
kk-55
13094a3f1c
AttributeError: 'numpy.ndarray' object has no attribute 'get_shape' when running with framework config tf2 or tfe (#16868) 2021-07-15 10:47:16 +01:00
Chen Shen
ba70d8dbc6
[RFC] Fix object size inconsistency caused by object-marked-failed. (#16976) 2021-07-14 23:33:36 -07:00
Clark Zinzow
915c426515
[Dataset] Fix S3FileSystem subsystem initialization on deserialization. (#17103)
* Add S3FileSystem wrapper that initializes the S3 subsystem on deserialization, use it for file-based datasources.

* Use S3FileSystem wrapper for read_binary_files.
2021-07-14 23:32:48 -07:00
Eric Liang
38bddc3f2b
First cut at dataset documentation (#16956) 2021-07-14 23:27:13 -07:00
chenk008
42e6c9b020
[Core] Use shim process in dedicated_workers_to_tasks (#17076)
* use shim process in dedicated_workers_to_tasks

* lint
2021-07-15 13:50:54 +08:00
Chris K. W
bd9d7bbbaa
[client] Add support for protocol (ray://, local://, custom://) to ray.init (#16946) 2021-07-14 21:45:46 -07:00
Antoni Baum
ac54164e73
[Datasets] Implement to_tf (#17000)
Co-authored-by: Clark Zinzow <clarkzinzow@gmail.com>
2021-07-14 20:48:25 -07:00
Amog Kamsetty
caa78a3cff
Revert "[Core] Remove gpustat from core dependencies (#17059)" (#17106)
This reverts commit 7ec18f671a.
2021-07-14 20:19:33 -07:00
Eric Liang
85725f2018
[data] Move Block to public API so that datasource API doesn't reference a private interface (#17098) 2021-07-14 18:59:13 -07:00
Ian Fox
673d6822c7
[readme] Add raydistributed twitter link to readme (#17099) 2021-07-14 18:55:24 -07:00
Yi Cheng
2088540ffb
[workflow] Get rid of asyncio.run for py36 (#17065) 2021-07-14 17:34:27 -07:00
Simon Mo
86d0159c0a
[Docs] Trial Fathom analytics for doc pages (#17056)
* [Docs] Trial Fathom analytics for doc pages

* Update custom domain
2021-07-14 14:11:52 -07:00
Kai Fricke
ed131f87da
[release] move release testing end to end script to main ray repo (#17070) 2021-07-14 12:39:07 -07:00
Chen Shen
92f19170ab
[error message] change noisy missing object error to debug (#17081) 2021-07-14 12:36:30 -07:00
Yi Cheng
75bdccb86e
fix bad test (#17057) 2021-07-14 11:42:08 -07:00
Antoni Baum
6e780ebf07
[tune] ResourceChangingScheduler dynamic resource allocation during tuning (#16787) 2021-07-14 10:45:13 +01:00
Antoni Baum
cfc5806c2d
[release] LightGBM release tests (#17043) 2021-07-14 08:38:55 +01:00
Philipp Moritz
0f79ebbd75
[Ray debugger] Sort breakpoints chronologically and print in table format (#17051) 2021-07-14 00:37:04 -07:00
Amog Kamsetty
c17e171f92
Revert "[Dashboard][event] Basic event module (#16985)" (#17068)
This reverts commit f1faa79a04.
2021-07-13 23:18:43 -07:00
Amog Kamsetty
7ec18f671a
[Core] Remove gpustat from core dependencies (#17059) 2021-07-13 21:22:02 -07:00
SangBin Cho
536537cd1a
[Test] Update large scale data processing tests (#16967)
* in progress

* in progress
2021-07-13 19:15:13 -07:00
Yi Cheng
dc0f948cb9
[workflow] S3 support for workflow (#16993)
* up

* up

* up

* format

* up

* fix comment

* up

* update

* update

* move dep

* bump pytest versin

* use lazy_fixture explicitly

* format
2021-07-13 19:14:41 -07:00
Chen Shen
645d8fcaf0
[logging][rfc] add RAY_LOG_EVERY_N and RAY_LOG_EVERY_MS (#17018)
* introduce log-every-n

* add n

* linter

* add license
2021-07-13 19:14:28 -07:00
fyrestone
f1faa79a04
[Dashboard][event] Basic event module (#16985)
* Basic event module

* Fix comments

* Set the SCAN_EVENT_DIR_INTERVAL_SECONDS defaults to 2

* Fix lint

* Fix lint

* Clean code

* Try to fix flaky

* Fix test

* Disable event module by default

* Make monitor events task cancellable

* Fix error

Co-authored-by: 刘宝 <po.lb@antfin.com>
2021-07-13 19:08:39 -07:00
Sven Mika
ce6dfc9b2d
[RLlib] Update tf1.x vs tf2.x documentation and eager example script. (#17030) 2021-07-13 20:02:17 -04:00
SangBin Cho
63ebfe2f2d
Revert back to ray.init (#17047) 2021-07-13 14:36:27 -07:00
Philipp Moritz
ac912f0ce1
Allow using breakpoint() to drop into Ray debugger (#17025)
* Set PYTHONBREAKPOINT

* update tests

* update

* update docs

* fix docs

* skip ray functions

* ok

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>

* breakpoint() is only working in Python > 3.6

* add note

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2021-07-13 13:52:17 -07:00
Grzegorz Bartyzel
d553d4da6c
[RLlib] DQN (Rainbow): Fix torch noisy layer support and loss (#16716) 2021-07-13 16:48:06 -04:00
Sven Mika
1fd0eb805e
[RLlib] Redo fix bug normalize vs unsquash actions (original PR made log-likelihood test flakey). (#17014) 2021-07-13 14:01:30 -04:00
Antoine Galataud
16f1011c07
[RLlib] Issue 15910: APEX current learning rate not updated on local worker (#15911) 2021-07-13 14:01:00 -04:00
Ian Rodney
fac6045c87
[GCP] Allow Head Node to Launch Workers with IAM Role (#17027) 2021-07-13 10:44:34 -07:00
Amog Kamsetty
38b5b6d24c
Revert "[RLlib] Simplify multiagent config (automatically infer class/spaces/config). (#16565)" (#17036)
This reverts commit e4123fff27.
2021-07-13 09:57:15 -07:00
Kai Fricke
27d80c4c88
[RLlib] ONNX export for tensorflow (1.x) and torch (#16805) 2021-07-13 12:38:11 -04:00
Kai Fricke
3380b68b54
[RLlib] Issue 16683: Fix last infos dict (#16999). 2021-07-13 11:33:48 -04:00
Edward Oakes
f7759fa484
[core] Add ray.util.list_actors() API (#16642) 2021-07-13 10:00:28 -05:00
Sven Mika
e4123fff27
[RLlib] Simplify multiagent config (automatically infer class/spaces/config). (#16565) 2021-07-13 06:38:14 -04:00
Tao Wang
90187433b1
[Java] Remove redis dependency(jedis) in java lang layer (#17029) 2021-07-13 17:34:10 +08:00
Ian Rodney
9cb80fcf17
[Client][Proxy] Handle Non-Default Redis Password (#16885) 2021-07-12 23:57:51 -07:00
Tao Wang
5b7e76770d
[Java] Use gcs client instead of redis client to get session dir (#16773)
* Use gcs client instead of redis client to get session dir

* fix compile and add comments

* fix compile

* lint

* fix

* lint

* lint

* Update src/ray/gcs/gcs_client/global_state_accessor.h

Co-authored-by: Qing Wang <kingchin1218@126.com>

* Update java/runtime/src/main/java/io/ray/runtime/RayNativeRuntime.java

Co-authored-by: Qing Wang <kingchin1218@126.com>

* per comment

Co-authored-by: Qing Wang <kingchin1218@126.com>
2021-07-13 14:01:22 +08:00
Eric Liang
e7350ff828
Fix flaky test_plasma_unlimited::test_fallback_allocation_failure (#17016)
* fix

* fix catch
2021-07-12 20:17:23 -07:00