Commit graph

8716 commits

Author SHA1 Message Date
Eric Liang
f03b43c532
[dataset] Support callable classes to simplify state initialization (#17136) 2021-07-15 23:06:14 -07:00
SongGuyang
dcb1baabd7
[C++ API] support loading C++ dynamic libraries from code search path (#16828) 2021-07-16 13:02:45 +08:00
SongGuyang
a57de0e224
support build different python wheel in setup.py (#16998) 2021-07-16 13:01:48 +08:00
Eric Liang
e69987bc96
Improve dataset error message (#17129) 2021-07-15 20:58:15 -07:00
Edward Oakes
90a1667b29
[debugger] Clean up breakpoint state for dead jobs (#17095) 2021-07-15 22:20:09 -05:00
Chen Shen
2a53d22438
[nightly-test] add test shuffle_data_loader (#16972)
* test shuffle_data_loader

* address comments

* update
2021-07-15 20:03:35 -07:00
Tao Wang
8b922a8517
[Cpp] Use global state accessor to get session dir, if necessary (#17105)
* [Cpp]use global state accessor to get session dir, if necessary

* per comment
2021-07-16 10:36:56 +08:00
Ian Rodney
23d43919cd
[Client] Except TypeError when Deserializing (#17035)
Co-authored-by: Amog Kamsetty <amogkamsetty@yahoo.com>
2021-07-15 19:10:21 -07:00
chenk008
8b4df111e4
[CI] Fix share file leak (#17122) 2021-07-16 09:36:39 +08:00
Chen Shen
c39571a1f2
Fix GCS shutdown order (#17135) 2021-07-15 17:41:19 -07:00
Amog Kamsetty
860addeafa
Remove pyspy dependency (#17061)
* remove pyspy

* fix

* update-for-posterity

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>

* update

Signed-off-by: Richard Liaw <rliaw@berkeley.edu>

* Update setup.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2021-07-15 17:36:42 -07:00
Yi Cheng
138676295f
[core] Add bundle id as a label; (#16819)
* check

* up

* up

* up

* up

* up

* up

* format

* up

* up

* add test

* format

* up

* format

* up

* format

* up

* up

* up

* rollback

* uncomment

* format

* fix comments

* fix mac build
2021-07-15 16:05:42 -07:00
Lixin Wei
06f6f4e0ec
[Core] Limit Batch Size When Broadcasting Resources (#17072) 2021-07-15 14:28:57 -07:00
Amog Kamsetty
d607a894de
[Autoscaler] Remove jsonschema from core dependencies (#17052) 2021-07-15 13:56:44 -07:00
Siyuan (Ryans) Zhuang
dd1427548c
[Workflow] Readonly Virtual Actor (#16963)
* readonly virtual actor

* create and get actor

* add TODO

* mapping between actor ID and workflow ID

* update doc

* ensure storage serializable

* get_latest_output

* update storage

* tests
2021-07-15 13:44:51 -07:00
Michael Galarnyk
880aa7d8a8
Update index.rst (#17123) 2021-07-15 13:11:42 -07:00
Amog Kamsetty
6ff4d1ddb1
[Datasets] to_torch implementation (#17113) 2021-07-15 13:02:07 -07:00
Stephanie Wang
bdaa96bf43
[core] Fix bugs in worker cleanup on driver exit (#17049)
* unit test

* cleanup test

* Don't kill workers when job finishes

* better test

* lint

* lint

* comment

* check
2021-07-15 12:53:51 -07:00
Chen Shen
02f58a5c6b
[nightly-test] increase timeout to 1 hour (#17125) 2021-07-15 12:30:08 -07:00
Qing Wang
d4635836ba
Port python API on get_current_actor_handle. (#17110)
* Port python API on get current actor handle.

* Address comment.
2021-07-15 11:22:46 -07:00
Eric Liang
3d764d7b4b
[data] Fix the ObjectRef type in the dataset docs (#17111)
* fix reft

* remove exp

* fix
2021-07-15 09:50:37 -07:00
architkulkarni
95a7c28ed5
[Core] [runtime env] Use global lock for conda install instead of per-env lock (#17101) 2021-07-15 11:33:30 -05:00
architkulkarni
8ece30246f
[Core] [runtime env] [Test] Partially deflake test_runtime_env_complicated by bumping 0.1s timeout to 0.5s (#17109) 2021-07-15 11:26:37 -05:00
Edward Oakes
58f62dbc52
[flaky test] Skip some flaky list_named_actors tests on windows (#17087) 2021-07-15 08:40:54 -07:00
Antoni Baum
f20311f194
[tune] ResourceChangingScheduler improvements (#17082) 2021-07-15 15:03:27 +01:00
Sven Mika
649580d735
[RLlib] Redo simplify multi agent config dict: Reverted b/c seemed to break test_typing (non RLlib test). (#17046) 2021-07-15 05:51:24 -04:00
kk-55
13094a3f1c
AttributeError: 'numpy.ndarray' object has no attribute 'get_shape' when running with framework config tf2 or tfe (#16868) 2021-07-15 10:47:16 +01:00
Chen Shen
ba70d8dbc6
[RFC] Fix object size inconsistency caused by object-marked-failed. (#16976) 2021-07-14 23:33:36 -07:00
Clark Zinzow
915c426515
[Dataset] Fix S3FileSystem subsystem initialization on deserialization. (#17103)
* Add S3FileSystem wrapper that initializes the S3 subsystem on deserialization, use it for file-based datasources.

* Use S3FileSystem wrapper for read_binary_files.
2021-07-14 23:32:48 -07:00
Eric Liang
38bddc3f2b
First cut at dataset documentation (#16956) 2021-07-14 23:27:13 -07:00
chenk008
42e6c9b020
[Core] Use shim process in dedicated_workers_to_tasks (#17076)
* use shim process in dedicated_workers_to_tasks

* lint
2021-07-15 13:50:54 +08:00
Chris K. W
bd9d7bbbaa
[client] Add support for protocol (ray://, local://, custom://) to ray.init (#16946) 2021-07-14 21:45:46 -07:00
Antoni Baum
ac54164e73
[Datasets] Implement to_tf (#17000)
Co-authored-by: Clark Zinzow <clarkzinzow@gmail.com>
2021-07-14 20:48:25 -07:00
Amog Kamsetty
caa78a3cff
Revert "[Core] Remove gpustat from core dependencies (#17059)" (#17106)
This reverts commit 7ec18f671a.
2021-07-14 20:19:33 -07:00
Eric Liang
85725f2018
[data] Move Block to public API so that datasource API doesn't reference a private interface (#17098) 2021-07-14 18:59:13 -07:00
Ian Fox
673d6822c7
[readme] Add raydistributed twitter link to readme (#17099) 2021-07-14 18:55:24 -07:00
Yi Cheng
2088540ffb
[workflow] Get rid of asyncio.run for py36 (#17065) 2021-07-14 17:34:27 -07:00
Simon Mo
86d0159c0a
[Docs] Trial Fathom analytics for doc pages (#17056)
* [Docs] Trial Fathom analytics for doc pages

* Update custom domain
2021-07-14 14:11:52 -07:00
Kai Fricke
ed131f87da
[release] move release testing end to end script to main ray repo (#17070) 2021-07-14 12:39:07 -07:00
Chen Shen
92f19170ab
[error message] change noisy missing object error to debug (#17081) 2021-07-14 12:36:30 -07:00
Yi Cheng
75bdccb86e
fix bad test (#17057) 2021-07-14 11:42:08 -07:00
Antoni Baum
6e780ebf07
[tune] ResourceChangingScheduler dynamic resource allocation during tuning (#16787) 2021-07-14 10:45:13 +01:00
Antoni Baum
cfc5806c2d
[release] LightGBM release tests (#17043) 2021-07-14 08:38:55 +01:00
Philipp Moritz
0f79ebbd75
[Ray debugger] Sort breakpoints chronologically and print in table format (#17051) 2021-07-14 00:37:04 -07:00
Amog Kamsetty
c17e171f92
Revert "[Dashboard][event] Basic event module (#16985)" (#17068)
This reverts commit f1faa79a04.
2021-07-13 23:18:43 -07:00
Amog Kamsetty
7ec18f671a
[Core] Remove gpustat from core dependencies (#17059) 2021-07-13 21:22:02 -07:00
SangBin Cho
536537cd1a
[Test] Update large scale data processing tests (#16967)
* in progress

* in progress
2021-07-13 19:15:13 -07:00
Yi Cheng
dc0f948cb9
[workflow] S3 support for workflow (#16993)
* up

* up

* up

* format

* up

* fix comment

* up

* update

* update

* move dep

* bump pytest versin

* use lazy_fixture explicitly

* format
2021-07-13 19:14:41 -07:00
Chen Shen
645d8fcaf0
[logging][rfc] add RAY_LOG_EVERY_N and RAY_LOG_EVERY_MS (#17018)
* introduce log-every-n

* add n

* linter

* add license
2021-07-13 19:14:28 -07:00
fyrestone
f1faa79a04
[Dashboard][event] Basic event module (#16985)
* Basic event module

* Fix comments

* Set the SCAN_EVENT_DIR_INTERVAL_SECONDS defaults to 2

* Fix lint

* Fix lint

* Clean code

* Try to fix flaky

* Fix test

* Disable event module by default

* Make monitor events task cancellable

* Fix error

Co-authored-by: 刘宝 <po.lb@antfin.com>
2021-07-13 19:08:39 -07:00