SangBin Cho
d6b6356173
[Core] Properly call shutdown instead of deleting a reference ( #17096 )
...
* Properly call shutdown instead of deleting a reference
* Add unit tests
* Add test ray shutdown
* Formatting
* format2
* Revert main logic to see if windows issue still fail
* Skip tests for windows.
* formatting
* Try fixing flakiness
* Remove node removed code path
Co-authored-by: Eric Liang <ekhliang@gmail.com>
2021-07-20 08:22:33 -07:00
Antoni Baum
5e9b680e39
[docs] Add LightGBM-Ray docs, update XGBoost-Ray docs ( #17188 )
2021-07-20 16:06:47 +01:00
Siyuan (Ryans) Zhuang
8efc04a8a6
[Core] Actor namespace ( #17178 )
...
* set actor namespace in Python on creation
* get actor with namespace in Python
* update message
2021-07-19 21:51:04 -07:00
matthewdeng
fef74aa94f
[sgd] add placement group support ( #17037 )
...
* [sgd] add placement group support
* add logic for removing placement group upon shutdown
* set placement group; add tests
* address comments - add timeout and improve error handling
* remove unused import
* mock SGD_PLACEMENT_GROUP_TIMEOUT_S
2021-07-19 21:50:37 -07:00
Siyuan (Ryans) Zhuang
9ca6bda3a1
[Workflow] Fix recovery storage mismatch issue ( #17166 )
...
* fix recovery path issue and add test
* add TODOs
2021-07-19 21:49:12 -07:00
dependabot[bot]
2de7b8f084
[tune](deps): Bump tune-sklearn in /python/requirements/tune ( #17173 )
...
Bumps [tune-sklearn](https://github.com/ray-project/tune-sklearn ) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/ray-project/tune-sklearn/releases )
- [Commits](https://github.com/ray-project/tune-sklearn/compare/v0.3.0...v0.4.0 )
---
updated-dependencies:
- dependency-name: tune-sklearn
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-07-19 19:42:54 -07:00
Amog Kamsetty
c9522e9a6f
Remove requests from Core Dependencies ( #17066 )
...
* remove requests
* update
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
* update
* lint
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2021-07-19 19:38:29 -07:00
Eric Liang
fabba96fad
Re-merge large function def, skipping test failing on Windows ( #17191 )
2021-07-19 18:03:26 -07:00
Amog Kamsetty
777921b2e7
[dependencies] vendor colorama ( #17183 )
2021-07-19 16:29:29 -07:00
Eric Liang
d59da075a6
Re-merge TMPDIR support, but only for Linux. OSX requires RAY_TMPDIR ( #17190 )
2021-07-19 15:45:03 -07:00
Patrick Ames
34789b3e56
[autoscaler] Add support for custom EC2 instance network interfaces ( #14080 )
...
* [autoscaler] Add support for custom EC2 instance network interfaces.
* [autoscaler] Add unit tests for custom EC2 network interfaces and support for AWS node provider stubs.
2021-07-19 17:21:21 -04:00
Amog Kamsetty
cb74053ee5
Retry remove gpustat dependency ( #17115 )
...
* remove gpustat
* move psutil imports
2021-07-19 11:14:10 -07:00
Siyuan (Ryans) Zhuang
1fbfbfc55b
[Serializatioin] Bump pickle5 version ( #17124 )
2021-07-19 10:40:38 -07:00
Siyuan (Ryans) Zhuang
9b110f9344
[Workflow] Update API ( #17165 )
...
* actor_id same as workflow_id
* @workflow.actor -> @workflow.virtual_actor
* readonly decorator
* run/run_async for virtual actor
* get_or_create for virtual actor
* update doc
* run/run_async for steps
* update tests
* update comments
2021-07-19 10:19:46 -07:00
architkulkarni
4069686e0f
Revert "Improve error message for oversized function ( #17133 )" ( #17184 )
...
This reverts commit 3e53619d64
2021-07-19 09:28:33 -07:00
dependabot[bot]
bda1a37e93
[tune](deps): Bump mlflow in /python/requirements/tune ( #17168 )
...
Bumps [mlflow](https://github.com/mlflow/mlflow ) from 1.17.0 to 1.19.0.
- [Release notes](https://github.com/mlflow/mlflow/releases )
- [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.rst )
- [Commits](https://github.com/mlflow/mlflow/compare/v1.17.0...v1.19.0 )
---
updated-dependencies:
- dependency-name: mlflow
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-07-17 16:23:10 -07:00
Eric Liang
3e53619d64
Improve error message for oversized function ( #17133 )
2021-07-17 11:04:05 -07:00
Philipp Moritz
c5c167300b
[Ray debugger] Set breakpoint() hook only for tasks and actors ( #17164 )
2021-07-17 10:27:51 -07:00
Simon Mo
9da49a7fb8
Revert "ray ignores TMPDIR variable" ( #17161 )
...
This reverts commit c27f43d9b8
2021-07-16 20:28:38 -07:00
Eric Liang
94f17ec099
[RFC] API stability annotations ( #17100 )
2021-07-16 17:09:20 -07:00
Alex Wu
93c16346bf
[Dataset] imagenet nightly test ( #17069 )
2021-07-16 14:15:49 -07:00
Eric Liang
c27f43d9b8
ray ignores TMPDIR variable ( #17130 )
2021-07-16 13:23:44 -07:00
Clark Zinzow
8302b5a335
[Core] Reverts full dispatch queue iteration PRs. ( #17127 )
...
* Revert "[Core] iterate over entire dispatch queue instead of returning when worker unavailable (#16535 )"
This reverts commit 54d66ac637#16751 )"
This reverts commit 13a133817b
2021-07-16 10:28:00 -07:00
Siyuan (Ryans) Zhuang
fd3742bb63
[Workflow] Update tests ( #17147 )
...
* update workflow tests
* use conftest
* use conftest
* use conftest
2021-07-16 09:25:40 -07:00
Eric Liang
40f1ee6e1b
Fix prefetch typo in Dataset ( #17143 )
2021-07-16 09:22:42 -07:00
fyrestone
e2808a35cf
Dashboard job module uses attrs instead of pydantic for job description ( #17116 )
2021-07-16 22:26:00 +08:00
Kai Fricke
49b72eec16
[tune] filter placement group resources if not in use ( #16996 )
2021-07-16 00:35:04 -07:00
Zhi Lin
6d9fb421c6
[tracing] Do not wrap when tracing is not enabled ( #16607 )
2021-07-16 00:27:54 -07:00
Eric Liang
f03b43c532
[dataset] Support callable classes to simplify state initialization ( #17136 )
2021-07-15 23:06:14 -07:00
SongGuyang
dcb1baabd7
[C++ API] support loading C++ dynamic libraries from code search path ( #16828 )
2021-07-16 13:02:45 +08:00
SongGuyang
a57de0e224
support build different python wheel in setup.py ( #16998 )
2021-07-16 13:01:48 +08:00
Eric Liang
e69987bc96
Improve dataset error message ( #17129 )
2021-07-15 20:58:15 -07:00
Edward Oakes
90a1667b29
[debugger] Clean up breakpoint state for dead jobs ( #17095 )
2021-07-15 22:20:09 -05:00
Ian Rodney
23d43919cd
[Client] Except TypeError when Deserializing ( #17035 )
...
Co-authored-by: Amog Kamsetty <amogkamsetty@yahoo.com>
2021-07-15 19:10:21 -07:00
Amog Kamsetty
860addeafa
Remove pyspy dependency ( #17061 )
...
* remove pyspy
* fix
* update-for-posterity
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
* update
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
* Update setup.py
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2021-07-15 17:36:42 -07:00
Yi Cheng
138676295f
[core] Add bundle id as a label; ( #16819 )
...
* check
* up
* up
* up
* up
* up
* up
* format
* up
* up
* add test
* format
* up
* format
* up
* format
* up
* up
* up
* rollback
* uncomment
* format
* fix comments
* fix mac build
2021-07-15 16:05:42 -07:00
Amog Kamsetty
d607a894de
[Autoscaler] Remove jsonschema from core dependencies ( #17052 )
2021-07-15 13:56:44 -07:00
Siyuan (Ryans) Zhuang
dd1427548c
[Workflow] Readonly Virtual Actor ( #16963 )
...
* readonly virtual actor
* create and get actor
* add TODO
* mapping between actor ID and workflow ID
* update doc
* ensure storage serializable
* get_latest_output
* update storage
* tests
2021-07-15 13:44:51 -07:00
Amog Kamsetty
6ff4d1ddb1
[Datasets] to_torch implementation ( #17113 )
2021-07-15 13:02:07 -07:00
Stephanie Wang
bdaa96bf43
[core] Fix bugs in worker cleanup on driver exit ( #17049 )
...
* unit test
* cleanup test
* Don't kill workers when job finishes
* better test
* lint
* lint
* comment
* check
2021-07-15 12:53:51 -07:00
Qing Wang
d4635836ba
Port python API on get_current_actor_handle. ( #17110 )
...
* Port python API on get current actor handle.
* Address comment.
2021-07-15 11:22:46 -07:00
Eric Liang
3d764d7b4b
[data] Fix the ObjectRef type in the dataset docs ( #17111 )
...
* fix reft
* remove exp
* fix
2021-07-15 09:50:37 -07:00
architkulkarni
95a7c28ed5
[Core] [runtime env] Use global lock for conda install instead of per-env lock ( #17101 )
2021-07-15 11:33:30 -05:00
architkulkarni
8ece30246f
[Core] [runtime env] [Test] Partially deflake test_runtime_env_complicated by bumping 0.1s timeout to 0.5s ( #17109 )
2021-07-15 11:26:37 -05:00
Edward Oakes
58f62dbc52
[flaky test] Skip some flaky list_named_actors tests on windows ( #17087 )
2021-07-15 08:40:54 -07:00
Antoni Baum
f20311f194
[tune] ResourceChangingScheduler improvements ( #17082 )
2021-07-15 15:03:27 +01:00
Sven Mika
649580d735
[RLlib] Redo simplify multi agent config dict: Reverted b/c seemed to break test_typing (non RLlib test). ( #17046 )
2021-07-15 05:51:24 -04:00
Clark Zinzow
915c426515
[Dataset] Fix S3FileSystem subsystem initialization on deserialization. ( #17103 )
...
* Add S3FileSystem wrapper that initializes the S3 subsystem on deserialization, use it for file-based datasources.
* Use S3FileSystem wrapper for read_binary_files.
2021-07-14 23:32:48 -07:00
Eric Liang
38bddc3f2b
First cut at dataset documentation ( #16956 )
2021-07-14 23:27:13 -07:00
Chris K. W
bd9d7bbbaa
[client] Add support for protocol (ray://, local://, custom://) to ray.init ( #16946 )
2021-07-14 21:45:46 -07:00
