Alexis DUBURCQ
362f7b7c56
[RLlib] Do not deepcopy input dict for efficiency and consistency with similar methods. ( #15709 )
...
Co-authored-by: Alexis Duburcq <alexis.duburcq@wandercraft.eu>
2021-07-21 20:09:41 -07:00
Chen Shen
70ab8aa1d4
Revert "[core] Do not spill back tasks blocked on args to blocked nodes ( #16488 )" ( #17247 )
...
This reverts commit dad8db46e1
2021-07-21 19:41:35 -07:00
Vince Jankovics
05c9dfbbda
[RLlib] CV2 to Skimage dependency change ( #16841 )
2021-07-21 22:24:18 -04:00
Clark Zinzow
05a7102104
[Datasets] Port read_parquet to Datasource API. ( #17230 )
...
* Port read_parquet to Datasource API.
* Update to new block representation.
* Remove unused _parse_paths.
* Support column selection.
* Formatting.
* Add column selection test.
2021-07-21 17:39:39 -07:00
Simon Mo
7b44dd8ecb
Revert "[core] remove opencensus/prometheus_exporter dependencies" ( #17251 )
...
This reverts commit 64874e1877
2021-07-21 16:57:47 -07:00
Yi Cheng
5accfa662c
[workflow] Test for better coverage ( #17233 )
...
* update
* workflow.init
* update
* update
* update tests
* check
* up
* update
* update
* check
* merge
* fix tests
* update
* add tests
* up
* format
* add space
* Update test_storage.py
Co-authored-by: Siyuan <suquark@gmail.com>
2021-07-21 14:48:36 -07:00
Antoni Baum
2e37826458
[tune] Function API support for ResourceChangingScheduler ( #17150 )
...
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2021-07-21 14:14:12 -07:00
Siyuan (Ryans) Zhuang
a550eb3e97
[Workflow] Update workflow storage interface ( #17222 )
...
* workflow.init
* update tests
* add tests
2021-07-21 11:51:01 -07:00
chenk008
afd59be8ca
[Core] Add worker resource limit ( #17179 )
...
* add resource restricted
* fix test
* lint
* lint
2021-07-21 22:00:34 +08:00
Kai Fricke
64874e1877
[core] remove opencensus/prometheus_exporter dependencies ( #17182 )
2021-07-21 12:57:31 +01:00
Simon Mo
250f0c24e0
[Runtime Env] Refactor local dev mode for linking ray packages ( #17227 )
2021-07-21 00:48:58 -07:00
Kai Fricke
e881c6cff8
[core] remove aiohttp dependencies ( #17181 )
2021-07-21 07:18:19 +01:00
Stephanie Wang
dad8db46e1
[core] Do not spill back tasks blocked on args to blocked nodes ( #16488 )
2021-07-20 17:13:02 -07:00
Eric Liang
877076160e
[data] Enable zero-copy access to underlying Arrow tables ( #17192 )
2021-07-20 16:38:21 -07:00
Eric Liang
d6e91a5b46
Update PublicAPI annotations #17224
2021-07-20 16:37:53 -07:00
Clark Zinzow
09f32b68d3
[Datasets] Slice off S3 protocol from S3 URIs. ( #17219 )
...
* Ensure that S3 protocols are sliced off of S3 URIs.
* Use urllib to parse and trim URI to path.
2021-07-20 15:23:35 -07:00
Clark Zinzow
08a50bf3b7
[Datasets] Allow for Parquet metadata file to be missing. ( #17217 )
...
* Allow for Parquet metadata file to be missing.
* Remove for-else.
2021-07-20 15:20:26 -07:00
Ian Rodney
e6bf0a8ea6
[autoscaler][docstring] Add Docstring for StandardAutoscaler ctor ( #17213 )
2021-07-20 12:19:54 -07:00
Patrick Ames
efed07023f
[autoscaler] Custom AWS network interface error condition tests and missing security group bug fix. ( #17207 )
2021-07-20 11:17:27 -07:00
Jialing He
492076806d
[object store] Assign the object owner in ray.put() ( #16833 )
2021-07-20 11:06:00 -07:00
Amog Kamsetty
4ece5247d6
[Datasets] to_torch no DataLoader ( #17211 )
2021-07-20 11:05:17 -07:00
Siyuan (Ryans) Zhuang
859cba7655
[Workflow] Remove namespace in workflow
2021-07-20 11:04:46 -07:00
Yi Cheng
8253064163
[workflow] workflow error handling ( #17175 )
2021-07-20 11:03:53 -07:00
Simon Mo
908aa2c7f3
Fix runtime env and dispatch queue take 2 ( #17163 )
2021-07-20 10:24:08 -07:00
SangBin Cho
d6b6356173
[Core] Properly call shutdown instead of deleting a reference ( #17096 )
...
* Properly call shutdown instead of deleting a reference
* Add unit tests
* Add test ray shutdown
* Formatting
* format2
* Revert main logic to see if windows issue still fail
* Skip tests for windows.
* formatting
* Try fixing flakiness
* Remove node removed code path
Co-authored-by: Eric Liang <ekhliang@gmail.com>
2021-07-20 08:22:33 -07:00
Antoni Baum
5e9b680e39
[docs] Add LightGBM-Ray docs, update XGBoost-Ray docs ( #17188 )
2021-07-20 16:06:47 +01:00
Siyuan (Ryans) Zhuang
8efc04a8a6
[Core] Actor namespace ( #17178 )
...
* set actor namespace in Python on creation
* get actor with namespace in Python
* update message
2021-07-19 21:51:04 -07:00
matthewdeng
fef74aa94f
[sgd] add placement group support ( #17037 )
...
* [sgd] add placement group support
* add logic for removing placement group upon shutdown
* set placement group; add tests
* address comments - add timeout and improve error handling
* remove unused import
* mock SGD_PLACEMENT_GROUP_TIMEOUT_S
2021-07-19 21:50:37 -07:00
Siyuan (Ryans) Zhuang
9ca6bda3a1
[Workflow] Fix recovery storage mismatch issue ( #17166 )
...
* fix recovery path issue and add test
* add TODOs
2021-07-19 21:49:12 -07:00
dependabot[bot]
2de7b8f084
[tune](deps): Bump tune-sklearn in /python/requirements/tune ( #17173 )
...
Bumps [tune-sklearn](https://github.com/ray-project/tune-sklearn ) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/ray-project/tune-sklearn/releases )
- [Commits](https://github.com/ray-project/tune-sklearn/compare/v0.3.0...v0.4.0 )
---
updated-dependencies:
- dependency-name: tune-sklearn
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-07-19 19:42:54 -07:00
Amog Kamsetty
c9522e9a6f
Remove requests from Core Dependencies ( #17066 )
...
* remove requests
* update
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
* update
* lint
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2021-07-19 19:38:29 -07:00
Eric Liang
fabba96fad
Re-merge large function def, skipping test failing on Windows ( #17191 )
2021-07-19 18:03:26 -07:00
Amog Kamsetty
777921b2e7
[dependencies] vendor colorama ( #17183 )
2021-07-19 16:29:29 -07:00
Eric Liang
d59da075a6
Re-merge TMPDIR support, but only for Linux. OSX requires RAY_TMPDIR ( #17190 )
2021-07-19 15:45:03 -07:00
Patrick Ames
34789b3e56
[autoscaler] Add support for custom EC2 instance network interfaces ( #14080 )
...
* [autoscaler] Add support for custom EC2 instance network interfaces.
* [autoscaler] Add unit tests for custom EC2 network interfaces and support for AWS node provider stubs.
2021-07-19 17:21:21 -04:00
Amog Kamsetty
cb74053ee5
Retry remove gpustat dependency ( #17115 )
...
* remove gpustat
* move psutil imports
2021-07-19 11:14:10 -07:00
Siyuan (Ryans) Zhuang
1fbfbfc55b
[Serializatioin] Bump pickle5 version ( #17124 )
2021-07-19 10:40:38 -07:00
Siyuan (Ryans) Zhuang
9b110f9344
[Workflow] Update API ( #17165 )
...
* actor_id same as workflow_id
* @workflow.actor -> @workflow.virtual_actor
* readonly decorator
* run/run_async for virtual actor
* get_or_create for virtual actor
* update doc
* run/run_async for steps
* update tests
* update comments
2021-07-19 10:19:46 -07:00
architkulkarni
4069686e0f
Revert "Improve error message for oversized function ( #17133 )" ( #17184 )
...
This reverts commit 3e53619d64
2021-07-19 09:28:33 -07:00
dependabot[bot]
bda1a37e93
[tune](deps): Bump mlflow in /python/requirements/tune ( #17168 )
...
Bumps [mlflow](https://github.com/mlflow/mlflow ) from 1.17.0 to 1.19.0.
- [Release notes](https://github.com/mlflow/mlflow/releases )
- [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.rst )
- [Commits](https://github.com/mlflow/mlflow/compare/v1.17.0...v1.19.0 )
---
updated-dependencies:
- dependency-name: mlflow
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-07-17 16:23:10 -07:00
Eric Liang
3e53619d64
Improve error message for oversized function ( #17133 )
2021-07-17 11:04:05 -07:00
Philipp Moritz
c5c167300b
[Ray debugger] Set breakpoint() hook only for tasks and actors ( #17164 )
2021-07-17 10:27:51 -07:00
Simon Mo
9da49a7fb8
Revert "ray ignores TMPDIR variable" ( #17161 )
...
This reverts commit c27f43d9b8
2021-07-16 20:28:38 -07:00
Eric Liang
94f17ec099
[RFC] API stability annotations ( #17100 )
2021-07-16 17:09:20 -07:00
Alex Wu
93c16346bf
[Dataset] imagenet nightly test ( #17069 )
2021-07-16 14:15:49 -07:00
Eric Liang
c27f43d9b8
ray ignores TMPDIR variable ( #17130 )
2021-07-16 13:23:44 -07:00
Clark Zinzow
8302b5a335
[Core] Reverts full dispatch queue iteration PRs. ( #17127 )
...
* Revert "[Core] iterate over entire dispatch queue instead of returning when worker unavailable (#16535 )"
This reverts commit 54d66ac637#16751 )"
This reverts commit 13a133817b
2021-07-16 10:28:00 -07:00
Siyuan (Ryans) Zhuang
fd3742bb63
[Workflow] Update tests ( #17147 )
...
* update workflow tests
* use conftest
* use conftest
* use conftest
2021-07-16 09:25:40 -07:00
Eric Liang
40f1ee6e1b
Fix prefetch typo in Dataset ( #17143 )
2021-07-16 09:22:42 -07:00
fyrestone
e2808a35cf
Dashboard job module uses attrs instead of pydantic for job description ( #17116 )
2021-07-16 22:26:00 +08:00
