Jiajun Yao
47744d282c
[data] Fix arrow dataset sort on empty blocks ( #19707 )
2021-10-26 15:30:23 -07:00
Eric Liang
2652ae7905
[client] Put of a list should not return a list, this is a client bug ( #19737 )
2021-10-26 13:51:37 -07:00
iasoon
b5158ca0ab
[serve] Correctly set num_replicas when deploying autoscaling deployment ( #19520 )
2021-10-26 12:10:59 -05:00
SangBin Cho
00ea716ada
Revert "Revert "[Core] [Placement Group] Fix bundle reconstruction when raylet fo after gcs fo ( #19452 )" ( #19724 )" ( #19736 )
...
This reverts commit d453afbab8
2021-10-26 08:25:09 -07:00
Jiao
aaef82920d
[serve] Add periodic timeouts to long poll client to avoid accumulating concurrent tasks in the controller ( #19728 )
2021-10-26 09:44:00 -05:00
Kai Fricke
3081488a99
[tune] Fix local checkpoint deletion for remote trials ( #19632 )
2021-10-26 09:18:07 +01:00
Eric Liang
81b0eb297c
Un-revert size estimator and fix Train test ( #19719 )
2021-10-25 22:09:24 -07:00
SangBin Cho
d453afbab8
Revert "[Core] [Placement Group] Fix bundle reconstruction when raylet fo after gcs fo ( #19452 )" ( #19724 )
...
This reverts commit e3ced0e59e
2021-10-26 09:14:25 +09:00
Simon Mo
5330aab27a
[CI] Deflake test metrics ( #19711 )
2021-10-25 16:34:20 -07:00
Eric Liang
66818d11b8
Revert "[data] Add serialized size estimator to block builder ( #19681 )" ( #19717 )
...
This reverts commit 8c37311c41
2021-10-25 15:06:58 -07:00
Eric Liang
8c37311c41
[data] Add serialized size estimator to block builder ( #19681 )
2021-10-25 14:58:49 -07:00
SangBin Cho
ecd5a622ef
[Tests] Add a memory usage on dask on ray tests ( #19674 )
2021-10-25 14:58:26 -07:00
SangBin Cho
544f774245
[Autoscaler/Core] Drain node API ( #19350 )
...
* Initial version done. Graceful shutdown is possible with direct raylet RPCs
* .
* .
* ip
* Done.
* done tests might fail
* fix lint + cpp tests
* fix 2
* Fix issues.
* Addressed code review.
* Fix another cpp test failure
* completed
* Skip windows tests
* Update the comment
* complete
* addressed code review.
2021-10-25 14:57:50 -07:00
Linsong Chu
13d4894789
[workflow] Add get_metadata() for workflow ( #19372 )
...
## Why are these changes needed?
Add the functionality to retrieve metadata for a workflow or workflow step.
Design:
- Similar to `get_output`, this will either return the metadata for workflow (`workflow.get_metadata(workflow_id)`) or the metadata for a specific step (`workflow.get_metadata(workflow_id, step_id)`)
- Exceptions will only be raised if workflow id or step id not exist. Canceled job, running job, etc. will return proper metadata by retrieving information from checkpoint. See [here](8c8ca609d7/python/ray/workflow/tests/test_metadata_get.py (L67)https://github.com/ray-project/ray/issues/17090#issuecomment-920481789 ). The aggregation logic is [here for step metadata](8c8ca609d7/python/ray/workflow/workflow_storage.py (L451)8c8ca609d7/python/ray/workflow/workflow_storage.py (L484)https://github.com/ray-project/ray/issues/17090
2021-10-25 14:52:51 -07:00
Alex Wu
58b28f04cd
[docs/usability] Apple Silicon support ( #19705 )
...
This PR puts the final touches on apple silicon support. There are 3 main caveats to supporting M1 macs right now (described in the docs):
Requires using forge.
Requires special installation instructions to get grpc working (this is an underlying grpc issue, so ideally it will be fixed upstream).
We're only publishing release wheels, not nightlies right now.
This also includes a grpc import check to ensure that we provide an actionable error message if the user tries the regular pip install ray process to properly install grpcio.
2021-10-25 14:49:28 -07:00
DK.Pino
e3ced0e59e
[Core] [Placement Group] Fix bundle reconstruction when raylet fo after gcs fo ( #19452 )
...
* fixed
* lint
* add cxx ut
* fix comment
* Revert "fix comment"
This reverts commit 32ea2558166a7674d7efe2e0c0a66ea7409c7d99.
* fix comment
2021-10-25 14:15:36 -07:00
Eric Liang
27a5b546ad
Make ArrowRow less scary ( #19686 )
2021-10-25 12:18:42 -07:00
Tao Wang
ff7d35d246
[Core]Add test case for cached named actor ( #19510 )
...
## Why are these changes needed?
Recently we found a bug about named actor cache, only in internal codebase but not community, and the case is not covered by test case so we didn't know before user telling us.
This add an extra test to cover it.
Bug Detail: we didn't publish actor's name when the actor is dead so the cache keep the name to the old actor handle. The owner of this actor cannot sense this bug because the cache didn't apply to the owner currently.
2021-10-25 11:37:41 -07:00
xwjiang2010
46266b15f0
[tune] Avoid looping through _live_trials twice in _get_next_trial. ( #19596 )
2021-10-25 19:26:55 +01:00
chenk008
b65aca9002
flush stdout/stderr to avoid empty log in docker start block ( #19546 )
2021-10-25 10:58:48 -07:00
architkulkarni
f101f7cc02
[runtime_env] Allow specifying runtime env in @ray.remote decorator with Ray Client ( #19626 )
2021-10-25 10:32:31 -05:00
Kai Fricke
6e455e59d8
[tune] Verbosely/gracefully handle empty experiment checkpoints ( #19641 )
2021-10-25 13:41:18 +01:00
Kai Fricke
0cfa267fde
[tune] Fix shim error message for scheduler ( #19642 )
2021-10-25 11:16:16 +01:00
SangBin Cho
aa9eb6499c
[Test] skip pg restart test ( #19670 )
2021-10-24 16:53:29 -07:00
Philipp Moritz
22eef65134
[Windows] Suppress 'Windows fatal exception: access violation' ( #19561 )
...
* suppress 'Windows fatal exception: access violation'
* lint
* update
* Update python/ray/_private/log_monitor.py
Co-authored-by: Matti Picus <matti.picus@gmail.com>
* fixE
* re-introduce mattip's fix again
* update
* .
Co-authored-by: Matti Picus <matti.picus@gmail.com>
Co-authored-by: Alex <alex@anyscale.com>
2021-10-24 11:23:23 -07:00
dependabot[bot]
5ed1530170
[tune](deps): Bump starlette in /python/requirements/ml ( #18691 )
...
Bumps [starlette](https://github.com/encode/starlette ) from 0.14.2 to 0.16.0.
- [Release notes](https://github.com/encode/starlette/releases )
- [Changelog](https://github.com/encode/starlette/blob/master/docs/release-notes.md )
- [Commits](https://github.com/encode/starlette/compare/0.14.2...0.16.0 )
---
updated-dependencies:
- dependency-name: starlette
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-23 18:34:14 -07:00
dependabot[bot]
55ab8da3c8
[tune](deps): Bump accelerate in /python/requirements/ml ( #19057 )
...
Bumps [accelerate](https://github.com/huggingface/accelerate ) from 0.3.0 to 0.5.1.
- [Release notes](https://github.com/huggingface/accelerate/releases )
- [Commits](https://github.com/huggingface/accelerate/compare/v0.3.0...v0.5.1 )
---
updated-dependencies:
- dependency-name: accelerate
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-23 18:33:01 -07:00
dependabot[bot]
9201687b34
[tune](deps): Bump pytorch-lightning in /python/requirements/ml ( #19059 )
...
Bumps [pytorch-lightning](https://github.com/PyTorchLightning/pytorch-lightning ) from 1.4.5 to 1.4.9.
- [Release notes](https://github.com/PyTorchLightning/pytorch-lightning/releases )
- [Changelog](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/CHANGELOG.md )
- [Commits](https://github.com/PyTorchLightning/pytorch-lightning/compare/1.4.5...1.4.9 )
---
updated-dependencies:
- dependency-name: pytorch-lightning
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-23 18:30:59 -07:00
dependabot[bot]
bea802cb80
[tune](deps): Bump wandb in /python/requirements/ml ( #19646 )
...
Bumps [wandb](https://github.com/wandb/client ) from 0.10.29 to 0.12.5.
- [Release notes](https://github.com/wandb/client/releases )
- [Changelog](https://github.com/wandb/client/blob/master/CHANGELOG.md )
- [Commits](https://github.com/wandb/client/compare/v0.10.29...v0.12.5 )
---
updated-dependencies:
- dependency-name: wandb
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-23 18:23:52 -07:00
Amog Kamsetty
97878162f4
[ActorGroup] Retry ActorGroup ( #19658 )
2021-10-23 16:37:29 -05:00
Edward Oakes
445fb0ee99
[runtime_env] Deflake test_runtime_env_working_dir.py ( #19665 )
2021-10-23 16:35:42 -05:00
Eric Liang
875d19f838
[data] Fix inconsistent naming of to_refs() methods, remove to_arrow() ( #19620 )
2021-10-23 12:20:23 -07:00
Jiao
e53fecfbd5
[jobs] Initial http jobs server on head node ( #19657 )
2021-10-23 12:48:16 -05:00
Philipp Moritz
dbd61b9e6b
[Dashboard] Include the dashboard in Windows wheels ( #19575 )
2021-10-22 17:57:36 -07:00
Jiajun Yao
a7b219fea1
[Core] Don't unpickle and run functions exported by other jobs ( #19576 )
2021-10-22 17:13:20 -07:00
Gagandeep Singh
358aa57474
Fixed usage of `cv_.wait_for` ( #19582 )
...
* Fixed usage of cv.wait_for
* Changed method to calculate remaining time out
* Modify timeout_ms -> remaining_timeout_ms
2021-10-22 16:23:13 -07:00
Edward Oakes
b4673daac6
[ray client] Add test that ray.init doesn't require resources to connect ( #19635 )
2021-10-22 18:21:53 -05:00
Alex Wu
31d89be926
[Workflow] Basic event support ( #19239 )
...
* basics
* .
* .
* a test
* a test
* tests
* cleanup
* concepts page
* docs
* polish
* fix sleep
* fix yi things
* lint
* fix
* .
* .
* .
* fix?
* .
Co-authored-by: Alex Wu <alex@anyscale.com>
2021-10-22 15:27:33 -07:00
Edward Oakes
c9258aff0f
Revert "[ActorGroup] Add ActorGroup ( #18960 )" ( #19655 )
...
This reverts commit 4f05bac8fb
2021-10-22 14:55:17 -07:00
shrekris-anyscale
cfae64ebe8
[multiprocessing] Modify Ray's map_async() to match Multiprocessing's map_async() behavior ( #19403 )
2021-10-22 16:31:34 -05:00
Gagandeep Singh
2f8da8f8c8
Bumped timeout due to slow test times in Windows ( #19595 )
2021-10-22 13:48:15 -07:00
Jiao
f0be4cb390
[jobs] Add job manager class for simple jobs python APIs ( #19567 )
2021-10-22 14:18:11 -05:00
Jiajun Yao
43b8f8e522
Revert "Revert "[Test] Fix flaky test_gpu test ( #19524 )" ( #19562 )" ( #19643 )
...
This reverts commit 7daf28f348
2021-10-22 11:48:57 -07:00
Edward Oakes
0760fe869d
[runtime_env] Clean up working dir tests, add more test cases ( #19597 )
2021-10-22 12:35:27 -05:00
Amog Kamsetty
4f05bac8fb
[ActorGroup] Add ActorGroup ( #18960 )
...
* move
* fix
* Revert "fix"
This reverts commit 532660fc334ae96a0ff34c8ab1288488312300a3.
* Revert "move"
This reverts commit 54321f4a539c2ee873f17d988da5627588aeff97.
* add
* wip
* wip
* wip
* wip
* address comments
* wip
* add to build
* fix
* fix
* fix
2021-10-22 10:22:31 -07:00
Simon Mo
1eb142b57c
[Serve] Fix shutdown protocol again ( #19609 )
2021-10-22 09:27:32 -07:00
Jiajun Yao
256bf0bf3a
[Release] Bump up dask to latest compatible version 2021.9.1 ( #19592 )
...
* Bump up dask to latest compatible version 2021.9.1
* Bump up dask to latest compatible version 2021.9.1
2021-10-22 09:16:28 -07:00
architkulkarni
030acf3857
[Serve] [Serve Autoscaler] Add upscale and downscale delay ( #19290 )
2021-10-22 10:33:28 -05:00
xwjiang2010
a632cb439f
[Tune] Remove queue_trials. ( #19472 )
2021-10-22 09:24:54 +01:00
Stephanie Wang
499d6e9fc1
Turn on reconstruction tests in CI ( #19497 )
2021-10-21 22:34:44 -07:00
