Jules S. Damji
e6343f0e69
Fixed a broken code snippet with a missing method ( #20130 )
...
Signed-off-by: Jules S.Damji <jules@anyscale.com>
Co-authored-by: Jules S.Damji <jules@anyscale.com>
2021-11-08 07:56:32 +09:00
dependabot[bot]
adf39941f4
[data](deps): Bump dask[complete] ( #20125 )
...
Bumps [dask[complete]](https://github.com/dask/dask ) from 2021.9.1 to 2021.11.0.
- [Release notes](https://github.com/dask/dask/releases )
- [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md )
- [Commits](https://github.com/dask/dask/compare/2021.09.1...2021.11.0 )
---
updated-dependencies:
- dependency-name: dask[complete]
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-07 11:55:39 -08:00
Alex Wu
45d7ef7c08
[windows][ci] Skip test_multi_node_failure_2 ( #20117 )
2021-11-07 09:17:46 -08:00
Sven Mika
76f8a9f125
[RLlib; testing] Increase size of two time-out'ing test cases from medium to large. ( #20128 )
2021-11-06 21:48:28 +01:00
Jiao
9ef75b27ac
[Job Submission] Add stop API to http & sdk, with better status code + stacktrace ( #20094 )
2021-11-06 12:37:54 -05:00
SangBin Cho
7a18d90a25
Revert "[scheduler] Update local object store usage ( #20026 )" ( #20118 )
...
This reverts commit 7e013366ac
2021-11-06 07:34:27 -07:00
SangBin Cho
f65cc72b4c
Revert "Set default max_pending_lease_requests_per_scheduling_category to 10 ( #19924 )" ( #20124 )
...
This reverts commit 0d850f3302
2021-11-05 23:35:30 -07:00
Yi Cheng
6a6cc434ba
[nightly] Remove grpc staging test since nightly is stable #20119 ( #20119 )
2021-11-05 21:36:58 -07:00
Amog Kamsetty
3408b60d2b
[Release] Refactor User Tests ( #20028 )
...
* wip
* add directory
* wip
* try again
* Revert "try again"
This reverts commit 82d33ccea6f92848df025e019b87df73cea49e5d.
* finish
* formatting
* fix merge
* fix path
* chmod
* check
* sudo
* wip
* update
* fix horovod
* try
* typo
* reduce num workers
2021-11-05 17:28:37 -07:00
Alex Wu
81194f5660
[workflow][docs] Fix api comparison formatting ( #20069 )
...
## Why are these changes needed?
The API comparison formatting uses \`code\` which is rendered as italicization not code. This PR puts the code in code blocks instead of italics.
## Related issue number
## Checks
2021-11-05 17:05:35 -07:00
mwtian
4d70ce1c86
[Core][Pubsub] add worker failure message to gcs pubsub ( #20075 )
...
## Why are these changes needed?
This is to demonstrate the steps needed to add a GCS pubsub channel, with GCS publisher and C++ subscribers subscribing via GCS client. For new channels, a unit test exercising the publishing and subscribing logic should also be added to `gcs_client_test.cc`.
## Related issue number
2021-11-05 14:52:49 -07:00
Jiajun Yao
0d850f3302
Set default max_pending_lease_requests_per_scheduling_category to 10 ( #19924 )
2021-11-05 14:24:20 -07:00
xwjiang2010
866fa9590f
[tune] clean up legacy branch in update_avail_resources. ( #20071 )
2021-11-05 10:28:46 -07:00
matthewdeng
78e9ff7c91
[train][datasets] add example for big data training ( #20042 )
...
* [train][datasets] add example for big data training
* add title docstring
* lint and dependencies
* add dask_ml requirement
2021-11-05 09:28:48 -07:00
Chen Shen
320f9dc234
[Core][CoreWorker] increase the default port range ( #19541 )
...
* increase the port range
* Update doc/source/configure.rst
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
2021-11-05 09:25:44 -07:00
Alex Wu
146b3d6bcc
[scheduler] Include depth and function descriptor in scheduling class ( #20004 )
2021-11-05 08:19:48 -07:00
Simon Mo
3d5cbc6e62
[Serve] Fix HTTP error handling behavior and add tests ( #20093 )
2021-11-05 10:15:54 -05:00
Sven Mika
a931076f59
[RLlib] Tf2 + eager-tracing same speed as framework=tf; Add more test coverage for tf2+tracing. ( #19981 )
2021-11-05 16:10:00 +01:00
gjoliver
1341bb59bf
[RLlib; Release testing] long_running_tests should use RLlib's app_config. ( #20095 )
2021-11-05 15:18:56 +01:00
SangBin Cho
8299aae918
[Placement Group] Add stats to pg scheduling ( #19841 )
...
* Add an e2e stats to pg scheduling
* Fix bugs.
* fix a bug.
* Revert "fix a bug."
This reverts commit dd7e03d1346fa39e54898effaaf8a2771103176e.
* done except unit tests.
* done except unit tests.
* Add unit tests.
* Address code review.
* done
* Fix
* done
* Fixed the test
2021-11-05 06:51:42 -07:00
Sven Mika
f3397b6f48
[RLlib] Minor fixes/cleanups; chop_into_sequences now handles nested data. ( #19408 )
2021-11-05 14:39:28 +01:00
Amog Kamsetty
adb8d77b2b
[Deps] Bump tensorflow on Docker image and add Codeowners ( #20041 )
2021-11-05 00:58:34 -07:00
dependabot[bot]
60e9737679
[tune](deps): Bump mlflow in /python/requirements/ml ( #19913 )
...
Bumps [mlflow](https://github.com/mlflow/mlflow ) from 1.19.0 to 1.21.0.
- [Release notes](https://github.com/mlflow/mlflow/releases )
- [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.rst )
- [Commits](https://github.com/mlflow/mlflow/compare/v1.19.0...v1.21.0 )
---
updated-dependencies:
- dependency-name: mlflow
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-04 23:37:01 -07:00
dependabot[bot]
9897ee0eab
[tune](deps): Bump onnxruntime in /python/requirements/ml ( #19666 )
...
Bumps [onnxruntime](https://github.com/microsoft/onnxruntime ) from 1.8.0 to 1.9.0.
- [Release notes](https://github.com/microsoft/onnxruntime/releases )
- [Changelog](https://github.com/microsoft/onnxruntime/blob/master/docs/ReleaseManagement.md )
- [Commits](https://github.com/microsoft/onnxruntime/compare/v1.8.0...v1.9.0 )
---
updated-dependencies:
- dependency-name: onnxruntime
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-04 23:34:48 -07:00
dependabot[bot]
f214c4a4ab
[tune](deps): Bump datasets from 1.11.0 to 1.14.0 in /python/requirements/ml ( #19645 )
...
* [tune](deps): Bump datasets in /python/requirements/ml
Bumps [datasets](https://github.com/huggingface/datasets ) from 1.11.0 to 1.14.0.
- [Release notes](https://github.com/huggingface/datasets/releases )
- [Commits](https://github.com/huggingface/datasets/compare/1.11.0...1.14.0 )
---
updated-dependencies:
- dependency-name: datasets
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update requirements_tune.txt
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>
2021-11-04 23:33:55 -07:00
Clark Zinzow
6ade6f0be6
[Datasets] Multi-aggregations [1/3]: Add basic support for groupby multi-aggregations. ( #20044 )
2021-11-04 22:48:49 -07:00
mwtian
fb0ede38ba
[CI] [macOS] avoid installing latest setuptools ( #20064 )
2021-11-04 21:35:03 -07:00
architkulkarni
c5175073b2
[runtime env] Add garbage collection for conda envs ( #20072 )
2021-11-04 23:13:34 -05:00
Edward Oakes
360993612c
[serve] Remove lingering backend references ( #20085 )
2021-11-04 20:32:13 -05:00
Eric Liang
6102912494
Dataset doc updates ( #19815 )
2021-11-04 18:13:40 -07:00
SangBin Cho
44b38e9aa1
Add Chaos testing fixture + test actor tasks chaos test in CI ( #19975 )
...
* Basic CI tests done
* Fix an issue
* shutdown to conftest
* Addressed code review.
2021-11-04 16:27:35 -07:00
Simon Mo
4d583da7d5
[Serve] Add verbose log for nightly test only ( #20088 )
2021-11-04 16:15:22 -07:00
Edward Oakes
65161fe9b4
[job submission] Move HTTP routes to /api/jobs prefix ( #19995 )
2021-11-04 17:45:25 -05:00
javi-redondo
11371768c1
Update Ray client docs with working_dir explanation ( #18294 )
2021-11-04 14:52:28 -07:00
SangBin Cho
56bab61fba
[Placement group] Raise an exception when invalid resources are specified with the placement group. ( #19680 )
...
* done
* Make it work
* Fix issues
* done
* try
* done
* Fix remaining bugs.
2021-11-04 14:41:00 -07:00
Eric Liang
585d472fdf
Add configuration context to dataset ( #19907 )
2021-11-04 14:36:51 -07:00
Alex Wu
4ffb7ccfac
[scheduler][cleanup] Remove one cpu optimization ( #20022 )
...
* .
* remove test
* Update cluster_task_manager.cc
* Update cluster_task_manager.cc
* lint
* lint
* .
Co-authored-by: Alex Wu <alex@anyscale.com>
2021-11-04 14:18:13 -07:00
Edward Oakes
49d308138f
[serve] Rename backend_state -> deployment_state ( #20040 )
2021-11-04 15:46:45 -05:00
javi-redondo
5781f44cc9
Update config.rst to reflect min nodes on autoscaling up ( #15589 )
2021-11-04 13:40:50 -07:00
Yi Cheng
04f60c998e
[nightly] Fix pytest missing in nightly test ( #20076 )
...
## Why are these changes needed?
In the nightly test we see
```
Command returned non-success status: 1; Command logs:Traceback (most recent call last): File "dask_on_ray/large_scale_test.py", line 17, in from ray._private.test_utils import monitor_memory_usage File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/_private/test_utils.py", line 18, in import pytest ModuleNotFoundError: No module named 'pytest'
```
This PR fixes this error.
## Related issue number
2021-11-04 13:38:05 -07:00
Philipp Moritz
a64e32c53b
[docs] Fix broken links in documentation and add linkcheck to documentation ( #20030 )
...
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2021-11-04 13:19:43 -07:00
Sven Mika
50c30f89c6
[Tune; RLlib] Move Tune tests that use RLlib into separate buildkite job. ( #20016 )
2021-11-04 20:40:57 +01:00
Jiao
6cfb52ff1d
[job submission] Add stop API + subprocess cleanup ( #19860 )
2021-11-04 13:59:47 -05:00
Alex Wu
36a214386f
[docs] PyData ray dataset talk ( #20038 )
2021-11-04 10:33:18 -07:00
Yi Cheng
65d3054a09
[build] fix the wrong flag for gcs ha test ( #20052 )
...
## Why are these changes needed?
It should be `RAY_gcs_grpc_based_pubsub` instead of `Ray_gcs_grpc_based_pubsub`
## Related issue number
2021-11-04 09:59:11 -07:00
Yi Cheng
7bb4c87780
[gcs] use gcs kv in internal kv ( #19933 )
...
## Why are these changes needed?
It's part of redis removal project. This PR focus on using gcs kv in internal kv.
- gcs client is introduced
- internal kv is updated to use gcs rpc client based kv
- related code got updated.
The other PR will update components using redis to use internal kv.
## Related issue number
https://github.com/ray-project/ray/issues/19443
2021-11-04 09:57:39 -07:00
Chen Shen
5c0e012ba3
[Core][Core-worker] de-escalate the error message when worker is accessed after shutdown. ( #20049 )
...
* de-escalate
* lint
2021-11-04 09:56:39 -07:00
Yi Cheng
b3b88a46f7
[pg] Fix the test case which hangs because of scheduling dead lock ( #20048 )
...
## Why are these changes needed?
In this test case, the following case could happen:
1. actor creation first uses all resource in local node which is a GPU node
2. the actor need GPU will not be able to be scheduled since we only have one GPU node
The fixing is just a short term fix and only tries to connect to the head node with CPU resources.
## Related issue number
#19438
2021-11-04 09:56:23 -07:00
Amog Kamsetty
f67b526b7a
[Tune] Fix PTL tutorial docs ( #19999 )
2021-11-04 09:21:28 -07:00
xwjiang2010
f1179cbccd
[tune] Remove unused clean_trial_placement_group. ( #19960 )
2021-11-04 08:55:42 -07:00
