Antoni Baum
d449f8db27
[CI] Update upstream requirements for XGB/LGBM-Ray ( #27908 )
...
To include these in the latest docker images (and get rid of deprecation warnings), bump in requirements_upstream.txt.
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
2022-08-17 10:55:02 +02:00
matthewdeng
83fb2fb21d
[tune] pin pymoo ( #27311 )
...
Signed-off-by: Matthew Deng <matt@anyscale.com>
2022-07-31 01:06:27 -07:00
Steven Morad
259429bdc3
Bump gym dep to 0.24 ( #26190 )
...
Co-authored-by: Steven Morad <smorad@anyscale.com>
Co-authored-by: Avnish <avnishnarayan@gmail.com>
Co-authored-by: Avnish Narayan <38871737+avnishn@users.noreply.github.com>
2022-07-22 12:37:16 -07:00
Antoni Baum
cc7115f6a2
[Tune/CI] Fix tune-sklearn notebook example ( #26470 )
...
Fixes the tune-sklearn notebook example as found in #26410
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
2022-07-13 18:14:36 +01:00
Kai Fricke
9b49417a72
[ci/hotfix] Pin raydp-nightly ( #26358 )
...
Alternative to #26356 - here we just pin raydp-nightly and resolve the dependency issues in follow-up PRs.
This is to quickly unblock CI.
Signed-off-by: Kai Fricke <kai@anyscale.com>
2022-07-07 14:54:01 +01:00
Kai Fricke
4b9a89ad90
[air] Move python/ray/ml to python/ray/air ( #25449 )
...
The package "ml" should be renamed to "air".
Main question: Keep a `ml.py` with `from ray.air import *` for some level of backwards compatibility?
I'd go for no to force people to use the new structure.
2022-06-03 21:53:44 +01:00
Jiajun Yao
be93bb340d
Remove MLDataset ( #25041 )
...
MLDataset is replaced by Ray Dataset.
2022-05-25 09:33:29 -07:00
Sven Mika
09886d7ab8
[RLlib] Upgrade gym 0.23 ( #24171 )
2022-05-23 08:18:44 +02:00
Antoni Baum
1d5e6d908d
[AIR] HuggingFace Text Classification example ( #24402 )
2022-05-18 09:35:12 -07:00
Antoni Baum
668049492c
[Datasets] Add from_huggingface for Hugging Face datasets integration ( #24464 )
...
Adds a from_huggingface method to Datasets, which allows the conversion of a Hugging Face Dataset to a Ray Dataset. As a Hugging Face Dataset is backed by an Arrow table, the conversion is trivial.
2022-05-06 13:09:28 -07:00
Adrish Dey
d02b4cb2d6
Adding support to wandb service ( #24017 )
...
Updating W&B Ray Tune Integration with new standards. Adding support to wandb service, the soon to be default way for multiprocessing + wandb run logging.
Co-authored-by: Kai Fricke <krfricke@users.noreply.github.com>
2022-05-02 15:47:08 +01:00
Antoni Baum
ff0ced1a64
[AIR] HuggingFaceTrainer&Predictor implementation ( #23876 )
...
Implements HuggingFaceTrainer & HuggingFacePredictor.
2022-04-29 14:31:54 -07:00
Balaji Veeramani
2190f7ff25
[Datsets] Add SimpleTensorFlowDatasource ( #24022 )
...
This PR makes it easier to use TensorFlow datasets with Ray Datasets.
2022-04-29 12:15:30 -07:00
Shawn
43ed78f6fd
[Datasets] Integrate Mars-on-Ray with Datasets; improve docs and add tests ( #23402 )
...
Add Mars-on-Ray + Datasets integration; improve Mars-on-Ray docs and add tests.
2022-04-29 09:43:52 -07:00
Amog Kamsetty
732175e245
[AIR] Add distributed torch_geometric example ( #23580 )
...
Add example for distributed pytorch geometric (graph learning) with Ray AIR
This only showcases distributed training, but with data small enough that it can be loaded in by each training worker individually. Distributed data ingest is out of scope for this PR.
Co-authored-by: matthewdeng <matthew.j.deng@gmail.com>
2022-04-21 09:48:43 -07:00
Avnish Narayan
c9df6ce70c
[RLlib] Pinning gym to 0.21 to fix test issues ( #24000 )
2022-04-19 08:33:31 +02:00
Kai Fricke
e3bd59882d
[air] Move storage handling to pyarrow.fs.FileSystem ( #23370 )
2022-04-13 14:31:30 -07:00
Kai Fricke
d27e73f851
[ci] Pin prometheus_client to fix current test outages ( #23749 )
...
What: Pins prometheus_client to < 0.14.0, hopefully fixing today's CI outages
Why: New version of the python client (https://github.com/prometheus/client_python/releases ) breaks our CI
2022-04-06 14:22:22 -07:00
dependabot[bot]
e69f7f33ee
[tune](deps): Bump optuna in /python/requirements/ml ( #19669 )
...
Bumps [optuna](https://github.com/optuna/optuna ) from 2.9.1 to 2.10.0.
- [Release notes](https://github.com/optuna/optuna/releases )
- [Commits](https://github.com/optuna/optuna/compare/v2.9.1...v2.10.0 )
---
updated-dependencies:
- dependency-name: optuna
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kai Fricke <kai@anyscale.com>
2022-03-25 17:58:22 +00:00
ddelange
e109c13b83
[ci] Clean up ray-ml requirements ( #23325 )
...
In https://github.com/ray-project/ray/blob/ray-1.11.0/docker/ray-ml/Dockerfile , the order of pip install commands currently matters (potentially a lot). It would be good to run one big pip install command to avoid ending up with a broken env.
Co-authored-by: Kai Fricke <krfricke@users.noreply.github.com>
2022-03-25 15:59:54 +00:00
dependabot[bot]
05bfcdbaf8
[tune](deps): Bump ax-platform in /python/requirements/ml ( #23098 )
...
Bumps [ax-platform](https://github.com/facebook/Ax ) from 0.2.1 to 0.2.4.
- [Release notes](https://github.com/facebook/Ax/releases )
- [Changelog](https://github.com/facebook/Ax/blob/main/CHANGELOG.md )
- [Commits](https://github.com/facebook/Ax/compare/0.2.1...0.2.4 )
---
updated-dependencies:
- dependency-name: ax-platform
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-23 10:13:00 +00:00
Avnish Narayan
e008a48ef2
[release tests] Pin gym everywhere ( #23349 )
2022-03-19 02:52:54 -07:00
Amog Kamsetty
d31d6bc9bb
[Docker] Add Train requirements to ray-ml docker image ( #22645 )
2022-03-17 15:07:32 -07:00
siddgoel
0722cbb37e
Add support for snappy text decompression #22298 ( #22486 )
...
Adds a streaming based reading option for Snappy-compressed files. Arrow doesn't support streaming Snappy decompression since the canonical C++ Snappy library doesn't natively support streaming decompression. This PR works around this by doing streaming reads of snappy-compressed files using the streaming decompression API provided in the [python-snappy](https://github.com/andrix/python-snappy ) package.
This commit supplies a custom datasource that uses Arrow + [python-snappy](https://github.com/andrix/python-snappy ) to read and decompress Snappy-compressed files.
Co-authored-by: siddharth.goel <siddharth.goel@bytedance.com>
Co-authored-by: Chen Shen <scv119@gmail.com>
2022-03-15 13:52:22 -07:00
mwtian
aad6f41593
[Tune] Remove unused autogluon requirement ( #16587 )
...
`autogluon` does not support Python 3.9. And Ray seems to not import it anywhere.
2022-03-11 16:54:23 -08:00
Daniel
8d1f1b0a64
[RLlib] Update pettingzoo==1.15.0 supersuit==3.3.3 ( #22519 )
2022-03-01 11:23:27 +01:00
Sven Mika
7b687e6cd8
[RLlib] SlateQ: Add a hard-task learning test to weekly regression suite. ( #22544 )
2022-02-25 21:58:16 +01:00
Amog Kamsetty
04feea4afe
[rllib] Upper bound gym version ( #22510 )
...
gym had 0.22 release today which is breaking a lot of the rllib tests and examples. Temporarily pins gym version for now.
2022-02-18 17:39:22 -08:00
dependabot[bot]
35ae459434
[tune](deps): Bump flaml from 0.6.7 to 0.9.7 in /python/requirements/ml ( #22071 )
...
* [tune](deps): Bump flaml from 0.6.7 to 0.9.6 in /python/requirements/ml
Bumps [flaml](https://github.com/microsoft/FLAML ) from 0.6.7 to 0.9.6.
- [Release notes](https://github.com/microsoft/FLAML/releases )
- [Commits](https://github.com/microsoft/FLAML/compare/v0.6.7...v0.9.6 )
---
updated-dependencies:
- dependency-name: flaml
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
2022-02-15 09:53:58 +00:00
dependabot[bot]
767b349b99
[data](deps): Bump dask[complete] ( #22334 )
...
Bumps [dask[complete]](https://github.com/dask/dask ) from 2022.1.0 to 2022.2.0.
- [Release notes](https://github.com/dask/dask/releases )
- [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md )
- [Commits](https://github.com/dask/dask/compare/2022.01.0...2022.02.0 )
---
updated-dependencies:
- dependency-name: dask[complete]
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-14 12:44:20 -08:00
Kaushik B
8515fdd6db
[tune] Update Lightning examples to support PTL 1.5 ( #20562 )
...
To helps resolve the issues users are facing with running Lightning examples with Ray Tune PyTorchLightning/pytorch-lightning#10407
Co-authored-by: Amog Kamsetty <amogkamsetty@yahoo.com>
2022-02-11 17:45:06 -08:00
SangBin Cho
20ab9188c6
[Ray Usage Stats] Record cluster metadata + Refactoring. ( #22170 )
...
This is the first PR to implement usage stats on Ray. Please refer to the file `usage_lib.py` for more details.
The full specification is here https://docs.google.com/document/d/1ZT-l9YbGHh-iWRUC91jS-ssQ5Qe2UQ43Lsoc1edCalc/edit#heading=h.17dss3b9evbj .
You can see the full PR for phase 1 from here; https://github.com/rkooo567/ray/pull/108/files .
The PR is doing some basic refactoring + adding cluster metadata to GCS instead of the version numbers.
After this PR, we will add code to enable usage report "off by default".
2022-02-08 22:12:36 -08:00
Sven Mika
c17a44cdfa
Revert "Revert "[RLlib] AlphaStar: Parallelized, multi-agent/multi-GPU learni…" ( #22153 )
2022-02-08 16:43:00 +01:00
SangBin Cho
a887763b38
Revert "[RLlib] AlphaStar: Parallelized, multi-agent/multi-GPU learni… ( #22105 )
...
This reverts commit 3f03ef8ba8
2022-02-04 00:54:50 -08:00
Sven Mika
3f03ef8ba8
[RLlib] AlphaStar: Parallelized, multi-agent/multi-GPU learning via league-based self-play. ( #21356 )
2022-02-03 09:32:09 +01:00
Dhruv Nair
3d79815cd0
Comet Integration ( #20766 )
...
This PR adds a `CometLoggerCallback` to the Tune Integrations, allowing users to log runs from Ray to [Comet](https://www.comet.ml/site/ ).
Co-authored-by: Michael Cullan <mjcullan@gmail.com>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
2022-01-25 11:42:00 -08:00
Sven Mika
c288b97e5f
[RLlib] Issue 21629: Video recorder env wrapper not working. Added test case. ( #21670 )
2022-01-24 19:38:21 +01:00
dependabot[bot]
1f563aaf9b
[data](deps): Bump dask[complete] from 2021.11.0 to 2022.1.0 in /python/requirements/data_processing ( #21621 )
...
Bumps [dask[complete]](https://github.com/dask/dask ) from 2021.11.0 to 2022.1.0.
2022-01-18 15:32:07 -08:00
Antoni Baum
6ba4513777
[tune] Load experiment into searcher ( #21506 )
...
This PR adds a new method to the Searcher class, add_evaluated_trials. This method wraps around add_evaluated_point and allows the user to pass a Trial, list of Trials or ExperimentAnalysis to load into the searcher. Furthermore, this PR updates the HEBO version to the latest and removes outdated documentation, and adds add_evaluated_point methods to Dragonfly and SkOpt searchers.
2022-01-11 15:58:20 -08:00
Yi Cheng
d2d749b6f9
[workflow] Fix test_serialization.py ( #21522 )
...
The new version of responses will introduce some errors in the test. This PR fixed responses.
It also fixed moto in case of future updates upstream.
2022-01-11 11:45:18 -08:00
Sven Mika
c01245763e
[RLlib] Revert "Revert "updated pettingzoo wrappers, env versions, urls"" ( #21339 )
2022-01-04 18:30:26 +01:00
Kai Fricke
489e6945a6
Revert "[RLlib] Updated pettingzoo wrappers, env versions, urls ( #20113 )" ( #21338 )
...
This reverts commit 327eb84154
2022-01-03 10:21:25 +00:00
Benjamin Black
327eb84154
[RLlib] Updated pettingzoo wrappers, env versions, urls ( #20113 )
2022-01-02 21:29:09 +01:00
Amog Kamsetty
57db4640ca
[Train] [Tune] Refactor MLflow ( #20802 )
...
Pulls out Tune's MLflow logging logic to a shared MLflow util.
Adds an MLflow logger callback to Ray Train
Closes #20642
2021-12-21 17:17:52 -08:00
matthewdeng
0de105d42f
[train] update Trainer._is_tune_enabled to work when Tune is not installed ( #20767 )
2021-11-29 20:08:51 -08:00
matthewdeng
caa4ff3783
[train][datasets] update example and remove dask ( #20592 )
2021-11-21 17:06:44 -08:00
shrekris-anyscale
a91ddbdeb9
Add smart_open dependency to ray[default] ( #20420 )
2021-11-18 10:00:30 -06:00
gjoliver
724a140795
[rllib] Make sure json can serialize result dict ( #20439 )
...
We may have fields in the result dict that are or None.
Make sure our results are json serializable.
2021-11-17 10:27:00 -08:00
Teofilo Zosa
abf0eb53cc
Fix aiohttp 3.8.0 breaking changes (and unpin from 3.7) ( #20261 )
2021-11-11 15:35:20 -08:00
Amog Kamsetty
f8430e6eca
[CI] Pin shortuuid to fix CI ( #20153 )
2021-11-08 12:08:32 -08:00
