hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-06 10:31:39 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
13929 commits 227 branches 67 tags 234 MiB
Commit graph

176 commits

Author SHA1 Message Date
matthewdeng
78e9ff7c91
[train][datasets] add example for big data training (#20042) 
* [train][datasets] add example for big data training

* add title docstring

* lint and dependencies

* add dask_ml requirement
 2021-11-05 09:28:48 -07:00
Amog Kamsetty
adb8d77b2b
[Deps] Bump tensorflow on Docker image and add Codeowners (#20041) 2021-11-05 00:58:34 -07:00
dependabot[bot]
60e9737679
[tune](deps): Bump mlflow in /python/requirements/ml (#19913) 
Bumps [mlflow](https://github.com/mlflow/mlflow) from 1.19.0 to 1.21.0.
- [Release notes](https://github.com/mlflow/mlflow/releases)
- [Changelog](https://github.com/mlflow/mlflow/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/mlflow/mlflow/compare/v1.19.0...v1.21.0)

---
updated-dependencies:
- dependency-name: mlflow
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-04 23:37:01 -07:00
dependabot[bot]
9897ee0eab
[tune](deps): Bump onnxruntime in /python/requirements/ml (#19666) 
Bumps [onnxruntime](https://github.com/microsoft/onnxruntime) from 1.8.0 to 1.9.0.
- [Release notes](https://github.com/microsoft/onnxruntime/releases)
- [Changelog](https://github.com/microsoft/onnxruntime/blob/master/docs/ReleaseManagement.md)
- [Commits](https://github.com/microsoft/onnxruntime/compare/v1.8.0...v1.9.0)

---
updated-dependencies:
- dependency-name: onnxruntime
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-11-04 23:34:48 -07:00
dependabot[bot]
f214c4a4ab
[tune](deps): Bump datasets from 1.11.0 to 1.14.0 in /python/requirements/ml (#19645) 
* [tune](deps): Bump datasets in /python/requirements/ml

Bumps [datasets](https://github.com/huggingface/datasets) from 1.11.0 to 1.14.0.
- [Release notes](https://github.com/huggingface/datasets/releases)
- [Commits](https://github.com/huggingface/datasets/compare/1.11.0...1.14.0)

---
updated-dependencies:
- dependency-name: datasets
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update requirements_tune.txt

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>
 2021-11-04 23:33:55 -07:00
Amog Kamsetty
ede9d0ed76
[CI] Pin keras (#20032) 
* try fix

* try again

* revert back

* add todo
 2021-11-03 15:32:10 -07:00
Avnish Narayan
026bf01071
[RLlib] Upgrade gym version to 0.21 and deprecate pendulum-v0. (#19535) 
* Fix QMix, SAC, and MADDPA too.

* Unpin gym and deprecate pendulum v0

Many tests in rllib depended on pendulum v0,
however in gym 0.21, pendulum v0 was deprecated
in favor of pendulum v1. This may change reward
thresholds, so will have to potentially rerun
all of the pendulum v1 benchmarks, or use another
environment in favor. The same applies to frozen
lake v0 and frozen lake v1

Lastly, all of the RLlib tests and have
been moved to python 3.7

* Add gym installation based on python version.

Pin python<= 3.6 to gym 0.19 due to install
issues with atari roms in gym 0.20

* Reformatting

* Fixing tests

* Move atari-py install conditional to req.txt

* migrate to new ale install method

* Fix QMix, SAC, and MADDPA too.

* Unpin gym and deprecate pendulum v0

Many tests in rllib depended on pendulum v0,
however in gym 0.21, pendulum v0 was deprecated
in favor of pendulum v1. This may change reward
thresholds, so will have to potentially rerun
all of the pendulum v1 benchmarks, or use another
environment in favor. The same applies to frozen
lake v0 and frozen lake v1

Lastly, all of the RLlib tests and have
been moved to python 3.7
* Add gym installation based on python version.

Pin python<= 3.6 to gym 0.19 due to install
issues with atari roms in gym 0.20

Move atari-py install conditional to req.txt

migrate to new ale install method

Make parametric_actions_cartpole return float32 actions/obs

Adding type conversions if obs/actions don't match space

Add utils to make elements match gym space dtypes

Co-authored-by: Jun Gong <jungong@anyscale.com>
Co-authored-by: sven1977 <svenmika1977@gmail.com>
 2021-11-03 16:24:00 +01:00
Simon Mo
6040319d02
[CI] Pin aiohttp version to fix master branch (#19948) 2021-11-01 23:00:08 -07:00
Sven Mika
4d945fe651
[RLlib] Issue 19878: Re-instate bare_metal_policy example script (#19881) 2021-10-30 12:50:39 -07:00
dependabot[bot]
5ed1530170
[tune](deps): Bump starlette in /python/requirements/ml (#18691) 
Bumps [starlette](https://github.com/encode/starlette) from 0.14.2 to 0.16.0.
- [Release notes](https://github.com/encode/starlette/releases)
- [Changelog](https://github.com/encode/starlette/blob/master/docs/release-notes.md)
- [Commits](https://github.com/encode/starlette/compare/0.14.2...0.16.0)

---
updated-dependencies:
- dependency-name: starlette
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-23 18:34:14 -07:00
dependabot[bot]
55ab8da3c8
[tune](deps): Bump accelerate in /python/requirements/ml (#19057) 
Bumps [accelerate](https://github.com/huggingface/accelerate) from 0.3.0 to 0.5.1.
- [Release notes](https://github.com/huggingface/accelerate/releases)
- [Commits](https://github.com/huggingface/accelerate/compare/v0.3.0...v0.5.1)

---
updated-dependencies:
- dependency-name: accelerate
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-23 18:33:01 -07:00
dependabot[bot]
9201687b34
[tune](deps): Bump pytorch-lightning in /python/requirements/ml (#19059) 
Bumps [pytorch-lightning](https://github.com/PyTorchLightning/pytorch-lightning) from 1.4.5 to 1.4.9.
- [Release notes](https://github.com/PyTorchLightning/pytorch-lightning/releases)
- [Changelog](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/CHANGELOG.md)
- [Commits](https://github.com/PyTorchLightning/pytorch-lightning/compare/1.4.5...1.4.9)

---
updated-dependencies:
- dependency-name: pytorch-lightning
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-23 18:30:59 -07:00
dependabot[bot]
bea802cb80
[tune](deps): Bump wandb in /python/requirements/ml (#19646) 
Bumps [wandb](https://github.com/wandb/client) from 0.10.29 to 0.12.5.
- [Release notes](https://github.com/wandb/client/releases)
- [Changelog](https://github.com/wandb/client/blob/master/CHANGELOG.md)
- [Commits](https://github.com/wandb/client/compare/v0.10.29...v0.12.5)

---
updated-dependencies:
- dependency-name: wandb
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-10-23 18:23:52 -07:00
Jiajun Yao
256bf0bf3a
[Release] Bump up dask to latest compatible version 2021.9.1 (#19592) 
* Bump up dask to latest compatible version 2021.9.1

* Bump up dask to latest compatible version 2021.9.1
 2021-10-22 09:16:28 -07:00
matthewdeng
4674c78050
[Train] Rename Ray SGD v2 to Ray Train (#19436) 2021-10-18 22:27:46 -07:00
Kai Fricke
6798bdbb5d
Revert "Revert "[RLlib](deps): Bump tensorflow from 2.5.0 to 2.6.0 in /python/requirements/rllib"" (#19352) 
This reverts commit bde9e058da.
 2021-10-18 22:29:16 +01:00
Antoni Baum
e9df253f5d
[CI/docs] Remove [default] from xgboost-ray (#19186) 
Co-authored-by: Kai Fricke <kai@anyscale.com>
 2021-10-14 16:29:55 +01:00
Edward Oakes
888fb24c25
Remove deprecated ray.services package (#18475) 
Co-authored-by: Kai Fricke <kai@anyscale.com>
 2021-10-14 16:28:16 +01:00
Kai Fricke
bde9e058da
Revert "[RLlib](deps): Bump tensorflow from 2.5.0 to 2.6.0 in /python/requirements/rllib (#18183)" (#19351) 
This reverts commit 74ee99ff99.
 2021-10-13 13:06:36 +01:00
dependabot[bot]
74ee99ff99
[RLlib](deps): Bump tensorflow from 2.5.0 to 2.6.0 in /python/requirements/rllib (#18183) 
* [RLlib](deps): Bump tensorflow in /python/requirements/rllib

Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.5.0 to 2.6.0.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.5.0...v2.6.0)

---
updated-dependencies:
- dependency-name: tensorflow
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* wip.

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: sven1977 <svenmika1977@gmail.com>
 2021-10-12 17:56:36 +02:00
Carlo Grisetti
2d0355548e
[Dashboard] Try to work around aiohttp 4.0.0 breaking changes (#19120) 2021-10-11 16:25:52 -07:00
Sven Mika
d439fd7f17
[RLlib] TF2/eager memory leak fixes. (#19198) 2021-10-09 00:11:53 +02:00
dependabot[bot]
40aec916d7
[RLlib](deps): Bump pyglet from 1.5.0 to 1.5.18 in /python/requirements/rllib (#16687) 
* [RLlib](deps): Bump pyglet in /python/requirements/rllib

Bumps [pyglet](https://github.com/pyglet/pyglet) from 1.5.0 to 1.5.18.
- [Release notes](https://github.com/pyglet/pyglet/releases)
- [Changelog](https://github.com/pyglet/pyglet/blob/v1.5.18/RELEASE_NOTES)
- [Commits](https://github.com/pyglet/pyglet/compare/v1.5.0...v1.5.18)

---
updated-dependencies:
- dependency-name: pyglet
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Amog Kamsetty <amogkamsetty@yahoo.com>
Co-authored-by: sven1977 <svenmika1977@gmail.com>
 2021-09-24 19:32:55 +02:00
Kai Fricke
cee18152f1
[tune] Remove deprecated features, promote warnings to errors (#18595) 2021-09-20 22:54:28 +01:00
Amog Kamsetty
84e958f330
[ML] Consolidate and upgrade Deep Learning Dependencies (#18574) 
* wip
'

* upgrade requirements

* add file

* fix

* fixes

* Apply suggestions from code review

Try mlagents==0.21.0 for now (works with torch 1.9).

* Apply suggestions from code review

* wip

* wip

* fix

* fix

* upgrade lightning bolts

* address comment

Co-authored-by: Sven Mika <sven@anyscale.io>
 2021-09-16 20:16:40 -07:00
Antoni Baum
d50ff16ccf
[ci] Fix HEBO breaking Tune tests (#18629) 2021-09-15 10:01:29 -07:00
Kai Fricke
0223ae9605
[xgboost] Bump xgboost_ray requirements_upstream.txt version to 0.1.3 (#18632) 2021-09-15 18:01:15 +01:00
Sven Mika
8a00154038
[RLlib] Bump tf version in ML docker to tf==2.5.0; add tfp to ML-docker. (#18544) 2021-09-15 08:46:37 +02:00
dependabot[bot]
30012c990f
[tune](deps): Bump matplotlib in /python/requirements/tune (#18025) 
Bumps [matplotlib](https://github.com/matplotlib/matplotlib) from 3.4.2 to 3.4.3.
- [Release notes](https://github.com/matplotlib/matplotlib/releases)
- [Commits](https://github.com/matplotlib/matplotlib/compare/v3.4.2...v3.4.3)

---
updated-dependencies:
- dependency-name: matplotlib
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-10 16:00:16 -07:00
dependabot[bot]
42794c7a3e
[tune](deps): Bump pytorch-lightning in /python/requirements/tune (#18359) 
Bumps [pytorch-lightning](https://github.com/PyTorchLightning/pytorch-lightning) from 1.4.3 to 1.4.5.
- [Release notes](https://github.com/PyTorchLightning/pytorch-lightning/releases)
- [Changelog](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/CHANGELOG.md)
- [Commits](https://github.com/PyTorchLightning/pytorch-lightning/compare/1.4.3...1.4.5)

---
updated-dependencies:
- dependency-name: pytorch-lightning
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-09-10 15:59:26 -07:00
xwjiang2010
ae689ecc6b
[tune] Add optional Experiment to Searcher/SearchAlgo. (#17724) 2021-09-10 09:30:18 +01:00
Zhi Lin
2fcd1bcb4b
[Dataset] implement from_spark, to_spark and some optimizations (#17340) 2021-09-09 11:43:47 -07:00
Sven Mika
cabaa3b3c6
[RLlib Testing] Add A3C/APPO/BC/DDPPO/MARWIL/CQL/ES/ARS/TD3 to weekly learning tests. (#18381) 2021-09-07 11:48:41 +02:00
Kai Fricke
fb38d06cfb
Move RLLib GPU release test dependencies to ml docker (#18208) 2021-09-03 09:35:18 +01:00
Wesley Gifford
6133a561e9
Dataset from modin (#18122) 2021-08-31 11:19:35 -07:00
Yi Cheng
f579822790
[workflow] Workflow inside virtual actor (#18066) 2021-08-30 10:40:22 -07:00
Amog Kamsetty
3b77840c1b
PyTorch Lightning Updates (#17876) 2021-08-27 23:15:51 -07:00
dependabot[bot]
15adedc72c
[tune](deps): Bump sigopt in /python/requirements/tune (#17996) 
Bumps [sigopt](https://sigopt.com/) from 7.4.0 to 7.5.0.

---
updated-dependencies:
- dependency-name: sigopt
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-23 15:18:59 -07:00
dependabot[bot]
f97a292867
[tune](deps): Bump dask[complete] in /python/requirements/tune (#17997) 
Bumps [dask[complete]](https://github.com/dask/dask) from 2021.06.1 to 2021.8.1.
- [Release notes](https://github.com/dask/dask/releases)
- [Changelog](https://github.com/dask/dask/blob/main/docs/release-procedure.md)
- [Commits](https://github.com/dask/dask/compare/2021.06.1...2021.08.1)

---
updated-dependencies:
- dependency-name: dask[complete]
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-23 15:18:37 -07:00
dependabot[bot]
91d01f7211
[RLlib](deps): Bump tensorflow from 2.4.1 to 2.5.0 in /python/requirements/rllib (#15849) 2021-08-16 10:55:48 +02:00
dependabot[bot]
f6922b1768
[tune](deps): Bump optuna in /python/requirements/tune (#17853) 
Bumps [optuna](https://github.com/optuna/optuna) from 2.8.0 to 2.9.1.
- [Release notes](https://github.com/optuna/optuna/releases)
- [Commits](https://github.com/optuna/optuna/compare/v2.8.0...v2.9.1)

---
updated-dependencies:
- dependency-name: optuna
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>
 2021-08-15 01:22:58 -07:00
dependabot[bot]
b29d05c79e
[tune](deps): Bump smart-open in /python/requirements/tune (#17852) 
Bumps [smart-open](https://github.com/piskvorky/smart_open) from 5.0.0 to 5.1.0.
- [Release notes](https://github.com/piskvorky/smart_open/releases)
- [Changelog](https://github.com/RaRe-Technologies/smart_open/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/piskvorky/smart_open/compare/v5.0.0...v5.1.0)

---
updated-dependencies:
- dependency-name: smart-open
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-14 23:48:34 -07:00
dependabot[bot]
5d486e4214
[tune](deps): Bump h5py from 3.2.1 to 3.3.0 in /python/requirements/tune (#17850) 
Bumps [h5py](https://github.com/h5py/h5py) from 3.2.1 to 3.3.0.
- [Release notes](https://github.com/h5py/h5py/releases)
- [Changelog](https://github.com/h5py/h5py/blob/master/docs/release_guide.rst)
- [Commits](https://github.com/h5py/h5py/compare/3.2.1...3.3.0)

---
updated-dependencies:
- dependency-name: h5py
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-08-14 23:32:16 -07:00
dependabot[bot]
afe6ec658e
[tune](deps): Bump nevergrad in /python/requirements/tune (#17817) 
Bumps [nevergrad](https://github.com/facebookresearch/nevergrad) from 0.4.3.post3 to 0.4.3.post7.
- [Release notes](https://github.com/facebookresearch/nevergrad/releases)
- [Changelog](https://github.com/facebookresearch/nevergrad/blob/master/CHANGELOG.md)
- [Commits](https://github.com/facebookresearch/nevergrad/compare/0.4.3.post3...0.4.3.post7)

---
updated-dependencies:
- dependency-name: nevergrad
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Amog Kamsetty <amogkamsetty@yahoo.com>
 2021-08-14 21:01:48 -07:00
dependabot[bot]
030214ce5b
[tune](deps): Bump ax-platform in /python/requirements/tune (#17815) 
Bumps [ax-platform](https://github.com/facebook/Ax) from 0.1.20 to 0.2.1.
- [Release notes](https://github.com/facebook/Ax/releases)
- [Changelog](https://github.com/facebook/Ax/blob/master/CHANGELOG.md)
- [Commits](https://github.com/facebook/Ax/compare/v0.1.20...0.2.1)

---
updated-dependencies:
- dependency-name: ax-platform
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Amog Kamsetty <amogkamsetty@yahoo.com>
 2021-08-14 21:01:28 -07:00
xwjiang2010
0be9f06ab6
[tune] Output insufficent resources warning msg when trials are in pending for extended amount of time. (#17533) 2021-08-13 01:37:56 -07:00
Clark Zinzow
d6eeb5dc70
[Datasets] Add local and S3 filesystem test coverage for file-based datasources. (#17158) 2021-08-12 08:39:31 -07:00
J K Terry
48e32555c8
[rllib] Update PettingZoo dependency versions (#17702) 
* update pettingzoo dependency versions

* pettingzoo verison

* fix tests
 2021-08-11 01:19:19 -07:00
matthewdeng
e89195bfb9
[SGD] add SGDv2 Trainer prototype implementation (#17440) 
* wip

* formatting

* increase timeouts

* wip

* address comments

* comments

* fix

* address comments

* Update python/ray/util/sgd/v2/worker_group.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Update python/ray/util/sgd/v2/worker_group.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* address comments

* formatting

* fix

* wip

* finish

* fix

* formatting

* remove reporting

* split TorchBackend

* fix tests

* address comments

* add file

* more fixes

* remove default value

* update run method doc

* add comment

* minor doc fixes

* lint

* add args to BaseWorker.execute

* address comments

* remove extra parentheses

* properly instantiate backend

* fix some of the tests

* fix torch setup

* fix type hint

* [SGD] add SGDv2 Trainer prototype implementation

* add fashion mnist test

* add HuggingFace example

* format

* formatting

* address comment

* address comments

* update comment

* Update python/ray/util/sgd/v2/examples/transformers/cluster.yaml

Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>

* update huggingface transformers

* update hugging face transformers

* fix shutdown on worker failure

* Update python/requirements/tune/requirements_tune.txt

* Update python/requirements/tune/requirements_tune.txt

* Update python/requirements/tune/requirements_tune.txt

* Update python/requirements/tune/requirements_tune.txt

* address comment and fix test

Co-authored-by: Amog Kamsetty <amogkamsetty@yahoo.com>
Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
 2021-08-02 15:27:42 -07:00
SangBin Cho
9a696cc66a
Pin aioredis version (#17472) 2021-07-30 12:04:14 -07:00