Commit graph

406 commits

Author SHA1 Message Date
Sven Mika
50c30f89c6
[Tune; RLlib] Move Tune tests that use RLlib into separate buildkite job. (#20016) 2021-11-04 20:40:57 +01:00
Jiao
6cfb52ff1d
[job submission] Add stop API + subprocess cleanup (#19860) 2021-11-04 13:59:47 -05:00
Sven Mika
4cb23d1c95
[Tune; Testing] Revert to 3.7 (undone by accident by previous PR); + some minor comment cleanups. (#20031) 2021-11-04 10:58:34 +01:00
Avnish Narayan
026bf01071
[RLlib] Upgrade gym version to 0.21 and deprecate pendulum-v0. (#19535)
* Fix QMix, SAC, and MADDPA too.

* Unpin gym and deprecate pendulum v0

Many tests in rllib depended on pendulum v0,
however in gym 0.21, pendulum v0 was deprecated
in favor of pendulum v1. This may change reward
thresholds, so will have to potentially rerun
all of the pendulum v1 benchmarks, or use another
environment in favor. The same applies to frozen
lake v0 and frozen lake v1

Lastly, all of the RLlib tests and have
been moved to python 3.7

* Add gym installation based on python version.

Pin python<= 3.6 to gym 0.19 due to install
issues with atari roms in gym 0.20

* Reformatting

* Fixing tests

* Move atari-py install conditional to req.txt

* migrate to new ale install method

* Fix QMix, SAC, and MADDPA too.

* Unpin gym and deprecate pendulum v0

Many tests in rllib depended on pendulum v0,
however in gym 0.21, pendulum v0 was deprecated
in favor of pendulum v1. This may change reward
thresholds, so will have to potentially rerun
all of the pendulum v1 benchmarks, or use another
environment in favor. The same applies to frozen
lake v0 and frozen lake v1

Lastly, all of the RLlib tests and have
been moved to python 3.7
* Add gym installation based on python version.

Pin python<= 3.6 to gym 0.19 due to install
issues with atari roms in gym 0.20

Move atari-py install conditional to req.txt

migrate to new ale install method

Make parametric_actions_cartpole return float32 actions/obs

Adding type conversions if obs/actions don't match space

Add utils to make elements match gym space dtypes

Co-authored-by: Jun Gong <jungong@anyscale.com>
Co-authored-by: sven1977 <svenmika1977@gmail.com>
2021-11-03 16:24:00 +01:00
Sven Mika
e6ae08f416
[RLlib] Optionally don't drop last ts in v-trace calculations (APPO and IMPALA). (#19601) 2021-11-03 10:01:34 +01:00
Simon Mo
6040319d02
[CI] Pin aiohttp version to fix master branch (#19948) 2021-11-01 23:00:08 -07:00
mwtian
7afdfdc6dd
[CI] narrow down tests that run when files change (#19656) 2021-10-29 16:47:54 -07:00
matthewdeng
bfb0ef1b08
move jsonschema to core dependencies and update default AutoscalerPrometheusMetrics (#19831) 2021-10-28 13:04:22 -07:00
Amog Kamsetty
5d54412f1c
[Docker] Alias ray-ml:nightly to ray-ml:nightly-gpu (#19726)
* wip

* wip

* update

* finish

* deprecate

* debug

* fix and address comments

* try catch

* fix

* split tests

* force

* merge

* docs

* wip

* fix and check

* update readme

* fix

* fix

* fix sanity checking

* format

* alias

* fix

* comment
2021-10-27 11:30:49 -07:00
Simon Mo
40d52edabc
[CI] Upload wheels to artifact store in all jobs (#19778) 2021-10-27 10:27:56 +01:00
Amog Kamsetty
db863aafc0
Revert "Revert "[Docker] Support multiple CUDA Versions (#19505)" (#19756)" (#19763)
This reverts commit e58fcca404.
2021-10-26 17:32:56 -07:00
Amog Kamsetty
e58fcca404
Revert "[Docker] Support multiple CUDA Versions (#19505)" (#19756)
This reverts commit f0053d405b.
2021-10-26 12:55:20 -07:00
Amog Kamsetty
f0053d405b
[Docker] Support multiple CUDA Versions (#19505)
* wip

* wip

* update

* finish

* deprecate

* debug

* fix and address comments

* try catch

* fix

* split tests

* force

* merge

* docs

* wip

* fix and check

* update readme

* fix

* fix

* fix sanity checking

* format
2021-10-25 18:57:05 -07:00
Simon Mo
48cf366dca
[Hotfix] Pin node version to 14 (#19522) 2021-10-19 14:13:06 -07:00
gjoliver
2bd7932830
Add a script to analyze python module dependencies using static analysis (#18965) 2021-10-19 13:33:02 -07:00
mwtian
3260330e45
Disable clang-tidy until ergonomic issues are resolved (#19499)
Why are these changes needed?
Currently clang-tidy does not run inside scripts/format.sh. Also clang-tidy can produce false positive warnings. Maybe we can disable clang-tidy until ergonomic issues are resolved.
2021-10-19 08:45:25 -07:00
matthewdeng
4674c78050
[Train] Rename Ray SGD v2 to Ray Train (#19436) 2021-10-18 22:27:46 -07:00
Kai Fricke
d8d8901192
[ci/tune] Remove deprecated jenkins_only tag from test tags (#19287) 2021-10-12 10:05:46 +01:00
Avnish Narayan
bbc64a7c3d
[RLlib] Pin Gym to 0.19 (#19170)
Gym appears to have cut a release, 0.21.
It isn't clear what changes were made
between 0.19/0.20 and 0.21, as there is
no change log available for the 0.21 release,
so for now we'll pin gym to 0.19 until we
can fully understand the breaking changes
in gym 0.21. I suspect some things have
just been removed from the regular gym installation
that rllib has previously relied on. Will address
later.
2021-10-07 07:59:02 +02:00
Kai Fricke
234b015b42
[ci] Clean wheels directory before build, validate wheel commit strings (#19097) 2021-10-06 13:48:24 +01:00
Eric Liang
f8a91c7fad
Revert "[Lint] run clang-tidy in scripts/format.h, update clang-tidy rules (#19055)" (#19119)
This reverts commit 5d9e3a0121.
2021-10-05 16:33:12 -07:00
mwtian
5d9e3a0121
[Lint] run clang-tidy in scripts/format.h, update clang-tidy rules (#19055) 2021-10-05 14:03:27 -07:00
Jiajun Yao
7588bfd315
[Lint] Add flake8-bugbear (#19053)
* Add flake8-bugbear

* Add flake8-bugbear
2021-10-03 23:24:11 -07:00
Simon Mo
9b2a368c8c
[Runtime Env] Implement basic runtime env plugin mechanism (#19044) 2021-10-01 17:22:54 -07:00
architkulkarni
0f0b161ea1
Revert "Revert "[Serve] [doc] Improve runtime env doc"" (#18943)
* Revert "Revert "[Serve] [doc] Improve runtime env doc (#18782)" (#18935)"

This reverts commit e4f4c79252.
2021-09-30 13:28:44 -05:00
gjoliver
e61f2c72d7
Upgrade bazel version to 4.2.1 (#18996) 2021-09-30 10:50:54 -07:00
Eric Liang
1ffec66cd5
[CI] Disable test_reference_counting on Windows build 2021-09-25 13:40:00 -07:00
mwtian
43ac18bbc0
[Build] include minimal debug info in C++ build; upgrade clang-format to 12 (#18888)
* Revert "Revert "[Build] include minimal debug info in C++ build; upgrade clang-format to 12 (#18840)" (#18886)"

This reverts commit f851a072f3.

* use gcc 8
2021-09-24 17:59:05 -07:00
Chen Shen
f851a072f3
Revert "[Build] include minimal debug info in C++ build; upgrade clang-format to 12 (#18840)" (#18886)
This reverts commit 07e1366383.
2021-09-24 12:55:08 -07:00
mwtian
07e1366383
[Build] include minimal debug info in C++ build; upgrade clang-format to 12 (#18840)
* debug info and clang-format

* doc

* fix

* no clang-format on all files

* gcc

* keep gcc 7
2021-09-24 12:26:33 -07:00
Guyang Song
337005d5a5
[C++ API][hotfix] fix C++ worker dynamic library loading issue on macOS (#18877)
* fix C++ worker in macox

* fix
2021-09-24 23:39:00 +08:00
Guyang Song
739cf64115
[C++ API] support head_args config in C++ API (#18709) 2021-09-23 19:30:53 +08:00
mwtian
efdbfcfdfb
[Build] Generate Bazel config for compiling with clang and libc++ in CI (#18622)
* Add Bazel config for building with llvm. Upgrade C++ std to 17.

* Fix redis. Try fixing asan and tsan

* Fix asan and format

* Update comments.

Co-authored-by: Chen Shen <scv119@gmail.com>
2021-09-17 19:01:07 -07:00
qicosmos
4af3d86d8a
Remove abi flag (#18538) 2021-09-17 12:13:56 +08:00
Amog Kamsetty
84e958f330
[ML] Consolidate and upgrade Deep Learning Dependencies (#18574)
* wip
'

* upgrade requirements

* add file

* fix

* fixes

* Apply suggestions from code review

Try mlagents==0.21.0 for now (works with torch 1.9).

* Apply suggestions from code review

* wip

* wip

* fix

* fix

* upgrade lightning bolts

* address comment

Co-authored-by: Sven Mika <sven@anyscale.io>
2021-09-16 20:16:40 -07:00
Chen Shen
28c9c1fd98
fix windows pg test by skipping (#18649) 2021-09-15 11:39:13 -07:00
Edward Oakes
7736cdd91d
[dashboard] Rename "new_dashboard" -> "dashboard" (#18214) 2021-09-15 11:17:15 -05:00
Simon Mo
497c5f56fa
[CI] Temporary disable worker-in-container test (#18606)
* revert again

* disable tmp
2021-09-14 22:38:20 -07:00
mwtian
26fd10c9e8
[CI] Add clang-tidy to lint (#18124)
* clang-tidy

* fix

* fix script

* test clang compiler

* fix clang-tidy rules

* Fix windows and other issues.

* Fix

* Improve information when running check-git-clang-tidy-output.sh on different OS
2021-09-09 00:41:53 -07:00
ellimac54
772d25cc38
Add Initial Windows Dockerfile (#17474) 2021-09-03 11:41:06 -07:00
Kai Fricke
fb38d06cfb
Move RLLib GPU release test dependencies to ml docker (#18208) 2021-09-03 09:35:18 +01:00
mwtian
134ac0ef55
[CI] Fix clang-format to always compare against master (#18140) 2021-08-31 10:16:33 -07:00
Kai Fricke
a8dbc44f9a
[ci] minimal dependency install test (#18071) 2021-08-31 15:26:25 +02:00
Antoni Baum
5be6bda4cf
[tests] Add Ludwig CI test (#18126) 2021-08-30 12:27:39 -07:00
Simon Mo
7c01ea7d2f
[Serve][Hotfix] Skip get_deployment on windows (#18029) 2021-08-25 11:27:31 -05:00
Chen Shen
08193b8e0a
fix more windows failures (#18058) 2021-08-24 21:19:45 -07:00
Kai Fricke
d058f98546
[RLlib] Add GPU tests to CI (run per-PR). (#17891)
Co-authored-by: simon-mo <simon.mo@hey.com>
2021-08-24 09:20:45 +02:00
Clark Zinzow
5ca28b1cc8
[Core] Update Bazel (to 3.4.1), gRPC, boringssl, and absl as a precursor to gRPC streaming PR. (#17903)
* Update Bazel (to 3.4.1), gRPC, boringssl, absl.

* Always reinstall Bazel if needing to upgrade to a new Bazel version.

* Add patch for properly detecting Windows Python headers when building gRPC.

* Add minimum Bazel version check.

* Update docs with new Bazel version.
2021-08-21 11:33:11 -07:00
Sven Mika
a2d96c513a
[RLlib] Expand machine for nightly multi-gpu learning tests. (#17955) 2021-08-19 22:27:30 +02:00
Simon Mo
b573864928
[CI] Add test owners (#17893) 2021-08-18 18:38:31 -07:00