ray/release
gjoliver d8a61f801f
[RLlib] Create a set of performance benchmark tests to run nightly. (#19945)
* Create a core set of algorithms tests to run nightly.

* Run release tests under tf, tf2, and torch frameworks.

* Fix

* Add eager_tracing option for tf2 framework.

* make sure core tests can run in parallel.

* cql

* Report progress while running nightly/weekly tests.

* Innclude SAC in nightly lineup.

* Revert changes to learning_tests

* rebrand to performance test.

* update build_pipeline.py with new performance_tests name.

* Record stats.

* bug fix, need to populate experiments dict.

* Alphabetize yaml files.

* Allow specifying frameworks. And do not run tf2 by default.

* remove some debugging code.

* fix

* Undo testing changes.

* Do not run CQL regression for now.

* LINT.

Co-authored-by: sven1977 <svenmika1977@gmail.com>
2021-11-08 18:15:13 +01:00
..
.buildkite [RLlib] Create a set of performance benchmark tests to run nightly. (#19945) 2021-11-08 18:15:13 +01:00
alerts [ci/release] Bump long running release test timeouts to 6 minutes (#19458) 2021-10-18 16:27:53 +01:00
golden_notebook_tests [ci/release] wrap pip github installs in quotation marks to prevent comment errors (#19464) 2021-10-18 18:55:56 +01:00
horovod_tests [Release] Refactor User Tests (#20028) 2021-11-05 17:28:37 -07:00
kubernetes_manual_tests [test][k8s] Restore kubernetes test directory, adds some info (#18982) 2021-10-01 11:23:22 +01:00
lightgbm_tests [CI/docs] Remove [default] from xgboost-ray (#19186) 2021-10-14 16:29:55 +01:00
long_running_distributed_tests [Tune] Remove queue_trials. (#19472) 2021-10-22 09:24:54 +01:00
long_running_tests [Release] Increase node memory by 2X in many_ppo test. (#19591) 2021-11-08 08:10:09 +09:00
microbenchmark [ci] upgrade microbenchmark base image to 1.6.0 (#18542) 2021-09-13 17:13:01 +01:00
ml_user_tests [Release] Refactor User Tests (#20028) 2021-11-05 17:28:37 -07:00
nightly_tests [nightly] Remove grpc staging test since nightly is stable #20119 (#20119) 2021-11-05 21:36:58 -07:00
release_logs Adding release logs for 1.8.0. (#19867) 2021-11-01 10:26:04 -07:00
rllib_tests [RLlib] Create a set of performance benchmark tests to run nightly. (#19945) 2021-11-08 18:15:13 +01:00
runtime_env_tests [runtime env] [test] Enable runtime env nightly test with working_dir reconnection (#19906) 2021-10-31 10:48:48 -05:00
serve_tests [Serve] Add verbose log for nightly test only (#20088) 2021-11-04 16:15:22 -07:00
sgd_tests [ci/release] Add sanity check for ray wheels hash to release tests (#18489) 2021-09-10 17:50:31 +01:00
tune_tests [tune] Cloud checkpointing release tests (#19638) 2021-10-29 12:12:01 +02:00
util Use bash to run sanity_check_cpp.sh (#19179) 2021-10-07 13:34:25 +01:00
xgboost_tests [Release] Refactor User Tests (#20028) 2021-11-05 17:28:37 -07:00
__init__.py [release] move release testing end to end script to main ray repo (#17070) 2021-07-14 12:39:07 -07:00
alert.py [release/alert] less results (#17637) 2021-08-06 10:26:07 +01:00
config_generator.html [release] move release testing end to end script to main ray repo (#17070) 2021-07-14 12:39:07 -07:00
e2e.py Support different s3 url formats (#20133) 2021-11-07 14:58:51 -08:00
README.md [Release] Remove release process doc (#19312) 2021-10-18 11:24:03 -07:00
requirements.txt [CI/docs] Remove [default] from xgboost-ray (#19186) 2021-10-14 16:29:55 +01:00
test_owners.yaml [ci] Add release test owner overview file (#18590) 2021-09-14 11:00:31 -07:00

Release Tests

While the exact release process relies on Anyscale internal tooling, the tests we run during the releases are located at https://github.com/ray-project/ray/blob/master/release/.buildkite/build_pipeline.py