ray/release
SangBin Cho 2c2d96eeb1
[Nightly tests] Improve k8s testing (#23108)
This PR improves broken k8s tests.

Use exponential backoff on the unstable HTTP path (getting job status sometimes has broken connection from the server. I couldn't really find the relevant logs to figure out why this is happening, unfortunately).
Fix benchmark tests resource leak check. The existing one was broken because the job submission uses 0.001 node IP resource, which means the cluster_resources can never be the same as available resources. I fixed the issue by not checking node IP resources
K8s infra doesn't support instances < 8 CPUs. I used m5.2xlarge instead of xlarge. It will increase the cost a bit, but it wouldn't be very big.
2022-03-14 03:49:15 -07:00
..
.buildkite [ci/release] Migrate golden notebook tests (#22949) 2022-03-13 21:39:41 +00:00
alerts Revert "[Nightly Test] Add more metadata to test result (#21990)" (#22052) 2022-02-02 12:56:42 -08:00
benchmarks [Nightly tests] Improve k8s testing (#23108) 2022-03-14 03:49:15 -07:00
golden_notebook_tests [ci/release] Migrate golden notebook tests (#22949) 2022-03-13 21:39:41 +00:00
horovod_tests [horovod] remove deprecated slot concept, use worker instead (#22708) 2022-03-10 08:16:42 +00:00
kubernetes_manual_tests [test][k8s] Restore kubernetes test directory, adds some info (#18982) 2021-10-01 11:23:22 +01:00
lightgbm_tests [ci/release] Migrate LightGBM tests (#22952) 2022-03-10 08:14:31 +00:00
long_running_distributed_tests [CI] Format Python code with Black (#21975) 2022-01-29 18:41:57 -08:00
long_running_tests [serve] Implement Serve Application object (#22917) 2022-03-10 10:28:29 -06:00
microbenchmark [Tests] Add microbenchmark to the new infra test (#22861) 2022-03-08 05:58:49 -08:00
ml_user_tests [release test] fix horovod release test. (#22781) 2022-03-03 08:24:26 -08:00
nightly_tests [ci/release] Always use full cluster address (#23067) 2022-03-11 16:31:21 +00:00
ray_release [Nightly tests] Improve k8s testing (#23108) 2022-03-14 03:49:15 -07:00
release_logs [Release 1.11.0] Release logs for 1.11.0rc1 (#22443) 2022-02-16 17:03:49 -08:00
rllib_tests [RLlib] SlateQ: Add a hard-task learning test to weekly regression suite. (#22544) 2022-02-25 21:58:16 +01:00
runtime_env_tests [CI] Format Python code with Black (#21975) 2022-01-29 18:41:57 -08:00
serve_tests [release tests] Fix minor bug in multi-deployment serve test (#22961) 2022-03-09 14:37:27 -06:00
sgd_tests [CI] Format Python code with Black (#21975) 2022-01-29 18:41:57 -08:00
tune_tests [ci/release] Migrate golden notebook tests (#22949) 2022-03-13 21:39:41 +00:00
util [CI] Format Python code with Black (#21975) 2022-01-29 18:41:57 -08:00
xgboost_tests [ci/release] Migrate LightGBM tests (#22952) 2022-03-10 08:14:31 +00:00
__init__.py [release] move release testing end to end script to main ray repo (#17070) 2021-07-14 12:39:07 -07:00
alert.py [CI] Format Python code with Black (#21975) 2022-01-29 18:41:57 -08:00
BUILD [serve][release tests] Add smoke test to CI for remaining tests (#22962) 2022-03-09 23:36:32 -06:00
config_generator.html [release] move release testing end to end script to main ray repo (#17070) 2021-07-14 12:39:07 -07:00
e2e.py [core] Enable lineage reconstruction by default (#22816) 2022-03-07 17:40:30 -05:00
README.md [Release] Remove release process doc (#19312) 2021-10-18 11:24:03 -07:00
release_tests.yaml [ci/release] Migrate golden notebook tests (#22949) 2022-03-13 21:39:41 +00:00
requirements.txt [ci/release] Refactor release test e2e into package (#22351) 2022-02-16 17:35:02 +00:00
requirements_buildkite.txt [ci/release] Refactor release test e2e into package (#22351) 2022-02-16 17:35:02 +00:00
run_e2e.sh [ci/release] Increase long running timeout, fix artifacts copy (#21905) 2022-01-26 21:25:03 +00:00
run_release_test.sh [ci/release] Fix result output in Buildkite pipeline run (#22946) 2022-03-09 17:29:31 +00:00
setup.py [ci/release] Refactor release test e2e into package (#22351) 2022-02-16 17:35:02 +00:00
test_owners.yaml [ci] Add release test owner overview file (#18590) 2021-09-14 11:00:31 -07:00

Release Tests

While the exact release process relies on Anyscale internal tooling, the tests we run during the releases are located at https://github.com/ray-project/ray/blob/master/release/.buildkite/build_pipeline.py