ray/release/rllib_tests
Kai Fricke 1ed8bd0345
[release/xgboost/lightgbm] Fix app config dependency install overwriting ray (#25307)
This line:

```
pip3 install -U --force-reinstall xgboost xgboost_ray lightgbm_ray petastorm
```

also re-installs the dependencies of these packages, and the `--force-reinstall` means we overwrite existing ones. This leads us to re-install the latest ray release, overwriting the wheels to be tested:

```
[INFO] 5/31/2022, 12:12:16 AM: Successfully installed ... ray-1.12.1 ...
[INFO] 5/31/2022, 12:12:17 AM: * Executed RUN pip3 install -U --force-reinstall xgboost xgboost_ray petastorm  (ff6ae9f9)
```

Instead, we should use `--no-deps` to avoid re-installing dependencies. Also, the wheels sanity check is moved to after installing additional packages in order to catch these errors earlier.
2022-05-31 13:46:17 +02:00
..
learning_tests [RLlib]: Rename input_evaluation to off_policy_estimation_methods. (#25107) 2022-05-27 13:14:54 +02:00
multi_gpu_learning_tests [RLlib] Replay Buffer API and Ape-X. (#24506) 2022-05-17 13:43:49 +02:00
multi_gpu_with_attention_learning_tests [CI] Format Python code with Black (#21975) 2022-01-29 18:41:57 -08:00
multi_gpu_with_lstm_learning_tests [CI] Format Python code with Black (#21975) 2022-01-29 18:41:57 -08:00
stress_tests [CI] Format Python code with Black (#21975) 2022-01-29 18:41:57 -08:00
unit_gpu_tests [ci] Clean up ci/ directory (refactor ci/travis) (#23866) 2022-04-13 18:11:30 +01:00
1gpu_4cpus.yaml [RLlib] Add an RLlib Tune experiment to UserTest suite. (#19807) 2021-11-03 17:04:27 -07:00
2gpus_32cpus.yaml [RLlib] Add an RLlib Tune experiment to UserTest suite. (#19807) 2021-11-03 17:04:27 -07:00
4gpus_64cpus.yaml [RLlib] Add an RLlib Tune experiment to UserTest suite. (#19807) 2021-11-03 17:04:27 -07:00
4gpus_544_cpus.yaml [Tune release test] Set 500G disk space for rllib_tests. (#19730) 2021-10-26 10:12:03 -07:00
8gpus_64cpus.yaml [Tune release test] Set 500G disk space for rllib_tests. (#19730) 2021-10-26 10:12:03 -07:00
8gpus_96cpus.yaml [Tune release test] Set 500G disk space for rllib_tests. (#19730) 2021-10-26 10:12:03 -07:00
12gpus_192cpus.yaml [RLlib] Create a set of performance benchmark tests to run nightly. (#19945) 2021-11-08 18:15:13 +01:00
app_config.yaml [release/xgboost/lightgbm] Fix app config dependency install overwriting ray (#25307) 2022-05-31 13:46:17 +02:00
README.rst [RLlib] Upgrade RLlib regression test scripts to new testing tool - RLlib release logs for 1.4. (#16080) 2021-06-01 17:39:18 +02:00

RLlib Tests
===========

This directory contains various RLlib release tests.

You should run these tests with the `releaser <https://github.com/ray-project/releaser>`_ tool.

Overview
--------
Currently, there are 3 RLlib tests:

1. ``learning_tests`` - Tests, whether major algos (tf+torch) can learn in Atari or PyBullet envs in ~30-60min.
1. ``stress_tests`` - Runs 4 IMPALA Atari jobs, each one using 1GPU and 128CPUs (needs autoscaling to succeed).
1. ``unit_gpu_tests`` - Tests, whether all of RLlib's example scripts can be run on a GPU.

Generally the releaser tool will run all tests in parallel.

Acceptance criteria
-------------------
These tests are considered passing when they throw no error at the end of the output log.