hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-07 02:51:39 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
ray/release/rllib_tests
History
gjoliver d8a61f801f
[RLlib] Create a set of performance benchmark tests to run nightly. (#19945) 
* Create a core set of algorithms tests to run nightly.

* Run release tests under tf, tf2, and torch frameworks.

* Fix

* Add eager_tracing option for tf2 framework.

* make sure core tests can run in parallel.

* cql

* Report progress while running nightly/weekly tests.

* Innclude SAC in nightly lineup.

* Revert changes to learning_tests

* rebrand to performance test.

* update build_pipeline.py with new performance_tests name.

* Record stats.

* bug fix, need to populate experiments dict.

* Alphabetize yaml files.

* Allow specifying frameworks. And do not run tf2 by default.

* remove some debugging code.

* fix

* Undo testing changes.

* Do not run CQL regression for now.

* LINT.

Co-authored-by: sven1977 <svenmika1977@gmail.com>
2021-11-08 18:15:13 +01:00
..
learning_tests [RLlib] Create a set of performance benchmark tests to run nightly. (#19945) 2021-11-08 18:15:13 +01:00
multi_gpu_learning_tests [RLlib] Add multi-GPU attention net tests to nightly test suite (+ R2D2 tests for LSTM and attention nets). (#18368) 2021-09-06 17:48:05 +02:00
multi_gpu_with_attention_learning_tests [RLlib Testing] Lower --smoke-test "time_total_s" to make sure it doesn't time out. (#18670) 2021-09-16 18:22:23 +02:00
multi_gpu_with_lstm_learning_tests [RLlib] Fix R2D2 (torch) multi-GPU issue. (#18550) 2021-09-14 19:58:10 +02:00
performance_tests [RLlib] Create a set of performance benchmark tests to run nightly. (#19945) 2021-11-08 18:15:13 +01:00
stress_tests [RLlib] Create a set of performance benchmark tests to run nightly. (#19945) 2021-11-08 18:15:13 +01:00
unit_gpu_tests [RLlib] Add multi-GPU learning tests to nightly. (#17778) 2021-08-18 17:21:01 +02:00
1gpu_4cpus.yaml [RLlib] Add an RLlib Tune experiment to UserTest suite. (#19807) 2021-11-03 17:04:27 -07:00
2gpus_32cpus.yaml [RLlib] Add an RLlib Tune experiment to UserTest suite. (#19807) 2021-11-03 17:04:27 -07:00
4gpus_64cpus.yaml [RLlib] Add an RLlib Tune experiment to UserTest suite. (#19807) 2021-11-03 17:04:27 -07:00
4gpus_544_cpus.yaml [Tune release test] Set 500G disk space for rllib_tests. (#19730) 2021-10-26 10:12:03 -07:00
8gpus_64cpus.yaml [Tune release test] Set 500G disk space for rllib_tests. (#19730) 2021-10-26 10:12:03 -07:00
8gpus_96cpus.yaml [Tune release test] Set 500G disk space for rllib_tests. (#19730) 2021-10-26 10:12:03 -07:00
12gpus_192cpus.yaml [RLlib] Create a set of performance benchmark tests to run nightly. (#19945) 2021-11-08 18:15:13 +01:00
app_config.yaml [RLlib] Add an RLlib Tune experiment to UserTest suite. (#19807) 2021-11-03 17:04:27 -07:00
README.rst [RLlib] Upgrade RLlib regression test scripts to new testing tool - RLlib release logs for 1.4. (#16080) 2021-06-01 17:39:18 +02:00
rllib_tests.yaml [RLlib] Create a set of performance benchmark tests to run nightly. (#19945) 2021-11-08 18:15:13 +01:00
wait_cluster.py [RLlib Testing] Lower --smoke-test "time_total_s" to make sure it doesn't time out. (#18670) 2021-09-16 18:22:23 +02:00

README.rst 

RLlib Tests
===========

This directory contains various RLlib release tests.

You should run these tests with the `releaser <https://github.com/ray-project/releaser>`_ tool.

Overview
--------
Currently, there are 3 RLlib tests:

1. ``learning_tests`` - Tests, whether major algos (tf+torch) can learn in Atari or PyBullet envs in ~30-60min.
1. ``stress_tests`` - Runs 4 IMPALA Atari jobs, each one using 1GPU and 128CPUs (needs autoscaling to succeed).
1. ``unit_gpu_tests`` - Tests, whether all of RLlib's example scripts can be run on a GPU.

Generally the releaser tool will run all tests in parallel.

Acceptance criteria
-------------------
These tests are considered passing when they throw no error at the end of the output log.