ray/ci/performance_tests
Sven 60d4d5e1aa Remove future imports (#6724)
* Remove all __future__ imports from RLlib.

* Remove (object) again from tf_run_builder.py::TFRunBuilder.

* Fix 2xLINT warnings.

* Fix broken appo_policy import (must be appo_tf_policy)

* Remove future imports from all other ray files (not just RLlib).

* Remove future imports from all other ray files (not just RLlib).

* Remove future import blocks that contain `unicode_literals` as well.
Revert appo_tf_policy.py to appo_policy.py (belongs to another PR).

* Add two empty lines before Schedule class.

* Put back __future__ imports into determine_tests_to_run.py. Fails otherwise on a py2/print related error.
2020-01-09 00:15:48 -08:00
..
README.rst Remove public facing references to --redis-address. (#6631) 2019-12-31 13:21:53 -08:00
test_performance.py Remove future imports (#6724) 2020-01-09 00:15:48 -08:00

Performance Tests
=================

This directory contains scripts for running performance benchmarks. These
benchmarks are intended to be used by Ray developers to check if a given pull
request introduces a performance regression.

To check if a pull request introduces a performance regression, it is necessary
to run these benchmarks on the codebase before and after the change.

Running the Workloads
---------------------

To run the workload on a single machine, do the following.

.. code-block:: bash

    python test_performance.py --num-nodes=3

This will start simulate a 3 node cluster on your local machine, attach to it,
and run the benchmarks. To run the benchmarks on an existing cluster, do the
following.

.. code-block:: bash

    python test_performance.py --num-nodes=3 --address=<redis-address>

The ``--num-nodes`` flag must match the number of nodes in the cluster. The
nodes in the cluster must be configured with the appropriate resource labels. In
particular, the ith node in the cluster must have a resource named ``"i"``
with quantity ``500``.