mirror of
https://github.com/vale981/ray
synced 2025-03-06 10:31:39 -05:00
f9b71a8bf6
This PR consolidates both #21667 and #21759 (look there for features), but improves on them in the following way: - [x] we reverted renaming of existing projects `tune`, `rllib`, `train`, `cluster`, `serve`, `raysgd` and `data` so that links won't break. I think my consolidation efforts with the `ray-` prefix were a little overeager in that regard. It's better like this. Only the creation of `ray-core` was a necessity, and some files moved into the `rllib` folder, so that should be relatively benign. - [x] Additionally, we added Algolia `docsearch`, screenshot below. This is _much_ better than our current search. Caveat: there's a sphinx dependency that needs to be replaced (`sphinx-tabs`) by another, newer one (`sphinx-panels`), as the former prevents loading of the `algolia.js` library. Will follow-up in the next PR (hoping this one doesn't get re-re-re-re-reverted).
40 lines
1.9 KiB
ReStructuredText
40 lines
1.9 KiB
ReStructuredText
.. _evaluation-reference-docs:
|
|
|
|
Evaluation and Environment Rollout
|
|
==================================
|
|
|
|
Data ingest via either environment rollouts or other data-generating methods
|
|
(e.g. reading from offline files) is done in RLlib by :py:class:`~ray.rllib.evaluation.rollout_worker.RolloutWorker`s,
|
|
which sit inside a :py:class:`~ray.rllib.evaluation.worker_set.WorkerSet`
|
|
(together with other parallel ``RolloutWorkers``) in the RLlib :py:class:`~ray.rllib.agents.trainer.Trainer`
|
|
(under the ``self.workers`` property):
|
|
|
|
|
|
.. https://docs.google.com/drawings/d/1OewMLAu6KZNon7zpDfZnTh9qiT6m-3M9wnkqWkQQMRc/edit
|
|
.. figure:: ../images/rollout_worker_class_overview.svg
|
|
:width: 600
|
|
:align: left
|
|
|
|
**A typical RLlib WorkerSet setup inside an RLlib Trainer:** Each :py:class:`~ray.rllib.evaluation.worker_set.WorkerSet` contains
|
|
exactly one local :py:class:`~ray.rllib.evaluation.rollout_worker.RolloutWorker` object and n ray remote
|
|
:py:class:`~ray.rllib.evaluation.rollout_worker.RolloutWorker` (ray actors).
|
|
The workers contain a policy map (with one or more policies), and - in case a simulator
|
|
(env) is available - a vectorized :py:class:`~ray.rllib.env.base_env.BaseEnv`
|
|
(containing m sub-environments) and a :py:class:`~ray.rllib.evaluation.sampler.SamplerInput` (either synchronous or asynchronous) which controls
|
|
the environment data collection loop.
|
|
In the online (environment is available) as well as the offline case (no environment),
|
|
:py:class:`~ray.rllib.agents.trainer.Trainer` uses the :py:meth:`~ray.rllib.evaluation.rollout_worker.RolloutWorker.sample` method to
|
|
get :py:class:`~ray.rllib.policy.sample_batch.SampleBatch` objects for training.
|
|
|
|
|
|
RolloutWorker- and related APIs
|
|
-------------------------------
|
|
|
|
.. toctree::
|
|
:maxdepth: 1
|
|
|
|
evaluation/rollout_worker.rst
|
|
evaluation/sample_batch.rst
|
|
evaluation/worker_set.rst
|
|
evaluation/samplers.rst
|
|
evaluation/policy_map.rst
|