mirror of
https://github.com/vale981/ray
synced 2025-03-06 18:41:40 -05:00
1c3aaf7189
* Update documentation for serialization. * Update documentation for reusable variables. * Update documentation for using Ray with TensorFlow. This change is to allow code blocks to be copied and pasted into a Python interpreter. * Fix documentation for hyperparameter optimization example.
101 lines
4.1 KiB
Markdown
101 lines
4.1 KiB
Markdown
# Reusable Variables
|
|
|
|
This document explains how to create and use reusable variables in Ray.
|
|
|
|
A reusable variable is a per-worker variable which is (1) created when the worker starts, and (2) is reinitialized before a task reuses it. Thus, while a task can modify a reusable variable, the variable is reinitialized before the next task uses it. Reusable variables obviates the need for serialization/deserialization and, like Ray objects, avoid side effects.
|
|
|
|
Reusable variables are Python objects that are created once on each worker and
|
|
can be used by all subsequent tasks that run on that worker. Reusable variables
|
|
will be reinitialized between tasks. There are several primary reasons for doing
|
|
this.
|
|
|
|
1. Reusable variables are created once on each worker and are not shipped
|
|
between machines, so they do not need to be serialized or deserialized (however,
|
|
the code that creates the reusable variable does need to be pickled).
|
|
2. Objects that are slow to construct (like a TensorFlow graph) only need to be
|
|
constructed once on each worker.
|
|
3. By reinitializing between tasks that use them, they help avoid side effects.
|
|
|
|
To elaborate on the first point, standard Python serialization libraries like
|
|
pickle fail on some objects. With these kinds of objects, it may be easier to
|
|
ship the code that creates the object to each worker and to run the code on each
|
|
worker than it would be to create the object on the driver and ship the object
|
|
to each worker.
|
|
|
|
## Creating a Reusable Variable
|
|
|
|
To give an example, consider a gym environment, which essentially provides a
|
|
Python wrapper for an Atari simulator.
|
|
|
|
```python
|
|
import gym
|
|
import ray
|
|
|
|
ray.init(start_ray_local=True, num_workers=5)
|
|
|
|
# Define a function to create the gym environment.
|
|
def env_initializer():
|
|
return gym.make("Pong-v0")
|
|
|
|
# Create the reusable variable. This line will cause env_initializer to run on
|
|
# each worker and on the driver.
|
|
ray.reusables.env = ray.Reusable(env_initializer)
|
|
|
|
# Define a remote function that uses the gym environment.
|
|
@ray.remote
|
|
def step():
|
|
env = ray.reusables.env
|
|
# Choose a random action.
|
|
action = env.action_space.sample()
|
|
# Take the action and return the result.
|
|
return env.step(action)
|
|
|
|
# Call the remote function.
|
|
step.remote()
|
|
```
|
|
|
|
When the gym is created, it prints something like `Making new env: Pong-v0`. You
|
|
may notice that this is printed once for each worker. Calling `step.remote()`
|
|
will run a remote function that uses the `env` variable. You may notice that
|
|
calling `step.remote()` causes the line `Making new env: Pong-v0` to be printed
|
|
again. That occurs because, by default, every time a remote function uses a
|
|
reusable variable, the worker will rerun the code that initializes the reusable
|
|
variable to prevent side effects from leaking between tasks and introducing
|
|
non-determinism into the program.
|
|
|
|
Of course, rerunning the initialization code can be expensive, so a custom
|
|
reinitializer can be passed into the creation of a reusable variable. If the
|
|
state of the reusable variable is not mutated by any remote function, then the
|
|
reinitialization code can just be the identity function.
|
|
|
|
```python
|
|
# Define a function to create the gym environment.
|
|
def env_initializer():
|
|
return gym.make("Pong-v0")
|
|
|
|
# Define a function to reinitialize the gym environment.
|
|
def env_reinitializer(env):
|
|
env.reset()
|
|
return env
|
|
|
|
# Create the reusable variable. This line will cause env_initializer to run on
|
|
# each worker and on the driver. Every time a remote function uses the reusable
|
|
# variable, env_reinitializer will run to reset the state of the variable.
|
|
ray.reusables.env = ray.Reusable(env_initializer, env_reinitializer)
|
|
|
|
# Define a remote function that uses the gym environment.
|
|
@ray.remote
|
|
def step():
|
|
env = ray.reusables.env
|
|
# Choose a random action.
|
|
action = env.action_space.sample()
|
|
# Take the action and return the result.
|
|
return env.step(action)
|
|
|
|
# Call the remote function.
|
|
step.remote()
|
|
```
|
|
|
|
**Note:** It may sometimes look like Ray is hanging and not responding. This can
|
|
occur when print statements happen in the background on workers and hide the
|
|
interpreter prompt. Try pressing enter, and see if that fixes it.
|