hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-05 18:11:42 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

[Doc] [runtime env] Move runtime env section up one level, add inbound links (#19863)

Browse source
This commit is contained in:
architkulkarni 2021-10-29 10:02:39 -07:00 • committed by GitHub
parent 4586ced5e4
commit fdefd875c3
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
9 changed files with 173 additions and 130 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
doc/examples/doc_code/runtime_env_example.py
View file

@ -23,6 +23,7 @@ runtime_env = {
# __runtime_env_conda_def_end__
# __ray_init_start__
# Running on the Ray cluster
ray.init(runtime_env=runtime_env)
# __ray_init_end__

9
doc/source/actors.rst
View file

@ -179,8 +179,8 @@ Methods of the actor can be called remotely.
.. _actor-resource-guide:
Resources with Actors
---------------------
Specifying Resources
--------------------
You can specify that an actor requires CPUs or GPUs in the decorator. While Ray has built-in support for CPUs and GPUs, Ray can also handle custom resources.
@ -310,6 +310,11 @@ requirements, you can do so as follows.
Note that to create these actors successfully, Ray will need to be started with
sufficient CPU resources and the relevant custom resources.
.. tip::
Besides compute resources, you can also specify an environment for an actor to run in,
which can include Python packages, local files, environment variables, and more--see :ref:`Runtime Environments <runtime-environments>` for details.
Terminating Actors
------------------

111
doc/source/advanced.rst
View file

@ -428,114 +428,9 @@ To get information about the current available resource capacity of your cluster
.. autofunction:: ray.available_resources
:noindex:
.. _runtime-environments:
.. _runtime-environments-old:
Runtime Environments
-----------------------------------
--------------------
.. note::
This API is in beta and may change before becoming stable.
.. note::
This feature requires a full installation of Ray using ``pip install "ray[default]"``.
On Mac OS and Linux, Ray 1.4+ supports dynamically setting the runtime environment of tasks, actors, and jobs so that they can depend on different Python libraries (e.g., conda environments, pip dependencies) while all running on the same Ray cluster.
The ``runtime_env`` is a (JSON-serializable) dictionary that can be passed as an option to tasks and actors, and can also be passed to ``ray.init()``.
The runtime environment defines the dependencies required for your workload.
You can specify a runtime environment for your whole job using ``ray.init()`` or Ray Client:
.. literalinclude:: ../examples/doc_code/runtime_env_example.py
:language: python
:start-after: __ray_init_start__
:end-before: __ray_init_end__
..
TODO(architkulkarni): run Ray Client doc example in CI
.. code-block:: python
# Using Ray Client
ray.init("ray://localhost:10001", runtime_env=runtime_env)
Or specify per-actor or per-task in the ``@ray.remote()`` decorator or by using ``.options()``:
.. literalinclude:: ../examples/doc_code/runtime_env_example.py
:language: python
:start-after: __per_task_per_actor_start__
:end-before: __per_task_per_actor_end__
The ``runtime_env`` is a Python dictionary including one or more of the following arguments:
- ``working_dir`` (Path): Specifies the working directory for your job. This must be an existing local directory with total size at most 100 MiB.
It will be cached on the cluster, so the next time you connect with Ray Client you will be able to skip uploading the directory contents.
All Ray workers for your job will be started in their node's local copy of this working directory.
- Examples
- ``"." # cwd``
- ``"/code/my_project"``
Note: Setting this option per-task or per-actor is currently unsupported.
Note: If your working directory contains a `.gitignore` file, the files and paths specified therein will not be uploaded to the cluster.
- ``excludes`` (List[str]): When used with ``working_dir``, specifies a list of files or paths to exclude from being uploaded to the cluster.
This field also supports the pattern-matching syntax used by ``.gitignore`` files: see `<https://git-scm.com/docs/gitignore>`_ for details.
- Example: ``["my_file.txt", "path/to/dir", "*.log"]``
- ``pip`` (List[str] | str): Either a list of pip packages, or a string containing the path to a pip
`“requirements.txt” <https://pip.pypa.io/en/stable/user_guide/#requirements-files>`_ file. The path may be an absolute path or a relative path.
This will be dynamically installed in the ``runtime_env``.
To use a library like Ray Serve or Ray Tune, you will need to include ``"ray[serve]"`` or ``"ray[tune]"`` here.
- Example: ``["requests==1.0.0", "aiohttp"]``
- Example: ``"./requirements.txt"``
- ``conda`` (dict | str): Either (1) a dict representing the conda environment YAML, (2) a string containing the absolute or relative path to a
`conda “environment.yml” <https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually>`_ file,
or (3) the name of a local conda environment already installed on each node in your cluster (e.g., ``"pytorch_p36"``).
In the first two cases, the Ray and Python dependencies will be automatically injected into the environment to ensure compatibility, so there is no need to manually include them.
Note that the ``conda`` and ``pip`` keys of ``runtime_env`` cannot both be specified at the same time---to use them together, please use ``conda`` and add your pip dependencies in the ``"pip"`` field in your conda ``environment.yaml``.
- Example: ``{"dependencies": ["pytorch", “torchvision”, "pip", {"pip": ["pendulum"]}]}``
- Example: ``"./environment.yml"``
- Example: ``"pytorch_p36"``
- ``env_vars`` (Dict[str, str]): Environment variables to set.
- Example: ``{"OMP_NUM_THREADS": "32", "TF_WARNINGS": "none"}``
- ``eager_install`` (bool): A boolean indicates whether to install runtime env eagerly before the workers are leased. This flag is set to True by default and only job level is supported now.
- Example: ``{"eager_install": False}``
The runtime environment is inheritable, so it will apply to all tasks/actors within a job and all child tasks/actors of a task or actor, once set.
If a child actor or task specifies a new ``runtime_env``, it will be merged with the parent’s ``runtime_env`` via a simple dict update.
For example, if ``runtime_env["pip"]`` is specified, it will override the ``runtime_env["pip"]`` field of the parent.
The one exception is the field ``runtime_env["env_vars"]``. This field will be `merged` with the ``runtime_env["env_vars"]`` dict of the parent.
This allows for an environment variables set in the parent's runtime environment to be automatically propagated to the child, even if new environment variables are set in the child's runtime environment.
Here are some examples of runtime environments combining multiple options:
..
TODO(architkulkarni): run working_dir doc example in CI
.. code-block:: python
runtime_env = {"working_dir": "/code/my_project", "pip": ["pendulum=2.1.2"]}
.. literalinclude:: ../examples/doc_code/runtime_env_example.py
:language: python
:start-after: __runtime_env_conda_def_start__
:end-before: __runtime_env_conda_def_end__
:ref:`This section has moved.<runtime-environments>`

7
doc/source/cluster/guide.rst
View file

@ -26,7 +26,7 @@ any cloud. It will:
* provision a new instance/machine using the cloud provider's SDK.
* execute shell commands to set up Ray with the provided options.
* (optionally) run any custom, user defined setup commands.
* (optionally) run any custom, user defined setup commands. (To dynamically set up environments after the cluster has been deployed, you can use :ref:`Runtime Environments<runtime-environments>`.)
* Initialize the Ray cluster.
* Deploy an autoscaler process.
@ -89,7 +89,7 @@ Deploying an application
------------------------
The recommended way of connecting to a Ray cluster is to use the
``ray.init("ray://<host>:<port>")`` API and connect via the Ray Client.
``ray.init("ray://<host>:<port>")`` API and connect via the :ref:`Ray Client<ray-client>`.
.. note::
@ -102,6 +102,9 @@ address of the Ray client server.
:ref:`Learn more about setting up the Ray client server here <Ray-client>`.
You can dynamically specify local files, Python packages, and environment variables for your
application using :ref:`Runtime Environments <runtime-environments>`.
.. note::
When deploying an application, the job will be killed if the driver

40
doc/source/cluster/ray-client.rst
View file

@ -5,9 +5,9 @@ Ray Client
**What is the Ray Client?**
The Ray Client is an API that connects a python script to a **remote** Ray cluster. Effectively, it allows you to leverage a remote Ray cluster just like you would with Ray running on your local machine.
The Ray Client is an API that connects a Python script to a **remote** Ray cluster. Effectively, it allows you to leverage a remote Ray cluster just like you would with Ray running on your local machine.
By changing ``ray.init()`` to ``ray.init("ray://<head_node_host>:<port>")``, you can connect from your laptop (or anywhere) directly to a remote cluster and scale-out your Ray code, while maintaining the ability to develop interactively in a python shell. **This will only work with Ray 1.5+.** If you're using an older version of ray, see the `1.4.1 docs <https://docs.ray.io/en/releases-1.4.1/cluster/ray-client.html>`_
By changing ``ray.init()`` to ``ray.init("ray://<head_node_host>:<port>")``, you can connect from your laptop (or anywhere) directly to a remote cluster and scale-out your Ray code, while maintaining the ability to develop interactively in a Python shell. **This will only work with Ray 1.5+.** If you're using an older version of ray, see the `1.4.1 docs <https://docs.ray.io/en/releases-1.4.1/cluster/ray-client.html>`_
.. code-block:: python
@ -31,36 +31,42 @@ By changing ``ray.init()`` to ``ray.init("ray://<head_node_host>:<port>")``, you
Client arguments
----------------
Ray client is used when the address passed into ``ray.init`` is prefixed with ``ray://``. Client mode currently accepts two arguments:
Ray Client is used when the address passed into ``ray.init`` is prefixed with ``ray://``. Besides the address, Client mode currently accepts two other arguments:
- ``namespace``: Sets the namespace for the session
- ``runtime_env``: Sets the `runtime environment <../advanced.html?highlight=runtime environment#runtime-environments-experimental>`_ for the session
- ``namespace`` (optional): Sets the namespace for the session.
- ``runtime_env`` (optional): Sets the `runtime environment <../advanced.html?highlight=runtime environment#runtime-environments-experimental>`_ for the session, allowing you to dynamically specify environment variables, packages, local files, and more.
.. code-block:: python
# Connects to an existing cluster at 1.2.3.4 listening on port 10001, using
# the namespace "my_namespace"
ray.init("ray://1.2.3.4:10001", namespace="my_namespace")
# the namespace "my_namespace". The Ray workers will run inside a cluster-side
# copy of the local directory "files/my_project", in a Python environment with
# `toolz` and `requests` installed.
ray.init(
"ray://1.2.3.4:10001",
namespace="my_namespace",
runtime_env={"working_dir": "files/my_project", "pip": ["toolz", "requests"]},
)
#....
When to use Ray client
When to use Ray Client
----------------------
Ray client should be used when you want to connect a script or an interactive shell session to a **remote** cluster.
Ray Client should be used when you want to connect a script or an interactive shell session to a **remote** cluster.
* Use ``ray.init("ray://<head_node_host>:10001")`` (Ray client) if you've set up a remote cluster at ``<head_node_host>``. This will connect your local script or shell to the cluster. See the section on :ref:`using ray client<how-do-you-use-the-ray-client>` for more details on setting up your cluster.
* Use ``ray.init("ray://<head_node_host>:10001")`` (Ray Client) if you've set up a remote cluster at ``<head_node_host>``. This will connect your local script or shell to the cluster. See the section on :ref:`using Ray Client<how-do-you-use-the-ray-client>` for more details on setting up your cluster.
* Use ``ray.init("localhost:<port>")`` (non-client connection, local address) if you're developing locally or on the head node of your cluster and you have already started the cluster (i.e. ``ray start --head`` has already been run)
* Use ``ray.init()`` (non-client connection, no address specified) if you're developing locally and want to automatically create a local cluster and attach directly to it.
.. _how-do-you-use-the-ray-client:
How do you use the Ray client?
How do you use the Ray Client?
------------------------------
Step 1: Set up your Ray cluster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you have a running Ray cluster (version >= 1.5), Ray client server is likely already running on port ``10001`` of the head node by default. Otherwise, you'll want to create a Ray cluster. To start a Ray cluster locally, you can run
If you have a running Ray cluster (version >= 1.5), Ray Client server is likely already running on port ``10001`` of the head node by default. Otherwise, you'll want to create a Ray cluster. To start a Ray cluster locally, you can run
.. code-block:: bash
@ -68,7 +74,7 @@ If you have a running Ray cluster (version >= 1.5), Ray client server is likely
To start a Ray cluster remotely, you can follow the directions in :ref:`ref-cluster-quick-start`.
If necessary, you can modify the Ray client server port to be other than ``10001``, by specifying ``--ray-client-server-port=...`` to the ``ray start`` :ref:`command <ray-start-doc>`.
If necessary, you can modify the Ray Client server port to be other than ``10001``, by specifying ``--ray-client-server-port=...`` to the ``ray start`` :ref:`command <ray-start-doc>`.
Step 2: Check ports
~~~~~~~~~~~~~~~~~~~
@ -163,7 +169,7 @@ Then, you can connect to the Ray cluster **from another terminal** using ``loca
Connect to multiple ray clusters (Experimental)
-----------------------------------------------
Ray client allows connecting to multiple ray clusters in one Python process. To do this, just pass ``allow_multiple=True`` to ``ray.init``:
Ray Client allows connecting to multiple Ray clusters in one Python process. To do this, just pass ``allow_multiple=True`` to ``ray.init``:
.. code-block:: python
@ -202,11 +208,11 @@ Ray client allows connecting to multiple ray clusters in one Python process. To
cli2.disconnect()
When using ray multi-client, there are some different behaviors to pay attention to:
When using Ray multi-client, there are some different behaviors to pay attention to:
* The client won't be disconnected automatically. Call ``disconnect`` explicitly to close the connection.
* Object references can only be used by the client from which it was obtained.
* ``ray.init`` without ``allow_multiple`` will create a default global ray client.
* ``ray.init`` without ``allow_multiple`` will create a default global Ray client.
Things to know
--------------
@ -227,7 +233,7 @@ Similarly, the minor Python (e.g., 3.6 vs 3.7) must match between the client and
Starting a connection on older Ray versions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you encounter ``socket.gaierror: [Errno -2] Name or service not known`` when using ``ray.init("ray://...")`` then you may be on a version of Ray prior to 1.5 that does not support starting client connections through ``ray.init``. If this is the case, see the `1.4.1 docs <https://docs.ray.io/en/releases-1.4.1/cluster/ray-client.html>`_ for Ray client.
If you encounter ``socket.gaierror: [Errno -2] Name or service not known`` when using ``ray.init("ray://...")`` then you may be on a version of Ray prior to 1.5 that does not support starting client connections through ``ray.init``. If this is the case, see the `1.4.1 docs <https://docs.ray.io/en/releases-1.4.1/cluster/ray-client.html>`_ for Ray Client.
Connection through the Ingress
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

2
doc/source/cross-language.rst
View file

@ -1,6 +1,6 @@
.. _cross_language:
Cross-language programming
Cross-Language Programming
==========================
This page will show you how to use Ray's cross-language programming feature.

127
doc/source/dependency-management.rst Normal file
View file

@ -0,0 +1,127 @@
.. _dependency_management:
Dependency Management
=====================
Your Ray project may depend on environment variables, local files, and Python packages.
Ray makes managing these dependencies easy, even when working with a remote cluster.
You can specify dependencies dynamically at runtime using :ref:`Runtime Environments<runtime-environments>`. This is useful for quickly iterating on a project with changing dependencies and local code files, or for running jobs, tasks and actors with different environments all on the same Ray cluster.
Alternatively, you can prepare your Ray cluster's environment once, when your cluster nodes start up. This can be
accomplished using ``setup_commands`` in the Ray Cluster launcher; see the :ref:`documentation<cluster-configuration-setup-commands>` for details.
You can still use
runtime environments on top of this, but they will not inherit anything from the base
cluster environment.
.. _runtime-environments:
Runtime Environments
--------------------
.. note::
This API is in beta and may change before becoming stable.
.. note::
This feature requires a full installation of Ray using ``pip install "ray[default]"``.
On Mac OS and Linux, Ray 1.4+ supports dynamically setting the runtime environment of tasks, actors, and jobs so that they can depend on different Python libraries (e.g., conda environments, pip dependencies) while all running on the same Ray cluster.
The ``runtime_env`` is a (JSON-serializable) dictionary that can be passed as an option to tasks and actors, and can also be passed to ``ray.init()``.
The runtime environment defines the dependencies required for your workload.
You can specify a runtime environment for your whole job, whether running a script directly on the cluster or using :ref:`Ray Client<ray-client>`:
.. literalinclude:: ../examples/doc_code/runtime_env_example.py
:language: python
:start-after: __ray_init_start__
:end-before: __ray_init_end__
..
TODO(architkulkarni): run Ray Client doc example in CI
.. code-block:: python
# Running on a local machine, connecting to remote cluster using Ray Client
ray.init("ray://123.456.7.89:10001", runtime_env=runtime_env)
Or specify per-actor or per-task in the ``@ray.remote()`` decorator or by using ``.options()``:
.. literalinclude:: ../examples/doc_code/runtime_env_example.py
:language: python
:start-after: __per_task_per_actor_start__
:end-before: __per_task_per_actor_end__
The ``runtime_env`` is a Python dictionary including one or more of the following arguments:
- ``working_dir`` (Path): Specifies the working directory for your job. This must be an existing local directory with total size at most 100 MiB.
It will be cached on the cluster, so the next time you connect with Ray Client you will be able to skip uploading the directory contents.
All Ray workers for your job will be started in their node's local copy of this working directory.
- Examples
- ``"." # cwd``
- ``"/code/my_project"``
Note: Setting this option per-task or per-actor is currently unsupported.
Note: If your working directory contains a `.gitignore` file, the files and paths specified therein will not be uploaded to the cluster.
- ``excludes`` (List[str]): When used with ``working_dir``, specifies a list of files or paths to exclude from being uploaded to the cluster.
This field also supports the pattern-matching syntax used by ``.gitignore`` files: see `<https://git-scm.com/docs/gitignore>`_ for details.
- Example: ``["my_file.txt", "path/to/dir", "*.log"]``
- ``pip`` (List[str] | str): Either a list of pip packages, or a string containing the path to a pip
`“requirements.txt” <https://pip.pypa.io/en/stable/user_guide/#requirements-files>`_ file. The path may be an absolute path or a relative path.
This will be dynamically installed in the ``runtime_env``.
To use a library like Ray Serve or Ray Tune, you will need to include ``"ray[serve]"`` or ``"ray[tune]"`` here.
- Example: ``["requests==1.0.0", "aiohttp"]``
- Example: ``"./requirements.txt"``
- ``conda`` (dict | str): Either (1) a dict representing the conda environment YAML, (2) a string containing the absolute or relative path to a
`conda “environment.yml” <https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually>`_ file,
or (3) the name of a local conda environment already installed on each node in your cluster (e.g., ``"pytorch_p36"``).
In the first two cases, the Ray and Python dependencies will be automatically injected into the environment to ensure compatibility, so there is no need to manually include them.
Note that the ``conda`` and ``pip`` keys of ``runtime_env`` cannot both be specified at the same time---to use them together, please use ``conda`` and add your pip dependencies in the ``"pip"`` field in your conda ``environment.yaml``.
- Example: ``{"dependencies": ["pytorch", “torchvision”, "pip", {"pip": ["pendulum"]}]}``
- Example: ``"./environment.yml"``
- Example: ``"pytorch_p36"``
- ``env_vars`` (Dict[str, str]): Environment variables to set.
- Example: ``{"OMP_NUM_THREADS": "32", "TF_WARNINGS": "none"}``
- ``eager_install`` (bool): A boolean indicates whether to install runtime env eagerly before the workers are leased. This flag is set to True by default and only job level is supported now.
- Example: ``{"eager_install": False}``
The runtime environment is inheritable, so it will apply to all tasks/actors within a job and all child tasks/actors of a task or actor, once set.
If a child actor or task specifies a new ``runtime_env``, it will be merged with the parent’s ``runtime_env`` via a simple dict update.
For example, if ``runtime_env["pip"]`` is specified, it will override the ``runtime_env["pip"]`` field of the parent.
The one exception is the field ``runtime_env["env_vars"]``. This field will be `merged` with the ``runtime_env["env_vars"]`` dict of the parent.
This allows for an environment variables set in the parent's runtime environment to be automatically propagated to the child, even if new environment variables are set in the child's runtime environment.
Here are some examples of runtime environments combining multiple options:
..
TODO(architkulkarni): run working_dir doc example in CI
.. code-block:: python
runtime_env = {"working_dir": "/files/my_project", "pip": ["pendulum=2.1.2"]}
.. literalinclude:: ../examples/doc_code/runtime_env_example.py
:language: python
:start-after: __runtime_env_conda_def_start__
:end-before: __runtime_env_conda_def_end__

1
doc/source/using-ray.rst
View file

@ -13,6 +13,7 @@ Finally, we've also included some content on using core Ray APIs with `Tensorflo
starting-ray.rst
actors.rst
namespaces.rst
dependency-management.rst
async_api.rst
concurrency_group_api.rst
using-ray-with-gpus.rst

5
doc/source/walkthrough.rst
View file

@ -341,6 +341,11 @@ Below are more examples of resource specifications:
// Ray aslo supports fractional and custom resources.
ray::Task(MyFunction).SetResource("GPU", 0.5).SetResource("Custom", 1.0).Remote();
.. tip::
Besides compute resources, you can also specify an environment for a task to run in,
which can include Python packages, local files, environment variables, and more--see :ref:`Runtime Environments <runtime-environments>` for details.
Multiple returns
~~~~~~~~~~~~~~~~