hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-06 10:31:39 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

[Serve] [Doc] Add note about serving multiple deployments defined by the same class (#19118)

Browse source
This commit is contained in:
architkulkarni 2021-10-06 08:24:42 -07:00 • committed by GitHub
parent 234b015b42
commit 281fcaa91a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 19 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
doc/source/serve/core-apis.rst
View file

@ -35,7 +35,14 @@ Deployments can be exposed in two ways: over HTTP or in Python via the :ref:`ser
By default, HTTP requests will be forwarded to the ``__call__`` method of the class (or the function) and a ``Starlette Request`` object will be the sole argument.
You can also define a deployment that wraps a FastAPI app for more flexible handling of HTTP requests. See :ref:`serve-fastapi-http` for details.
We can also list all available deployments and dynamically get a reference to them:
To serve multiple deployments defined by the same class, use the ``name`` option:
.. code-block:: python
MyFirstDeployment.options(name="hello_service").deploy("Hello!")
MyFirstDeployment.options(name="hi_service").deploy("Hi!)
You can also list all available deployments and dynamically get references to them:
.. code-block:: python

15
doc/source/serve/ml-models.rst
View file

@ -70,10 +70,10 @@ Integration with Model Registries
Ray Serve is flexible. If you can load your model as a Python
function or class, then you can scale it up and serve it with Ray Serve.
For example, if you are using the
For example, if you are using the
`MLflow Model Registry <https://www.mlflow.org/docs/latest/model-registry.html>`_
to manage your models, the following wrapper
class will allow you to load a model using its MLflow `Model URI`:
class will allow you to load a model using its MLflow `Model URI`:
.. code-block:: python
@ -93,12 +93,19 @@ class will allow you to load a model using its MLflow `Model URI`:
model_uri = "model:/my_registered_model/Production"
MLflowDeployment.deploy(model_uri)
.. tip::
To serve multiple different MLflow models in the same program, use the ``name`` option:
.. code-block:: python
MLflowDeployment.options(name="my_mlflow_model_1").deploy(model_uri)
.. tip::
The above approach will work for any model registry, not just MLflow.
Namely, load the model from the registry in ``__init__``, and forward the request to the model in ``__call__``.
For an even more hands-off and seamless integration with MLflow, check out the
For an even more hands-off and seamless integration with MLflow, check out the
`Ray Serve MLflow deployment plugin <https://github.com/ray-project/mlflow-ray-serve>`__. A full
tutorial is available `here <https://github.com/mlflow/mlflow/tree/master/examples/ray_serve>`__.