ray/doc/source/serve/http-adapters.md

(serve-http-adapters)=

HTTP Adapters

HTTP adapters are functions that convert raw HTTP requests to basic Python types that you know and recognize.

For example, here is an adapter that extracts the JSON content from a request:

async def json_resolver(request: starlette.requests.Request):
    return await request.json()

The input arguments to an HTTP adapter should be type-annotated. At a minimum, the adapter should accept a starlette.requests.Request type (https://www.starlette.io/requests/#request), but it can also accept any type that's recognized by FastAPI's dependency injection framework.

Here is an HTTP adapter that accepts two HTTP query parameters:

def parse_query_args(field_a: int, field_b: str):
    return YourDataClass(field_a, field_b)

You can specify different type signatures to facilitate the extraction of HTTP fields, including

• query parameters,
• body parameters, and
• many other data types.

For more details, you can take a look at the FastAPI documentation.

You can use adapters in different scenarios within Serve, which we will go over one by one:

• Ray AIR Predictor
• Serve Deployment Graph DAGDriver
• Embedded in Bring Your Own FastAPI Application

Ray AIR Predictor

Ray Serve provides a suite of adapters to convert HTTP requests to ML inputs like numpy arrays. You can use them together with the Ray AI Runtime (AIR) model wrapper feature to one-click deploy pre-trained models.

As an example, we provide a simple adapter for an n-dimensional array.

When using model wrappers, you can specify your HTTP adapter via the http_adapter field:

from ray import serve
from ray.serve.http_adapters import json_to_ndarray
from ray.serve import PredictorDeployment

PredictorDeployment.options(name="my_model").deploy(
    my_ray_air_predictor,
    my_ray_air_checkpoint,
    http_adapter=json_to_ndarray
)

Serve Deployment Graph DAGDriver

When using a Serve Deployment Graph, you can configure ray.serve.drivers.DAGDriver to accept an HTTP adapter via its http_adapter field.

For example, the json_request adapter parses JSON in the HTTP body:

from ray.serve.drivers import DAGDriver
from ray.serve.http_adapters import json_request
from ray.dag.input_node import InputNode

with InputNode() as input_node:
    # ...
    dag = DAGDriver.bind(other_node, http_adapter=json_request)

Embedded in your existing FastAPI Application

You can also bring the adapter to your own FastAPI app using Depends. The input schema will automatically be part of the generated OpenAPI schema with FastAPI.

from fastapi import FastAPI, Depends
from ray.serve.http_adapters import json_to_ndarray

app = FastAPI()

@app.post("/endpoint")
async def endpoint(np_array = Depends(json_to_ndarray)):
    ...

It has the following schema for input:

(serve-ndarray-schema)=

.. autopydantic_model:: ray.serve.http_adapters.NdArray

Pydantic models as adapters

Serve also supports pydantic models as a shorthand for HTTP adapters in model wrappers. Instead of using a function to define your HTTP adapter as in the examples above, you can directly pass in a pydantic model class to effectively tell Ray Serve "validate the HTTP body with this schema." Once validated, the model instance will passed to the predictor.

from pydantic import BaseModel

class User(BaseModel):
    user_id: int
    user_name: str

# ...

PredictorDeployment.deploy(..., http_adapter=User)
# Or:
DAGDriver.bind(other_node, http_adapter=User)

List of Built-in Adapters

Here is a list of adapters; please feel free to contribute more!

.. automodule:: ray.serve.http_adapters
    :members: json_to_ndarray, image_to_ndarray, starlette_request, json_request, pandas_read_json, json_to_multi_ndarray