hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-09 12:56:46 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
ray/doc/source/serve/package-ref.md
Archit Kulkarni 9f0d8e364d
[Doc] Update Serve architecture doc for 2.0 (#26861) 
- Move autoscaling architecture from autoscaling page to architecture page
- Update architecture page
    - Remove "Router" actor
    - Update description of ServeHandle
    - Update defaults about HTTPproxy (default one on each node -> default just one per cluster, on the head node)
- Add note about fault tolerance in different failure scenarios
- Assorted typos/usage nits

Co-authored-by: shrekris-anyscale <92341594+shrekris-anyscale@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
2022-08-03 14:30:33 -05:00

4.5 KiB
Raw Blame History

(serve-api)=

Ray Serve API

(core-apis)=

Core APIs

.. autofunction:: ray.serve.start

.. autofunction:: ray.serve.deployment

.. autofunction:: ray.serve.list_deployments

.. autofunction:: ray.serve.get_deployment

.. autofunction:: ray.serve.shutdown

(deployment-api)=

Deployment API

.. autoclass:: ray.serve.deployment.Deployment
    :members: deploy, delete, options, get_handle

(servehandle-api)=

ServeHandle API

.. autoclass:: ray.serve.handle.RayServeHandle
    :members: remote, options

Batching Requests

.. autofunction:: ray.serve.batch(max_batch_size=10, batch_wait_timeout_s=0.0)

(serve-rest-api)=

Serve REST API

REST API

GET "/api/serve/deployments/"

Gets latest config that Serve has received. This config represents the current goal state for the Serve application. Starts a Serve application on the Ray cluster if it's not already running. See the config schema for the response's JSON schema.

Example Request:

GET /api/serve/deployments/ HTTP 1.1
Host: http://localhost:8265/
Accept: application/json

Example Response:

HTTP/1.1 200 OK
Content-Type: application/json

{
    "import_path": "fruit.deployment_graph",
    "runtime_env": {
        "working_dir": "https://github.com/ray-project/serve_config_examples/archive/HEAD.zip"
    },
    "deployments": [
        {"name": "MangoStand", "user_config": {"price": 1}},
        {"name": "OrangeStand", "user_config": {"price": 2}},
        {"name": "PearStand", "user_config": {"price": 3}}
    ]
}

PUT "/api/serve/deployments/"

Declaratively deploys the Serve application. Starts Serve on the Ray cluster if it's not already running. See the config schema for the request's JSON schema.

Example Request:

PUT /api/serve/deployments/ HTTP 1.1
Host: http://localhost:8265/
Accept: application/json

{
    "import_path": "fruit.deployment_graph",
    "runtime_env": {
        "working_dir": "https://github.com/ray-project/serve_config_examples/archive/HEAD.zip"
    },
    "deployments": [
        {"name": "MangoStand", "user_config": {"price": 1}},
        {"name": "OrangeStand", "user_config": {"price": 2}},
        {"name": "PearStand", "user_config": {"price": 3}}
    ]
}

Example Response

HTTP/1.1 200 OK
Content-Type: application/json

GET "/api/serve/deployments/status"

Gets the Serve application's current status, including all the deployment statuses. This config represents the current goal state for the Serve application. Starts a Serve application on the Ray cluster if it's not already running. See the status schema for the response's JSON schema.

Example Request:

GET /api/serve/deployments/ HTTP 1.1
Host: http://localhost:8265/
Accept: application/json

Example Response

HTTP/1.1 200 OK
Content-Type: application/json

{
    "app_status": {
        "status": "RUNNING",
        "message": "",
        "deployment_timestamp": 1855994527.146304
    },
    "deployment_statuses": [
        {
            "name": "MangoStand",
            "status": "HEALTHY",
            "message": ""
        },
        {
            "name": "OrangeStand",
            "status": "HEALTHY",
            "message": ""
        },
        {
            "name": "PearStand",
            "status": "HEALTHY",
            "message": ""
        },
        {
            "name": "FruitMarket",
            "status": "HEALTHY",
            "message": ""
        },
        {
            "name": "DAGDriver",
            "status": "HEALTHY",
            "message": ""
        }
    ]
}

DELETE "/api/serve/deployments/"

Shuts down the Serve application running on the Ray cluster. Has no effect if Serve is not running on the Ray cluster.

Example Request:

DELETE /api/serve/deployments/ HTTP 1.1
Host: http://localhost:8265/
Accept: application/json

Example Response

HTTP/1.1 200 OK
Content-Type: application/json

(serve-rest-api-config-schema)=

Config Schema

.. autopydantic_model:: ray.serve.schema.ServeApplicationSchema

(serve-rest-api-status-schema)=

Status Schema

.. autopydantic_model:: ray.serve.schema.ServeStatusSchema

(serve-cli)=

Serve CLI

.. click:: ray.serve.scripts:cli
   :prog: serve
   :show-nested: