ray/dashboard/modules/job/pydantic_models.py

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

82 lines
2.9 KiB
Python
Raw Normal View History

Fix the jobs tab in the beta dashboard and fill it with data from both "submission" jobs and "driver" jobs (#25902) ## Why are these changes needed? - Fixes the jobs tab in the new dashboard. Previously it didn't load. - Combines the old job concept, "driver jobs" and the new job submission conception into a single concept called "jobs". Jobs tab shows information about both jobs. - Updates all job APIs: They now returns both submission jobs and driver jobs. They also contains additional data in the response including "id", "job_id", "submission_id", and "driver". They also accept either job_id or submission_id as input. - Job ID is the same as the "ray core job id" concept. It is in the form of "0100000" and is the primary id to represent jobs. - Submission ID is an ID that is generated for each ray job submission. It is in the form of "raysubmit_12345...". It is a secondary id that can be used if a client needs to provide a self-generated id. or if the job id doesn't exist (ex: if the submission job doesn't create a ray driver) This PR has 2 deprecations - The `submit_job` sdk now accepts a new kwarg `submission_id`. `job_id is deprecated. - The `ray job submit` CLI now accepts `--submission-id`. `--job-id` is deprecated. **This PR has 4 backwards incompatible changes:** - list_jobs sdk now returns a list instead of a dictionary - the `ray job list` CLI now prints a list instead of a dictionary - The `/api/jobs` endpoint returns a list instead of a dictionary - The `POST api/jobs` endpoint (submit job) now returns a json with `submission_id` field instead of `job_id`.
2022-07-27 02:39:52 -07:00
from enum import Enum
from typing import Any, Dict, Optional
from pydantic import BaseModel, Field
from ray.dashboard.modules.job.common import JobStatus
class DriverInfo(BaseModel):
"""A class for recording information about the driver related to the job."""
id: str = Field(..., description="The id of the driver")
node_ip_address: str = Field(
..., description="The ip address of the node the driver is running on"
)
pid: str = Field(
..., description="The pid of the worker process the driver is using."
)
# TODO(aguo): Add node_id as a field.
class JobType(str, Enum):
"""An enumeration for describing the different job types."""
#: A job that was initiated by the job submission apis
SUBMISSION = "SUBMISSION"
#: A job that was initiated by a driver script.
DRIVER = "DRIVER"
class JobDetails(BaseModel):
"""
Job data with extra details about its driver and its submission.
"""
type: JobType = Field(..., description="The type of job.")
entrypoint: Optional[str] = Field(
None, description="The entrypoint command for this job."
)
job_id: Optional[str] = Field(
None,
description="The job id. An id that is created for every job that is "
"launched in ray. This can be used to fetch data about jobs using ray "
"core apis.",
)
submission_id: Optional[str] = Field(
None,
description="A submission id is an id created for every submission job. It can "
"be used to fetch data about jobs using the job submission apis.",
)
driver_info: Optional[DriverInfo] = Field(
None,
description="The driver related to this job. For submission jobs, "
"it is the last driver launched by that job submission, "
"or None if there is no driver.",
)
# The following fields are copied from JobInfo.
# TODO(aguo): Inherit from JobInfo once it's migrated to pydantic.
status: JobStatus = Field(..., description="The status of the job.")
entrypoint: str = Field(..., description="The entrypoint command for this job.")
message: Optional[str] = Field(
None, description="A message describing the status in more detail."
)
error_type: Optional[str] = Field(
None, description="Internal error, user script error"
)
start_time: Optional[int] = Field(
None,
description="The time when the job was started. " "A Unix timestamp in ms.",
)
end_time: Optional[int] = Field(
None,
description="The time when the job moved into a terminal state. "
"A Unix timestamp in ms.",
)
metadata: Optional[Dict[str, str]] = Field(
None, description="Arbitrary user-provided metadata for the job."
)
runtime_env: Optional[Dict[str, Any]] = Field(
None, description="The runtime environment for the job."
)