mirror of
https://github.com/vale981/ray
synced 2025-03-06 18:41:40 -05:00
331b71ea8d
Adds a unit-tested and restructured ray_release package for running release tests.
Relevant changes in behavior:
Per default, Buildkite will wait for the wheels of the current commit to be available. Alternatively, users can a) specify a different commit hash, b) a wheels URL (which we will also wait for to be available) or c) specify a branch (or user/branch combination), in which case the latest available wheels will be used (e.g. if master is passed, behavior matches old default behavior).
The main subpackages are:
Cluster manager: Creates cluster envs/computes, starts cluster, terminates cluster
Command runner: Runs commands, e.g. as client command or sdk command
File manager: Uploads/downloads files to/from session
Reporter: Reports results (e.g. to database)
Much of the code base is unit tested, but there are probably some pieces missing.
Example build (waited for wheels to be built): https://buildkite.com/ray-project/kf-dev/builds/51#_
Wheel build: https://buildkite.com/ray-project/ray-builders-branch/builds/6023
20 lines
600 B
Python
20 lines
600 B
Python
import abc
|
|
from typing import Optional
|
|
|
|
from ray_release.cluster_manager.cluster_manager import ClusterManager
|
|
|
|
|
|
class FileManager(abc.ABC):
|
|
def __init__(self, cluster_manager: ClusterManager):
|
|
self.cluster_manager = cluster_manager
|
|
|
|
def upload(self, source: Optional[str] = None, target: Optional[str] = None):
|
|
"""Upload source to target.
|
|
|
|
Infers target dir from basename if not stated.
|
|
"""
|
|
raise NotImplementedError
|
|
|
|
def download(self, source: str, target: str):
|
|
"""Download source_dir to target_dir."""
|
|
raise NotImplementedError
|