ray/.buildkite
Kai Fricke 331b71ea8d
[ci/release] Refactor release test e2e into package (#22351)
Adds a unit-tested and restructured ray_release package for running release tests.

Relevant changes in behavior:

Per default, Buildkite will wait for the wheels of the current commit to be available. Alternatively, users can a) specify a different commit hash, b) a wheels URL (which we will also wait for to be available) or c) specify a branch (or user/branch combination), in which case the latest available wheels will be used (e.g. if master is passed, behavior matches old default behavior).

The main subpackages are:

    Cluster manager: Creates cluster envs/computes, starts cluster, terminates cluster
    Command runner: Runs commands, e.g. as client command or sdk command
    File manager: Uploads/downloads files to/from session
    Reporter: Reports results (e.g. to database)

Much of the code base is unit tested, but there are probably some pieces missing.

Example build (waited for wheels to be built): https://buildkite.com/ray-project/kf-dev/builds/51#_
Wheel build: https://buildkite.com/ray-project/ray-builders-branch/builds/6023
2022-02-16 17:35:02 +00:00
..
copy_files.py [CI] Format Python code with Black (#21975) 2022-01-29 18:41:57 -08:00
Dockerfile [ci/buildkite] Add ml pipeline to speed up ML/RLLib tests (#20895) 2021-12-09 21:14:10 +00:00
Dockerfile.gpu [ci/gpu] Move ML dependency install to Dockerfile (#21711) 2022-02-01 12:04:55 +00:00
pipeline.gpu.large.yml [ci/gpu] Move ML dependency install to Dockerfile (#21711) 2022-02-01 12:04:55 +00:00
pipeline.gpu.yml [ci/gpu] Move ML dependency install to Dockerfile (#21711) 2022-02-01 12:04:55 +00:00
pipeline.macos.yml [Test] Shard "Small & Large" tests (#21351) 2022-01-05 10:49:14 -08:00
pipeline.ml.yml Split workflow and dataset tests (#22415) 2022-02-16 01:47:55 -08:00
pipeline.windows.yml [CI] Migrate Windows Wheels to Buildkite (#21388) 2022-01-05 12:49:19 -08:00
pipeline.yml [ci/release] Refactor release test e2e into package (#22351) 2022-02-16 17:35:02 +00:00