mirror of
https://github.com/vale981/ray
synced 2025-03-06 02:21:39 -05:00
940c028540
We sometimes end up with stale wheel uploads from previous runs of a Buildkite agent. The result is that commit wheels are being overwritten from old build jobs - effectively breaking the wheel build logic. Example: This Agent: https://buildkite.com/organizations/ray-project/agents/4b955117-2f6c-4849-b703-3457daf69f89 - builds wheels (in post-wheels tests) for a35ebc945b - and then runs both the Ray CPP worker and the Train + Tune tests in 6746e9f - Usually these two tests shouldn't provide artifacts at all, but they do - these are the wheels from a35ebc945b though! Meaning these are uncleaned leftovers from the first build task. - See here for proof of artifact upload: https://buildkite.com/ray-project/ray-builders-pr/builds/27622#d11bc514-ebd8-4e0c-a2ce-826b9bad27de The solution is thus to always clean up the artifacts directory in the worker, i.e. `rm -rf /artifact-mount/*` This PR adds two of such clean up instructions - once before commands are run and once after artifacts are uploaded. We can probably just do either, but it doesn't hurt to have both. |
||
|---|---|---|
| .. | ||
| hooks | ||
| copy_files.py | ||
| Dockerfile | ||
| Dockerfile.gpu | ||
| pipeline.gpu.large.yml | ||
| pipeline.gpu.yml | ||
| pipeline.macos.yml | ||
| pipeline.ml.yml | ||
| pipeline.windows.yml | ||
| pipeline.yml | ||