ray/release/air_tests/horovod/app_config_master.yaml

base_image: {{ env["RAY_IMAGE_ML_NIGHTLY_GPU"] | default("anyscale/ray-ml:nightly-py37-gpu") }}
env_vars: {}
debian_packages:
  - curl

python:
  pip_packages:
    - pytest
    - awscli
  conda_packages: []

post_build_cmds:
  - pip3 uninstall ray -y || true
  - pip3 install -U {{ env["RAY_WHEELS"] | default("ray") }}
  - pip3 install 'ray[tune]'
  - HOROVOD_WITH_GLOO=1 HOROVOD_WITHOUT_MPI=1 HOROVOD_WITHOUT_TENSORFLOW=1 HOROVOD_WITHOUT_MXNET=1 HOROVOD_WITH_PYTORCH=1 pip3 install -U git+https://github.com/horovod/horovod.git
  - {{ env["RAY_WHEELS_SANITY_CHECK"] | default("echo No Ray wheels sanity check") }}
[ci/release] Support running tests with different python versions (#24843) OSS release tests currently run with hardcoded Python 3.7 base. In the future we will want to run tests on different python versions. This PR adds support for a new `python` field in the test configuration. The python field will determine both the base image used in the Buildkite runner docker container (for Ray client compatibility) and the base image for the Anyscale cluster environments. Note that in Buildkite, we will still only wait for the python 3.7 base image before kicking off tests. That is acceptable, as we can assume that most wheels finish in a similar time, so even if we wait for the 3.7 image and kick off a 3.8 test, that runner will wait maybe for 5-10 more minutes. 2022-05-17 17:03:12 +01:00			`base_image: {{ env["RAY_IMAGE_ML_NIGHTLY_GPU"] \| default("anyscale/ray-ml:nightly-py37-gpu") }}`
[release] convert tune release tests (#15913) 2021-06-01 20:19:15 +02:00			`env_vars: {}`
			`debian_packages:`
			`- curl`

			`python:`
			`pip_packages:`
			`- pytest`
			`- awscli`
			`conda_packages: []`

			`post_build_cmds:`
[release] Uninstall old ray in all release test app configs to fix commit mismatch error (#21175) * uninstall old ray in all release test app configs * add instruction to e2e.py dosctring 2021-12-18 16:58:49 -08:00			`- pip3 uninstall ray -y \|\| true`
[release] convert tune release tests (#15913) 2021-06-01 20:19:15 +02:00			`- pip3 install -U {{ env["RAY_WHEELS"] \| default("ray") }}`
[Release/Horovod] Add user test for Horovod (#19661) * infra * wip * add test * typo * typo * update * rename * fix * full path * formatting * reorder * update * update * Update release/horovod_tests/workloads/horovod_user_test.py Co-authored-by: matthewdeng <matthew.j.deng@gmail.com> * bump num_workers * update installs * try * add pip_packages * min_workers * fix * bump pg timeout * Fix symlink * fix * fix * cmake * fix * pin filelock * final * update * fix * Update release/horovod_tests/workloads/horovod_user_test.py * fix * fix * separate compute template * test latest and master Co-authored-by: matthewdeng <matthew.j.deng@gmail.com> 2021-11-01 18:28:07 -07:00			`- pip3 install 'ray[tune]'`
[release test] fix horovod release test. (#22781) horovod_user_test_master is failing with recent horovod release[[link](https://buildkite.com/ray-project/periodic-ci/builds/2960#61dabda8-eea0-4b7b-93bf-9e341926d3fd)]. Error message is saying: ``` AttributeError: Can't get attribute '_ExecutorDriver' on <module 'horovod.ray.runner' from '/home/ray/anaconda3/lib/python3.7/site-packages/horovod/ray/runner.py'> ``` The horovod test is set up in such a way that it has the "driver" (a.k.a. client) part (which is the code that runs in a buildkite agent) and the "cluster" (a.k.a. server) part (which runs in Anyscale cluster). Driver's dependency is specified by `release/ml_user_tests/horovod/driver_setup_master.sh` while cluster's dependency is specified by `release/horovod_tests/app_config_master.yaml`. The two communicate via Anyscale client. The above error message is complaining that while client's horovod version has _ExecutorDriver in runner.py, the server's horovod doesn't. This is due to the version mismatch of the above two files. This PR brings the two horovod dependency to both point to horovod master. 2022-03-03 08:24:26 -08:00			`- HOROVOD_WITH_GLOO=1 HOROVOD_WITHOUT_MPI=1 HOROVOD_WITHOUT_TENSORFLOW=1 HOROVOD_WITHOUT_MXNET=1 HOROVOD_WITH_PYTORCH=1 pip3 install -U git+https://github.com/horovod/horovod.git`
[ci/release] Add sanity check for ray wheels hash to release tests (#18489) 2021-09-10 17:50:31 +01:00			`- {{ env["RAY_WHEELS_SANITY_CHECK"] \| default("echo No Ray wheels sanity check") }}`