hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-06 10:31:39 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
ray/benchmarks/benchmark_tests.yaml

206 lines
5.4 KiB
YAML
Raw Normal View History

Integrate scalability envelope with releaser (#16417) * . * . * . * . * . * . * . * success Co-authored-by: Alex Wu <alex@anyscale.com>
2021-06-15 10:42:55 -07:00
- name: single_node
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
Integrate scalability envelope with releaser (#16417) * . * . * . * . * . * . * . * success Co-authored-by: Alex Wu <alex@anyscale.com>
2021-06-15 10:42:55 -07:00
cluster:
app_config: app_config.yaml
compute_template: single_node.yaml
run:
timeout: 12000
prepare: sleep 0
script: python single_node/test_single_node.py
- name: object_store
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
Integrate scalability envelope with releaser (#16417) * . * . * . * . * . * . * . * success Co-authored-by: Alex Wu <alex@anyscale.com>
2021-06-15 10:42:55 -07:00
cluster:
app_config: app_config.yaml
compute_template: object_store.yaml
run:
[nightly-test] increase timeout to 1 hour (#17125)
2021-07-15 12:30:08 -07:00
timeout: 3600
[Nightly Test] Fix a wrong prepare script for object store nightly test (#20739) By mistake, we are running sleep 0 instead of wait_cluster.py
2021-11-29 13:40:59 +09:00
prepare: python distributed/wait_cluster.py --num-nodes=50
Integrate scalability envelope with releaser (#16417) * . * . * . * . * . * . * . * success Co-authored-by: Alex Wu <alex@anyscale.com>
2021-06-15 10:42:55 -07:00
script: python object_store/test_object_store.py
Split scalability envelope + smoke tests (#17455) * . * done? * done? * sang comments * . Co-authored-by: Alex Wu <alex@anyscale.com>
2021-07-30 10:20:19 -07:00
- name: many_actors
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
Integrate scalability envelope with releaser (#16417) * . * . * . * . * . * . * . * success Co-authored-by: Alex Wu <alex@anyscale.com>
2021-06-15 10:42:55 -07:00
cluster:
app_config: app_config.yaml
compute_template: distributed.yaml
run:
timeout: 3600 # 1hr
Split scalability envelope + smoke tests (#17455) * . * done? * done? * sang comments * . Co-authored-by: Alex Wu <alex@anyscale.com>
2021-07-30 10:20:19 -07:00
prepare: python distributed/wait_cluster.py --num-nodes=65
script: python distributed/test_many_actors.py
- name: many_actors_smoke_test
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
Split scalability envelope + smoke tests (#17455) * . * done? * done? * sang comments * . Co-authored-by: Alex Wu <alex@anyscale.com>
2021-07-30 10:20:19 -07:00
cluster:
app_config: app_config.yaml
compute_template: distributed_smoke_test.yaml
run:
timeout: 3600 # 1hr
prepare: python distributed/wait_cluster.py --num-nodes=2
script: SMOKE_TEST=1 python distributed/test_many_actors.py
- name: many_tasks
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
Split scalability envelope + smoke tests (#17455) * . * done? * done? * sang comments * . Co-authored-by: Alex Wu <alex@anyscale.com>
2021-07-30 10:20:19 -07:00
cluster:
app_config: app_config.yaml
compute_template: distributed.yaml
run:
timeout: 3600 # 1hr
prepare: python distributed/wait_cluster.py --num-nodes=65
script: python distributed/test_many_tasks.py --num-tasks=10000
- name: many_tasks_smoke_test
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
Split scalability envelope + smoke tests (#17455) * . * done? * done? * sang comments * . Co-authored-by: Alex Wu <alex@anyscale.com>
2021-07-30 10:20:19 -07:00
cluster:
app_config: app_config.yaml
compute_template: distributed_smoke_test.yaml
run:
timeout: 3600 # 1hr
prepare: python distributed/wait_cluster.py --num-nodes=2
script: python distributed/test_many_tasks.py --num-tasks=100
- name: many_pgs
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
Split scalability envelope + smoke tests (#17455) * . * done? * done? * sang comments * . Co-authored-by: Alex Wu <alex@anyscale.com>
2021-07-30 10:20:19 -07:00
cluster:
app_config: app_config.yaml
compute_template: distributed.yaml
run:
timeout: 3600 # 1hr
prepare: python distributed/wait_cluster.py --num-nodes=65
script: python distributed/test_many_pgs.py
- name: many_pgs_smoke_test
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
Split scalability envelope + smoke tests (#17455) * . * done? * done? * sang comments * . Co-authored-by: Alex Wu <alex@anyscale.com>
2021-07-30 10:20:19 -07:00
cluster:
app_config: app_config.yaml
compute_template: distributed_smoke_test.yaml
run:
timeout: 3600 # 1hr
prepare: python distributed/wait_cluster.py --num-nodes=2
script: SMOKE_TEST=1 python distributed/test_many_pgs.py
# NOTE: No smoke test since this shares a script with the many_tasks_smoke_test
- name: many_nodes
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
Split scalability envelope + smoke tests (#17455) * . * done? * done? * sang comments * . Co-authored-by: Alex Wu <alex@anyscale.com>
2021-07-30 10:20:19 -07:00
cluster:
app_config: app_config.yaml
compute_template: many_nodes.yaml
run:
timeout: 3600 # 1hr
prepare: python distributed/wait_cluster.py --num-nodes=250
script: python distributed/test_many_tasks.py --num-tasks=1000
[nightly] Add staging nightly test for gcs ha (#21004) This PR adds four staging nightly tests for gcs : - many_actors - many_tasks - many_pgs - many_nodes These are benchmark tests that are highly related to gcs ha. To make it easier to add tests, this PR also change e2e.py a little bit to include testing flags to app config.
2021-12-09 23:07:23 -08:00
- name: many_tasks_redis_ha
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
[nightly] Add staging nightly test for gcs ha (#21004) This PR adds four staging nightly tests for gcs : - many_actors - many_tasks - many_pgs - many_nodes These are benchmark tests that are highly related to gcs ha. To make it easier to add tests, this PR also change e2e.py a little bit to include testing flags to app config.
2021-12-09 23:07:23 -08:00
cluster:
app_config: app_config.yaml
compute_template: distributed.yaml
run:
timeout: 3600 # 1hr
prepare: python distributed/wait_cluster.py --num-nodes=65
script: python distributed/test_many_tasks.py --num-tasks=10000
app_env_vars:
[nightly] Enable GCS HA nightly test with bootstrap (#21389) After https://github.com/ray-project/ray/pull/21232 we are able to start ray without redis. We need to bake the test for a while before turning on the flag by default. This PR add tests for this.
2022-01-05 10:53:07 -08:00
RAY_gcs_grpc_based_pubsub: "1"
[nightly] Add staging nightly test for gcs ha (#21004) This PR adds four staging nightly tests for gcs : - many_actors - many_tasks - many_pgs - many_nodes These are benchmark tests that are highly related to gcs ha. To make it easier to add tests, this PR also change e2e.py a little bit to include testing flags to app config.
2021-12-09 23:07:23 -08:00
RAY_gcs_storage: "memory"
- name: many_actors_redis_ha
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
[nightly] Add staging nightly test for gcs ha (#21004) This PR adds four staging nightly tests for gcs : - many_actors - many_tasks - many_pgs - many_nodes These are benchmark tests that are highly related to gcs ha. To make it easier to add tests, this PR also change e2e.py a little bit to include testing flags to app config.
2021-12-09 23:07:23 -08:00
cluster:
app_config: app_config.yaml
compute_template: distributed.yaml
run:
timeout: 3600 # 1hr
prepare: python distributed/wait_cluster.py --num-nodes=65
script: python distributed/test_many_actors.py
app_env_vars:
[nightly] Enable GCS HA nightly test with bootstrap (#21389) After https://github.com/ray-project/ray/pull/21232 we are able to start ray without redis. We need to bake the test for a while before turning on the flag by default. This PR add tests for this.
2022-01-05 10:53:07 -08:00
RAY_gcs_grpc_based_pubsub: "1"
[nightly] Add staging nightly test for gcs ha (#21004) This PR adds four staging nightly tests for gcs : - many_actors - many_tasks - many_pgs - many_nodes These are benchmark tests that are highly related to gcs ha. To make it easier to add tests, this PR also change e2e.py a little bit to include testing flags to app config.
2021-12-09 23:07:23 -08:00
RAY_gcs_storage: "memory"
- name: many_nodes_redis_ha
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
[nightly] Add staging nightly test for gcs ha (#21004) This PR adds four staging nightly tests for gcs : - many_actors - many_tasks - many_pgs - many_nodes These are benchmark tests that are highly related to gcs ha. To make it easier to add tests, this PR also change e2e.py a little bit to include testing flags to app config.
2021-12-09 23:07:23 -08:00
cluster:
app_config: app_config.yaml
compute_template: many_nodes.yaml
run:
timeout: 3600 # 1hr
prepare: python distributed/wait_cluster.py --num-nodes=250
script: python distributed/test_many_tasks.py --num-tasks=1000
app_env_vars:
[nightly] Enable GCS HA nightly test with bootstrap (#21389) After https://github.com/ray-project/ray/pull/21232 we are able to start ray without redis. We need to bake the test for a while before turning on the flag by default. This PR add tests for this.
2022-01-05 10:53:07 -08:00
RAY_gcs_grpc_based_pubsub: "1"
[nightly] Add staging nightly test for gcs ha (#21004) This PR adds four staging nightly tests for gcs : - many_actors - many_tasks - many_pgs - many_nodes These are benchmark tests that are highly related to gcs ha. To make it easier to add tests, this PR also change e2e.py a little bit to include testing flags to app config.
2021-12-09 23:07:23 -08:00
RAY_gcs_storage: "memory"
- name: many_pgs_redis_ha
[Test Infra] Unrevert team col (#21700) This fixes the previous problems from team column revert. This has 2 additional changes; alert handler receives the team argument, which was the root cause of breakage; https://github.com/ray-project/ray/pull/21289 Previously, tests without a team column were raising an exception, but I made the condition weaker (warning logs). I will eventually change it to raise an exception, but for smoother transition, we will log warning instead for a short time
2022-01-20 06:29:53 +09:00
team: core
[nightly] Add staging nightly test for gcs ha (#21004) This PR adds four staging nightly tests for gcs : - many_actors - many_tasks - many_pgs - many_nodes These are benchmark tests that are highly related to gcs ha. To make it easier to add tests, this PR also change e2e.py a little bit to include testing flags to app config.
2021-12-09 23:07:23 -08:00
cluster:
app_config: app_config.yaml
compute_template: distributed.yaml
run:
timeout: 3600 # 1hr
prepare: python distributed/wait_cluster.py --num-nodes=65
script: python distributed/test_many_pgs.py
app_env_vars:
[nightly] Enable GCS HA nightly test with bootstrap (#21389) After https://github.com/ray-project/ray/pull/21232 we are able to start ray without redis. We need to bake the test for a while before turning on the flag by default. This PR add tests for this.
2022-01-05 10:53:07 -08:00
RAY_gcs_grpc_based_pubsub: "1"
[nightly] Add staging nightly test for gcs ha (#21004) This PR adds four staging nightly tests for gcs : - many_actors - many_tasks - many_pgs - many_nodes These are benchmark tests that are highly related to gcs ha. To make it easier to add tests, this PR also change e2e.py a little bit to include testing flags to app config.
2021-12-09 23:07:23 -08:00
RAY_gcs_storage: "memory"
[nightly] Add more many tasks tests (#21727) This PR add four tests for many tasks: many short tasks send from the single node many short tasks send from multiple nodes many long tasks send from multiple nodes many long tasks send from the single node TODO: migrate many nodes actor tests to this one. scheduling envelop should contain: (tasks): scheduling_test_many_xx_tasks_yy_nodes (actors):many_nodes_actor_test (to be combined with this one) (shuffle): pipelined_ingestion_1500_gb_15_windows (shuffle): dask_on_ray_1tb_sort
2022-01-20 14:52:26 -08:00
- name: scheduling_test_many_0s_tasks_single_node
team: core
cluster:
app_config: app_config.yaml
compute_template: scheduling.yaml
run:
timeout: 3600
prepare: python distributed/wait_cluster.py --num-nodes=32
script: python distributed/test_scheduling.py --total-num-task=1984000 --num-cpu-per-task=1 --task-duration-s=0 --total-num-actors=1 --num-actors-per-nodes=1
- name: scheduling_test_many_0s_tasks_many_nodes
team: core
cluster:
app_config: app_config.yaml
compute_template: scheduling.yaml
run:
timeout: 3600
prepare: python distributed/wait_cluster.py --num-nodes=32
script: python distributed/test_scheduling.py --total-num-task=1984000 --num-cpu-per-task=1 --task-duration-s=0 --total-num-actors=32 --num-actors-per-nodes=1
- name: scheduling_test_many_5s_tasks_single_node
team: core
cluster:
app_config: app_config.yaml
compute_template: scheduling.yaml
run:
timeout: 3600
prepare: python distributed/wait_cluster.py --num-nodes=32
script: python distributed/test_scheduling.py --total-num-task=1984000 --num-cpu-per-task=1 --task-duration-s=5 --total-num-actors=1 --num-actors-per-nodes=1
stable: false
- name: scheduling_test_many_5s_tasks_many_nodes
team: core
cluster:
app_config: app_config.yaml
compute_template: scheduling.yaml
run:
timeout: 3600
prepare: python distributed/wait_cluster.py --num-nodes=32
script: python distributed/test_scheduling.py --total-num-task=1984000 --num-cpu-per-task=1 --task-duration-s=5 --total-num-actors=32 --num-actors-per-nodes=1
stable: false
Reference in a new issue Copy permalink