hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-11 13:46:40 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
ray/release/nightly_tests/dataset/dataset_test.yaml

92 lines
2.3 KiB
YAML
Raw Normal View History

[Dataset] imagenet nightly test (#17069)
2021-07-16 14:15:49 -07:00
- name: inference
[Nightly Test] Add a team column to each test config. (#21198) Please review **e2e.py and test_suite belonging to your team**! This is the first part of https://docs.google.com/document/d/16IrwerYi2oJugnRf5hvzukgpJ6FAVEpB6stH_CiNMjY/edit# This PR adds a team name to each test suite. If the name is not specified, it will be reported as unspecified. If you are running a local test, and if the new test suite doesn't have a team name specified, it will raise an exception (in this way, we can avoid missing team names in the future). Note that we will aggregate all of test config into a single file, nightly_test.yaml.
2021-12-28 07:42:41 +09:00
team: core
[Dataset] imagenet nightly test (#17069)
2021-07-16 14:15:49 -07:00
cluster:
app_config: app_config.yaml
compute_template: inference.yaml
run:
timeout: 600
[dataset][nightly-test] add pipelined ingestion/training nightly test
2021-09-23 20:39:03 -07:00
prepare: python wait_cluster.py 2 600
[Dataset] imagenet nightly test (#17069)
2021-07-16 14:15:49 -07:00
script: python inference.py
Dataset speed up read (#17435)
2021-08-01 18:03:46 -07:00
add dataset shuffle data loader (#17917)
2021-08-20 11:26:01 -07:00
- name: shuffle_data_loader
[Nightly Test] Add a team column to each test config. (#21198) Please review **e2e.py and test_suite belonging to your team**! This is the first part of https://docs.google.com/document/d/16IrwerYi2oJugnRf5hvzukgpJ6FAVEpB6stH_CiNMjY/edit# This PR adds a team name to each test suite. If the name is not specified, it will be reported as unspecified. If you are running a local test, and if the new test suite doesn't have a team name specified, it will raise an exception (in this way, we can avoid missing team names in the future). Note that we will aggregate all of test config into a single file, nightly_test.yaml.
2021-12-28 07:42:41 +09:00
team: core
add dataset shuffle data loader (#17917)
2021-08-20 11:26:01 -07:00
cluster:
app_config: shuffle_app_config.yaml
compute_template: shuffle_compute.yaml
run:
timeout: 1800
script: python dataset_shuffle_data_loader.py
[dataset][nightly-test] add pipelined ingestion/training nightly test
2021-09-23 20:39:03 -07:00
- name: pipelined_training_50_gb
[Nightly Test] Add a team column to each test config. (#21198) Please review **e2e.py and test_suite belonging to your team**! This is the first part of https://docs.google.com/document/d/16IrwerYi2oJugnRf5hvzukgpJ6FAVEpB6stH_CiNMjY/edit# This PR adds a team name to each test suite. If the name is not specified, it will be reported as unspecified. If you are running a local test, and if the new test suite doesn't have a team name specified, it will raise an exception (in this way, we can avoid missing team names in the future). Note that we will aggregate all of test config into a single file, nightly_test.yaml.
2021-12-28 07:42:41 +09:00
team: core
[dataset][nightly-test] add pipelined ingestion/training nightly test
2021-09-23 20:39:03 -07:00
cluster:
app_config: pipelined_training_app.yaml
compute_template: pipelined_training_compute.yaml
run:
timeout: 4800
prepare: python wait_cluster.py 15 1200
[Dataset][nighlyt-test] spend less money #19488 Reduce the epoch and ensure everything runs in the same datacenter.
2021-10-18 18:53:50 -07:00
script: python pipelined_training.py --epochs 1
[dataset][nightly-test] add pipelined ingestion/training nightly test
2021-09-23 20:39:03 -07:00
- name: pipelined_ingestion_1500_gb_15_windows
[Nightly Test] Add a team column to each test config. (#21198) Please review **e2e.py and test_suite belonging to your team**! This is the first part of https://docs.google.com/document/d/16IrwerYi2oJugnRf5hvzukgpJ6FAVEpB6stH_CiNMjY/edit# This PR adds a team name to each test suite. If the name is not specified, it will be reported as unspecified. If you are running a local test, and if the new test suite doesn't have a team name specified, it will raise an exception (in this way, we can avoid missing team names in the future). Note that we will aggregate all of test config into a single file, nightly_test.yaml.
2021-12-28 07:42:41 +09:00
team: core
[dataset][nightly-test] add pipelined ingestion/training nightly test
2021-09-23 20:39:03 -07:00
cluster:
app_config: pipelined_ingestion_app.yaml
compute_template: pipelined_ingestion_compute.yaml
run:
timeout: 4800
prepare: python wait_cluster.py 21 2400
script: python pipelined_training.py --epochs 2 --num-windows 15 --num-files 915 --debug
[CUJ2] add nightly tests for running 500GB ray train (#20195) * add * update cluster env * fix build Co-authored-by: Matthew Deng <matthew.j.deng@gmail.com>
2021-11-21 20:04:45 -08:00
- name: datasets_ingest_train_infer
[Nightly Test] Add a team column to each test config. (#21198) Please review **e2e.py and test_suite belonging to your team**! This is the first part of https://docs.google.com/document/d/16IrwerYi2oJugnRf5hvzukgpJ6FAVEpB6stH_CiNMjY/edit# This PR adds a team name to each test suite. If the name is not specified, it will be reported as unspecified. If you are running a local test, and if the new test suite doesn't have a team name specified, it will raise an exception (in this way, we can avoid missing team names in the future). Note that we will aggregate all of test config into a single file, nightly_test.yaml.
2021-12-28 07:42:41 +09:00
team: core
[CUJ2] add nightly tests for running 500GB ray train (#20195) * add * update cluster env * fix build Co-authored-by: Matthew Deng <matthew.j.deng@gmail.com>
2021-11-21 20:04:45 -08:00
cluster:
app_config: ray_sgd_training_app.yaml
compute_template: ray_sgd_training_compute.yaml
run:
timeout: 14400
prepare: python wait_cluster.py 66 2400
script: python ray_sgd_training.py --address auto --use-s3 --num-workers 16 --use-gpu --large-dataset
[cuj2] merge latest change to cuj2 (groupby based filtering) and add a debug mode. (#20742) This PR does two things: merge latest groupby based filtering to CUJ2 add a debug mode so we only run dummy trainer for measure data processing performance.
2021-11-29 19:10:17 -08:00
stable: false
[CUJ2] add nightly tests for running 500GB ray train (#20195) * add * update cluster env * fix build Co-authored-by: Matthew Deng <matthew.j.deng@gmail.com>
2021-11-21 20:04:45 -08:00
smoke_test:
cluster:
app_config: ray_sgd_training_app.yaml
compute_template: ray_sgd_training_smoke_compute.yaml
run:
timeout: 3600
prepare: python wait_cluster.py 8 2400
script: python ray_sgd_training.py --address auto --use-s3 --num-workers 8 --use-gpu
[cuj2] merge latest change to cuj2 (groupby based filtering) and add a debug mode. (#20742) This PR does two things: merge latest groupby based filtering to CUJ2 add a debug mode so we only run dummy trainer for measure data processing performance.
2021-11-29 19:10:17 -08:00
- name: datasets_preprocess_ingest
[Nightly Test] Add a team column to each test config. (#21198) Please review **e2e.py and test_suite belonging to your team**! This is the first part of https://docs.google.com/document/d/16IrwerYi2oJugnRf5hvzukgpJ6FAVEpB6stH_CiNMjY/edit# This PR adds a team name to each test suite. If the name is not specified, it will be reported as unspecified. If you are running a local test, and if the new test suite doesn't have a team name specified, it will raise an exception (in this way, we can avoid missing team names in the future). Note that we will aggregate all of test config into a single file, nightly_test.yaml.
2021-12-28 07:42:41 +09:00
team: core
[cuj2] merge latest change to cuj2 (groupby based filtering) and add a debug mode. (#20742) This PR does two things: merge latest groupby based filtering to CUJ2 add a debug mode so we only run dummy trainer for measure data processing performance.
2021-11-29 19:10:17 -08:00
cluster:
app_config: ray_sgd_training_app.yaml
[nighly-test] update cuj2 to reflect latest change #20889 we fixed groupby issue in cuj2; sync the change into nightly test. this test doesn't need to use gpu at all. it returns soon after data ingestion finishes.
2021-12-06 09:59:21 -08:00
compute_template: ray_sgd_training_compute_no_gpu.yaml
[cuj2] merge latest change to cuj2 (groupby based filtering) and add a debug mode. (#20742) This PR does two things: merge latest groupby based filtering to CUJ2 add a debug mode so we only run dummy trainer for measure data processing performance.
2021-11-29 19:10:17 -08:00
run:
[nighly-test] update cuj2 to reflect latest change #20889 we fixed groupby issue in cuj2; sync the change into nightly test. this test doesn't need to use gpu at all. it returns soon after data ingestion finishes.
2021-12-06 09:59:21 -08:00
timeout: 7200
prepare: python wait_cluster.py 21 2400
[cuj2] merge latest change to cuj2 (groupby based filtering) and add a debug mode. (#20742) This PR does two things: merge latest groupby based filtering to CUJ2 add a debug mode so we only run dummy trainer for measure data processing performance.
2021-11-29 19:10:17 -08:00
script: python ray_sgd_training.py --address auto --use-s3 --num-workers 16 --use-gpu --large-dataset --debug
stable: false
[dataset][cuj2] add another single node ingestion example (#20754) * add runner * fix bugs * add configs * add time
2021-12-07 02:50:17 -08:00
- name: datasets_ingest_400G
[Nightly Test] Add a team column to each test config. (#21198) Please review **e2e.py and test_suite belonging to your team**! This is the first part of https://docs.google.com/document/d/16IrwerYi2oJugnRf5hvzukgpJ6FAVEpB6stH_CiNMjY/edit# This PR adds a team name to each test suite. If the name is not specified, it will be reported as unspecified. If you are running a local test, and if the new test suite doesn't have a team name specified, it will raise an exception (in this way, we can avoid missing team names in the future). Note that we will aggregate all of test config into a single file, nightly_test.yaml.
2021-12-28 07:42:41 +09:00
team: core
[dataset][cuj2] add another single node ingestion example (#20754) * add runner * fix bugs * add configs * add time
2021-12-07 02:50:17 -08:00
cluster:
app_config: ray_sgd_training_app.yaml
compute_template: dataset_ingest_400G_compute.yaml
run:
timeout: 7200
script: python ray_sgd_runner.py --address auto --use-gpu --num-epochs 1
stable: false
Reference in a new issue Copy permalink