hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-10 13:26:39 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
ray/ci/jenkins_tests/run_multi_node_tests.sh

47 lines
1.6 KiB
Bash
Raw Normal View History

Add multinode tests by simulating multiple nodes using Docker. (#378) * run test workloads for a Docker cluster * better manage docker image versions * Changes to make multinode docker tests work with Python 3. * option to mount local test directory on head node to speed development * Attempt to simplify multinode test setup. * Small change. * Add in development-mode to run multinode docker tests more easily during development. * add jenkins test script that links to Docker hash * Read docker SHA from build_docker.sh and add test that should fail. * Consolidate implementations and remove duplicate files. * Allow test to retry if it fails to schedule on all nodes. * Remove sleep when in docker multinode tests.
2017-03-18 23:44:54 -07:00
#!/usr/bin/env bash
# Cause the script to exit if a single command fails.
set -e
Add script for building MacOS wheels. (#601) * Add script for building MacOS wheels. * Small cleanups to script. * Fix setting of PATH before building wheel. * Create symbolic link to correct Python executable so Ray installation finds the right Python. * Address comments. * Rename readme.
2017-05-31 17:30:46 -07:00
# Show explicitly which commands are currently running.
adding -x for clearer output in build console log (#565)
2017-05-18 17:04:56 -07:00
set -x
increase container memory and shm to 20G (#3475) * increase container memory and shm to 20G * variables are POWERFUL
2018-12-05 14:59:07 -08:00
MEMORY_SIZE="20G"
SHM_SIZE="20G"
Add multinode tests by simulating multiple nodes using Docker. (#378) * run test workloads for a Docker cluster * better manage docker image versions * Changes to make multinode docker tests work with Python 3. * option to mount local test directory on head node to speed development * Attempt to simplify multinode test setup. * Small change. * Add in development-mode to run multinode docker tests more easily during development. * add jenkins test script that links to Docker hash * Read docker SHA from build_docker.sh and add test that should fail. * Consolidate implementations and remove duplicate files. * Allow test to retry if it fails to schedule on all nodes. * Remove sleep when in docker multinode tests.
2017-03-18 23:44:54 -07:00
ROOT_DIR=$(cd "$(dirname "${BASH_SOURCE:-$0}")"; pwd)
Test example applications and rllib in jenkins tests. (#707) * Test example applications in Jenkins. * Fix default upload_dir argument for Algorithm class. * Fix evolution strategies. * Comment out policy gradient example which doesn't seem to work. * Set --env-name for evolution strategies.
2017-07-16 11:51:33 -07:00
DOCKER_SHA=$($ROOT_DIR/../../build-docker.sh --output-sha --no-cache)
[build] Make travis logs not as long (#4213) * clean it up * Update .travis.yml * Update .travis.yml * update * fix example * suppress * timeout * print periodic progress * Update suppress_output * Update run_silent.sh * Update suppress_output * Update suppress_output * manually do timeout * sleep 300 * fix test * Update run_silent.sh * Update suppress_output * Update .travis.yml
2019-03-07 12:09:03 -08:00
SUPPRESS_OUTPUT=$ROOT_DIR/../suppress_output
Add multinode tests by simulating multiple nodes using Docker. (#378) * run test workloads for a Docker cluster * better manage docker image versions * Changes to make multinode docker tests work with Python 3. * option to mount local test directory on head node to speed development * Attempt to simplify multinode test setup. * Small change. * Add in development-mode to run multinode docker tests more easily during development. * add jenkins test script that links to Docker hash * Read docker SHA from build_docker.sh and add test that should fail. * Consolidate implementations and remove duplicate files. * Allow test to retry if it fails to schedule on all nodes. * Remove sleep when in docker multinode tests.
2017-03-18 23:44:54 -07:00
echo "Using Docker image" $DOCKER_SHA
Set RAY_FORCE_DIRECT=1 for run_rllib_tests, test_basic (#6171)
2019-11-25 14:12:11 -08:00
######################## RLLIB TESTS #################################
source $ROOT_DIR/run_rllib_tests.sh
Force tune tests to run in direct call mode (#6301) * force tune direct mode * force tune * fix * Update run_multi_node_tests.sh
2019-11-27 19:58:33 -08:00
######################## TUNE TESTS #################################
bash $ROOT_DIR/run_tune_tests.sh ${MEMORY_SIZE} ${SHM_SIZE} $DOCKER_SHA
[docs] Convert Examples to Gallery (#5414)
2019-09-24 15:46:56 -07:00
######################## EXAMPLE TESTS #################################
$SUPPRESS_OUTPUT docker run --rm --shm-size=${SHM_SIZE} --memory=${MEMORY_SIZE} $DOCKER_SHA \
python /ray/doc/examples/plot_pong_example.py
$SUPPRESS_OUTPUT docker run --rm --shm-size=${SHM_SIZE} --memory=${MEMORY_SIZE} $DOCKER_SHA \
python /ray/doc/examples/plot_parameter_server.py
$SUPPRESS_OUTPUT docker run --rm --shm-size=${SHM_SIZE} --memory=${MEMORY_SIZE} $DOCKER_SHA \
python /ray/doc/examples/plot_hyperparameter.py
Update TF documentation (#5918)
2019-10-16 01:31:27 -07:00
$SUPPRESS_OUTPUT docker run --rm --shm-size=${SHM_SIZE} --memory=${MEMORY_SIZE} $DOCKER_SHA \
python /ray/doc/examples/doc_code/torch_example.py
$SUPPRESS_OUTPUT docker run --rm --shm-size=${SHM_SIZE} --memory=${MEMORY_SIZE} $DOCKER_SHA \
python /ray/doc/examples/doc_code/tf_example.py
Move large mem test to end (#4664)
2019-04-19 11:43:22 -07:00
######################## RAY BACKEND TESTS #################################
$SUPPRESS_OUTPUT docker run --rm --shm-size=60G --memory=60G $DOCKER_SHA \
python /ray/ci/jenkins_tests/miscellaneous/large_memory_test.py
Reference in a new issue Copy permalink