ray/ci/jenkins_tests/run_multi_node_tests.sh

#!/usr/bin/env bash

# Cause the script to exit if a single command fails.
set -e

# Show explicitly which commands are currently running.
set -x

MEMORY_SIZE="20G"
SHM_SIZE="20G"

ROOT_DIR=$(cd "$(dirname "${BASH_SOURCE:-$0}")"; pwd)

DOCKER_SHA=$($ROOT_DIR/../../build-docker.sh --output-sha --no-cache)
SUPPRESS_OUTPUT=$ROOT_DIR/../suppress_output
echo "Using Docker image" $DOCKER_SHA

######################## RLLIB TESTS #################################

source $ROOT_DIR/run_rllib_tests.sh

######################## TUNE TESTS #################################

bash $ROOT_DIR/run_tune_tests.sh ${MEMORY_SIZE} ${SHM_SIZE} $DOCKER_SHA

######################## RAY BACKEND TESTS #################################

$SUPPRESS_OUTPUT docker run --rm --shm-size=60G --memory=60G $DOCKER_SHA \
    python /ray/ci/jenkins_tests/miscellaneous/large_memory_test.py

######################## SGD TESTS #################################

$SUPPRESS_OUTPUT docker run --rm --shm-size=${SHM_SIZE} --memory=${MEMORY_SIZE} $DOCKER_SHA \
    python -m pytest /ray/python/ray/experimental/sgd/tests
Add multinode tests by simulating multiple nodes using Docker. (#378) * run test workloads for a Docker cluster * better manage docker image versions * Changes to make multinode docker tests work with Python 3. * option to mount local test directory on head node to speed development * Attempt to simplify multinode test setup. * Small change. * Add in development-mode to run multinode docker tests more easily during development. * add jenkins test script that links to Docker hash * Read docker SHA from build_docker.sh and add test that should fail. * Consolidate implementations and remove duplicate files. * Allow test to retry if it fails to schedule on all nodes. * Remove sleep when in docker multinode tests. 2017-03-18 23:44:54 -07:00			`#!/usr/bin/env bash`

			`# Cause the script to exit if a single command fails.`
			`set -e`

Add script for building MacOS wheels. (#601) * Add script for building MacOS wheels. * Small cleanups to script. * Fix setting of PATH before building wheel. * Create symbolic link to correct Python executable so Ray installation finds the right Python. * Address comments. * Rename readme. 2017-05-31 17:30:46 -07:00			`# Show explicitly which commands are currently running.`
adding -x for clearer output in build console log (#565) 2017-05-18 17:04:56 -07:00			`set -x`

increase container memory and shm to 20G (#3475) * increase container memory and shm to 20G * variables are POWERFUL 2018-12-05 14:59:07 -08:00			`MEMORY_SIZE="20G"`
			`SHM_SIZE="20G"`

Add multinode tests by simulating multiple nodes using Docker. (#378) * run test workloads for a Docker cluster * better manage docker image versions * Changes to make multinode docker tests work with Python 3. * option to mount local test directory on head node to speed development * Attempt to simplify multinode test setup. * Small change. * Add in development-mode to run multinode docker tests more easily during development. * add jenkins test script that links to Docker hash * Read docker SHA from build_docker.sh and add test that should fail. * Consolidate implementations and remove duplicate files. * Allow test to retry if it fails to schedule on all nodes. * Remove sleep when in docker multinode tests. 2017-03-18 23:44:54 -07:00			`ROOT_DIR=$(cd "$(dirname "${BASH_SOURCE:-$0}")"; pwd)`

Test example applications and rllib in jenkins tests. (#707) * Test example applications in Jenkins. * Fix default upload_dir argument for Algorithm class. * Fix evolution strategies. * Comment out policy gradient example which doesn't seem to work. * Set --env-name for evolution strategies. 2017-07-16 11:51:33 -07:00			`DOCKER_SHA=$($ROOT_DIR/../../build-docker.sh --output-sha --no-cache)`
[build] Make travis logs not as long (#4213) * clean it up * Update .travis.yml * Update .travis.yml * update * fix example * suppress * timeout * print periodic progress * Update suppress_output * Update run_silent.sh * Update suppress_output * Update suppress_output * manually do timeout * sleep 300 * fix test * Update run_silent.sh * Update suppress_output * Update .travis.yml 2019-03-07 12:09:03 -08:00			`SUPPRESS_OUTPUT=$ROOT_DIR/../suppress_output`
Add multinode tests by simulating multiple nodes using Docker. (#378) * run test workloads for a Docker cluster * better manage docker image versions * Changes to make multinode docker tests work with Python 3. * option to mount local test directory on head node to speed development * Attempt to simplify multinode test setup. * Small change. * Add in development-mode to run multinode docker tests more easily during development. * add jenkins test script that links to Docker hash * Read docker SHA from build_docker.sh and add test that should fail. * Consolidate implementations and remove duplicate files. * Allow test to retry if it fails to schedule on all nodes. * Remove sleep when in docker multinode tests. 2017-03-18 23:44:54 -07:00			`echo "Using Docker image" $DOCKER_SHA`

[rllib] Silent tests (#4151) 2019-02-28 16:32:22 -08:00			`######################## RLLIB TESTS #################################`
Exploration with Parameter Space Noise (#4048) * enable parameter space noise for exploration * enable parameter space noise for exploration * yapf formatted * remove the usage of scipy softmax avialable in the latest version only * enable subclass that has no parameter_noise in the config * run user specified callbacks and test parameter space noise in multi node setting * formatted by yapf * Update dqn.py * lint 2019-02-21 14:35:18 +08:00
[rllib] Silent tests (#4151) 2019-02-28 16:32:22 -08:00			`source $ROOT_DIR/run_rllib_tests.sh`
[rllib] Support batch norm layers (#3369) * batch norm * lint * fix dqn/ddpg update ops * bn model * Update tf_policy_graph.py * Update multi_gpu_impl.py * Apply suggestions from code review Co-Authored-By: ericl <ekhliang@gmail.com> 2018-11-29 13:33:39 -08:00
Modularize Tune tests from multi-node tests (#4204) 2019-03-02 19:21:08 -08:00			`######################## TUNE TESTS #################################`

			`bash $ROOT_DIR/run_tune_tests.sh ${MEMORY_SIZE} ${SHM_SIZE} $DOCKER_SHA`
[rllib] Support batch norm layers (#3369) * batch norm * lint * fix dqn/ddpg update ops * bn model * Update tf_policy_graph.py * Update multi_gpu_impl.py * Apply suggestions from code review Co-Authored-By: ericl <ekhliang@gmail.com> 2018-11-29 13:33:39 -08:00
Move large mem test to end (#4664) 2019-04-19 11:43:22 -07:00			`######################## RAY BACKEND TESTS #################################`

			`$SUPPRESS_OUTPUT docker run --rm --shm-size=60G --memory=60G $DOCKER_SHA \`
			`python /ray/ci/jenkins_tests/miscellaneous/large_memory_test.py`

[rllib] Silent tests (#4151) 2019-02-28 16:32:22 -08:00			`######################## SGD TESTS #################################`
[rllib] Q-Mix implementation (Q-Mix, VDN, IQN, and Ape-X variants) (#3548) 2018-12-18 10:40:01 -08:00
[build] Make travis logs not as long (#4213) * clean it up * Update .travis.yml * Update .travis.yml * update * fix example * suppress * timeout * print periodic progress * Update suppress_output * Update run_silent.sh * Update suppress_output * Update suppress_output * manually do timeout * sleep 300 * fix test * Update run_silent.sh * Update suppress_output * Update .travis.yml 2019-03-07 12:09:03 -08:00			`$SUPPRESS_OUTPUT docker run --rm --shm-size=${SHM_SIZE} --memory=${MEMORY_SIZE} $DOCKER_SHA \`
[sgd] Distributed Training via PyTorch (#4797) Implements distributed SGD using distributed PyTorch. 2019-06-01 21:39:22 -07:00			`python -m pytest /ray/python/ray/experimental/sgd/tests`