hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-06 02:21:39 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

[docs] search algorithm notebook examples (#23924)

Browse source 
Co-authored-by: brettskymind <brett@pathmind.com>
Co-authored-by: Max Pumperla <max.pumperla@googlemail.com>
This commit is contained in:
Brett Göhre 2022-04-25 11:10:58 -07:00 • committed by GitHub
parent 55b1d857ab
commit 9e0a59d94a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
26 changed files with 4114 additions and 166 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

318
doc/source/tune/examples/ax_example.ipynb Normal file
View file

@ -0,0 +1,318 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "47de02e1",
"metadata": {},
"source": [
"# Running Tune experiments with AxSearch\n",
"In this tutorial we introduce Ax, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with Ax and, as a result, allow you to seamlessly scale up a Ax optimization process - without sacrificing performance.\n",
"\n",
"Ax is a platform for optimizing any kind of experiment, including machine learning experiments, A/B tests, and simulations. Ax can optimize discrete configurations (e.g., variants of an A/B test) using multi-armed bandit optimization, and continuous/ordered configurations (e.g. float/int parameters) using Bayesian optimization. Results of A/B tests and simulations with reinforcement learning agents often exhibit high amounts of noise. Ax supports state-of-the-art algorithms which work better than traditional Bayesian optimization in high-noise settings. Ax also supports multi-objective and constrained optimization which are common to real-world problems (e.g. improving load time without increasing data use). Ax belongs to the domain of \"derivative-free\" and \"black-box\" optimization.\n",
"\n",
"In this example we minimize a simple objective to briefly demonstrate the usage of AxSearch with Ray Tune via `AxSearch`. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `ax-platform==0.2.4` library is installed withe python version >= 3.7. To learn more, please refer to the [Ax website](https://ax.dev/)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "297d8b18",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# !pip install ray[tune]\n",
"!pip install ax-platform==0.2.4"
]
},
{
"cell_type": "markdown",
"id": "59b1e0d1",
"metadata": {},
"source": [
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cbae6dbe",
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"import numpy as np\n",
"import time\n",
"\n",
"import ray\n",
"from ray import tune\n",
"from ray.tune.suggest.ax import AxSearch"
]
},
{
"cell_type": "markdown",
"id": "7b2b6af7",
"metadata": {},
"source": [
"Let's start by defining a classic benchmark for global optimization.\n",
"The form here is explicit for demonstration, yet it is typically a black-box.\n",
"We artificially sleep for a bit (`0.02` seconds) to simulate a long-running ML experiment.\n",
"This setup assumes that we're running multiple `step`s of an experiment and try to tune 6-dimensions of the `x` hyperparameter."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0f7fbe0f",
"metadata": {},
"outputs": [],
"source": [
"def landscape(x):\n",
" \"\"\"\n",
" Hartmann 6D function containing 6 local minima.\n",
" It is a classic benchmark for developing global optimization algorithms.\n",
" \"\"\"\n",
" alpha = np.array([1.0, 1.2, 3.0, 3.2])\n",
" A = np.array(\n",
" [\n",
" [10, 3, 17, 3.5, 1.7, 8],\n",
" [0.05, 10, 17, 0.1, 8, 14],\n",
" [3, 3.5, 1.7, 10, 17, 8],\n",
" [17, 8, 0.05, 10, 0.1, 14],\n",
" ]\n",
" )\n",
" P = 10 ** (-4) * np.array(\n",
" [\n",
" [1312, 1696, 5569, 124, 8283, 5886],\n",
" [2329, 4135, 8307, 3736, 1004, 9991],\n",
" [2348, 1451, 3522, 2883, 3047, 6650],\n",
" [4047, 8828, 8732, 5743, 1091, 381],\n",
" ]\n",
" )\n",
" y = 0.0\n",
" for j, alpha_j in enumerate(alpha):\n",
" t = 0\n",
" for k in range(6):\n",
" t += A[j, k] * ((x[k] - P[j, k]) ** 2)\n",
" y -= alpha_j * np.exp(-t)\n",
" return y"
]
},
{
"cell_type": "markdown",
"id": "0b1ae9df",
"metadata": {},
"source": [
"Next, our `objective` function takes a Tune `config`, evaluates the `landscape` of our experiment in a training loop,\n",
"and uses `tune.report` to report the `landscape` back to Tune."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8c3f252e",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" for i in range(config[\"iterations\"]):\n",
" x = np.array([config.get(\"x{}\".format(i + 1)) for i in range(6)])\n",
" tune.report(\n",
" timesteps_total=i, landscape=landscape(x), l2norm=np.sqrt((x ** 2).sum())\n",
" )\n",
" time.sleep(0.02)"
]
},
{
"cell_type": "markdown",
"id": "d9982d95",
"metadata": {},
"source": [
"Next we define a search space. The critical assumption is that the optimal hyperparamters live within this space. Yet, if the space is very large, then those hyperparamters may be difficult to find in a short amount of time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "30f75f5a",
"metadata": {},
"outputs": [],
"source": [
"search_space = {\n",
" \"iterations\":100,\n",
" \"x1\": tune.uniform(0.0, 1.0),\n",
" \"x2\": tune.uniform(0.0, 1.0),\n",
" \"x3\": tune.uniform(0.0, 1.0),\n",
" \"x4\": tune.uniform(0.0, 1.0),\n",
" \"x5\": tune.uniform(0.0, 1.0),\n",
" \"x6\": tune.uniform(0.0, 1.0)\n",
"}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "106d8578",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.init(configure_logging=False)"
]
},
{
"cell_type": "markdown",
"id": "932f74e6",
"metadata": {},
"source": [
"Now we define the search algorithm from `AxSearch`. If you want to constrain your parameters or even the space of outcomes, that can be easily done by passing the argumentsas below."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "34dd5c95",
"metadata": {},
"outputs": [],
"source": [
"algo = AxSearch(\n",
" parameter_constraints=[\"x1 + x2 <= 2.0\"],\n",
" outcome_constraints=[\"l2norm <= 1.25\"],\n",
")"
]
},
{
"cell_type": "markdown",
"id": "f6d18a99",
"metadata": {},
"source": [
"We also use `ConcurrencyLimiter` to constrain to 4 concurrent trials. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dcd905ef",
"metadata": {},
"outputs": [],
"source": [
"algo = tune.suggest.ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "10fd5427",
"metadata": {},
"source": [
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"You can decrease this if it takes too long on your machine, or you can set a time limit easily through `stop` argument in `tune.run()` as we will show here."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c53349a5",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 100\n",
"stop_timesteps = 200"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6c661045",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# Reducing samples for smoke tests\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "91076c5a",
"metadata": {},
"source": [
"Finally, we run the experiment to find the global minimum of the provided landscape (which contains 5 false minima). The argument to metric, `\"landscape\"`, is provided via the `objective` function's `tune.report`. The experiment `\"min\"`imizes the \"mean_loss\" of the `landscape` by searching within `search_space` via `algo`, `num_samples` times or when `\"timesteps_total\": stop_timesteps`. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, notice how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2f519d63",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" name=\"ax\",\n",
" metric=\"landscape\",\n",
" mode=\"min\",\n",
" search_alg=algo,\n",
" num_samples=num_samples,\n",
" config=search_space,\n",
" stop={\"timesteps_total\": stop_timesteps}\n",
")"
]
},
{
"cell_type": "markdown",
"id": "860b53b0",
"metadata": {},
"source": [
"And now we have the hyperparameters found to minimize the mean loss."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "12906421",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "68872424",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.shutdown()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"orphan": true
},
"nbformat": 4,
"nbformat_minor": 5
}

264
doc/source/tune/examples/bayesopt_example.ipynb Normal file
View file

@ -0,0 +1,264 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "db54cdf9",
"metadata": {},
"source": [
"# Running Tune experiments with BayesOpt\n",
"\n",
"In this tutorial we introduce BayesOpt, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with BayesOpt and, as a result, allow you to seamlessly scale up a BayesOpt optimization process - without sacrificing performance.\n",
"\n",
"BayesOpt is a constrained global optimization package utilizing Bayesian inference on gaussian processes, where the emphasis is on finding the maximum value of an unknown function in as few iterations as possible. BayesOpt's techniques are particularly suited for optimization of high cost functions, situations where the balance between exploration and exploitation is important. Therefore BayesOpt falls in the domain of \"derivative-free\" and \"black-box\" optimization. In this example we minimize a simple objective to briefly demonstrate the usage of BayesOpt with Ray Tune via `BayesOptSearch`, including conditional search spaces. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `bayesian-optimization==1.2.0` library is installed. To learn more, please refer to [BayesOpt website](https://github.com/fmfn/BayesianOptimization)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7ed16354",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# !pip install ray[tune]\n",
"!pip install bayesian-optimization==1.2.0"
]
},
{
"cell_type": "markdown",
"id": "2236f834",
"metadata": {},
"source": [
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6d36c78b",
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"import time\n",
"\n",
"import ray\n",
"from ray import tune\n",
"from ray.tune.suggest import ConcurrencyLimiter\n",
"from ray.tune.suggest.bayesopt import BayesOptSearch"
]
},
{
"cell_type": "markdown",
"id": "6257a3a8",
"metadata": {},
"source": [
"Let's start by defining a simple evaluation function.\n",
"We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
"This setup assumes that we're running multiple `step`s of an experiment and try to tune two hyperparameters,\n",
"namely `width` and `height`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "646c75a9",
"metadata": {},
"outputs": [],
"source": [
"def evaluate(step, width, height):\n",
" time.sleep(0.1)\n",
" return (0.1 + width * step / 100) ** (-1) + height * 0.1"
]
},
{
"cell_type": "markdown",
"id": "d89b7fdc",
"metadata": {},
"source": [
"Next, our ``objective`` function takes a Tune ``config``, evaluates the `score` of your experiment in a training loop,\n",
"and uses `tune.report` to report the `score` back to Tune."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e9adf637",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" for step in range(config[\"steps\"]):\n",
" score = evaluate(step, config[\"width\"], config[\"height\"])\n",
" tune.report(iterations=step, mean_loss=score)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bc634b1d",
"metadata": {
"lines_to_next_cell": 0,
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.init(configure_logging=False)"
]
},
{
"cell_type": "markdown",
"id": "0b9a2c4d",
"metadata": {},
"source": [
"Now we define the search algorithm built from `BayesOptSearch`, constrained to a maximum of `4` concurrent trials with a `ConcurrencyLimiter`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6f1d2fe7",
"metadata": {},
"outputs": [],
"source": [
"algo = BayesOptSearch(utility_kwargs={\"kind\": \"ucb\", \"kappa\": 2.5, \"xi\": 0.0})\n",
"algo = ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "27963e39",
"metadata": {},
"source": [
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d777201c",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 1000"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bb5f39a6",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# If 1000 samples take too long, you can reduce this number.\n",
"# We override this number here for our smoke tests.\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "752523c8",
"metadata": {},
"source": [
"Next we define a search space. The critical assumption is that the optimal hyperparameters live within this space. Yet, if the space is very large, then those hyperparameters may be difficult to find in a short amount of time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "116f8757",
"metadata": {},
"outputs": [],
"source": [
"search_space = {\n",
" \"steps\": 100,\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "1754bf85",
"metadata": {},
"source": [
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_config` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, notice how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5c44a0c5",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" name=\"bayesopt_exp\",\n",
" num_samples=num_samples,\n",
" config=search_space,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "477f099b",
"metadata": {},
"source": [
"Here are the hyperparamters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3488aefa",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2936353a",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.shutdown()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"orphan": true
},
"nbformat": 4,
"nbformat_minor": 5
}

357
doc/source/tune/examples/bohb_example.ipynb Normal file
View file

@ -0,0 +1,357 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "986bcaab",
"metadata": {},
"source": [
"# Running Tune experiments with BOHB\n",
"\n",
"In this tutorial we introduce BOHB, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with BOHB and, as a result, allow you to seamlessly scale up a BOHB optimization process - without sacrificing performance.\n",
"\n",
"Bayesian Optimization HyperBand (BOHB) combines the benefits of Bayesian optimization together with Bandit-based methods (e.g. HyperBand). BOHB does not rely on the gradient of the objective function, but instead, learns from samples of the search space. It is suitable for optimizing functions that are nondifferentiable, with many local minima, or even unknown but only testable. Therefore, this approach belongs to the domain of \"derivative-free optimization\" and \"black-box optimization\".\n",
"\n",
"In this example we minimize a simple objective to briefly demonstrate the usage of BOHB with Ray Tune via `BOHBSearch`. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `ConfigSpace==0.4.18` and `hpbandster==0.7.4` libraries are installed. To learn more, please refer to the [BOHB website](https://github.com/automl/HpBandSter)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d12bd979",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# !pip install ray[tune]\n",
"!pip install ConfigSpace==0.4.18\n",
"!pip install hpbandster==0.7.4"
]
},
{
"cell_type": "markdown",
"id": "96641e94",
"metadata": {},
"source": [
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0e65ccdb",
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"import time\n",
"\n",
"import ray\n",
"from ray import tune\n",
"from ray.tune.suggest import ConcurrencyLimiter\n",
"from ray.tune.schedulers.hb_bohb import HyperBandForBOHB\n",
"from ray.tune.suggest.bohb import TuneBOHB\n",
"import ConfigSpace as CS"
]
},
{
"cell_type": "markdown",
"id": "edba942a",
"metadata": {},
"source": [
"Let's start by defining a simple evaluation function.\n",
"We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
"This setup assumes that we're running multiple `step`s of an experiment and try to tune two hyperparameters,\n",
"namely `width` and `height`, and `activation`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "af512205",
"metadata": {},
"outputs": [],
"source": [
"def evaluate(step, width, height, activation):\n",
" time.sleep(0.1)\n",
" activation_boost = 10 if activation==\"relu\" else 1\n",
" return (0.1 + width * step / 100) ** (-1) + height * 0.1 + activation_boost"
]
},
{
"cell_type": "markdown",
"id": "c073ea21",
"metadata": {},
"source": [
"Next, our `objective` function takes a Tune `config`, evaluates the `score` of your experiment in a training loop,\n",
"and uses `tune.report` to report the `score` back to Tune."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8a086e87",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" for step in range(config[\"steps\"]):\n",
" score = evaluate(step, config[\"width\"], config[\"height\"], config[\"activation\"])\n",
" tune.report(iterations=step, mean_loss=score)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "05d07329",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.init(configure_logging=False)"
]
},
{
"cell_type": "markdown",
"id": "32ee1ba7",
"metadata": {},
"source": [
"Next we define a search space. The critical assumption is that the optimal hyperparameters live within this space. Yet, if the space is very large, then those hyperparameters may be difficult to find in a short amount of time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "21598e54",
"metadata": {},
"outputs": [],
"source": [
"search_space = {\n",
" \"steps\": 100,\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
" \"activation\": tune.choice([\"relu\", \"tanh\"]),\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "def82932",
"metadata": {},
"source": [
"Next we define the search algorithm built from `TuneBOHB`, constrained to a maximum of `4` concurrent trials with a `ConcurrencyLimiter`. Below `algo` will take care of the BO (Bayesian optimization) part of BOHB, while scheduler will take care the HB (HyperBand) part."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e847b5b6",
"metadata": {},
"outputs": [],
"source": [
"algo = TuneBOHB()\n",
"algo = tune.suggest.ConcurrencyLimiter(algo, max_concurrent=4)\n",
"scheduler = HyperBandForBOHB(\n",
" time_attr=\"training_iteration\",\n",
" max_t=100,\n",
" reduction_factor=4,\n",
" stop_last_trials=False,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "1787a842",
"metadata": {},
"source": [
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "702eb3d4",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 1000"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dfb3ecad",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# If 1000 samples take too long, you can reduce this number.\n",
"# We override this number here for our smoke tests.\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "aa5936df",
"metadata": {},
"source": [
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching within `\"steps\": 100` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, notice how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4bdfb12d",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" scheduler=scheduler,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" name=\"bohb_exp\",\n",
" num_samples=num_samples,\n",
" config=search_space\n",
")"
]
},
{
"cell_type": "markdown",
"id": "3e89853c",
"metadata": {},
"source": [
"Here are the hyperparameters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4be691d5",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "markdown",
"id": "800a19d9",
"metadata": {},
"source": [
"## Optional: Passing the search space via the TuneBOHB algorithm\n",
"\n",
"We can define the hyperparameter search space using `ConfigSpace`, which is the format accepted by BOHB."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b96cb496",
"metadata": {},
"outputs": [],
"source": [
"config_space = CS.ConfigurationSpace()\n",
"config_space.add_hyperparameter(\n",
" CS.UniformFloatHyperparameter(\"width\", lower=0, upper=20)\n",
")\n",
"config_space.add_hyperparameter(\n",
" CS.UniformFloatHyperparameter(\"height\", lower=-100, upper=100)\n",
")\n",
"config_space.add_hyperparameter(\n",
" CS.CategoricalHyperparameter(\n",
" \"activation\", choices=[\"relu\", \"tanh\"]\n",
" )\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9cb77270",
"metadata": {},
"outputs": [],
"source": [
"algo = TuneBOHB(\n",
" space=config_space,\n",
" metric=\"episode_reward_mean\",\n",
" mode=\"max\",\n",
")\n",
"algo = tune.suggest.ConcurrencyLimiter(algo, max_concurrent=4)\n",
"scheduler = HyperBandForBOHB(\n",
" time_attr=\"training_iteration\",\n",
" max_t=100,\n",
" reduction_factor=4,\n",
" stop_last_trials=False,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "8305c975",
"metadata": {},
"source": [
"analysis = tune.run(\n",
" objective,\n",
" config=config_space,\n",
" scheduler=bohb_hyperband,\n",
" search_alg=algo,\n",
" num_samples=num_samples,\n",
" name=\"bohb_exp_2\",\n",
" stop={\"training_iteration\": 100},\n",
")\n",
"\n",
"Here again are the hyperparameters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1ae613e4",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6b83ef6d",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.shutdown()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"orphan": true
},
"nbformat": 4,
"nbformat_minor": 5
}

255
doc/source/tune/examples/dragonfly_example.ipynb Normal file
View file

@ -0,0 +1,255 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "0332db08",
"metadata": {},
"source": [
"# Running Tune experiments with Dragonfly\n",
"\n",
"In this tutorial we introduce Dragonfly, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with Dragonfly and, as a result, allow you to seamlessly scale up a Dragonfly optimization process - without sacrificing performance.\n",
"\n",
"Dragonfly is an open source python library for scalable Bayesian optimization. Bayesian optimization is used optimizing black-box functions whose evaluations are usually expensive. Beyond vanilla optimization techniques, Dragonfly provides an array of tools to scale up Bayesian optimisation to expensive large scale problems. These include features/functionality that are especially suited for high dimensional spaces (optimizing with a large number of variables), parallel evaluations in synchronous or asynchronous settings (conducting multiple evaluations in parallel), multi-fidelity optimization (using cheap approximations to speed up the optimization process), and multi-objective optimisation (optimizing multiple functions simultaneously).\n",
"\n",
"Bayesian optimization does not rely on the gradient of the objective function, but instead, learns from samples of the search space. It is suitable for optimizing functions that are nondifferentiable, with many local minima, or even unknown but only testable. Therefore, it belongs to the domain of \"derivative-free optimization\" and \"black-box optimization\". In this example we minimize a simple objective to briefly demonstrate the usage of Dragonfly with Ray Tune via `DragonflySearch`. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `dragonfly-opt==0.1.6` library is installed. To learn more, please refer to the [Dragonfly website](https://dragonfly-opt.readthedocs.io/)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9878e2bd",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# !pip install ray[tune]\n",
"!pip install dragonfly-opt==0.1.6"
]
},
{
"cell_type": "markdown",
"id": "3acad068",
"metadata": {},
"source": [
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4ea7aefa",
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"import numpy as np\n",
"import time\n",
"\n",
"import ray\n",
"from ray import tune\n",
"from ray.tune.suggest import ConcurrencyLimiter\n",
"from ray.tune.suggest.dragonfly import DragonflySearch"
]
},
{
"cell_type": "markdown",
"id": "dc67fc3f",
"metadata": {},
"source": [
"Let's start by defining a optimization problem. Suppose we want to figure out the proportions of water and several salts to add to an ionic solution with the goal of maximizing it's ability to conduct electricity. The objective here is explicit for demonstration, yet in practice they often come out of a black-box (e.g. a physical device measuring conductivity, or reporting the results of a long-running ML experiment). We artificially sleep for a bit (`0.02` seconds) to simulate a more typical experiment. This setup assumes that we're running multiple `step`s of an experiment and try to tune relative proportions of 4 ingredients-- these proportions should be considered as hyperparameters. Our `objective` function will take a Tune `config`, evaluates the `conductivity` of our experiment in a training loop,\n",
"and uses `tune.report` to report the `conductivity` back to Tune."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f0d72404",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" \"\"\"\n",
" Simplistic model of electrical conductivity with added Gaussian noise to simulate experimental noise.\n",
" \"\"\"\n",
" for i in range(config[\"iterations\"]):\n",
" vol1 = config[\"LiNO3_vol\"] # LiNO3\n",
" vol2 = config[\"Li2SO4_vol\"] # Li2SO4\n",
" vol3 = config[\"NaClO4_vol\"] # NaClO4\n",
" vol4 = 10 - (vol1 + vol2 + vol3) # Water\n",
" conductivity = vol1 + 0.1 * (vol2 + vol3) ** 2 + 2.3 * vol4 * (vol1 ** 1.5)\n",
" conductivity += np.random.normal() * 0.01\n",
" tune.report(timesteps_total=i, objective=conductivity)\n",
" time.sleep(0.02)"
]
},
{
"cell_type": "markdown",
"id": "1808e8e0",
"metadata": {},
"source": [
"Next we define a search space. The critical assumption is that the optimal hyperparameters live within this space. Yet, if the space is very large, then those hyperparameters may be difficult to find in a short amount of time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6b867a19",
"metadata": {},
"outputs": [],
"source": [
"search_space = {\n",
" \"iterations\": 100,\n",
" \"LiNO3_vol\": tune.uniform(0, 7),\n",
" \"Li2SO4_vol\": tune.uniform(0, 7),\n",
" \"NaClO4_vol\": tune.uniform(0, 7)\n",
"}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6fe16177",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.init(configure_logging=False)"
]
},
{
"cell_type": "markdown",
"id": "ed5fc098",
"metadata": {},
"source": [
"Now we define the search algorithm from `DragonflySearch` with `optimizer` and `domain` arguments specified in a common way. We also use `ConcurrencyLimiter` to constrain to 4 concurrent trials."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a8075e34",
"metadata": {},
"outputs": [],
"source": [
"algo = DragonflySearch(\n",
" optimizer=\"bandit\",\n",
" domain=\"euclidean\",\n",
")\n",
"algo = ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "82240170",
"metadata": {},
"source": [
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c437c1e1",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 100"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "70192183",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# Reducing samples for smoke tests\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "3a823fb1",
"metadata": {},
"source": [
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_config` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, notice how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "97c69cbe",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" metric=\"objective\",\n",
" mode=\"max\",\n",
" name=\"dragonfly_search\",\n",
" search_alg=algo,\n",
" num_samples=num_samples,\n",
" config=search_space\n",
")"
]
},
{
"cell_type": "markdown",
"id": "fb1a7563",
"metadata": {},
"source": [
"Below are the recommended relative proportions of water and each salt found to maximize conductivity in the ionic solution (according to the simple model):"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ccb3b44b",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found: \", analysis.best_config)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "42eca06e",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.shutdown()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"orphan": true
},
"nbformat": 4,
"nbformat_minor": 5
}

465
doc/source/tune/examples/flaml_example.ipynb Normal file
View file

@ -0,0 +1,465 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "d3d9d39e",
"metadata": {},
"source": [
"# Running Tune experiments with BlendSearch and CFO\n",
"\n",
"In this tutorial we introduce BlendSearch and CFO, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with FLAML and, as a result, allow you to seamlessly scale up a BlendSearch and CFO optimization process - without sacrificing performance.\n",
"\n",
"Fast Library for Automated Machine Learning & Tuning (FLAML) does not rely on the gradient of the objective function, but instead, learns from samples of the search space. It is suitable for optimizing functions that are nondifferentiable, with many local minima, or even unknown but only testable. Therefore, it is necessarily belongs to the domain of \"derivative-free optimization\" and \"black-box optimization\".\n",
"\n",
"FLAML has two primary algorithms: (1) Frugal Optimization for Cost-related Hyperparameters (CFO) begins with a low-cost initial point and gradually moves to a high cost region as needed. It is a local search method that leverages randomized direct search method with an adaptive step-size and random restarts. As a local search method, it has an appealing provable convergence rate and bounded cost but may get trapped in suboptimal local minina. (2) Economical Hyperparameter Optimization With Blended Search Strategy (BlendSearch) combines CFO's local search with global search, making it less suspectable to local minima traps. It leverages the frugality of CFO and the space exploration ability of global search methods such as Bayesian optimization.\n",
"\n",
"In this example we minimize a simple objective to briefly demonstrate the usage of FLAML with Ray Tune via `BlendSearch` and `CFO`. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `flaml==0.4.1` and `optuna==2.9.1` libraries are installed. To learn more, please refer to the [FLAML website](https://github.com/microsoft/FLAML/tree/main/flaml/tune).\n",
" \n",
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1a9f10f0",
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"import time\n",
"\n",
"import ray\n",
"from ray import tune\n",
"from ray.tune.suggest import ConcurrencyLimiter\n",
"from ray.tune.suggest.flaml import BlendSearch, CFO"
]
},
{
"cell_type": "markdown",
"id": "f5a4bce8",
"metadata": {},
"source": [
"Let's start by defining a simple evaluation function.\n",
"We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
"This setup assumes that we're running multiple `step`s of an experiment and try to tune two hyperparameters,\n",
"namely `width` and `height`, and `activation`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9b80ac32",
"metadata": {},
"outputs": [],
"source": [
"def evaluate(step, width, height, activation):\n",
" time.sleep(0.1)\n",
" activation_boost = 10 if activation==\"relu\" else 1\n",
" return (0.1 + width * step / 100) ** (-1) + height * 0.1 + activation_boost"
]
},
{
"cell_type": "markdown",
"id": "391dcd0e",
"metadata": {},
"source": [
"Next, our `objective` function takes a Tune `config`, evaluates the `score` of your experiment in a training loop,\n",
"and uses `tune.report` to report the `score` back to Tune."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0773e711",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" for step in range(config[\"steps\"]):\n",
" score = evaluate(step, config[\"width\"], config[\"height\"], config[\"activation\"])\n",
" tune.report(iterations=step, mean_loss=score)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8dde8596",
"metadata": {
"lines_to_next_cell": 0,
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.init(configure_logging=False)"
]
},
{
"cell_type": "markdown",
"id": "a1007f97",
"metadata": {},
"source": [
"## Running Tune experiments with BlendSearch\n",
"\n",
"This example demonstrates the usage of Economical Hyperparameter Optimization With Blended Search Strategy (BlendSearch) with Ray Tune.\n",
"\n",
"Now we define the search algorithm built from `BlendSearch`, constrained to a maximum of `4` concurrent trials with a `ConcurrencyLimiter`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "37b5070e",
"metadata": {},
"outputs": [],
"source": [
"algo = BlendSearch()\n",
"algo = ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "6bb24c73",
"metadata": {},
"source": [
"The number of samples this Tune run is set to `1000`.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f219c33a",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 1000"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7ad72495",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# If 1000 samples take too long, you can reduce this number.\n",
"# We override this number here for our smoke tests.\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "6a206833",
"metadata": {},
"source": [
"Next we define a search space. The critical assumption is that the optimal hyperparameters live within this space. Yet, if the space is very large, then those hyperparameters may be difficult to find in a short amount of time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f2236f81",
"metadata": {},
"outputs": [],
"source": [
"search_config = {\n",
" \"steps\": 100,\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
" \"activation\": tune.choice([\"relu, tanh\"])\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "f77303bc",
"metadata": {},
"source": [
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_config` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, observe how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7a95373a",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" name=\"blendsearch_exp\",\n",
" num_samples=num_samples,\n",
" config=search_config,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "77a49ffb",
"metadata": {},
"source": [
"Here are the hyperparamters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "59cdf197",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "markdown",
"id": "1dca41c4",
"metadata": {},
"source": [
"## Incorporating a time budget to the experiment\n",
"\n",
"Define the time budget in seconds:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "878bd08f",
"metadata": {},
"outputs": [],
"source": [
"time_budget_s = 30"
]
},
{
"cell_type": "markdown",
"id": "5c9d1bdc",
"metadata": {},
"source": [
"Similarly we define a search space, but this time we feed it as an argument to `BlendSearch` rather than `tune.run`'s `config` argument.\n",
"\n",
"We next define the time budget via `set_search_properties`. And once again include the `ConcurrencyLimiter`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5846724f",
"metadata": {},
"outputs": [],
"source": [
"algo = BlendSearch(\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" space={\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
" \"activation\": tune.choice([\"relu\", \"tanh\"]),\n",
" },\n",
")\n",
"algo.set_search_properties(config={\"time_budget_s\": time_budget_s})\n",
"algo = ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "13564767",
"metadata": {},
"source": [
"Now we run the experiment, this time with the `time_budget` included as an argument. Note: We allow for virtually infinite `num_samples` by passing `-1`, so that the experiment is stopped according to the time budget rather than a sample limit."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b878b2be",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" time_budget_s=time_budget_s,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" name=\"blendsearch_exp\",\n",
" num_samples=-1,\n",
" config={\"steps\": 100},\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "48f06c5f",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "markdown",
"id": "47b906b7",
"metadata": {},
"source": [
"## Running Tune experiments with CFO\n",
"\n",
"This example demonstrates the usage of Frugal Optimization for Cost-related Hyperparameters (CFO) with Ray Tune.\n",
"\n",
"We now define the search algorithm as built from `CFO`, constrained to a maximum of `4` concurrent trials with a `ConcurrencyLimiter`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "29f2c0be",
"metadata": {},
"outputs": [],
"source": [
"algo = CFO()\n",
"algo = ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "a89cf9bb",
"metadata": {},
"source": [
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3d15f10d",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 1000"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "137f3ec0",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# If 1000 samples take too long, you can reduce this number.\n",
"# We override this number here for our smoke tests.\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "cfa2e413",
"metadata": {},
"source": [
"Next we define a search space. The critical assumption is that the optimal hyperparameters live within this space. Yet, if the space is very large, then those hyperparameters may be difficult to find in a short amount of time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ab4fbcbe",
"metadata": {},
"outputs": [],
"source": [
"search_config = {\n",
" \"steps\": 100,\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
" \"activation\": tune.choice([\"relu, tanh\"])\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "b19390a2",
"metadata": {},
"source": [
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_config` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, notice how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2acf19f5",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" name=\"cfo_exp\",\n",
" num_samples=num_samples,\n",
" config=search_config,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "71e6f6ec",
"metadata": {},
"source": [
"Here are the hyperparameters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b657dccf",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "505995af",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.shutdown()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"orphan": true
},
"nbformat": 4,
"nbformat_minor": 5
}

285
doc/source/tune/examples/hebo_example.ipynb Normal file
View file

@ -0,0 +1,285 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "83323340",
"metadata": {},
"source": [
"# Running Tune experiments with HEBOSearch\n",
"\n",
"In this tutorial we introduce HEBO, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with ZOOpt and, as a result, allow you to seamlessly scale up a HEBO optimization process - without sacrificing performance.\n",
"\n",
"Heteroscadastic Evolutionary Bayesian Optimization (HEBO) does not rely on the gradient of the objective function, but instead, learns from samples of the search space. It is suitable for optimizing functions that are nondifferentiable, with many local minima, or even unknown but only testable. This necessarily makes the algorithm belong to the domain of \"derivative-free optimization\" and \"black-box optimization\".\n",
"\n",
"In this example we minimize a simple objective to briefly demonstrate the usage of HEBO with Ray Tune via `HEBOSearch`. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `zoopt==0.4.1` library is installed. To learn more, please refer to the [HEBO website](https://github.com/huawei-noah/HEBO/tree/master/HEBO)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b8de7864",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# !pip install ray[tune]\n",
"!pip install HEBO==0.3.2"
]
},
{
"cell_type": "markdown",
"id": "37141309",
"metadata": {},
"source": [
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9dae789a",
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"import time\n",
"\n",
"import ray\n",
"from ray import tune\n",
"from ray.tune.suggest.hebo import HEBOSearch"
]
},
{
"cell_type": "markdown",
"id": "f5747767",
"metadata": {},
"source": [
"Let's start by defining a simple evaluation function.\n",
"We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
"This setup assumes that we're running multiple `step`s of an experiment and try to tune two hyperparameters,\n",
"namely `width` and `height`, and `activation`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d654948a",
"metadata": {},
"outputs": [],
"source": [
"def evaluate(step, width, height, activation):\n",
" time.sleep(0.1)\n",
" activation_boost = 10 if activation==\"relu\" else 1\n",
" return (0.1 + width * step / 100) ** (-1) + height * 0.1 + activation_boost"
]
},
{
"cell_type": "markdown",
"id": "e61acd5f",
"metadata": {},
"source": [
"Next, our ``objective`` function takes a Tune ``config``, evaluates the `score` of your experiment in a training loop,\n",
"and uses `tune.report` to report the `score` back to Tune."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b7e5f38b",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" for step in range(config[\"steps\"]):\n",
" score = evaluate(step, config[\"width\"], config[\"height\"], config[\"activation\"])\n",
" tune.report(iterations=step, mean_loss=score)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "49966a2e",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.init(configure_logging=False)"
]
},
{
"cell_type": "markdown",
"id": "79567a1b",
"metadata": {},
"source": [
"While defining the search algorithm, we may choose to provide an initial set of hyperparameters that we believe are especially promising or informative, and\n",
"pass this information as a helpful starting point for the `HyperOptSearch` object.\n",
"\n",
"We also set the maximum concurrent trials to `8`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9aaf222e",
"metadata": {},
"outputs": [],
"source": [
"\n",
"previously_run_params = [\n",
" {\"width\": 10, \"height\": 0, \"activation\": \"relu\"},\n",
" {\"width\": 15, \"height\": -20, \"activation\": \"tanh\"},\n",
"]\n",
"\n",
"known_rewards = [-189, -1144]\n",
"\n",
"max_concurrent = 8\n",
"\n",
"algo = HEBOSearch(\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" points_to_evaluate=previously_run_params,\n",
" evaluated_rewards=known_rewards,\n",
" random_state_seed=123,\n",
" max_concurrent=max_concurrent,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "19942c67",
"metadata": {},
"source": [
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ea2a405c",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 1000"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cdc1b707",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# If 1000 samples take too long, you can reduce this number.\n",
"# We override this number here for our smoke tests.\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "40fe2a91",
"metadata": {},
"source": [
"Next we define a search space. The critical assumption is that the optimal hyperparamters live within this space. Yet, if the space is very large, then those hyperparameters may be difficult to find in a short amount of time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0a530e21",
"metadata": {},
"outputs": [],
"source": [
"search_config = {\n",
" \"steps\": 100,\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
" \"activation\": tune.choice([\"relu, tanh\"])\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "3e38301f",
"metadata": {},
"source": [
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_config` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, notice how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3fb63ef3",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" name=\"hebo_exp_with_warmstart\",\n",
" search_alg=algo,\n",
" num_samples=num_samples,\n",
" config=search_config\n",
")"
]
},
{
"cell_type": "markdown",
"id": "963e547b",
"metadata": {},
"source": [
"Here are the hyperparamters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8b8a23e4",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d0882e22",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.shutdown()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"orphan": true
},
"nbformat": 4,
"nbformat_minor": 5
}

297
doc/source/tune/examples/hyperopt_example.ipynb
View file

@ -2,13 +2,53 @@
"cells": [
{
"cell_type": "markdown",
"id": "35712b90",
"id": "58fc50bc",
"metadata": {},
"source": [
"# Running Tune experiments with HyperOpt\n",
"\n",
"This example demonstrates the usage of HyperOpt with Ray Tune, using a `AsyncHyperBandScheduler` scheduler\n",
"together with `HyperOptSearch`.\n",
"In this tutorial we introduce HyperOpt, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with HyperOpt and, as a result, allow you to seamlessly scale up a Hyperopt optimization process - without sacrificing performance.\n",
"\n",
"HyperOpt provides gradient/derivative-free optimization able to handle noise over the objective landscape, including evolutionary, bandit, and Bayesian optimization algorithms. Nevergrad internally supports search spaces which are continuous, discrete or a mixture of thereof. It also provides a library of functions on which to test the optimization algorithms and compare with other benchmarks.\n",
"\n",
"In this example we minimize a simple objective to briefly demonstrate the usage of HyperOpt with Ray Tune via `HyperOptSearch`. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `hyperopt==0.2.5` library is installed. To learn more, please refer to [HyperOpt website](http://hyperopt.github.io/hyperopt).\n",
"\n",
"We include a important example on conditional search spaces (stringing together relationships among hyperparameters)."
]
},
{
"cell_type": "markdown",
"id": "e4586d28",
"metadata": {},
"source": [
"Background information:\n",
"- [HyperOpt website](http://hyperopt.github.io/hyperopt)\n",
"\n",
"Necessary requirements:\n",
"- `pip install ray[tune]`\n",
"- `pip install hyperopt==0.2.5`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6567f2dc",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# !pip install ray[tune]\n",
"!pip install hyperopt==0.2.5"
]
},
{
"cell_type": "markdown",
"id": "b8e9e0cd",
"metadata": {},
"source": [
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
@ -17,7 +57,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "42ed664d",
"id": "6592315e",
"metadata": {
"tags": [
"hide-input"
@ -30,36 +70,36 @@
"import ray\n",
"from ray import tune\n",
"from ray.tune.suggest import ConcurrencyLimiter\n",
"from ray.tune.schedulers import AsyncHyperBandScheduler\n",
"from ray.tune.suggest.hyperopt import HyperOptSearch"
"from ray.tune.suggest.hyperopt import HyperOptSearch\n",
"from hyperopt import hp"
]
},
{
"cell_type": "markdown",
"id": "7e4932f5",
"id": "d4b6d1d5",
"metadata": {},
"source": [
"Let's start by defining a simple evaluation function.\n",
"We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
"This setup assumes that we're running multiple `step`s of an experiment and try to tune two hyperparameters,\n",
"namely `x` and `y`."
"namely `width` and `height`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0854c730",
"id": "12d4efc8",
"metadata": {},
"outputs": [],
"source": [
"def evaluate(step, x, y):\n",
"def evaluate(step, width, height):\n",
" time.sleep(0.1)\n",
" return (0.1 + x * step / 100) ** (-1) + y * 0.1"
" return (0.1 + width * step / 100) ** (-1) + height * 0.1"
]
},
{
"cell_type": "markdown",
"id": "f3915b1b",
"id": "4f4f5aa2",
"metadata": {},
"source": [
"Next, our ``objective`` function takes a Tune ``config``, evaluates the `score` of your experiment in a training loop,\n",
@ -69,20 +109,20 @@
{
"cell_type": "code",
"execution_count": null,
"id": "9d7df36a",
"id": "c9818009",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" for step in range(config[\"steps\"]):\n",
" score = evaluate(step, config[\"x\"], config[\"y\"])\n",
" score = evaluate(step, config[\"width\"], config[\"height\"])\n",
" tune.report(iterations=step, mean_loss=score)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5354d2a1",
"id": "33eddcb9",
"metadata": {
"tags": [
"remove-cell"
@ -95,38 +135,55 @@
},
{
"cell_type": "markdown",
"id": "e761e836",
"id": "5be35d5e",
"metadata": {},
"source": [
"Let's say we have a hypothesis on what the best parameters currently are (`current_best_params`), then we can\n",
"pass this belief into a `HyperOptSearch` searcher and set the maximum concurrent trials to `4` with a `ConcurrencyLimiter`.\n",
"We can also define a `scheduler` to go along with our algorithm and set the number of samples for this Tune run to `1000`\n",
"(your can decrease this if it takes too long on your machine)."
"While defining the search algorithm, we may choose to provide an initial set of hyperparameters that we believe are especially promising or informative, and\n",
"pass this information as a helpful starting point for the `HyperOptSearch` object.\n",
"\n",
"We also set the maximum concurrent trials to `4` with a `ConcurrencyLimiter`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4ea0667d",
"id": "d4615bed",
"metadata": {
"lines_to_next_cell": 0
},
"outputs": [],
"source": [
"initial_params = [\n",
" {\"width\": 1, \"height\": 2},\n",
" {\"width\": 4, \"height\": 2},\n",
"]\n",
"algo = HyperOptSearch(points_to_evaluate=initial_params)\n",
"algo = ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "2a51e7c1",
"metadata": {},
"source": [
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2dbb2be0",
"metadata": {},
"outputs": [],
"source": [
"current_best_params = [\n",
" {\"x\": 1, \"y\": 2},\n",
" {\"x\": 4, \"y\": 2},\n",
"]\n",
"searcher = HyperOptSearch(points_to_evaluate=current_best_params)\n",
"\n",
"algo = ConcurrencyLimiter(searcher, max_concurrent=4)\n",
"scheduler = AsyncHyperBandScheduler()\n",
"\n",
"num_samples = 1000"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8fbd31df",
"id": "950558ed",
"metadata": {
"tags": [
"remove-cell"
@ -141,42 +198,196 @@
},
{
"cell_type": "markdown",
"id": "5edc6d9b",
"id": "6e3629cb",
"metadata": {},
"source": [
"Finally, all that's left is to define a search space, run the experiment and print the best parameters:"
"Next we define a search space. The critical assumption is that the optimal hyperparamters live within this space. Yet, if the space is very large, then those hyperparameters may be difficult to find in a short amount of time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0da436cb",
"id": "65189946",
"metadata": {},
"outputs": [],
"source": [
"search_space = {\n",
"search_config = {\n",
" \"steps\": 100,\n",
" \"x\": tune.uniform(0, 20),\n",
" \"y\": tune.uniform(-100, 100),\n",
"}\n",
"\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
" \"activation\": tune.choice([\"relu, tanh\"])\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "1b94c93b",
"metadata": {},
"source": [
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_config` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, notice how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9a99a3a7",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" scheduler=scheduler,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" name=\"hyperopt_exp\",\n",
" num_samples=num_samples,\n",
" config=search_space,\n",
")\n",
")"
]
},
{
"cell_type": "markdown",
"id": "49be6f01",
"metadata": {},
"source": [
"Here are the hyperparamters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7036798c",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "markdown",
"id": "504e9d2a",
"metadata": {},
"source": [
"## Conditional search spaces\n",
"\n",
"Sometimes we may want to build a more complicated search space that has conditional dependencies on other hyperparameters. In this case, we pass a nested dictionary to `objective_two`, which has been slightly adjusted from `objective` to deal with the conditional search space."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2f7b5449",
"metadata": {},
"outputs": [],
"source": [
"def evaluation_fn(step, width, height, mult=1):\n",
" return (0.1 + width * step / 100) ** (-1) + height * 0.1 * mult"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4b83b81c",
"metadata": {},
"outputs": [],
"source": [
"def objective_two(config):\n",
" width, height = config[\"width\"], config[\"height\"]\n",
" sub_dict = config[\"activation\"]\n",
" mult = sub_dict.get(\"mult\", 1)\n",
" \n",
" for step in range(config[\"steps\"]):\n",
" intermediate_score = evaluation_fn(step, width, height, mult)\n",
" tune.report(iterations=step, mean_loss=intermediate_score)\n",
" time.sleep(0.1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "75cea99e",
"metadata": {},
"outputs": [],
"source": [
"conditional_space = {\n",
" \"activation\": hp.choice(\n",
" \"activation\",\n",
" [\n",
" {\"activation\": \"relu\", \"mult\": hp.uniform(\"mult\", 1, 2)},\n",
" {\"activation\": \"tanh\"},\n",
" ],\n",
" ),\n",
" \"width\": hp.uniform(\"width\", 0, 20),\n",
" \"height\": hp.uniform(\"height\", -100, 100),\n",
" \"steps\": 100,\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "7df282c1",
"metadata": {},
"source": [
"Now we the define the search algorithm built from `HyperOptSearch` constrained by `ConcurrencyLimiter`. When the hyperparameter search space is conditional, we pass it (`conditional_space`) into `HyperOptSearch`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ea2c71a6",
"metadata": {},
"outputs": [],
"source": [
"algo = HyperOptSearch(space=conditional_space, metric=\"mean_loss\", mode=\"min\")\n",
"algo = ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "630f84ab",
"metadata": {},
"source": [
"Now we run the experiment, this time with an empty `config` because we instead provided `space` to the `HyperOptSearch` `search_alg`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "14111e9e",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective_two,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" search_alg=algo,\n",
" num_samples=num_samples\n",
")"
]
},
{
"cell_type": "markdown",
"id": "e6172afa",
"metadata": {},
"source": [
"Finally, we again show the hyperparameters that minimize the mean loss defined by the score of the objective function above. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "03c3fc49",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "81945658",
"id": "2f7b72d3",
"metadata": {
"tags": [
"remove-cell"
@ -190,7 +401,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},

159
doc/source/tune/examples/index.rst
View file

@ -151,40 +151,133 @@ At the end of these guides you will often find links to even more examples.
Search Algorithm Examples
-------------------------
.. TODO: make these panels with logos!
.. panels::
:container: container pb-4
:column: col-md-4 px-2 py-2
:img-top-cls: pt-5 w-75 d-block mx-auto
- :doc:`/tune/examples/includes/ax_example`:
Example script showing usage of :ref:`AxSearch <tune-ax>` [`Ax website <https://ax.dev/>`__]
- :doc:`/tune/examples/includes/dragonfly_example`:
Example script showing usage of :ref:`DragonflySearch <Dragonfly>` [`Dragonfly website <https://dragonfly-opt.readthedocs.io/>`__]
- :doc:`/tune/examples/includes/skopt_example`:
Example script showing usage of :ref:`SkoptSearch <skopt>` [`Scikit-Optimize website <https://scikit-optimize.github.io>`__]
- :doc:`/tune/examples/hyperopt_example`:
Example script showing usage of :ref:`HyperOptSearch <tune-hyperopt>` [`HyperOpt website <http://hyperopt.github.io/hyperopt>`__]
- :doc:`/tune/examples/includes/hyperopt_conditional_search_space_example`:
Example script showing usage of :ref:`HyperOptSearch <tune-hyperopt>` [`HyperOpt website <http://hyperopt.github.io/hyperopt>`__] with a conditional search space
- :doc:`/tune/examples/includes/bayesopt_example`:
Example script showing usage of :ref:`BayesOptSearch <bayesopt>` [`BayesianOptimization website <https://github.com/fmfn/BayesianOptimization>`__]
- :doc:`/tune/examples/includes/blendsearch_example`:
Example script showing usage of :ref:`BlendSearch <BlendSearch>` [`BlendSearch website <https://github.com/microsoft/FLAML/tree/main/flaml/tune>`__]
- :doc:`/tune/examples/includes/cfo_example`:
Example script showing usage of :ref:`CFO <CFO>` [`CFO website <https://github.com/microsoft/FLAML/tree/main/flaml/tune>`__]
- :doc:`/tune/examples/includes/bohb_example`:
Example script showing usage of :ref:`TuneBOHB <suggest-TuneBOHB>` [`BOHB website <https://github.com/automl/HpBandSter>`__]
- :doc:`/tune/examples/includes/nevergrad_example`:
Example script showing usage of :ref:`NevergradSearch <nevergrad>` [`Nevergrad website <https://github.com/facebookresearch/nevergrad>`__]
- :doc:`/tune/examples/optuna_example`:
Example script showing usage of :ref:`OptunaSearch <tune-optuna>` [`Optuna website <https://optuna.org/>`__]
- :doc:`/tune/examples/includes/zoopt_example`:
Example script showing usage of :ref:`ZOOptSearch <zoopt>` [`ZOOpt website <https://github.com/polixir/ZOOpt>`__]
- :doc:`/tune/examples/includes/sigopt_example`:
Example script showing usage of :ref:`SigOptSearch <sigopt>` [`SigOpt website <https://sigopt.com/>`__]
- :doc:`/tune/examples/includes/hebo_example`:
Example script showing usage of :ref:`HEBOSearch <tune-hebo>` [`HEBO website <https://github.com/huawei-noah/HEBO/tree/master/HEBO>`__]
- :doc:`/tune/examples/includes/sigopt_multi_objective_example`:
Example using Sigopt's multi-objective functionality (contributed).
- :doc:`/tune/examples/includes/sigopt_prior_beliefs_example`:
Example using Sigopt's support for prior beliefs (contributed).
---
:img-top: ../images/ax.png
+++
.. link-button:: ax_example
:type: ref
:text: How To Use Tune With Ax
:classes: btn-link btn-block stretched-link
---
:img-top: ../images/dragonfly.png
+++
.. link-button:: dragonfly_example
:type: ref
:text: How To Use Tune With Dragonfly
:classes: btn-link btn-block stretched-link
---
:img-top: ../images/skopt.png
+++
.. link-button:: skopt_example
:type: ref
:text: How To Use Tune With Scikit-Optimize
:classes: btn-link btn-block stretched-link
---
:img-top: ../images/hyperopt.png
+++
.. link-button:: hyperopt_example
:type: ref
:text: How To Use Tune With HyperOpt
:classes: btn-link btn-block stretched-link
---
:img-top: ../images/bayesopt.png
+++
.. link-button:: bayesopt_example
:type: ref
:text: How To Use Tune With BayesOpt
:classes: btn-link btn-block stretched-link
---
:img-top: ../images/flaml.png
+++
.. link-button:: flaml_example
:type: ref
:text: How To Use Tune With BlendSearch and CFO
:classes: btn-link btn-block stretched-link
---
:img-top: ../images/bohb.png
+++
.. link-button:: bohb_example
:type: ref
:text: How To Use Tune With TuneBOHB
:classes: btn-link btn-block stretched-link
---
:img-top: ../images/nevergrad.png
+++
.. link-button:: nevergrad_example
:type: ref
:text: How To Use Tune With Nevergrad
:classes: btn-link btn-block stretched-link
---
:img-top: ../images/optuna.png
+++
.. link-button:: optuna_example
:type: ref
:text: How To Use Tune With Optuna
:classes: btn-link btn-block stretched-link
---
:img-top: ../images/zoopt.png
+++
.. link-button:: zoopt_example
:type: ref
:text: How To Use Tune With ZOOpt
:classes: btn-link btn-block stretched-link
---
:img-top: ../images/sigopt.png
+++
.. link-button:: sigopt_example
:type: ref
:text: How To Use Tune With SigOpt
:classes: btn-link btn-block stretched-link
---
:img-top: ../images/hebo.png
+++
.. link-button:: hebo_example
:type: ref
:text: How To Use Tune With HEBO
:classes: btn-link btn-block stretched-link
API docs:
- :ref:`AxSearch <tune-ax>` [`Ax website <https://ax.dev/>`__]
- :ref:`DragonflySearch <Dragonfly>` [`Dragonfly website <https://dragonfly-opt.readthedocs.io/>`__]
- :ref:`SkoptSearch <skopt>` [`Scikit-Optimize website <https://scikit-optimize.github.io>`__]
- :ref:`HyperOptSearch <tune-hyperopt>` [`HyperOpt website <http://hyperopt.github.io/hyperopt>`__] including a conditional search space.
- :ref:`BayesOptSearch <bayesopt>` [`BayesianOptimization website <https://github.com/fmfn/BayesianOptimization>`__]
- :ref:`BlendSearch <BlendSearch>` [`BlendSearch website <https://github.com/microsoft/FLAML/tree/main/flaml/tune>`__] and :ref:`CFO <CFO>` [`CFO website <https://github.com/microsoft/FLAML/tree/main/flaml/tune>`__]
- :ref:`TuneBOHB <suggest-TuneBOHB>` [`BOHB website <https://github.com/automl/HpBandSter>`__]
- :ref:`NevergradSearch <nevergrad>` [`Nevergrad website <https://github.com/facebookresearch/nevergrad>`__]
- :ref:`OptunaSearch <tune-optuna>` [`Optuna website <https://optuna.org/>`__]
- :ref:`ZOOptSearch <zoopt>` [`ZOOpt website <https://github.com/polixir/ZOOpt>`__]
- :ref:`SigOptSearch <sigopt>` [`SigOpt website <https://sigopt.com/>`__] including support for prior beliefs and the multi-objective case.
- :ref:`HEBOSearch <tune-hebo>` [`HEBO website <https://github.com/huawei-noah/HEBO/tree/master/HEBO>`__]
.. _tune-general-examples:

347
doc/source/tune/examples/nevergrad_example.ipynb Normal file
View file

@ -0,0 +1,347 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "5a1d28f3",
"metadata": {},
"source": [
"# Running Tune experiments with Nevergrad\n",
"\n",
"In this tutorial we introduce Nevergrad, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with Nevergrad and, as a result, allow you to seamlessly scale up a Nevergrad optimization process - without sacrificing performance.\n",
"\n",
"Nevergrad provides gradient/derivative-free optimization able to handle noise over the objective landscape, including evolutionary, bandit, and Bayesian optimization algorithms. Nevergrad internally supports search spaces which are continuous, discrete or a mixture of thereof. It also provides a library of functions on which to test the optimization algorithms and compare with other benchmarks.\n",
"\n",
"In this example we minimize a simple objective to briefly demonstrate the usage of Nevergrad with Ray Tune via `NevergradSearch`. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `nevergrad==0.4.3.post7` library is installed. To learn more, please refer to [Nevergrad website](https://github.com/facebookresearch/nevergrad)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5ab54f85",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# !pip install ray[tune]\n",
"!pip install nevergrad==0.4.3.post7 "
]
},
{
"cell_type": "markdown",
"id": "66cb8206",
"metadata": {},
"source": [
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1f6d7a31",
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"import time\n",
"\n",
"import ray\n",
"import nevergrad as ng\n",
"from ray import tune\n",
"from ray.tune.suggest import ConcurrencyLimiter\n",
"from ray.tune.suggest.nevergrad import NevergradSearch"
]
},
{
"cell_type": "markdown",
"id": "41f2c881",
"metadata": {},
"source": [
"Let's start by defining a simple evaluation function.\n",
"We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
"This setup assumes that we're running multiple `step`s of an experiment and try to tune two hyperparameters,\n",
"namely `width` and `height`, and `activation`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "271bd5c5",
"metadata": {},
"outputs": [],
"source": [
"def evaluate(step, width, height, activation):\n",
" time.sleep(0.1)\n",
" activation_boost = 10 if activation==\"relu\" else 1\n",
" return (0.1 + width * step / 100) ** (-1) + height * 0.1 + activation_boost"
]
},
{
"cell_type": "markdown",
"id": "f060ea83",
"metadata": {},
"source": [
"Next, our `objective` function takes a Tune `config`, evaluates the `score` of your experiment in a training loop,\n",
"and uses `tune.report` to report the `score` back to Tune."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c71fc423",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" for step in range(config[\"steps\"]):\n",
" score = evaluate(step, config[\"width\"], config[\"height\"], config[\"activation\"])\n",
" tune.report(iterations=step, mean_loss=score)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "619263ee",
"metadata": {
"lines_to_next_cell": 0,
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.init(configure_logging=False)"
]
},
{
"cell_type": "markdown",
"id": "5b7a4b94",
"metadata": {},
"source": [
"Now we construct the hyperparameter search space using `ConfigSpace`"
]
},
{
"cell_type": "markdown",
"id": "e2405373",
"metadata": {},
"source": [
"Next we define the search algorithm built from `NevergradSearch`, constrained to a maximum of `4` concurrent trials with a `ConcurrencyLimiter`. Here we use `ng.optimizers.OnePlusOne`, a simple evolutionary algorithm."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f099b674",
"metadata": {},
"outputs": [],
"source": [
"algo = NevergradSearch(\n",
" optimizer=ng.optimizers.OnePlusOne,\n",
")\n",
"algo = tune.suggest.ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "4bddc4e5",
"metadata": {},
"source": [
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "adb807bc",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 1000"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "191c7f89",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# If 1000 samples take too long, you can reduce this number.\n",
"# We override this number here for our smoke tests.\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "a3956381",
"metadata": {},
"source": [
"Finally, all that's left is to define a search space."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8829ffc5",
"metadata": {},
"outputs": [],
"source": [
"search_config = {\n",
" \"steps\": 100,\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
" \"activation\": tune.choice([\"relu, tanh\"])\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "0b9d051f",
"metadata": {},
"source": [
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_space` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, observe how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "769f4368",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" name=\"nevergrad_exp\",\n",
" num_samples=num_samples,\n",
" config=search_config,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "950003be",
"metadata": {},
"source": [
"Here are the hyperparamters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0f021674",
"metadata": {
"lines_to_next_cell": 0
},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "markdown",
"id": "0d3824ae",
"metadata": {},
"source": [
"## Optional: passing the (hyper)parameter space into the search algorithm\n",
"\n",
"We can also pass the search space into `NevergradSearch` using their designed format."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "89ae7455",
"metadata": {},
"outputs": [],
"source": [
"space = ng.p.Dict(\n",
" width=ng.p.Scalar(lower=0, upper=20),\n",
" height=ng.p.Scalar(lower=-100, upper=100),\n",
" activation=ng.p.Choice(choices=[\"relu\", \"tanh\"])\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e52eeab1",
"metadata": {},
"outputs": [],
"source": [
"algo = NevergradSearch(\n",
" optimizer=ng.optimizers.OnePlusOne,\n",
" space=space,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\"\n",
")\n",
"algo = tune.suggest.ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "f8926177",
"metadata": {},
"source": [
"Again we run the experiment, this time with a less passed via the `config` and instead passed through `search_alg`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "64f39800",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
"# metric=\"mean_loss\",\n",
"# mode=\"min\",\n",
" name=\"nevergrad_exp\",\n",
" num_samples=num_samples,\n",
" config={\"steps\": 100},\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0478c1ea",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.shutdown()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"orphan": true
},
"nbformat": 4,
"nbformat_minor": 5
}

218
doc/source/tune/examples/optuna_example.ipynb
View file

@ -2,15 +2,40 @@
"cells": [
{
"cell_type": "markdown",
"id": "25705204",
"id": "a587ce4e",
"metadata": {},
"source": [
"# Running Tune experiments with Optuna\n",
"\n",
"This example demonstrates the usage of Optuna with Ray Tune via `OptunaSearch`, including conditional search spaces and the multi-objective use case.\n",
"In this tutorial we introduce Optuna, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with Optuna and, as a result, allow you to seamlessly scale up a Optuna optimization process - without sacrificing performance.\n",
"\n",
"While you may use a scheduler with `OptunaSearch`, e.g. `AsyncHyperBandScheduler`, please note that schedulers may not work correctly with multi-objective optimization, since they often expect a scalar score.\n",
"Similar to Ray Tune, Optuna is an automatic hyperparameter optimization software framework, particularly designed for machine learning. It features an imperative (\"how\" over \"what\" emphasis), define-by-run style user API. With Optuna, a user has the ability to dynamically construct the search spaces for the hyperparameters. Optuna falls in the domain of \"derivative-free optimization\" and \"black-box optimization\".\n",
"\n",
"In this example we minimize a simple objective to briefly demonstrate the usage of Optuna with Ray Tune via `OptunaSearch`, including examples of conditional search spaces (string together relationships between hyperparameters), and the multi-objective problem (measure trade-offs among all important metrics). It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `optuna==2.9.1` library is installed. To learn more, please refer to [Optuna website](https://optuna.org/).\n",
"\n",
"Please note that sophisticated schedulers, such as `AsyncHyperBandScheduler`, may not work correctly with multi-objective optimization, since they typically expect a scalar score to compare fitness among trials.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "aeaf9ff0",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# !pip install ray[tune]\n",
"!pip install optuna==2.9.1"
]
},
{
"cell_type": "markdown",
"id": "467466a3",
"metadata": {},
"source": [
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
@ -19,7 +44,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "4d2fbc3e",
"id": "fb1ad624",
"metadata": {
"tags": [
"hide-input"
@ -33,14 +58,13 @@
"import ray\n",
"from ray import tune\n",
"from ray.tune.suggest import ConcurrencyLimiter\n",
"from ray.tune.schedulers import AsyncHyperBandScheduler\n",
"from ray.tune.suggest.optuna import OptunaSearch"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "56404ca0",
"id": "e64f0b44",
"metadata": {
"tags": [
"remove-cell"
@ -53,7 +77,7 @@
},
{
"cell_type": "markdown",
"id": "ac0a86b6",
"id": "56b0c685",
"metadata": {},
"source": [
"Let's start by defining a simple evaluation function.\n",
@ -66,7 +90,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "3fa1a9a7",
"id": "90a11f98",
"metadata": {},
"outputs": [],
"source": [
@ -78,17 +102,17 @@
},
{
"cell_type": "markdown",
"id": "1f066da2",
"id": "bc579b83",
"metadata": {},
"source": [
"Next, our ``objective`` function to be optimized takes a Tune ``config``, evaluates the `score` of your experiment in a training loop,\n",
"Next, our `objective` function to be optimized takes a Tune `config`, evaluates the `score` of your experiment in a training loop,\n",
"and uses `tune.report` to report the `score` back to Tune."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dd6c61a1",
"id": "3a11d0e0",
"metadata": {
"lines_to_next_cell": 0
},
@ -103,7 +127,7 @@
},
{
"cell_type": "markdown",
"id": "2fcf8aef",
"id": "c58bd20b",
"metadata": {},
"source": [
"Next we define a search space. The critical assumption is that the optimal hyperparamters live within this space. Yet, if the space is very large, then those hyperparamters may be difficult to find in a short amount of time.\n",
@ -114,7 +138,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "8ba9fbfb",
"id": "c3e4eecb",
"metadata": {},
"outputs": [],
"source": [
@ -128,28 +152,25 @@
},
{
"cell_type": "markdown",
"id": "93afec28",
"id": "ef0c666d",
"metadata": {},
"source": [
"While defining the search algorithm, we may choose to provide an initial set of hyperparameters that we believe are especially promising or informative, and\n",
"pass this information as a helpful starting point for the `OptunaSearch` object.\n",
"\n",
"Here we define the Optuna search algorithm:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e8ded912",
"id": "f23cadc8",
"metadata": {},
"outputs": [],
"source": [
"searcher = OptunaSearch()"
"algo = OptunaSearch()"
]
},
{
"cell_type": "markdown",
"id": "a001c1be",
"id": "4287fa79",
"metadata": {},
"source": [
"We also constrain the the number of concurrent trials to `4` with a `ConcurrencyLimiter`."
@ -158,26 +179,26 @@
{
"cell_type": "code",
"execution_count": null,
"id": "9ada67a5",
"id": "68022ea4",
"metadata": {},
"outputs": [],
"source": [
"algo = ConcurrencyLimiter(searcher, max_concurrent=4)\n"
"algo = ConcurrencyLimiter(algo, max_concurrent=4)\n"
]
},
{
"cell_type": "markdown",
"id": "1ce35bf5",
"id": "4c250f74",
"metadata": {},
"source": [
"Lastly, we set the number of samples for this Tune run to `1000`\n",
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c6c39b98",
"id": "f6c21314",
"metadata": {},
"outputs": [],
"source": [
@ -187,7 +208,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "c8855163",
"id": "9533aabf",
"metadata": {
"tags": [
"remove-cell"
@ -201,55 +222,54 @@
},
{
"cell_type": "markdown",
"id": "410b7464",
"id": "92942b88",
"metadata": {},
"source": [
"Furthermore, we define a `scheduler` to go along with our algorithm. This is optional, and only to demonstrate that we don't need to compromise other great features of Ray Tune while using Optuna."
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_space` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, notice how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5b5dbea1",
"metadata": {},
"outputs": [],
"source": [
"scheduler = AsyncHyperBandScheduler()"
]
},
{
"cell_type": "markdown",
"id": "24792770",
"metadata": {},
"source": [
"Now all that's left is running the experiment."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "677eeb36",
"id": "4e224bb2",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" scheduler=scheduler,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" num_samples=num_samples,\n",
" config=search_space\n",
")\n",
"\n",
")"
]
},
{
"cell_type": "markdown",
"id": "b66aab6a",
"metadata": {},
"source": [
"And now we have the hyperparameters found to minimize the mean loss."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e69db02e",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "markdown",
"id": "574cca0b",
"id": "d545d30b",
"metadata": {},
"source": [
"## Providing an initial set of hyperparameters\n",
"\n",
"While defining the search algorithm, we may choose to provide an initial set of hyperparameters that we believe are especially promising or informative, and\n",
"pass this information as a helpful starting point for the `OptunaSearch` object."
]
@ -257,7 +277,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "a2e38018",
"id": "7596b7f4",
"metadata": {},
"outputs": [],
"source": [
@ -269,7 +289,7 @@
},
{
"cell_type": "markdown",
"id": "f7b25729",
"id": "f84bbff0",
"metadata": {},
"source": [
"Now the `search_alg` built using `OptunaSearch` takes `points_to_evaluate`."
@ -278,7 +298,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "d85142ae",
"id": "320d1935",
"metadata": {
"lines_to_next_cell": 0
},
@ -290,7 +310,7 @@
},
{
"cell_type": "markdown",
"id": "f7c8c0f7",
"id": "9147d9a2",
"metadata": {},
"source": [
"And run the experiment with initial hyperparameter evaluations:"
@ -299,7 +319,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "26815194",
"id": "ee442efd",
"metadata": {},
"outputs": [],
"source": [
@ -310,23 +330,41 @@
" mode=\"min\",\n",
" num_samples=num_samples,\n",
" config=search_space\n",
")\n",
"\n",
")"
]
},
{
"cell_type": "markdown",
"id": "ccfe15e2",
"metadata": {},
"source": [
"We take another look at the optimal hyperparamters."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fcfa0c2e",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "markdown",
"id": "d8b9278c",
"id": "88080576",
"metadata": {},
"source": [
"## Conditional search spaces \n",
"\n",
"Sometimes we may want to build a more complicated search space that has conditional dependencies on other hyperparameters. In this case, we pass a define-by-run function to the `search_alg` argument in `ray.tune()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d8ea7416",
"id": "f0acc2fc",
"metadata": {
"lines_to_next_cell": 0
},
@ -361,7 +399,7 @@
},
{
"cell_type": "markdown",
"id": "283aec6a",
"id": "4c9d0945",
"metadata": {},
"source": [
"As before, we create the `search_alg` from `OptunaSearch` and `ConcurrencyLimiter`, this time we define the scope of search via the `space` argument and provide no initialization. We also must specific metric and mode when using `space`. "
@ -370,7 +408,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "b4818fb1",
"id": "906f9ffc",
"metadata": {},
"outputs": [],
"source": [
@ -380,7 +418,7 @@
},
{
"cell_type": "markdown",
"id": "020f4e6e",
"id": "fea9399c",
"metadata": {},
"source": [
"Running the experiment with a define-by-run search space:"
@ -389,33 +427,51 @@
{
"cell_type": "code",
"execution_count": null,
"id": "f478ec68",
"metadata": {
"lines_to_next_cell": 0
},
"id": "bf0ee932",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" num_samples=num_samples\n",
")\n",
"\n",
")"
]
},
{
"cell_type": "markdown",
"id": "11e1ee04",
"metadata": {},
"source": [
"We take a look again at the optimal hyperparameters."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "13e4ce18",
"metadata": {
"lines_to_next_cell": 0
},
"outputs": [],
"source": [
"print(\"Best hyperparameters for loss found were: \", analysis.get_best_config(\"mean_loss\", \"min\"))"
]
},
{
"cell_type": "markdown",
"id": "fc1ac46f",
"id": "34bbd066",
"metadata": {},
"source": [
"## Multi-objective optimization\n",
"\n",
"Finally, let's take a look at the multi-objective case."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b08eefa4",
"id": "b233cbea",
"metadata": {},
"outputs": [],
"source": [
@ -434,7 +490,7 @@
},
{
"cell_type": "markdown",
"id": "73943695",
"id": "338e5108",
"metadata": {},
"source": [
"We define the `OptunaSearch` object this time with metric and mode as list arguments."
@ -443,7 +499,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "ab840970",
"id": "624d0bc8",
"metadata": {},
"outputs": [],
"source": [
@ -455,24 +511,28 @@
" search_alg=algo,\n",
" num_samples=num_samples,\n",
" config=search_space\n",
")\n",
"\n",
"print(\"Best hyperparameters for loss found were: \", analysis.get_best_config(\"loss\", \"min\"))\n",
"print(\"Best hyperparameters for gain found were: \", analysis.get_best_config(\"gain\", \"max\"))"
")"
]
},
{
"cell_type": "markdown",
"id": "4c06be1e",
"id": "df42b8b3",
"metadata": {},
"source": [
"Now there are two hyperparameter sets for the two objectives.\n",
"\n",
"```\n",
"print(\"Best hyperparameters for loss found were: \", analysis.get_best_config(\"loss\", \"min\"))\n",
"print(\"Best hyperparameters for gain found were: \", analysis.get_best_config(\"gain\", \"max\"))\n",
"```\n",
"\n",
"We can mix-and-match the use of initial hyperparameter evaluations, conditional search spaces via define-by-run functions, and multi-objective tasks. This is also true of scheduler usage, with the exception of multi-objective optimization-- schedulers typically rely on a single scalar score, rather than the two scores we use here: loss, gain."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e662ef66",
"id": "a058fdb3",
"metadata": {
"tags": [
"remove-cell"

550
doc/source/tune/examples/sigopt_example.ipynb Normal file
View file

@ -0,0 +1,550 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "fdacf514",
"metadata": {},
"source": [
"# Running Tune experiments with SigOpt\n",
"\n",
"In this tutorial we introduce SigOpt, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with SigOpt and, as a result, allow you to seamlessly scale up a SigOpt optimization process - without sacrificing performance.\n",
"\n",
"SigOpt is a model development platform with built in hyperparameter optimization algorithms. Their technology is closed source, but is designed for optimizing functions that are nondifferentiable, with many local minima, or even unknown but only testable. Therefore, SigOpt necessarily falls in the domain of \"derivative-free optimization\" and \"black-box optimization\". In this example we minimize a simple objective to briefly demonstrate the usage of SigOpt with Ray Tune via `SigOptSearch`. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `sigopt==7.5.0` library is installed and an API key exists. To learn more and to obtain the necessary API key, refer to [SigOpt website](https://sigopt.com/). \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6e1d6d8a",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# !pip install ray[tune]\n",
"!pip install sigopt==7.5.0"
]
},
{
"cell_type": "markdown",
"id": "28c03459",
"metadata": {},
"source": [
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "152fce99",
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"import time\n",
"import os\n",
"\n",
"import ray\n",
"import numpy as np\n",
"from ray import tune\n",
"from ray.tune.suggest.sigopt import SigOptSearch\n",
"\n",
"if \"SIGOPT_KEY\" not in os.environ:\n",
" raise ValueError(\n",
" \"SigOpt API Key not found. Please set the SIGOPT_KEY \"\n",
" \"environment variable.\"\n",
" )"
]
},
{
"cell_type": "markdown",
"id": "58b82407",
"metadata": {},
"source": [
"Let's start by defining a simple evaluation function.\n",
"We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
"This setup assumes that we're running multiple `step`s of an experiment and try to tune two hyperparameters,\n",
"namely `width` and `height`, and `activation`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2f9f7650",
"metadata": {},
"outputs": [],
"source": [
"def evaluate(step, width, height, activation):\n",
" time.sleep(0.1)\n",
" activation_boost = 10 if activation==\"relu\" else 1\n",
" return (0.1 + width * step / 100) ** (-1) + height * 0.1 + activation_boost"
]
},
{
"cell_type": "markdown",
"id": "dec8f15e",
"metadata": {},
"source": [
"Next, our ``objective`` function takes a Tune ``config``, evaluates the `score` of your experiment in a training loop,\n",
"and uses `tune.report` to report the `score` back to Tune."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7284f24a",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" for step in range(config[\"steps\"]):\n",
" score = evaluate(step, config[\"width\"], config[\"height\"], config[\"activation\"])\n",
" tune.report(iterations=step, mean_loss=score)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "394c722e",
"metadata": {
"lines_to_next_cell": 0,
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.init(configure_logging=False)"
]
},
{
"cell_type": "markdown",
"id": "a1040d00",
"metadata": {},
"source": [
"Next we define a search space. The critical assumption is that the optimal hyperparamters live within this space. Yet, if the space is very large, then those hyperparameters may be difficult to find in a short amount of time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "57dbed8d",
"metadata": {},
"outputs": [],
"source": [
"#search_config = {\n",
"# \"steps\": 100,\n",
"# \"width\": tune.uniform(0, 20),\n",
"# \"height\": tune.uniform(-100, 100),\n",
"# \"activation\": tune.choice([\"relu, tanh\"])\n",
"#}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7f542011",
"metadata": {},
"outputs": [],
"source": [
"space = [\n",
" {\n",
" \"name\": \"width\",\n",
" \"type\": \"int\",\n",
" \"bounds\": {\"min\": 0, \"max\": 20},\n",
" },\n",
" {\n",
" \"name\": \"height\",\n",
" \"type\": \"int\",\n",
" \"bounds\": {\"min\": -100, \"max\": 100},\n",
" },\n",
" {\n",
" \"name\": \"activation\",\n",
" \"type\": \"categorical\",\n",
" \"categorical_values\": [\"relu\",\"tanh\"]\n",
" }\n",
"]"
]
},
{
"cell_type": "markdown",
"id": "0493aaed",
"metadata": {},
"source": [
"Now we define the search algorithm built from `SigOptSearch`, constrained to a maximum of `1` concurrent trials."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a9737339",
"metadata": {},
"outputs": [],
"source": [
"algo = SigOptSearch(\n",
" space,\n",
" name=\"SigOpt Example Experiment\",\n",
" max_concurrent=1,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
")"
]
},
{
"cell_type": "markdown",
"id": "d844d3ae",
"metadata": {},
"source": [
"```\n",
"\n",
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"(you can decrease this if it takes too long on your machine).\n",
"\n",
"```python\n",
"num_samples = 1000\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "38991a3c",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# If 1000 samples take too long, you can reduce this number.\n",
"# We override this number here for our smoke tests.\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "13743e20",
"metadata": {},
"source": [
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `space` provided above to `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, notice how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a2b6d066",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" name=\"sigopt_exp\",\n",
" search_alg=algo,\n",
" num_samples=num_samples,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" config={\"steps\": 100}\n",
")"
]
},
{
"cell_type": "markdown",
"id": "b7577b9e",
"metadata": {},
"source": [
"Here are the hyperparamters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3567e642",
"metadata": {
"lines_to_next_cell": 0
},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "markdown",
"id": "d6e96a2a",
"metadata": {},
"source": [
"## Multi-objective optimization with Sigopt\n",
"\n",
"We define another simple objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8e13db27",
"metadata": {
"lines_to_next_cell": 0
},
"outputs": [],
"source": [
"np.random.seed(0)\n",
"vector1 = np.random.normal(0, 0.1, 100)\n",
"vector2 = np.random.normal(0, 0.1, 100)\n",
"\n",
"def evaluate(w1, w2):\n",
" total = w1 * vector1 + w2 * vector2\n",
" return total.mean(), total.std()\n",
"\n",
"def multi_objective(config):\n",
" w1 = config[\"w1\"]\n",
" w2 = config[\"total_weight\"] - w1\n",
" \n",
" average, std = evaluate(w1, w2)\n",
" tune.report(average=average, std=std, sharpe=average / std)\n",
" time.sleep(0.1)"
]
},
{
"cell_type": "markdown",
"id": "a9487562",
"metadata": {},
"source": [
"We define the space manually for `SigOptSearch`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "47ed922c",
"metadata": {},
"outputs": [],
"source": [
"space = [\n",
" {\n",
" \"name\": \"w1\",\n",
" \"type\": \"double\",\n",
" \"bounds\": {\"min\": 0, \"max\": 1},\n",
" },\n",
"]\n",
"\n",
"algo = SigOptSearch(\n",
" space,\n",
" name=\"sigopt_multiobj_exp\",\n",
" observation_budget=num_samples,\n",
" max_concurrent=1,\n",
" metric=[\"average\", \"std\", \"sharpe\"],\n",
" mode=[\"max\", \"min\", \"obs\"],\n",
")"
]
},
{
"cell_type": "markdown",
"id": "07b0f93d",
"metadata": {},
"source": [
"Finally, we run the experiment using Ray Tune, which in this case requires very little input since most of the construction has gone inside `search_algo`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "de17bd46",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" multi_objective,\n",
" name=\"sigopt_multiobj_exp\",\n",
" search_alg=algo,\n",
" num_samples=num_samples,\n",
" config={\"total_weight\": 1},\n",
")"
]
},
{
"cell_type": "markdown",
"id": "3d8441a6",
"metadata": {},
"source": [
"And here are they hyperparameters found to minimize the the objective on average."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fcd606dd",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.get_best_config(\"average\", \"min\"))"
]
},
{
"cell_type": "markdown",
"id": "66c34c60",
"metadata": {},
"source": [
"## Incorporating prior beliefs with Sigopt\n",
"\n",
"If we have information about beneficial hyperparameters within the search space, then we can incorporate this bias via a prior distribution. Without explicitly incorporating a prior, the default is a uniform distribution of preference over the search space. Below we highlight the hyperparamters we expect to be better with a Gaussian prior distribution.\n",
"\n",
"We start with defining another objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "870dfb0d",
"metadata": {},
"outputs": [],
"source": [
"np.random.seed(0)\n",
"vector1 = np.random.normal(0.0, 0.1, 100)\n",
"vector2 = np.random.normal(0.0, 0.1, 100)\n",
"vector3 = np.random.normal(0.0, 0.1, 100)\n",
"\n",
"def evaluate(w1, w2, w3):\n",
" total = w1 * vector1 + w2 * vector2 + w3 * vector3\n",
" return total.mean(), total.std()\n",
"\n",
"def multi_objective_two(config):\n",
" w1 = config[\"w1\"]\n",
" w2 = config[\"w2\"]\n",
" total = w1 + w2\n",
" if total > 1:\n",
" w3 = 0\n",
" w1 /= total\n",
" w2 /= total\n",
" else:\n",
" w3 = 1 - total\n",
" \n",
" average, std = evaluate(w1, w2, w3)\n",
" tune.report(average=average, std=std)"
]
},
{
"cell_type": "markdown",
"id": "7ae02ab4",
"metadata": {},
"source": [
"Now we begin setting up the SigOpt experiment and algorithm. Incorporating a prior distribution over hyperparameters requires establishing a connection with SigOpt via `\"SIGOPT_KEY\"` environment variable. Here we create a Gaussian prior over w1 and w2, each independently. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fa992019",
"metadata": {},
"outputs": [],
"source": [
"samples = num_samples\n",
"\n",
"conn = Connection(client_token=os.environ[\"SIGOPT_KEY\"])\n",
"\n",
"experiment = conn.experiments().create(\n",
" name=\"prior experiment example\",\n",
" parameters=[\n",
" {\n",
" \"name\": \"w1\",\n",
" \"bounds\": {\"max\": 1, \"min\": 0},\n",
" \"prior\": {\"mean\": 1 / 3, \"name\": \"normal\", \"scale\": 0.2},\n",
" \"type\": \"double\",\n",
" },\n",
" {\n",
" \"name\": \"w2\",\n",
" \"bounds\": {\"max\": 1, \"min\": 0},\n",
" \"prior\": {\"mean\": 1 / 3, \"name\": \"normal\", \"scale\": 0.2},\n",
" \"type\": \"double\",\n",
" }, \n",
" ],\n",
" metrics=[\n",
" dict(name=\"std\", objective=\"minimize\", strategy=\"optimize\"),\n",
" dict(name=\"average\", strategy=\"store\"),\n",
" ],\n",
" observation_budget=samples,\n",
" parallel_bandwidth=1,\n",
")\n",
"\n",
"algo = SigOptSearch(\n",
" connection=conn,\n",
" experiment_id=experiment.id,\n",
" name=\"sigopt_prior_multi_exp\",\n",
" max_concurrent=1,\n",
" metric=[\"average\", \"std\"],\n",
" mode=[\"obs\", \"min\"],\n",
")"
]
},
{
"cell_type": "markdown",
"id": "dbc4099f",
"metadata": {},
"source": [
"Finally, we run the experiment using Ray Tune, where `search_algo` establishes the search space."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "da778e0f",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" name=\"sigopt_prior_multi_exp\",\n",
" search_alg=algo,\n",
" num_samples=samples,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "6f9c2959",
"metadata": {},
"source": [
"And here are they hyperparameters found to minimize the the objective on average."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6b9142bc",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.get_best_config(\"average\", \"min\"))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3e0151e1",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.shutdown()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"orphan": true
},
"nbformat": 4,
"nbformat_minor": 5
}

356
doc/source/tune/examples/skopt_example.ipynb Normal file
View file

@ -0,0 +1,356 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "61966d26",
"metadata": {},
"source": [
"# Running Tune experiments with Skopt\n",
"\n",
"In this tutorial we introduce Skopt, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with Skopt and, as a result, allow you to seamlessly scale up a Skopt optimization process - without sacrificing performance.\n",
"\n",
"Scikit-Optimize, or skopt, is a simple and efficient library to optimize expensive and noisy black-box functions, e.g. large-scale ML experiments. It implements several methods for sequential model-based optimization. Noteably, skopt does not perform gradient-based optimization, and instead uses computationally cheap surrogate models to\n",
"approximate the expensive function. In this example we minimize a simple objective to briefly demonstrate the usage of Skopt with Ray Tune via `SkOptSearch`. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `scikit-opitmize==0.8.1` library is installed. To learn more, please refer to the [Scikit-Optimize website](https://scikit-optimize.github.io).\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7d3b7ff9",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# !pip install ray[tune]\n",
"!pip install scikit-optimize==0.8.1\n",
"!pip install sklearn==0.18.2"
]
},
{
"cell_type": "markdown",
"id": "a730341c",
"metadata": {},
"source": [
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1591bb7e",
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"import time\n",
"from typing import Dict, Optional, Any\n",
"\n",
"import ray\n",
"\n",
"import skopt\n",
"from ray import tune\n",
"from ray.tune.suggest import ConcurrencyLimiter\n",
"from ray.tune.suggest.skopt import SkOptSearch"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f538eecb",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.init(configure_logging=False)"
]
},
{
"cell_type": "markdown",
"id": "c18346e5",
"metadata": {},
"source": [
"Let's start by defining a simple evaluation function. Again, an explicit math formula is queried here for demonstration, yet in practice this is typically a black-box function-- e.g. the performance results after training an ML model. We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment. This setup assumes that we're running multiple `step`s of an experiment while tuning three hyperparameters, namely `width`, `height`, and `activation`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3cb44451",
"metadata": {},
"outputs": [],
"source": [
"def evaluate(step, width, height, activation):\n",
" time.sleep(0.1)\n",
" activation_boost = 10 if activation==\"relu\" else 0\n",
" return (0.1 + width * step / 100) ** (-1) + height * 0.1 + activation_boost"
]
},
{
"cell_type": "markdown",
"id": "9ac14837",
"metadata": {},
"source": [
"Next, our `objective` function to be optimized takes a Tune `config`, evaluates the `score` of your experiment in a training loop,\n",
"and uses `tune.report` to report the `score` back to Tune."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f4b27c17",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" for step in range(config[\"steps\"]):\n",
" score = evaluate(step, config[\"width\"], config[\"height\"], config[\"activation\"])\n",
" tune.report(iterations=step, mean_loss=score)"
]
},
{
"cell_type": "markdown",
"id": "2b48209c",
"metadata": {},
"source": [
"Next we define a search space. The critical assumption is that the optimal hyperparamters live within this space. Yet, if the space is very large, then those hyperparameters may be difficult to find in a short amount of time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e1cf2e16",
"metadata": {},
"outputs": [],
"source": [
"search_space = {\n",
" \"steps\": 100,\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
" \"activation\": tune.choice([\"relu\", \"tanh\"]),\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "2892b243",
"metadata": {},
"source": [
"The search algorithm is instantiated from the `SkOptSearch` class. We also constrain the the number of concurrent trials to `4` with a `ConcurrencyLimiter`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5b560697",
"metadata": {},
"outputs": [],
"source": [
"algo = SkOptSearch()\n",
"algo = ConcurrencyLimiter(algo, max_concurrent=4)\n"
]
},
{
"cell_type": "markdown",
"id": "573a6c97",
"metadata": {},
"source": [
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bebc40db",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 1000"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a86c5a19",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# We override here for our smoke tests.\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "35c5823b",
"metadata": {},
"source": [
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_config` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, notice how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c86c6919",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" name=\"skopt_exp\",\n",
" num_samples=num_samples,\n",
" config=search_space\n",
")"
]
},
{
"cell_type": "markdown",
"id": "5f387c95",
"metadata": {},
"source": [
"We now have hyperparameters found to minimize the mean loss."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "19fff170",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "markdown",
"id": "7e7f43bb",
"metadata": {},
"source": [
"## Providing an initial set of hyperparameters\n",
"\n",
"While defining the search algorithm, we may choose to provide an initial set of hyperparameters that we believe are especially promising or informative, and\n",
"pass this information as a helpful starting point for the `SkOptSearch` object. We also can pass the known rewards for these initial params to save on unnecessary computation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c44a3ad2",
"metadata": {},
"outputs": [],
"source": [
"initial_params = [\n",
" {\"width\": 10, \"height\": 0, \"activation\": \"relu\"},\n",
" {\"width\": 15, \"height\": -20, \"activation\": \"tanh\"}\n",
"]\n",
"known_rewards = [-189, -1144]"
]
},
{
"cell_type": "markdown",
"id": "30e19a4b",
"metadata": {},
"source": [
"Now the `search_alg` built using `SkOptSearch` takes `points_to_evaluate`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "19308593",
"metadata": {
"lines_to_next_cell": 0
},
"outputs": [],
"source": [
"algo = SkOptSearch(points_to_evaluate=initial_params)\n",
"algo = ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "bd022018",
"metadata": {},
"source": [
"And again run the experiment, this time with initial hyperparameter evaluations:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d9c7c8d4",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" name=\"skopt_exp_with_warmstart\",\n",
" num_samples=num_samples,\n",
" config=search_space\n",
")"
]
},
{
"cell_type": "markdown",
"id": "04c9d550",
"metadata": {},
"source": [
"And we again show the ideal hyperparameters."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d7267095",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "23739237",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.shutdown()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"orphan": true
},
"nbformat": 4,
"nbformat_minor": 5
}

391
doc/source/tune/examples/zoopt_example.ipynb Normal file
View file

@ -0,0 +1,391 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "a97c49a9",
"metadata": {},
"source": [
"# Running Tune experiments with ZOOpt\n",
"\n",
"In this tutorial we introduce ZOOpt, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with ZOOpt and, as a result, allow you to seamlessly scale up a ZOOpt optimization process - without sacrificing performance.\n",
"\n",
"Zeroth-order optimization (ZOOpt) does not rely on the gradient of the objective function, but instead, learns from samples of the search space. It is suitable for optimizing functions that are nondifferentiable, with many local minima, or even unknown but only testable. Therefore, zeroth-order optimization is commonly referred to as \"derivative-free optimization\" and \"black-box optimization\". In this example we minimize a simple objective to briefly demonstrate the usage of ZOOpt with Ray Tune via `ZOOptSearch`. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `zoopt==0.4.1` library is installed. To learn more, please refer to the [ZOOpt website](https://github.com/polixir/ZOOpt)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "58fee596",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# !pip install ray[tune]\n",
"!pip install zoopt==0.4.1"
]
},
{
"cell_type": "markdown",
"id": "a9c8a34b",
"metadata": {},
"source": [
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d05017fe",
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"import time\n",
"\n",
"import ray\n",
"from ray import tune\n",
"from ray.tune.suggest.zoopt import ZOOptSearch\n",
"from zoopt import ValueType"
]
},
{
"cell_type": "markdown",
"id": "5a41255a",
"metadata": {},
"source": [
"Let's start by defining a simple evaluation function.\n",
"We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
"This setup assumes that we're running multiple `step`s of an experiment and try to tune two hyperparameters,\n",
"namely `width` and `height`, and `activation`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "61db0806",
"metadata": {},
"outputs": [],
"source": [
"def evaluate(step, width, height):\n",
" time.sleep(0.1)\n",
" return (0.1 + width * step / 100) ** (-1) + height * 0.1"
]
},
{
"cell_type": "markdown",
"id": "a979f791",
"metadata": {},
"source": [
"Next, our ``objective`` function takes a Tune ``config``, evaluates the `score` of your experiment in a training loop,\n",
"and uses `tune.report` to report the `score` back to Tune."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3b451a2c",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" for step in range(config[\"steps\"]):\n",
" score = evaluate(step, config[\"width\"], config[\"height\"])\n",
" tune.report(iterations=step, mean_loss=score)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ad4f9faf",
"metadata": {
"lines_to_next_cell": 0,
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.init(configure_logging=False)"
]
},
{
"cell_type": "markdown",
"id": "036e0085",
"metadata": {},
"source": [
"Next we define a search space. The critical assumption is that the optimal hyperparameters live within this space. Yet, if the space is very large, then those hyperparameters may be difficult to find in a short amount of time."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b28469ce",
"metadata": {},
"outputs": [],
"source": [
"search_config = {\n",
" \"steps\": 100,\n",
" \"width\": tune.randint(0, 10),\n",
" \"height\": tune.quniform(-10, 10, 1e-2),\n",
" \"activation\": tune.choice([\"relu, tanh\"])\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "599b1ece",
"metadata": {},
"source": [
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "94fcbc63",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 1000"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "33f11052",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# If 1000 samples take too long, you can reduce this number.\n",
"# We override this number here for our smoke tests.\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "c5b47448",
"metadata": {},
"source": [
"Next we define the search algorithm built from `ZOOptSearch`, constrained to a maximum of `8` concurrent trials via ZOOpt's internal `\"parallel_num\"`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "22c15bc5",
"metadata": {},
"outputs": [],
"source": [
"zoopt_config = {\n",
" \"parallel_num\": 8\n",
"}\n",
"algo = ZOOptSearch(\n",
" algo=\"Asracos\", # only supports ASRacos currently\n",
" budget=num_samples,\n",
" **zoopt_config,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "f5147bec",
"metadata": {},
"source": [
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_config` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, notice how efficient it is to execute `tune.run()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b51e7a31",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" name=\"zoopt_exp\",\n",
" num_samples=num_samples,\n",
" config=search_config\n",
")"
]
},
{
"cell_type": "markdown",
"id": "8b1080a3",
"metadata": {},
"source": [
"Here are the hyperparamters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c2fb0f9c",
"metadata": {
"lines_to_next_cell": 0
},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "markdown",
"id": "48d0547d",
"metadata": {},
"source": [
"## Optional: passing the parameter space into the search algorithm\n",
"\n",
"We can also pass the parameter space ourselves in the following formats: \n",
"- continuous dimensions: (continuous, search_range, precision)\n",
"- discrete dimensions: (discrete, search_range, has_order)\n",
"- grid dimensions: (grid, grid_list)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "00763339",
"metadata": {},
"outputs": [],
"source": [
"space = {\n",
" \"height\": (ValueType.CONTINUOUS, [-10, 10], 1e-2),\n",
" \"width\": (ValueType.DISCRETE, [0, 10], True),\n",
" \"layers\": (ValueType.GRID, [4, 8, 16])\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "54ecd17b",
"metadata": {},
"source": [
"ZOOpt again handles constraining the amount of concurrent trials with `\"parallel_num\"`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9ea6d7be",
"metadata": {},
"outputs": [],
"source": [
"zoopt_search_config = {\n",
" \"parallel_num\": 8,\n",
" \"metric\": \"mean_loss\",\n",
" \"mode\": \"min\"\n",
"}\n",
"algo = ZOOptSearch(\n",
" algo=\"Asracos\",\n",
" budget=num_samples,\n",
" dim_dict=space,\n",
" **zoopt_search_config\n",
")"
]
},
{
"cell_type": "markdown",
"id": "b0b0be8b",
"metadata": {},
"source": [
"This time we pass only `\"steps\"` and `\"activation\"` to the Tune `config` because `\"height\"` and `\"width\"` have been passed into `ZOOptSearch` to create the `search_algo`. \n",
"Again, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_config` via `algo`, `num_samples` times."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7995ede5",
"metadata": {},
"outputs": [],
"source": [
"analysis = tune.run(\n",
" objective,\n",
" search_alg=algo,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" name=\"zoopt_exp\",\n",
" num_samples=num_samples,\n",
" config={\n",
" \"steps\": 100,\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"id": "01e3b036",
"metadata": {},
"source": [
"Here are the hyperparamters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c11178dd",
"metadata": {},
"outputs": [],
"source": [
"print(\"Best hyperparameters found were: \", analysis.best_config)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "23eab59f",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.shutdown()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8cd01164",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "f5fb2c1b",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"orphan": true
},
"nbformat": 4,
"nbformat_minor": 5
}

BIN
doc/source/tune/images/ax.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 7.1 KiB

BIN
doc/source/tune/images/bayesopt.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 3.8 KiB

BIN
doc/source/tune/images/bohb.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 3.7 KiB

BIN
doc/source/tune/images/dragonfly.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 9.3 KiB

BIN
doc/source/tune/images/flaml.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 2.9 KiB

BIN
doc/source/tune/images/hebo.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 11 KiB

BIN
doc/source/tune/images/hyperopt.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 6 KiB

BIN
doc/source/tune/images/nevergrad.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 7 KiB

BIN
doc/source/tune/images/optuna.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 8.3 KiB

BIN
doc/source/tune/images/sigopt.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 6.8 KiB

BIN
doc/source/tune/images/skopt.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 3.2 KiB

BIN
doc/source/tune/images/zoopt.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 2.9 KiB

16
doc/source/tune/index.rst
View file

@ -10,14 +10,10 @@ Tune: Scalable Hyperparameter Tuning
:align: center
Tune is a Python library for experiment execution and hyperparameter tuning at any scale.
You can tune your favorite machine learning framework, :ref:`including PyTorch, XGBoost, TensorFlow and Keras <tune-guides>`,
and choose among state of the art algorithms such as :ref:`Population Based Training (PBT) <tune-scheduler-pbt>`,
:ref:`BayesOptSearch <bayesopt>`, or :ref:`HyperBand/ASHA <tune-scheduler-hyperband>`.
Tune integrates with a wide range of hyperparameter optimization tools, like
:ref:`Optuna, Hyperopt, Ax, and Nevergrad <tune-search-alg>`, to name a few.
You can tune your favorite machine learning framework (:ref:`PyTorch <tune-pytorch-cifar-ref>`, :ref:`XGBoost <tune-xgboost-ref>`, :doc:`Scikit-Learn <examples/tune-sklearn>`, :doc:`TensorFlow and Keras <examples/tune_mnist_keras>`, and :doc:`more <examples/index>`) by running state of the art algorithms such as :ref:`Population Based Training (PBT) <tune-scheduler-pbt>` and :ref:`HyperBand/ASHA <tune-scheduler-hyperband>`.
Tune further integrates with a wide range of additional hyperparameter optimization tools, including :doc:`Ax <examples/ax_example>`, :doc:`BayesOpt <examples/bayesopt_example>`, :doc:`BOHB <examples/bohb_example>`, :doc:`Dragonfly <examples/dragonfly_example>`, :doc:`FLAML <examples/flaml_example>`, :doc:`HEBO <examples/hebo_example>`, :doc:`Hyperopt <examples/hyperopt_example>`, :doc:`Nevergrad <examples/nevergrad_example>`, :doc:`Optuna <examples/optuna_example>`, :doc:`SigOpt <examples/sigopt_example>`, :doc:`skopt <examples/skopt_example>`, and :doc:`ZOOpt <examples/zoopt_example>`.
.. tabbed:: Examples
.. tabbed:: Example
Learn how to use Ray Tune for various machine learning frameworks in just a few steps.
**Click on the tabs to see code examples**.
@ -123,8 +119,8 @@ And you can move your models from training to serving on the same infrastructure
**Examples**
^^^
In our examples you can find practical tutorials for
scikit-learn, Keras, TensorFlow, PyTorch, mlflow, and many more.
In our examples you can find practical tutorials for using frameworks such as
scikit-learn, Keras, TensorFlow, PyTorch, and mlflow, and state of the art search algorithm integrations.
+++
.. link-button:: tune-examples-ref
@ -204,7 +200,7 @@ If you're new to Tune, you're probably wondering, "what makes Tune different?"
In this situation, Tune actually allows you to power up your existing workflow.
Tune's :ref:`Search Algorithms <tune-search-alg>` integrate with a variety of popular hyperparameter tuning
libraries (such as Nevergrad or HyperOpt) and allow you to seamlessly scale up your optimization
libraries (see :ref:`examples <tune-examples-ref>`) and allow you to seamlessly scale up your optimization
process - without sacrificing performance.
Projects using Tune