mirror of
https://github.com/vale981/ray
synced 2025-03-06 18:41:40 -05:00
cd5218f831
Replaces #24225 and adds example navigation Signed-off-by: Max Pumperla <max.pumperla@googlemail.com>
397 lines
No EOL
10 KiB
Text
397 lines
No EOL
10 KiB
Text
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "986bcaab",
|
||
"metadata": {},
|
||
"source": [
|
||
"# Running Tune experiments with BOHB\n",
|
||
"\n",
|
||
"In this tutorial we introduce BOHB, while running a simple Ray Tune experiment.\n",
|
||
"Tune’s Search Algorithms integrate with BOHB and, as a result,\n",
|
||
"allow you to seamlessly scale up a BOHB optimization\n",
|
||
"process - without sacrificing performance.\n",
|
||
"\n",
|
||
"Bayesian Optimization HyperBand (BOHB) combines the benefits of Bayesian optimization\n",
|
||
"together with Bandit-based methods (e.g. HyperBand). BOHB does not rely on\n",
|
||
"the gradient of the objective function,\n",
|
||
"but instead, learns from samples of the search space.\n",
|
||
"It is suitable for optimizing functions that are non-differentiable,\n",
|
||
"with many local minima, or even unknown but only testable.\n",
|
||
"Therefore, this approach belongs to the domain of\n",
|
||
"\"derivative-free optimization\" and \"black-box optimization\".\n",
|
||
"\n",
|
||
"In this example we minimize a simple objective to briefly demonstrate the usage of\n",
|
||
"BOHB with Ray Tune via `BOHBSearch`. It's useful to keep in mind that despite\n",
|
||
"the emphasis on machine learning experiments, Ray Tune optimizes any implicit\n",
|
||
"or explicit objective. Here we assume `ConfigSpace==0.4.18` and `hpbandster==0.7.4`\n",
|
||
"libraries are installed. To learn more, please refer to the\n",
|
||
"[BOHB website](https://github.com/automl/HpBandSter)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "d12bd979",
|
||
"metadata": {
|
||
"tags": [
|
||
"remove-cell"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# !pip install ray[tune]\n",
|
||
"!pip install ConfigSpace==0.4.18\n",
|
||
"!pip install hpbandster==0.7.4"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "96641e94",
|
||
"metadata": {},
|
||
"source": [
|
||
"Click below to see all the imports we need for this example.\n",
|
||
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
|
||
"Just click on the rocket symbol at the top of the navigation."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "0e65ccdb",
|
||
"metadata": {
|
||
"tags": [
|
||
"hide-input"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"import time\n",
|
||
"\n",
|
||
"import ray\n",
|
||
"from ray import tune\n",
|
||
"from ray.tune.suggest import ConcurrencyLimiter\n",
|
||
"from ray.tune.schedulers.hb_bohb import HyperBandForBOHB\n",
|
||
"from ray.tune.suggest.bohb import TuneBOHB\n",
|
||
"import ConfigSpace as CS"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "edba942a",
|
||
"metadata": {},
|
||
"source": [
|
||
"Let's start by defining a simple evaluation function.\n",
|
||
"We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
|
||
"This setup assumes that we're running multiple `step`s of an experiment and try to tune\n",
|
||
"two hyperparameters, namely `width` and `height`, and `activation`."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "af512205",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def evaluate(step, width, height, activation):\n",
|
||
" time.sleep(0.1)\n",
|
||
" activation_boost = 10 if activation==\"relu\" else 1\n",
|
||
" return (0.1 + width * step / 100) ** (-1) + height * 0.1 + activation_boost"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c073ea21",
|
||
"metadata": {},
|
||
"source": [
|
||
"Next, our `objective` function takes a Tune `config`, evaluates the `score` of your\n",
|
||
"experiment in a training loop, and uses `tune.report` to report the `score` back to Tune."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "8a086e87",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def objective(config):\n",
|
||
" for step in range(config[\"steps\"]):\n",
|
||
" score = evaluate(step, config[\"width\"], config[\"height\"], config[\"activation\"])\n",
|
||
" tune.report(iterations=step, mean_loss=score)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "05d07329",
|
||
"metadata": {
|
||
"tags": [
|
||
"remove-cell"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"ray.init(configure_logging=False)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "32ee1ba7",
|
||
"metadata": {},
|
||
"source": [
|
||
"Next we define a search space. The critical assumption is that the optimal\n",
|
||
"hyperparameters live within this space. Yet, if the space is very large,\n",
|
||
"then those hyperparameters may be difficult to find in a short amount of time."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "21598e54",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"search_space = {\n",
|
||
" \"steps\": 100,\n",
|
||
" \"width\": tune.uniform(0, 20),\n",
|
||
" \"height\": tune.uniform(-100, 100),\n",
|
||
" \"activation\": tune.choice([\"relu\", \"tanh\"]),\n",
|
||
"}"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "def82932",
|
||
"metadata": {},
|
||
"source": [
|
||
"Next we define the search algorithm built from `TuneBOHB`, constrained\n",
|
||
"to a maximum of `4` concurrent trials with a `ConcurrencyLimiter`.\n",
|
||
"Below `algo` will take care of the BO (Bayesian optimization) part of BOHB,\n",
|
||
"while scheduler will take care the HB (HyperBand) part."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "e847b5b6",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"algo = TuneBOHB()\n",
|
||
"algo = tune.suggest.ConcurrencyLimiter(algo, max_concurrent=4)\n",
|
||
"scheduler = HyperBandForBOHB(\n",
|
||
" time_attr=\"training_iteration\",\n",
|
||
" max_t=100,\n",
|
||
" reduction_factor=4,\n",
|
||
" stop_last_trials=False,\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "1787a842",
|
||
"metadata": {},
|
||
"source": [
|
||
"The number of samples is the number of hyperparameter combinations\n",
|
||
"that will be tried out. This Tune run is set to `1000` samples.\n",
|
||
"(you can decrease this if it takes too long on your machine)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "702eb3d4",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"num_samples = 1000"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "dfb3ecad",
|
||
"metadata": {
|
||
"tags": [
|
||
"remove-cell"
|
||
],
|
||
"pycharm": {
|
||
"name": "#%%\n"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"num_samples = 10"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "aa5936df",
|
||
"metadata": {},
|
||
"source": [
|
||
"Finally, we run the experiment to `min`imize the \"mean_loss\" of the `objective`\n",
|
||
"by searching within `\"steps\": 100` via `algo`, `num_samples` times. This previous\n",
|
||
"sentence is fully characterizes the search problem we aim to solve.\n",
|
||
"With this in mind, notice how efficient it is to execute `tune.run()`."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "4bdfb12d",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"analysis = tune.run(\n",
|
||
" objective,\n",
|
||
" search_alg=algo,\n",
|
||
" scheduler=scheduler,\n",
|
||
" metric=\"mean_loss\",\n",
|
||
" mode=\"min\",\n",
|
||
" name=\"bohb_exp\",\n",
|
||
" num_samples=num_samples,\n",
|
||
" config=search_space\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3e89853c",
|
||
"metadata": {},
|
||
"source": [
|
||
"Here are the hyperparameters found to minimize the mean loss of the defined objective."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "4be691d5",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"print(\"Best hyperparameters found were: \", analysis.best_config)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "800a19d9",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Optional: Passing the search space via the TuneBOHB algorithm\n",
|
||
"\n",
|
||
"We can define the hyperparameter search space using `ConfigSpace`,\n",
|
||
"which is the format accepted by BOHB."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "b96cb496",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"config_space = CS.ConfigurationSpace()\n",
|
||
"config_space.add_hyperparameter(\n",
|
||
" CS.UniformFloatHyperparameter(\"width\", lower=0, upper=20)\n",
|
||
")\n",
|
||
"config_space.add_hyperparameter(\n",
|
||
" CS.UniformFloatHyperparameter(\"height\", lower=-100, upper=100)\n",
|
||
")\n",
|
||
"config_space.add_hyperparameter(\n",
|
||
" CS.CategoricalHyperparameter(\n",
|
||
" \"activation\", choices=[\"relu\", \"tanh\"]\n",
|
||
" )\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "9cb77270",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"algo = TuneBOHB(\n",
|
||
" space=config_space,\n",
|
||
" metric=\"episode_reward_mean\",\n",
|
||
" mode=\"max\",\n",
|
||
")\n",
|
||
"algo = tune.suggest.ConcurrencyLimiter(algo, max_concurrent=4)\n",
|
||
"scheduler = HyperBandForBOHB(\n",
|
||
" time_attr=\"training_iteration\",\n",
|
||
" max_t=100,\n",
|
||
" reduction_factor=4,\n",
|
||
" stop_last_trials=False,\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"id": "8305c975",
|
||
"metadata": {
|
||
"pycharm": {
|
||
"name": "#%%\n"
|
||
}
|
||
},
|
||
"source": [
|
||
"analysis = tune.run(\n",
|
||
" objective,\n",
|
||
" config=config_space,\n",
|
||
" scheduler=bohb_hyperband,\n",
|
||
" search_alg=algo,\n",
|
||
" num_samples=num_samples,\n",
|
||
" name=\"bohb_exp_2\",\n",
|
||
" stop={\"training_iteration\": 100},\n",
|
||
")"
|
||
],
|
||
"execution_count": null,
|
||
"outputs": []
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"Here again are the hyperparameters found to minimize the mean loss of the\n",
|
||
"defined objective."
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
}
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "1ae613e4",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"print(\"Best hyperparameters found were: \", analysis.best_config)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "6b83ef6d",
|
||
"metadata": {
|
||
"tags": [
|
||
"remove-cell"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"ray.shutdown()"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"orphan": true
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
} |