2022-04-25 11:10:58 -07:00
{
"cells": [
{
"cell_type": "markdown",
"id": "d3d9d39e",
"metadata": {},
"source": [
"# Running Tune experiments with BlendSearch and CFO\n",
"\n",
2022-05-13 15:39:18 +02:00
"In this tutorial we introduce BlendSearch and CFO, while running a simple Ray Tune\n",
"experiment. Tune’
"you to seamlessly scale up a BlendSearch and CFO optimization\n",
"process - without sacrificing performance.\n",
2022-04-25 11:10:58 -07:00
"\n",
2022-05-13 15:39:18 +02:00
"Fast Library for Automated Machine Learning & Tuning (FLAML) does not rely on the\n",
"gradient of the objective function, but instead, learns from samples of the\n",
"search space. It is suitable for optimizing functions that are non-differentiable,\n",
"with many local minima, or even unknown but only testable. Therefore, it is\n",
"necessarily belongs to the domain of \"derivative-free optimization\"\n",
"and \"black-box optimization\".\n",
2022-04-25 11:10:58 -07:00
"\n",
2022-05-13 15:39:18 +02:00
"FLAML has two primary algorithms: (1) Frugal Optimization for Cost-related\n",
"Hyperparameters (CFO) begins with a low-cost initial point and gradually moves to\n",
"a high-cost region as needed. It is a local search method that leverages randomized\n",
"direct search method with an adaptive step-size and random restarts.\n",
"As a local search method, it has an appealing provable convergence rate and bounded\n",
"cost but may get trapped in suboptimal local minima. (2) Economical Hyperparameter\n",
"Optimization With Blended Search Strategy (BlendSearch) combines CFO's local search\n",
"with global search, making it less suspectable to local minima traps.\n",
"It leverages the frugality of CFO and the space exploration ability of global search\n",
"methods such as Bayesian optimization.\n",
2022-04-25 11:10:58 -07:00
"\n",
2022-05-13 15:39:18 +02:00
"In this example we minimize a simple objective to briefly demonstrate the usage of\n",
"FLAML with Ray Tune via `BlendSearch` and `CFO`. It's useful to keep in mind that\n",
"despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit\n",
"or explicit objective. Here we assume `flaml==0.4.1` and `optuna==2.9.1` libraries\n",
"are installed. To learn more, please refer to\n",
"the [FLAML website](https://github.com/microsoft/FLAML/tree/main/flaml/tune).\n",
2022-04-25 11:10:58 -07:00
" \n",
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 1,
2022-04-25 11:10:58 -07:00
"id": "1a9f10f0",
"metadata": {
"tags": [
"hide-input"
]
},
2022-07-24 18:53:57 +01:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/flaml/searcher/blendsearch.py:14: DeprecationWarning: The module `ray.tune.suggest` has been moved to `ray.tune.search` and the old location will be deprecated soon. Please adjust your imports to point to the new location. Example: Do a global search and replace `ray.tune.suggest` with `ray.tune.search`.\n",
" from ray.tune.suggest import Searcher\n",
"/Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/flaml/searcher/blendsearch.py:15: DeprecationWarning: The module `ray.tune.suggest.optuna` has been moved to `ray.tune.search.optuna` and the old location will be deprecated soon. Please adjust your imports to point to the new location. Example: Do a global search and replace `ray.tune.suggest.optuna` with `ray.tune.search.optuna`.\n",
" from ray.tune.suggest.optuna import OptunaSearch as GlobalSearch\n",
"/Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/flaml/tune/sample.py:39: DeprecationWarning: The module `ray.tune.sample` has been moved to `ray.tune.search.sample` and the old location will be deprecated soon. Please adjust your imports to point to the new location. Example: Do a global search and replace `ray.tune.sample` with `ray.tune.search.sample`.\n",
" from ray.tune.sample import _BackwardsCompatibleNumpyRng\n",
"/Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/flaml/tune/space.py:6: DeprecationWarning: The module `ray.tune.suggest.variant_generator` has been moved to `ray.tune.search.variant_generator` and the old location will be deprecated soon. Please adjust your imports to point to the new location. Example: Do a global search and replace `ray.tune.suggest.variant_generator` with `ray.tune.search.variant_generator`.\n",
" from ray.tune.suggest.variant_generator import generate_variants\n"
]
}
],
2022-04-25 11:10:58 -07:00
"source": [
"import time\n",
"\n",
"import ray\n",
"from ray import tune\n",
2022-07-09 19:47:21 -07:00
"from ray.air import session\n",
2022-06-25 14:55:30 +01:00
"from ray.tune.search import ConcurrencyLimiter\n",
"from ray.tune.search.flaml import BlendSearch, CFO"
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "markdown",
"id": "f5a4bce8",
"metadata": {},
"source": [
"Let's start by defining a simple evaluation function.\n",
"We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
2022-05-13 15:39:18 +02:00
"This setup assumes that we're running multiple `step`s of an experiment and try to\n",
"tune three hyperparameters, namely `width` and `height`, and `activation`."
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 2,
2022-04-25 11:10:58 -07:00
"id": "9b80ac32",
"metadata": {},
"outputs": [],
"source": [
"def evaluate(step, width, height, activation):\n",
" time.sleep(0.1)\n",
" activation_boost = 10 if activation==\"relu\" else 1\n",
" return (0.1 + width * step / 100) ** (-1) + height * 0.1 + activation_boost"
]
},
{
"cell_type": "markdown",
"id": "391dcd0e",
"metadata": {},
"source": [
2022-05-13 15:39:18 +02:00
"Next, our `objective` function takes a Tune `config`, evaluates the `score` of your\n",
2022-06-30 10:37:31 -07:00
"experiment in a training loop, and uses `session.report` to report the `score` back to Tune."
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 3,
2022-04-25 11:10:58 -07:00
"id": "0773e711",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" for step in range(config[\"steps\"]):\n",
" score = evaluate(step, config[\"width\"], config[\"height\"], config[\"activation\"])\n",
2022-06-30 10:37:31 -07:00
" session.report({\"iterations\": step, \"mean_loss\": score})"
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 4,
2022-04-25 11:10:58 -07:00
"id": "8dde8596",
"metadata": {
"lines_to_next_cell": 0,
"tags": [
"remove-cell"
]
},
2022-07-24 18:53:57 +01:00
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
" <div style=\"margin-left: 50px;display: flex;flex-direction: row;align-items: center\">\n",
" <h3 style=\"color: var(--jp-ui-font-color0)\">Ray</h3>\n",
" <svg version=\"1.1\" id=\"ray\" width=\"3em\" viewBox=\"0 0 144.5 144.6\" style=\"margin-left: 3em;margin-right: 3em\">\n",
" <g id=\"layer-1\">\n",
" <path fill=\"#00a2e9\" class=\"st0\" d=\"M97.3,77.2c-3.8-1.1-6.2,0.9-8.3,5.1c-3.5,6.8-9.9,9.9-17.4,9.6S58,88.1,54.8,81.2c-1.4-3-3-4-6.3-4.1\n",
" c-5.6-0.1-9.9,0.1-13.1,6.4c-3.8,7.6-13.6,10.2-21.8,7.6C5.2,88.4-0.4,80.5,0,71.7c0.1-8.4,5.7-15.8,13.8-18.2\n",
" c8.4-2.6,17.5,0.7,22.3,8c1.3,1.9,1.3,5.2,3.6,5.6c3.9,0.6,8,0.2,12,0.2c1.8,0,1.9-1.6,2.4-2.8c3.5-7.8,9.7-11.8,18-11.9\n",
" c8.2-0.1,14.4,3.9,17.8,11.4c1.3,2.8,2.9,3.6,5.7,3.3c1-0.1,2,0.1,3,0c2.8-0.5,6.4,1.7,8.1-2.7s-2.3-5.5-4.1-7.5\n",
" c-5.1-5.7-10.9-10.8-16.1-16.3C84,38,81.9,37.1,78,38.3C66.7,42,56.2,35.7,53,24.1C50.3,14,57.3,2.8,67.7,0.5\n",
" C78.4-2,89,4.7,91.5,15.3c0.1,0.3,0.1,0.5,0.2,0.8c0.7,3.4,0.7,6.9-0.8,9.8c-1.7,3.2-0.8,5,1.5,7.2c6.7,6.5,13.3,13,19.8,19.7\n",
" c1.8,1.8,3,2.1,5.5,1.2c9.1-3.4,17.9-0.6,23.4,7c4.8,6.9,4.6,16.1-0.4,22.9c-5.4,7.2-14.2,9.9-23.1,6.5c-2.3-0.9-3.5-0.6-5.1,1.1\n",
" c-6.7,6.9-13.6,13.7-20.5,20.4c-1.8,1.8-2.5,3.2-1.4,5.9c3.5,8.7,0.3,18.6-7.7,23.6c-7.9,5-18.2,3.8-24.8-2.9\n",
" c-6.4-6.4-7.4-16.2-2.5-24.3c4.9-7.8,14.5-11,23.1-7.8c3,1.1,4.7,0.5,6.9-1.7C91.7,98.4,98,92.3,104.2,86c1.6-1.6,4.1-2.7,2.6-6.2\n",
" c-1.4-3.3-3.8-2.5-6.2-2.6C99.8,77.2,98.9,77.2,97.3,77.2z M72.1,29.7c5.5,0.1,9.9-4.3,10-9.8c0-0.1,0-0.2,0-0.3\n",
" C81.8,14,77,9.8,71.5,10.2c-5,0.3-9,4.2-9.3,9.2c-0.2,5.5,4,10.1,9.5,10.3C71.8,29.7,72,29.7,72.1,29.7z M72.3,62.3\n",
" c-5.4-0.1-9.9,4.2-10.1,9.7c0,0.2,0,0.3,0,0.5c0.2,5.4,4.5,9.7,9.9,10c5.1,0.1,9.9-4.7,10.1-9.8c0.2-5.5-4-10-9.5-10.3\n",
" C72.6,62.3,72.4,62.3,72.3,62.3z M115,72.5c0.1,5.4,4.5,9.7,9.8,9.9c5.6-0.2,10-4.8,10-10.4c-0.2-5.4-4.6-9.7-10-9.7\n",
" c-5.3-0.1-9.8,4.2-9.9,9.5C115,72.1,115,72.3,115,72.5z M19.5,62.3c-5.4,0.1-9.8,4.4-10,9.8c-0.1,5.1,5.2,10.4,10.2,10.3\n",
" c5.6-0.2,10-4.9,9.8-10.5c-0.1-5.4-4.5-9.7-9.9-9.6C19.6,62.3,19.5,62.3,19.5,62.3z M71.8,134.6c5.9,0.2,10.3-3.9,10.4-9.6\n",
" c0.5-5.5-3.6-10.4-9.1-10.8c-5.5-0.5-10.4,3.6-10.8,9.1c0,0.5,0,0.9,0,1.4c-0.2,5.3,4,9.8,9.3,10\n",
" C71.6,134.6,71.7,134.6,71.8,134.6z\"/>\n",
" </g>\n",
" </svg>\n",
" <table>\n",
" <tr>\n",
" <td style=\"text-align: left\"><b>Python version:</b></td>\n",
" <td style=\"text-align: left\"><b>3.7.7</b></td>\n",
" </tr>\n",
" <tr>\n",
" <td style=\"text-align: left\"><b>Ray version:</b></td>\n",
2022-07-27 18:43:27 +00:00
" <td style=\"text-align: left\"><b> 2.0.0rc0</b></td>\n",
2022-07-24 18:53:57 +01:00
" </tr>\n",
" <tr>\n",
" <td style=\"text-align: left\"><b>Dashboard:</b></td>\n",
" <td style=\"text-align: left\"><b><a href=\"http://127.0.0.1:8265\" target=\"_blank\">http://127.0.0.1:8265</a></b></td>\n",
"</tr>\n",
"\n",
" </table>\n",
" </div>\n",
"</div>\n"
],
"text/plain": [
2022-07-27 18:43:27 +00:00
"RayContext(dashboard_url='127.0.0.1:8265', python_version='3.7.7', ray_version='2.0.0rc0', ray_commit='{{RAY_COMMIT_SHA}}', address_info={'node_ip_address': '127.0.0.1', 'raylet_ip_address': '127.0.0.1', 'redis_address': None, 'object_store_address': '/tmp/ray/session_2022-07-22_15-18-05_176636_45728/sockets/plasma_store', 'raylet_socket_name': '/tmp/ray/session_2022-07-22_15-18-05_176636_45728/sockets/raylet', 'webui_url': '127.0.0.1:8265', 'session_dir': '/tmp/ray/session_2022-07-22_15-18-05_176636_45728', 'metrics_export_port': 63113, 'gcs_address': '127.0.0.1:63688', 'address': '127.0.0.1:63688', 'dashboard_agent_listen_port': 52365, 'node_id': '35eafee40228cabb8ae4ddafa3e39f41262af0440bf69a0d776f8fe8'})"
2022-07-24 18:53:57 +01:00
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
2022-04-25 11:10:58 -07:00
"source": [
"ray.init(configure_logging=False)"
]
},
{
"cell_type": "markdown",
"id": "a1007f97",
"metadata": {},
"source": [
"## Running Tune experiments with BlendSearch\n",
"\n",
2022-05-13 15:39:18 +02:00
"This example demonstrates the usage of Economical Hyperparameter Optimization\n",
"With Blended Search Strategy (BlendSearch) with Ray Tune.\n",
2022-04-25 11:10:58 -07:00
"\n",
2022-05-13 15:39:18 +02:00
"Now we define the search algorithm built from `BlendSearch`, constrained to a\n",
"maximum of `4` concurrent trials with a `ConcurrencyLimiter`."
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 5,
2022-04-25 11:10:58 -07:00
"id": "37b5070e",
"metadata": {},
"outputs": [],
"source": [
"algo = BlendSearch()\n",
"algo = ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "6bb24c73",
"metadata": {},
"source": [
"The number of samples this Tune run is set to `1000`.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 6,
2022-04-25 11:10:58 -07:00
"id": "f219c33a",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 1000"
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 7,
2022-04-25 11:10:58 -07:00
"id": "7ad72495",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# If 1000 samples take too long, you can reduce this number.\n",
"# We override this number here for our smoke tests.\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "6a206833",
"metadata": {},
"source": [
2022-05-13 15:39:18 +02:00
"Next we define a search space. The critical assumption is that the optimal\n",
"hyperparameters live within this space. Yet, if the space is very large, then those\n",
"hyperparameters may be difficult to find in a short amount of time."
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 8,
2022-04-25 11:10:58 -07:00
"id": "f2236f81",
"metadata": {},
"outputs": [],
"source": [
"search_config = {\n",
" \"steps\": 100,\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
" \"activation\": tune.choice([\"relu, tanh\"])\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "f77303bc",
"metadata": {},
"source": [
2022-05-13 15:39:18 +02:00
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by\n",
"searching `search_config` via `algo`, `num_samples` times. This previous sentence is\n",
"fully characterizes the search problem we aim to solve. With this in mind, observe\n",
2022-07-24 18:53:57 +01:00
"how efficient it is to execute `tuner.fit()`."
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 9,
2022-04-25 11:10:58 -07:00
"id": "7a95373a",
"metadata": {},
2022-07-24 18:53:57 +01:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Function checkpointing is disabled. This may result in unexpected behavior when using checkpointing features or certain schedulers. To enable, set the train function arguments to be `func(config, checkpoint_dir=None)`.\n",
"\u001b[32m[I 2022-07-22 15:18:10,659]\u001b[0m A new study created in memory with name: optuna\u001b[0m\n"
]
},
{
"data": {
"text/html": [
"== Status ==<br>Current time: 2022-07-22 15:18:55 (running for 00:00:43.79)<br>Memory usage on this node: 10.0/16.0 GiB<br>Using FIFO scheduling algorithm.<br>Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/4.69 GiB heap, 0.0/2.0 GiB objects<br>Current best trial: 1fc231f8 with mean_loss=-8.519908298508359 and parameters={'steps': 100, 'width': 15.42641286533492, 'height': -95.8496101281197, 'activation': 'relu, tanh'}<br>Result logdir: /Users/kai/ray_results/blendsearch_exp<br>Number of trials: 10/10 (10 TERMINATED)<br><table>\n",
"<thead>\n",
"<tr><th>Trial name </th><th>status </th><th>loc </th><th>activation </th><th style=\"text-align: right;\"> height</th><th style=\"text-align: right;\"> steps</th><th style=\"text-align: right;\"> width</th><th style=\"text-align: right;\"> loss</th><th style=\"text-align: right;\"> iter</th><th style=\"text-align: right;\"> total time (s)</th><th style=\"text-align: right;\"> iterations</th><th style=\"text-align: right;\"> neg_mean_loss</th></tr>\n",
"</thead>\n",
"<tbody>\n",
"<tr><td>objective_1e0f814e</td><td>TERMINATED</td><td>127.0.0.1:45801</td><td>relu, tanh </td><td style=\"text-align: right;\"> 29.4532 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 1.94864 </td><td style=\"text-align: right;\"> 4.43814</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.061 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -4.43814</td></tr>\n",
"<tr><td>objective_1fc231f8</td><td>TERMINATED</td><td>127.0.0.1:45809</td><td>relu, tanh </td><td style=\"text-align: right;\">-95.8496 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">15.4264 </td><td style=\"text-align: right;\">-8.51991</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.6807</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 8.51991</td></tr>\n",
"<tr><td>objective_1fc3b668</td><td>TERMINATED</td><td>127.0.0.1:45810</td><td>relu, tanh </td><td style=\"text-align: right;\"> 49.7608 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">12.673 </td><td style=\"text-align: right;\"> 6.05515</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.7372</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -6.05515</td></tr>\n",
"<tr><td>objective_1fc58b1e</td><td>TERMINATED</td><td>127.0.0.1:45811</td><td>relu, tanh </td><td style=\"text-align: right;\">-55.0407 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 9.97014 </td><td style=\"text-align: right;\">-4.40377</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.6335</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 4.40377</td></tr>\n",
"<tr><td>objective_265c09ee</td><td>TERMINATED</td><td>127.0.0.1:45837</td><td>relu, tanh </td><td style=\"text-align: right;\"> 52.1061 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 3.96126 </td><td style=\"text-align: right;\"> 6.45927</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7539</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -6.45927</td></tr>\n",
"<tr><td>objective_28587494</td><td>TERMINATED</td><td>127.0.0.1:45842</td><td>relu, tanh </td><td style=\"text-align: right;\">-82.332 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 3.38222 </td><td style=\"text-align: right;\">-6.94321</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7116</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 6.94321</td></tr>\n",
"<tr><td>objective_28682bfa</td><td>TERMINATED</td><td>127.0.0.1:45845</td><td>relu, tanh </td><td style=\"text-align: right;\">-94.9771 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">17.681 </td><td style=\"text-align: right;\">-8.4409 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7471</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 8.4409 </td></tr>\n",
"<tr><td>objective_28788388</td><td>TERMINATED</td><td>127.0.0.1:45848</td><td>relu, tanh </td><td style=\"text-align: right;\"> 90.6787 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">13.7072 </td><td style=\"text-align: right;\">10.141 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7525</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -10.141 </td></tr>\n",
"<tr><td>objective_2e3cd63e</td><td>TERMINATED</td><td>127.0.0.1:45864</td><td>relu, tanh </td><td style=\"text-align: right;\"> 2.43845</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 0.0789653</td><td style=\"text-align: right;\"> 6.85628</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7006</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -6.85628</td></tr>\n",
"<tr><td>objective_3046455a</td><td>TERMINATED</td><td>127.0.0.1:45869</td><td>relu, tanh </td><td style=\"text-align: right;\"> 22.5052 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">16.2524 </td><td style=\"text-align: right;\"> 3.31229</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7176</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -3.31229</td></tr>\n",
"</tbody>\n",
"</table><br><br>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_1e0f814e:\n",
" date: 2022-07-22_15-18-14\n",
" done: false\n",
" experiment_id: 623da6c1aa28400a9d85da19bdc5721d\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 13.945321626643842\n",
" neg_mean_loss: -13.945321626643842\n",
" node_ip: 127.0.0.1\n",
" pid: 45801\n",
" time_since_restore: 0.10402607917785645\n",
" time_this_iter_s: 0.10402607917785645\n",
" time_total_s: 0.10402607917785645\n",
" timestamp: 1658499494\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 1e0f814e\n",
" warmup_time: 0.003920078277587891\n",
" \n",
"Result for objective_1fc58b1e:\n",
" date: 2022-07-22_15-18-17\n",
" done: false\n",
" experiment_id: da508e209a1946e89ec15a3f1a5ce5ef\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 5.495932910616953\n",
" neg_mean_loss: -5.495932910616953\n",
" node_ip: 127.0.0.1\n",
" pid: 45811\n",
" time_since_restore: 0.10269379615783691\n",
" time_this_iter_s: 0.10269379615783691\n",
" time_total_s: 0.10269379615783691\n",
" timestamp: 1658499497\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 1fc58b1e\n",
" warmup_time: 0.004099130630493164\n",
" \n",
"Result for objective_1fc231f8:\n",
" date: 2022-07-22_15-18-17\n",
" done: false\n",
" experiment_id: 24c1fa443be246229e2c2ed9581452de\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 1.4150389871880282\n",
" neg_mean_loss: -1.4150389871880282\n",
" node_ip: 127.0.0.1\n",
" pid: 45809\n",
" time_since_restore: 0.10376811027526855\n",
" time_this_iter_s: 0.10376811027526855\n",
" time_total_s: 0.10376811027526855\n",
" timestamp: 1658499497\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 1fc231f8\n",
" warmup_time: 0.004148244857788086\n",
" \n",
"Result for objective_1fc3b668:\n",
" date: 2022-07-22_15-18-17\n",
" done: false\n",
" experiment_id: 5cdd489afb0e43b7a44aa4484765dc92\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 15.976077650772236\n",
" neg_mean_loss: -15.976077650772236\n",
" node_ip: 127.0.0.1\n",
" pid: 45810\n",
" time_since_restore: 0.1012420654296875\n",
" time_this_iter_s: 0.1012420654296875\n",
" time_total_s: 0.1012420654296875\n",
" timestamp: 1658499497\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 1fc3b668\n",
" warmup_time: 0.002666950225830078\n",
" \n",
"Result for objective_1e0f814e:\n",
" date: 2022-07-22_15-18-19\n",
" done: false\n",
" experiment_id: 623da6c1aa28400a9d85da19bdc5721d\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 44\n",
" iterations_since_restore: 45\n",
" mean_loss: 4.989814690087961\n",
" neg_mean_loss: -4.989814690087961\n",
" node_ip: 127.0.0.1\n",
" pid: 45801\n",
" time_since_restore: 5.1339499950408936\n",
" time_this_iter_s: 0.10842394828796387\n",
" time_total_s: 5.1339499950408936\n",
" timestamp: 1658499499\n",
" timesteps_since_restore: 0\n",
" training_iteration: 45\n",
" trial_id: 1e0f814e\n",
" warmup_time: 0.003920078277587891\n",
" \n",
"Result for objective_1fc58b1e:\n",
" date: 2022-07-22_15-18-22\n",
" done: false\n",
" experiment_id: da508e209a1946e89ec15a3f1a5ce5ef\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -4.295122851662673\n",
" neg_mean_loss: 4.295122851662673\n",
" node_ip: 127.0.0.1\n",
" pid: 45811\n",
" time_since_restore: 5.167006015777588\n",
" time_this_iter_s: 0.10817217826843262\n",
" time_total_s: 5.167006015777588\n",
" timestamp: 1658499502\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 1fc58b1e\n",
" warmup_time: 0.004099130630493164\n",
" \n",
"Result for objective_1fc231f8:\n",
" date: 2022-07-22_15-18-22\n",
" done: false\n",
" experiment_id: 24c1fa443be246229e2c2ed9581452de\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -8.44891425494968\n",
" neg_mean_loss: 8.44891425494968\n",
" node_ip: 127.0.0.1\n",
" pid: 45809\n",
" time_since_restore: 5.138737916946411\n",
" time_this_iter_s: 0.10868191719055176\n",
" time_total_s: 5.138737916946411\n",
" timestamp: 1658499502\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 1fc231f8\n",
" warmup_time: 0.004148244857788086\n",
" \n",
"Result for objective_1fc3b668:\n",
" date: 2022-07-22_15-18-22\n",
" done: false\n",
" experiment_id: 5cdd489afb0e43b7a44aa4484765dc92\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 6.141195146339472\n",
" neg_mean_loss: -6.141195146339472\n",
" node_ip: 127.0.0.1\n",
" pid: 45810\n",
" time_since_restore: 5.181504011154175\n",
" time_this_iter_s: 0.10758399963378906\n",
" time_total_s: 5.181504011154175\n",
" timestamp: 1658499502\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 1fc3b668\n",
" warmup_time: 0.002666950225830078\n",
" \n",
"Result for objective_1e0f814e:\n",
" date: 2022-07-22_15-18-24\n",
" done: false\n",
" experiment_id: 623da6c1aa28400a9d85da19bdc5721d\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 91\n",
" iterations_since_restore: 92\n",
" mean_loss: 4.479149291114477\n",
" neg_mean_loss: -4.479149291114477\n",
" node_ip: 127.0.0.1\n",
" pid: 45801\n",
" time_since_restore: 10.196602821350098\n",
" time_this_iter_s: 0.10757303237915039\n",
" time_total_s: 10.196602821350098\n",
" timestamp: 1658499504\n",
" timesteps_since_restore: 0\n",
" training_iteration: 92\n",
" trial_id: 1e0f814e\n",
" warmup_time: 0.003920078277587891\n",
" \n",
"Result for objective_1e0f814e:\n",
" date: 2022-07-22_15-18-25\n",
" done: true\n",
" experiment_id: 623da6c1aa28400a9d85da19bdc5721d\n",
" experiment_tag: 1_activation=relu_tanh,height=29.4532,steps=100,width=1.9486\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 4.438137591322411\n",
" neg_mean_loss: -4.438137591322411\n",
" node_ip: 127.0.0.1\n",
" pid: 45801\n",
" time_since_restore: 11.060971975326538\n",
" time_this_iter_s: 0.10748600959777832\n",
" time_total_s: 11.060971975326538\n",
" timestamp: 1658499505\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 1e0f814e\n",
" warmup_time: 0.003920078277587891\n",
" \n",
"Result for objective_1fc231f8:\n",
" date: 2022-07-22_15-18-26\n",
" done: false\n",
" experiment_id: 24c1fa443be246229e2c2ed9581452de\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 90\n",
" iterations_since_restore: 91\n",
" mean_loss: -8.513449547227397\n",
" neg_mean_loss: 8.513449547227397\n",
" node_ip: 127.0.0.1\n",
" pid: 45809\n",
" time_since_restore: 9.72878909111023\n",
" time_this_iter_s: 0.10644912719726562\n",
" time_total_s: 9.72878909111023\n",
" timestamp: 1658499506\n",
" timesteps_since_restore: 0\n",
" training_iteration: 91\n",
" trial_id: 1fc231f8\n",
" warmup_time: 0.004148244857788086\n",
" \n",
"Result for objective_1fc3b668:\n",
" date: 2022-07-22_15-18-26\n",
" done: false\n",
" experiment_id: 5cdd489afb0e43b7a44aa4484765dc92\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 89\n",
" iterations_since_restore: 90\n",
" mean_loss: 6.063959309753095\n",
" neg_mean_loss: -6.063959309753095\n",
" node_ip: 127.0.0.1\n",
" pid: 45810\n",
" time_since_restore: 9.6911301612854\n",
" time_this_iter_s: 0.10597920417785645\n",
" time_total_s: 9.6911301612854\n",
" timestamp: 1658499506\n",
" timesteps_since_restore: 0\n",
" training_iteration: 90\n",
" trial_id: 1fc3b668\n",
" warmup_time: 0.002666950225830078\n",
" \n",
"Result for objective_1fc58b1e:\n",
" date: 2022-07-22_15-18-27\n",
" done: false\n",
" experiment_id: da508e209a1946e89ec15a3f1a5ce5ef\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 91\n",
" iterations_since_restore: 92\n",
" mean_loss: -4.395049451529077\n",
" neg_mean_loss: 4.395049451529077\n",
" node_ip: 127.0.0.1\n",
" pid: 45811\n",
" time_since_restore: 9.873695850372314\n",
" time_this_iter_s: 0.10703802108764648\n",
" time_total_s: 9.873695850372314\n",
" timestamp: 1658499507\n",
" timesteps_since_restore: 0\n",
" training_iteration: 92\n",
" trial_id: 1fc58b1e\n",
" warmup_time: 0.004099130630493164\n",
" \n",
"Result for objective_265c09ee:\n",
" date: 2022-07-22_15-18-28\n",
" done: false\n",
" experiment_id: a7543e8d696745edbfe36a6f4ebe4071\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 16.210614243979172\n",
" neg_mean_loss: -16.210614243979172\n",
" node_ip: 127.0.0.1\n",
" pid: 45837\n",
" time_since_restore: 0.1049339771270752\n",
" time_this_iter_s: 0.1049339771270752\n",
" time_total_s: 0.1049339771270752\n",
" timestamp: 1658499508\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 265c09ee\n",
" warmup_time: 0.0026078224182128906\n",
" \n",
"Result for objective_1fc58b1e:\n",
" date: 2022-07-22_15-18-28\n",
" done: true\n",
" experiment_id: da508e209a1946e89ec15a3f1a5ce5ef\n",
" experiment_tag: 4_activation=relu_tanh,height=-55.0407,steps=100,width=9.9701\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -4.403770601366095\n",
" neg_mean_loss: 4.403770601366095\n",
" node_ip: 127.0.0.1\n",
" pid: 45811\n",
" time_since_restore: 11.633522987365723\n",
" time_this_iter_s: 0.10734701156616211\n",
" time_total_s: 11.633522987365723\n",
" timestamp: 1658499508\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 1fc58b1e\n",
" warmup_time: 0.004099130630493164\n",
" \n",
"Result for objective_1fc231f8:\n",
" date: 2022-07-22_15-18-28\n",
" done: true\n",
" experiment_id: 24c1fa443be246229e2c2ed9581452de\n",
" experiment_tag: 2_activation=relu_tanh,height=-95.8496,steps=100,width=15.4264\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -8.519908298508359\n",
" neg_mean_loss: 8.519908298508359\n",
" node_ip: 127.0.0.1\n",
" pid: 45809\n",
" time_since_restore: 11.680676937103271\n",
" time_this_iter_s: 0.10814285278320312\n",
" time_total_s: 11.680676937103271\n",
" timestamp: 1658499508\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 1fc231f8\n",
" warmup_time: 0.004148244857788086\n",
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_1fc3b668:\n",
" date: 2022-07-22_15-18-29\n",
" done: true\n",
" experiment_id: 5cdd489afb0e43b7a44aa4484765dc92\n",
" experiment_tag: 3_activation=relu_tanh,height=49.7608,steps=100,width=12.6730\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 6.055152568795227\n",
" neg_mean_loss: -6.055152568795227\n",
" node_ip: 127.0.0.1\n",
" pid: 45810\n",
" time_since_restore: 11.73720097541809\n",
" time_this_iter_s: 0.10816287994384766\n",
" time_total_s: 11.73720097541809\n",
" timestamp: 1658499509\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 1fc3b668\n",
" warmup_time: 0.002666950225830078\n",
" \n",
"Result for objective_28587494:\n",
" date: 2022-07-22_15-18-31\n",
" done: false\n",
" experiment_id: 750bd707fda147f0bd615c32243d64e5\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 2.766796283480206\n",
" neg_mean_loss: -2.766796283480206\n",
" node_ip: 127.0.0.1\n",
" pid: 45842\n",
" time_since_restore: 0.10265803337097168\n",
" time_this_iter_s: 0.10265803337097168\n",
" time_total_s: 0.10265803337097168\n",
" timestamp: 1658499511\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: '28587494'\n",
" warmup_time: 0.0028159618377685547\n",
" \n",
"Result for objective_28682bfa:\n",
" date: 2022-07-22_15-18-31\n",
" done: false\n",
" experiment_id: 5487e6a1cf70423096cf6285197fb824\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 1.5022935169945555\n",
" neg_mean_loss: -1.5022935169945555\n",
" node_ip: 127.0.0.1\n",
" pid: 45845\n",
" time_since_restore: 0.10289216041564941\n",
" time_this_iter_s: 0.10289216041564941\n",
" time_total_s: 0.10289216041564941\n",
" timestamp: 1658499511\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 28682bfa\n",
" warmup_time: 0.003072023391723633\n",
" \n",
"Result for objective_28788388:\n",
" date: 2022-07-22_15-18-31\n",
" done: false\n",
" experiment_id: e959dae493a240c489a3193352c46f3a\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 20.067866923898734\n",
" neg_mean_loss: -20.067866923898734\n",
" node_ip: 127.0.0.1\n",
" pid: 45848\n",
" time_since_restore: 0.1038503646850586\n",
" time_this_iter_s: 0.1038503646850586\n",
" time_total_s: 0.1038503646850586\n",
" timestamp: 1658499511\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: '28788388'\n",
" warmup_time: 0.0027070045471191406\n",
" \n",
"Result for objective_265c09ee:\n",
" date: 2022-07-22_15-18-33\n",
" done: false\n",
" experiment_id: a7543e8d696745edbfe36a6f4ebe4071\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 6.720352557756592\n",
" neg_mean_loss: -6.720352557756592\n",
" node_ip: 127.0.0.1\n",
" pid: 45837\n",
" time_since_restore: 5.148331165313721\n",
" time_this_iter_s: 0.10472607612609863\n",
" time_total_s: 5.148331165313721\n",
" timestamp: 1658499513\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 265c09ee\n",
" warmup_time: 0.0026078224182128906\n",
" \n",
"Result for objective_28587494:\n",
" date: 2022-07-22_15-18-36\n",
" done: false\n",
" experiment_id: 750bd707fda147f0bd615c32243d64e5\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -6.641362320132167\n",
" neg_mean_loss: 6.641362320132167\n",
" node_ip: 127.0.0.1\n",
" pid: 45842\n",
" time_since_restore: 5.175055980682373\n",
" time_this_iter_s: 0.10797286033630371\n",
" time_total_s: 5.175055980682373\n",
" timestamp: 1658499516\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: '28587494'\n",
" warmup_time: 0.0028159618377685547\n",
" \n",
"Result for objective_28682bfa:\n",
" date: 2022-07-22_15-18-36\n",
" done: false\n",
" experiment_id: 5487e6a1cf70423096cf6285197fb824\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -8.378801379858526\n",
" neg_mean_loss: 8.378801379858526\n",
" node_ip: 127.0.0.1\n",
" pid: 45845\n",
" time_since_restore: 5.192234039306641\n",
" time_this_iter_s: 0.10880398750305176\n",
" time_total_s: 5.192234039306641\n",
" timestamp: 1658499516\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 28682bfa\n",
" warmup_time: 0.003072023391723633\n",
" \n",
"Result for objective_28788388:\n",
" date: 2022-07-22_15-18-36\n",
" done: false\n",
" experiment_id: e959dae493a240c489a3193352c46f3a\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 10.22071644495505\n",
" neg_mean_loss: -10.22071644495505\n",
" node_ip: 127.0.0.1\n",
" pid: 45848\n",
" time_since_restore: 5.175441265106201\n",
" time_this_iter_s: 0.10689902305603027\n",
" time_total_s: 5.175441265106201\n",
" timestamp: 1658499516\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: '28788388'\n",
" warmup_time: 0.0027070045471191406\n",
" \n",
"Result for objective_265c09ee:\n",
" date: 2022-07-22_15-18-38\n",
" done: false\n",
" experiment_id: a7543e8d696745edbfe36a6f4ebe4071\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 6.472149118155721\n",
" neg_mean_loss: -6.472149118155721\n",
" node_ip: 127.0.0.1\n",
" pid: 45837\n",
" time_since_restore: 10.2187819480896\n",
" time_this_iter_s: 0.10822892189025879\n",
" time_total_s: 10.2187819480896\n",
" timestamp: 1658499518\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 265c09ee\n",
" warmup_time: 0.0026078224182128906\n",
" \n",
"Result for objective_265c09ee:\n",
" date: 2022-07-22_15-18-38\n",
" done: true\n",
" experiment_id: a7543e8d696745edbfe36a6f4ebe4071\n",
" experiment_tag: 5_activation=relu_tanh,height=52.1061,steps=100,width=3.9613\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 6.459268729660066\n",
" neg_mean_loss: -6.459268729660066\n",
" node_ip: 127.0.0.1\n",
" pid: 45837\n",
" time_since_restore: 10.753926277160645\n",
" time_this_iter_s: 0.10714435577392578\n",
" time_total_s: 10.753926277160645\n",
" timestamp: 1658499518\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 265c09ee\n",
" warmup_time: 0.0026078224182128906\n",
" \n",
"Result for objective_2e3cd63e:\n",
" date: 2022-07-22_15-18-41\n",
" done: false\n",
" experiment_id: a2f52d042c5d43bcbb59c6a16249fdb9\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 11.243845267715532\n",
" neg_mean_loss: -11.243845267715532\n",
" node_ip: 127.0.0.1\n",
" pid: 45864\n",
" time_since_restore: 0.10467910766601562\n",
" time_this_iter_s: 0.10467910766601562\n",
" time_total_s: 0.10467910766601562\n",
" timestamp: 1658499521\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 2e3cd63e\n",
" warmup_time: 0.002730131149291992\n",
" \n",
"Result for objective_28587494:\n",
" date: 2022-07-22_15-18-41\n",
" done: false\n",
" experiment_id: 750bd707fda147f0bd615c32243d64e5\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: -6.928259075209273\n",
" neg_mean_loss: 6.928259075209273\n",
" node_ip: 127.0.0.1\n",
" pid: 45842\n",
" time_since_restore: 10.179664850234985\n",
" time_this_iter_s: 0.10827279090881348\n",
" time_total_s: 10.179664850234985\n",
" timestamp: 1658499521\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: '28587494'\n",
" warmup_time: 0.0028159618377685547\n",
" \n",
"Result for objective_28682bfa:\n",
" date: 2022-07-22_15-18-41\n",
" done: false\n",
" experiment_id: 5487e6a1cf70423096cf6285197fb824\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: -8.437898356861577\n",
" neg_mean_loss: 8.437898356861577\n",
" node_ip: 127.0.0.1\n",
" pid: 45845\n",
" time_since_restore: 10.208577156066895\n",
" time_this_iter_s: 0.10761713981628418\n",
" time_total_s: 10.208577156066895\n",
" timestamp: 1658499521\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 28682bfa\n",
" warmup_time: 0.003072023391723633\n",
" \n",
"Result for objective_28788388:\n",
" date: 2022-07-22_15-18-41\n",
" done: false\n",
" experiment_id: e959dae493a240c489a3193352c46f3a\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 10.144880256980718\n",
" neg_mean_loss: -10.144880256980718\n",
" node_ip: 127.0.0.1\n",
" pid: 45848\n",
" time_since_restore: 10.215306997299194\n",
" time_this_iter_s: 0.10723686218261719\n",
" time_total_s: 10.215306997299194\n",
" timestamp: 1658499521\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: '28788388'\n",
" warmup_time: 0.0027070045471191406\n",
" \n",
"Result for objective_28587494:\n",
" date: 2022-07-22_15-18-42\n",
" done: true\n",
" experiment_id: 750bd707fda147f0bd615c32243d64e5\n",
" experiment_tag: 6_activation=relu_tanh,height=-82.3320,steps=100,width=3.3822\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -6.943213699003365\n",
" neg_mean_loss: 6.943213699003365\n",
" node_ip: 127.0.0.1\n",
" pid: 45842\n",
" time_since_restore: 10.711625099182129\n",
" time_this_iter_s: 0.10703706741333008\n",
" time_total_s: 10.711625099182129\n",
" timestamp: 1658499522\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: '28587494'\n",
" warmup_time: 0.0028159618377685547\n",
" \n",
"Result for objective_28682bfa:\n",
" date: 2022-07-22_15-18-42\n",
" done: true\n",
" experiment_id: 5487e6a1cf70423096cf6285197fb824\n",
" experiment_tag: 7_activation=relu_tanh,height=-94.9771,steps=100,width=17.6810\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -8.440901810803183\n",
" neg_mean_loss: 8.440901810803183\n",
" node_ip: 127.0.0.1\n",
" pid: 45845\n",
" time_since_restore: 10.747072219848633\n",
" time_this_iter_s: 0.1084749698638916\n",
" time_total_s: 10.747072219848633\n",
" timestamp: 1658499522\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 28682bfa\n",
" warmup_time: 0.003072023391723633\n",
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_28788388:\n",
" date: 2022-07-22_15-18-42\n",
" done: true\n",
" experiment_id: e959dae493a240c489a3193352c46f3a\n",
" experiment_tag: 8_activation=relu_tanh,height=90.6787,steps=100,width=13.7072\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 10.141019147716873\n",
" neg_mean_loss: -10.141019147716873\n",
" node_ip: 127.0.0.1\n",
" pid: 45848\n",
" time_since_restore: 10.752525091171265\n",
" time_this_iter_s: 0.10643196105957031\n",
" time_total_s: 10.752525091171265\n",
" timestamp: 1658499522\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: '28788388'\n",
" warmup_time: 0.0027070045471191406\n",
" \n",
"Result for objective_3046455a:\n",
" date: 2022-07-22_15-18-44\n",
" done: false\n",
" experiment_id: f282080699d848f69c4b8626bd4c2d91\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 13.250521336587763\n",
" neg_mean_loss: -13.250521336587763\n",
" node_ip: 127.0.0.1\n",
" pid: 45869\n",
" time_since_restore: 0.10329103469848633\n",
" time_this_iter_s: 0.10329103469848633\n",
" time_total_s: 0.10329103469848633\n",
" timestamp: 1658499524\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 3046455a\n",
" warmup_time: 0.0027010440826416016\n",
" \n",
"Result for objective_2e3cd63e:\n",
" date: 2022-07-22_15-18-46\n",
" done: false\n",
" experiment_id: a2f52d042c5d43bcbb59c6a16249fdb9\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 8.5370623175226\n",
" neg_mean_loss: -8.5370623175226\n",
" node_ip: 127.0.0.1\n",
" pid: 45864\n",
" time_since_restore: 5.129793882369995\n",
" time_this_iter_s: 0.10869097709655762\n",
" time_total_s: 5.129793882369995\n",
" timestamp: 1658499526\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 2e3cd63e\n",
" warmup_time: 0.002730131149291992\n",
" \n",
"Result for objective_3046455a:\n",
" date: 2022-07-22_15-18-49\n",
" done: false\n",
" experiment_id: f282080699d848f69c4b8626bd4c2d91\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 3.3797430580474837\n",
" neg_mean_loss: -3.3797430580474837\n",
" node_ip: 127.0.0.1\n",
" pid: 45869\n",
" time_since_restore: 5.139970064163208\n",
" time_this_iter_s: 0.10870695114135742\n",
" time_total_s: 5.139970064163208\n",
" timestamp: 1658499529\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 3046455a\n",
" warmup_time: 0.0027010440826416016\n",
" \n",
"Result for objective_2e3cd63e:\n",
" date: 2022-07-22_15-18-51\n",
" done: false\n",
" experiment_id: a2f52d042c5d43bcbb59c6a16249fdb9\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 6.983470378488619\n",
" neg_mean_loss: -6.983470378488619\n",
" node_ip: 127.0.0.1\n",
" pid: 45864\n",
" time_since_restore: 10.16285490989685\n",
" time_this_iter_s: 0.1078939437866211\n",
" time_total_s: 10.16285490989685\n",
" timestamp: 1658499531\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 2e3cd63e\n",
" warmup_time: 0.002730131149291992\n",
" \n",
"Result for objective_2e3cd63e:\n",
" date: 2022-07-22_15-18-51\n",
" done: true\n",
" experiment_id: a2f52d042c5d43bcbb59c6a16249fdb9\n",
" experiment_tag: 9_activation=relu_tanh,height=2.4385,steps=100,width=0.0790\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 6.8562837197212945\n",
" neg_mean_loss: -6.8562837197212945\n",
" node_ip: 127.0.0.1\n",
" pid: 45864\n",
" time_since_restore: 10.700610876083374\n",
" time_this_iter_s: 0.10786271095275879\n",
" time_total_s: 10.700610876083374\n",
" timestamp: 1658499531\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 2e3cd63e\n",
" warmup_time: 0.002730131149291992\n",
" \n",
"Result for objective_3046455a:\n",
" date: 2022-07-22_15-18-54\n",
" done: false\n",
" experiment_id: f282080699d848f69c4b8626bd4c2d91\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 3.315552368411653\n",
" neg_mean_loss: -3.315552368411653\n",
" node_ip: 127.0.0.1\n",
" pid: 45869\n",
" time_since_restore: 10.18196415901184\n",
" time_this_iter_s: 0.10727214813232422\n",
" time_total_s: 10.18196415901184\n",
" timestamp: 1658499534\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 3046455a\n",
" warmup_time: 0.0027010440826416016\n",
" \n",
"Result for objective_3046455a:\n",
" date: 2022-07-22_15-18-55\n",
" done: true\n",
" experiment_id: f282080699d848f69c4b8626bd4c2d91\n",
" experiment_tag: 10_activation=relu_tanh,height=22.5052,steps=100,width=16.2524\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 3.3122882595656677\n",
" neg_mean_loss: -3.3122882595656677\n",
" node_ip: 127.0.0.1\n",
" pid: 45869\n",
" time_since_restore: 10.717580080032349\n",
" time_this_iter_s: 0.1054232120513916\n",
" time_total_s: 10.717580080032349\n",
" timestamp: 1658499535\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 3046455a\n",
" warmup_time: 0.0027010440826416016\n",
" \n"
]
}
],
2022-04-25 11:10:58 -07:00
"source": [
2022-07-24 18:53:57 +01:00
"tuner = tune.Tuner(\n",
2022-04-25 11:10:58 -07:00
" objective,\n",
2022-07-24 18:53:57 +01:00
" tune_config=tune.TuneConfig(\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" search_alg=algo,\n",
" num_samples=num_samples,\n",
" ),\n",
" param_space=search_config,\n",
")\n",
"results = tuner.fit()"
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "markdown",
"id": "77a49ffb",
"metadata": {},
"source": [
"Here are the hyperparamters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 10,
2022-04-25 11:10:58 -07:00
"id": "59cdf197",
"metadata": {},
2022-07-24 18:53:57 +01:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Best hyperparameters found were: {'steps': 100, 'width': 15.42641286533492, 'height': -95.8496101281197, 'activation': 'relu, tanh'}\n"
]
}
],
2022-04-25 11:10:58 -07:00
"source": [
2022-07-24 18:53:57 +01:00
"print(\"Best hyperparameters found were: \", results.get_best_result().config)"
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "markdown",
"id": "1dca41c4",
"metadata": {},
"source": [
"## Incorporating a time budget to the experiment\n",
"\n",
"Define the time budget in seconds:"
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 11,
2022-04-25 11:10:58 -07:00
"id": "878bd08f",
"metadata": {},
"outputs": [],
"source": [
"time_budget_s = 30"
]
},
{
"cell_type": "markdown",
"id": "5c9d1bdc",
"metadata": {},
"source": [
2022-05-13 15:39:18 +02:00
"Similarly we define a search space, but this time we feed it as an argument to\n",
2022-07-24 18:53:57 +01:00
"`BlendSearch` rather than `Tuner()`'s `param_space` argument.\n",
2022-04-25 11:10:58 -07:00
"\n",
2022-05-13 15:39:18 +02:00
"We next define the time budget via `set_search_properties`.\n",
"And once again include the `ConcurrencyLimiter`."
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 12,
2022-04-25 11:10:58 -07:00
"id": "5846724f",
"metadata": {},
2022-07-24 18:53:57 +01:00
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"You passed a `space` parameter to OptunaSearch that contained unresolved search space definitions. OptunaSearch should however be instantiated with fully configured search spaces only. To use Ray Tune's automatic search space conversion, pass the space definition as part of the `param_space` argument to `Tuner()` instead.\n",
"\u001b[32m[I 2022-07-22 15:18:55,531]\u001b[0m A new study created in memory with name: optuna\u001b[0m\n"
]
}
],
2022-04-25 11:10:58 -07:00
"source": [
"algo = BlendSearch(\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" space={\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
" \"activation\": tune.choice([\"relu\", \"tanh\"]),\n",
" },\n",
")\n",
"algo.set_search_properties(config={\"time_budget_s\": time_budget_s})\n",
"algo = ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "13564767",
"metadata": {},
"source": [
2022-05-13 15:39:18 +02:00
"Now we run the experiment, this time with the `time_budget` included as an argument.\n",
"Note: We allow for virtually infinite `num_samples` by passing `-1`, so that the\n",
"experiment is stopped according to the time budget rather than a sample limit."
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 13,
2022-04-25 11:10:58 -07:00
"id": "b878b2be",
"metadata": {},
2022-07-24 18:53:57 +01:00
"outputs": [
{
"data": {
"text/html": [
"== Status ==<br>Current time: 2022-07-22 15:19:27 (running for 00:00:32.42)<br>Memory usage on this node: 10.2/16.0 GiB<br>Using FIFO scheduling algorithm.<br>Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/4.69 GiB heap, 0.0/2.0 GiB objects<br>Current best trial: 421d81ee with mean_loss=-8.596112689018389 and parameters={'steps': 100, 'width': 13.171830039895717, 'height': -96.72215542618497, 'activation': 'tanh'}<br>Result logdir: /Users/kai/ray_results/blendsearch_exp<br>Number of trials: 12/infinite (12 TERMINATED)<br><table>\n",
"<thead>\n",
"<tr><th>Trial name </th><th>status </th><th>loc </th><th>activation </th><th style=\"text-align: right;\"> height</th><th style=\"text-align: right;\"> steps</th><th style=\"text-align: right;\"> width</th><th style=\"text-align: right;\"> loss</th><th style=\"text-align: right;\"> iter</th><th style=\"text-align: right;\"> total time (s)</th><th style=\"text-align: right;\"> iterations</th><th style=\"text-align: right;\"> neg_mean_loss</th></tr>\n",
"</thead>\n",
"<tbody>\n",
"<tr><td>objective_38449df6</td><td>TERMINATED</td><td>127.0.0.1:45883</td><td>relu </td><td style=\"text-align: right;\"> 29.4532 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 1.94864 </td><td style=\"text-align: right;\">13.4381 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.6555 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -13.4381 </td></tr>\n",
"<tr><td>objective_39d44fea</td><td>TERMINATED</td><td>127.0.0.1:45889</td><td>tanh </td><td style=\"text-align: right;\"> -95.8496 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">15.4264 </td><td style=\"text-align: right;\">-8.51991 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.2945 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 8.51991 </td></tr>\n",
"<tr><td>objective_39d5c046</td><td>TERMINATED</td><td>127.0.0.1:45890</td><td>tanh </td><td style=\"text-align: right;\"> -55.0407 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 9.97014 </td><td style=\"text-align: right;\">-4.40377 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.169 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 4.40377 </td></tr>\n",
"<tr><td>objective_39d71608</td><td>TERMINATED</td><td>127.0.0.1:45891</td><td>tanh </td><td style=\"text-align: right;\"> -82.332 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 3.38222 </td><td style=\"text-align: right;\">-6.94321 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.3021 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 6.94321 </td></tr>\n",
"<tr><td>objective_402f9534</td><td>TERMINATED</td><td>127.0.0.1:45909</td><td>relu </td><td style=\"text-align: right;\"> 2.43845</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 0.0789653</td><td style=\"text-align: right;\">15.8563 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7275 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -15.8563 </td></tr>\n",
"<tr><td>objective_4208c6aa</td><td>TERMINATED</td><td>127.0.0.1:45914</td><td>relu </td><td style=\"text-align: right;\"> -41.6248 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">14.4351 </td><td style=\"text-align: right;\"> 5.90701 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.811 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -5.90701 </td></tr>\n",
"<tr><td>objective_421aeb64</td><td>TERMINATED</td><td>127.0.0.1:45917</td><td>relu </td><td style=\"text-align: right;\"> -94.9771 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">17.681 </td><td style=\"text-align: right;\"> 0.559098</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.8502 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -0.559098</td></tr>\n",
"<tr><td>objective_421d81ee</td><td>TERMINATED</td><td>127.0.0.1:45918</td><td>tanh </td><td style=\"text-align: right;\"> -96.7222 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">13.1718 </td><td style=\"text-align: right;\">-8.59611 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.8456 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 8.59611 </td></tr>\n",
"<tr><td>objective_481b0de6</td><td>TERMINATED</td><td>127.0.0.1:45933</td><td>relu </td><td style=\"text-align: right;\"> -81.3401 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">15.2514 </td><td style=\"text-align: right;\"> 2.48132 </td><td style=\"text-align: right;\"> 11</td><td style=\"text-align: right;\"> 1.17882</td><td style=\"text-align: right;\"> 10</td><td style=\"text-align: right;\"> -2.48132 </td></tr>\n",
"<tr><td>objective_4a00fc10</td><td>TERMINATED</td><td>127.0.0.1:45940</td><td>tanh </td><td style=\"text-align: right;\">-100 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">11.0922 </td><td style=\"text-align: right;\"> </td><td style=\"text-align: right;\"> </td><td style=\"text-align: right;\"> </td><td style=\"text-align: right;\"> </td><td style=\"text-align: right;\"> </td></tr>\n",
"<tr><td>objective_4a167946</td><td>TERMINATED</td><td>127.0.0.1:45945</td><td>tanh </td><td style=\"text-align: right;\"> -71.566 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">10.8509 </td><td style=\"text-align: right;\"> </td><td style=\"text-align: right;\"> </td><td style=\"text-align: right;\"> </td><td style=\"text-align: right;\"> </td><td style=\"text-align: right;\"> </td></tr>\n",
"<tr><td>objective_4a1908a0</td><td>TERMINATED</td><td>127.0.0.1:45946</td><td>relu </td><td style=\"text-align: right;\">-100 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">12.4575 </td><td style=\"text-align: right;\"> </td><td style=\"text-align: right;\"> </td><td style=\"text-align: right;\"> </td><td style=\"text-align: right;\"> </td><td style=\"text-align: right;\"> </td></tr>\n",
"</tbody>\n",
"</table><br><br>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_38449df6:\n",
" date: 2022-07-22_15-18-58\n",
" done: false\n",
" experiment_id: 22ed7d2e0f154e3d89fddc8f7f0b743f\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 22.945321626643842\n",
" neg_mean_loss: -22.945321626643842\n",
" node_ip: 127.0.0.1\n",
" pid: 45883\n",
" time_since_restore: 0.1004023551940918\n",
" time_this_iter_s: 0.1004023551940918\n",
" time_total_s: 0.1004023551940918\n",
" timestamp: 1658499538\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 38449df6\n",
" warmup_time: 0.0025548934936523438\n",
" \n",
"Result for objective_39d5c046:\n",
" date: 2022-07-22_15-19-00\n",
" done: false\n",
" experiment_id: faa00d9b1a2b4014bf01bab455bf9dbc\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 5.495932910616953\n",
" neg_mean_loss: -5.495932910616953\n",
" node_ip: 127.0.0.1\n",
" pid: 45890\n",
" time_since_restore: 0.10018515586853027\n",
" time_this_iter_s: 0.10018515586853027\n",
" time_total_s: 0.10018515586853027\n",
" timestamp: 1658499540\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 39d5c046\n",
" warmup_time: 0.0029740333557128906\n",
" \n",
"Result for objective_39d71608:\n",
" date: 2022-07-22_15-19-00\n",
" done: false\n",
" experiment_id: 43ea2d3839fb42cabf83009ddd82acbd\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 2.766796283480206\n",
" neg_mean_loss: -2.766796283480206\n",
" node_ip: 127.0.0.1\n",
" pid: 45891\n",
" time_since_restore: 0.10022497177124023\n",
" time_this_iter_s: 0.10022497177124023\n",
" time_total_s: 0.10022497177124023\n",
" timestamp: 1658499540\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 39d71608\n",
" warmup_time: 0.003571033477783203\n",
" \n",
"Result for objective_39d44fea:\n",
" date: 2022-07-22_15-19-00\n",
" done: false\n",
" experiment_id: d45c2138630140c3915a785b0e544a10\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 1.4150389871880282\n",
" neg_mean_loss: -1.4150389871880282\n",
" node_ip: 127.0.0.1\n",
" pid: 45889\n",
" time_since_restore: 0.10021686553955078\n",
" time_this_iter_s: 0.10021686553955078\n",
" time_total_s: 0.10021686553955078\n",
" timestamp: 1658499540\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 39d44fea\n",
" warmup_time: 0.003281831741333008\n",
" \n",
"Result for objective_38449df6:\n",
" date: 2022-07-22_15-19-03\n",
" done: false\n",
" experiment_id: 22ed7d2e0f154e3d89fddc8f7f0b743f\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 48\n",
" iterations_since_restore: 49\n",
" mean_loss: 13.91118054261762\n",
" neg_mean_loss: -13.91118054261762\n",
" node_ip: 127.0.0.1\n",
" pid: 45883\n",
" time_since_restore: 5.199498176574707\n",
" time_this_iter_s: 0.10724401473999023\n",
" time_total_s: 5.199498176574707\n",
" timestamp: 1658499543\n",
" timesteps_since_restore: 0\n",
" training_iteration: 49\n",
" trial_id: 38449df6\n",
" warmup_time: 0.0025548934936523438\n",
" \n",
"Result for objective_39d5c046:\n",
" date: 2022-07-22_15-19-05\n",
" done: false\n",
" experiment_id: faa00d9b1a2b4014bf01bab455bf9dbc\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -4.295122851662673\n",
" neg_mean_loss: 4.295122851662673\n",
" node_ip: 127.0.0.1\n",
" pid: 45890\n",
" time_since_restore: 5.137373924255371\n",
" time_this_iter_s: 0.1063838005065918\n",
" time_total_s: 5.137373924255371\n",
" timestamp: 1658499545\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 39d5c046\n",
" warmup_time: 0.0029740333557128906\n",
" \n",
"Result for objective_39d44fea:\n",
" date: 2022-07-22_15-19-05\n",
" done: false\n",
" experiment_id: d45c2138630140c3915a785b0e544a10\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -8.44891425494968\n",
" neg_mean_loss: 8.44891425494968\n",
" node_ip: 127.0.0.1\n",
" pid: 45889\n",
" time_since_restore: 5.140375852584839\n",
" time_this_iter_s: 0.10918521881103516\n",
" time_total_s: 5.140375852584839\n",
" timestamp: 1658499545\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 39d44fea\n",
" warmup_time: 0.003281831741333008\n",
" \n",
"Result for objective_39d71608:\n",
" date: 2022-07-22_15-19-05\n",
" done: false\n",
" experiment_id: 43ea2d3839fb42cabf83009ddd82acbd\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -6.641362320132167\n",
" neg_mean_loss: 6.641362320132167\n",
" node_ip: 127.0.0.1\n",
" pid: 45891\n",
" time_since_restore: 5.144212007522583\n",
" time_this_iter_s: 0.1064291000366211\n",
" time_total_s: 5.144212007522583\n",
" timestamp: 1658499545\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 39d71608\n",
" warmup_time: 0.003571033477783203\n",
" \n",
"Result for objective_38449df6:\n",
" date: 2022-07-22_15-19-08\n",
" done: false\n",
" experiment_id: 22ed7d2e0f154e3d89fddc8f7f0b743f\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 95\n",
" iterations_since_restore: 96\n",
" mean_loss: 13.457824286707531\n",
" neg_mean_loss: -13.457824286707531\n",
" node_ip: 127.0.0.1\n",
" pid: 45883\n",
" time_since_restore: 10.220572233200073\n",
" time_this_iter_s: 0.10792016983032227\n",
" time_total_s: 10.220572233200073\n",
" timestamp: 1658499548\n",
" timesteps_since_restore: 0\n",
" training_iteration: 96\n",
" trial_id: 38449df6\n",
" warmup_time: 0.0025548934936523438\n",
" \n",
"Result for objective_38449df6:\n",
" date: 2022-07-22_15-19-08\n",
" done: true\n",
" experiment_id: 22ed7d2e0f154e3d89fddc8f7f0b743f\n",
" experiment_tag: 1_activation=relu,height=29.4532,steps=100,width=1.9486\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 13.43813759132241\n",
" neg_mean_loss: -13.43813759132241\n",
" node_ip: 127.0.0.1\n",
" pid: 45883\n",
" time_since_restore: 10.655535221099854\n",
" time_this_iter_s: 0.10872602462768555\n",
" time_total_s: 10.655535221099854\n",
" timestamp: 1658499548\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 38449df6\n",
" warmup_time: 0.0025548934936523438\n",
" \n",
"Result for objective_39d5c046:\n",
" date: 2022-07-22_15-19-10\n",
" done: false\n",
" experiment_id: faa00d9b1a2b4014bf01bab455bf9dbc\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: -4.398492005249027\n",
" neg_mean_loss: 4.398492005249027\n",
" node_ip: 127.0.0.1\n",
" pid: 45890\n",
" time_since_restore: 10.1424720287323\n",
" time_this_iter_s: 0.10685110092163086\n",
" time_total_s: 10.1424720287323\n",
" timestamp: 1658499550\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 39d5c046\n",
" warmup_time: 0.0029740333557128906\n",
" \n",
"Result for objective_39d44fea:\n",
" date: 2022-07-22_15-19-10\n",
" done: false\n",
" experiment_id: d45c2138630140c3915a785b0e544a10\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: -8.515740400738455\n",
" neg_mean_loss: 8.515740400738455\n",
" node_ip: 127.0.0.1\n",
" pid: 45889\n",
" time_since_restore: 10.051257848739624\n",
" time_this_iter_s: 0.10633492469787598\n",
" time_total_s: 10.051257848739624\n",
" timestamp: 1658499550\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 39d44fea\n",
" warmup_time: 0.003281831741333008\n",
" \n",
"Result for objective_39d71608:\n",
" date: 2022-07-22_15-19-10\n",
" done: false\n",
" experiment_id: 43ea2d3839fb42cabf83009ddd82acbd\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: -6.925081133185133\n",
" neg_mean_loss: 6.925081133185133\n",
" node_ip: 127.0.0.1\n",
" pid: 45891\n",
" time_since_restore: 10.069098949432373\n",
" time_this_iter_s: 0.10451602935791016\n",
" time_total_s: 10.069098949432373\n",
" timestamp: 1658499550\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 39d71608\n",
" warmup_time: 0.003571033477783203\n",
" \n",
"Result for objective_402f9534:\n",
" date: 2022-07-22_15-19-11\n",
" done: false\n",
" experiment_id: 66af4092dc134e049ed47a56532133ce\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 20.243845267715532\n",
" neg_mean_loss: -20.243845267715532\n",
" node_ip: 127.0.0.1\n",
" pid: 45909\n",
" time_since_restore: 0.10260891914367676\n",
" time_this_iter_s: 0.10260891914367676\n",
" time_total_s: 0.10260891914367676\n",
" timestamp: 1658499551\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 402f9534\n",
" warmup_time: 0.0026721954345703125\n",
" \n",
"Result for objective_39d5c046:\n",
" date: 2022-07-22_15-19-11\n",
" done: true\n",
" experiment_id: faa00d9b1a2b4014bf01bab455bf9dbc\n",
" experiment_tag: 3_activation=tanh,height=-55.0407,steps=100,width=9.9701\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -4.403770601366095\n",
" neg_mean_loss: 4.403770601366095\n",
" node_ip: 127.0.0.1\n",
" pid: 45890\n",
" time_since_restore: 11.169048070907593\n",
" time_this_iter_s: 0.1081700325012207\n",
" time_total_s: 11.169048070907593\n",
" timestamp: 1658499551\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 39d5c046\n",
" warmup_time: 0.0029740333557128906\n",
" \n",
"Result for objective_39d44fea:\n",
" date: 2022-07-22_15-19-12\n",
" done: true\n",
" experiment_id: d45c2138630140c3915a785b0e544a10\n",
" experiment_tag: 2_activation=tanh,height=-95.8496,steps=100,width=15.4264\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -8.519908298508359\n",
" neg_mean_loss: 8.519908298508359\n",
" node_ip: 127.0.0.1\n",
" pid: 45889\n",
" time_since_restore: 11.29450273513794\n",
" time_this_iter_s: 0.1167898178100586\n",
" time_total_s: 11.29450273513794\n",
" timestamp: 1658499552\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 39d44fea\n",
" warmup_time: 0.003281831741333008\n",
" \n",
"Result for objective_39d71608:\n",
" date: 2022-07-22_15-19-12\n",
" done: true\n",
" experiment_id: 43ea2d3839fb42cabf83009ddd82acbd\n",
" experiment_tag: 4_activation=tanh,height=-82.3320,steps=100,width=3.3822\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -6.943213699003365\n",
" neg_mean_loss: 6.943213699003365\n",
" node_ip: 127.0.0.1\n",
" pid: 45891\n",
" time_since_restore: 11.302148818969727\n",
" time_this_iter_s: 0.12157797813415527\n",
" time_total_s: 11.302148818969727\n",
" timestamp: 1658499552\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 39d71608\n",
" warmup_time: 0.003571033477783203\n",
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_4208c6aa:\n",
" date: 2022-07-22_15-19-14\n",
" done: false\n",
" experiment_id: ce292c59f7c74e1ab42de6004eb3d846\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 15.837521363412662\n",
" neg_mean_loss: -15.837521363412662\n",
" node_ip: 127.0.0.1\n",
" pid: 45914\n",
" time_since_restore: 0.10442900657653809\n",
" time_this_iter_s: 0.10442900657653809\n",
" time_total_s: 0.10442900657653809\n",
" timestamp: 1658499554\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 4208c6aa\n",
" warmup_time: 0.002707958221435547\n",
" \n",
"Result for objective_421aeb64:\n",
" date: 2022-07-22_15-19-14\n",
" done: false\n",
" experiment_id: a4b264c05660423ea645cef7a9cd38bb\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 10.502293516994555\n",
" neg_mean_loss: -10.502293516994555\n",
" node_ip: 127.0.0.1\n",
" pid: 45917\n",
" time_since_restore: 0.1030268669128418\n",
" time_this_iter_s: 0.1030268669128418\n",
" time_total_s: 0.1030268669128418\n",
" timestamp: 1658499554\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 421aeb64\n",
" warmup_time: 0.0031020641326904297\n",
" \n",
"Result for objective_421d81ee:\n",
" date: 2022-07-22_15-19-14\n",
" done: false\n",
" experiment_id: 836a62c82d6c467fb232414586872fd0\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 1.3277844573815027\n",
" neg_mean_loss: -1.3277844573815027\n",
" node_ip: 127.0.0.1\n",
" pid: 45918\n",
" time_since_restore: 0.10497617721557617\n",
" time_this_iter_s: 0.10497617721557617\n",
" time_total_s: 0.10497617721557617\n",
" timestamp: 1658499554\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 421d81ee\n",
" warmup_time: 0.002989053726196289\n",
" \n",
"Result for objective_402f9534:\n",
" date: 2022-07-22_15-19-16\n",
" done: false\n",
" experiment_id: 66af4092dc134e049ed47a56532133ce\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 17.5370623175226\n",
" neg_mean_loss: -17.5370623175226\n",
" node_ip: 127.0.0.1\n",
" pid: 45909\n",
" time_since_restore: 5.13571572303772\n",
" time_this_iter_s: 0.1079568862915039\n",
" time_total_s: 5.13571572303772\n",
" timestamp: 1658499556\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 402f9534\n",
" warmup_time: 0.0026721954345703125\n",
" \n",
"Result for objective_4208c6aa:\n",
" date: 2022-07-22_15-19-19\n",
" done: false\n",
" experiment_id: ce292c59f7c74e1ab42de6004eb3d846\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 46\n",
" iterations_since_restore: 47\n",
" mean_loss: 5.98588603781584\n",
" neg_mean_loss: -5.98588603781584\n",
" node_ip: 127.0.0.1\n",
" pid: 45914\n",
" time_since_restore: 5.11784291267395\n",
" time_this_iter_s: 0.106842041015625\n",
" time_total_s: 5.11784291267395\n",
" timestamp: 1658499559\n",
" timesteps_since_restore: 0\n",
" training_iteration: 47\n",
" trial_id: 4208c6aa\n",
" warmup_time: 0.002707958221435547\n",
" \n",
"Result for objective_421aeb64:\n",
" date: 2022-07-22_15-19-19\n",
" done: false\n",
" experiment_id: a4b264c05660423ea645cef7a9cd38bb\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 46\n",
" iterations_since_restore: 47\n",
" mean_loss: 0.6237521179327707\n",
" neg_mean_loss: -0.6237521179327707\n",
" node_ip: 127.0.0.1\n",
" pid: 45917\n",
" time_since_restore: 5.119860887527466\n",
" time_this_iter_s: 0.10904288291931152\n",
" time_total_s: 5.119860887527466\n",
" timestamp: 1658499559\n",
" timesteps_since_restore: 0\n",
" training_iteration: 47\n",
" trial_id: 421aeb64\n",
" warmup_time: 0.0031020641326904297\n",
" \n",
"Result for objective_421d81ee:\n",
" date: 2022-07-22_15-19-19\n",
" done: false\n",
" experiment_id: 836a62c82d6c467fb232414586872fd0\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 46\n",
" iterations_since_restore: 47\n",
" mean_loss: -8.509852624896407\n",
" neg_mean_loss: 8.509852624896407\n",
" node_ip: 127.0.0.1\n",
" pid: 45918\n",
" time_since_restore: 5.103626012802124\n",
" time_this_iter_s: 0.10568881034851074\n",
" time_total_s: 5.103626012802124\n",
" timestamp: 1658499559\n",
" timesteps_since_restore: 0\n",
" training_iteration: 47\n",
" trial_id: 421d81ee\n",
" warmup_time: 0.002989053726196289\n",
" \n",
"Result for objective_402f9534:\n",
" date: 2022-07-22_15-19-21\n",
" done: false\n",
" experiment_id: 66af4092dc134e049ed47a56532133ce\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 15.983470378488619\n",
" neg_mean_loss: -15.983470378488619\n",
" node_ip: 127.0.0.1\n",
" pid: 45909\n",
" time_since_restore: 10.194732666015625\n",
" time_this_iter_s: 0.11167168617248535\n",
" time_total_s: 10.194732666015625\n",
" timestamp: 1658499561\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 402f9534\n",
" warmup_time: 0.0026721954345703125\n",
" \n",
"Result for objective_402f9534:\n",
" date: 2022-07-22_15-19-22\n",
" done: true\n",
" experiment_id: 66af4092dc134e049ed47a56532133ce\n",
" experiment_tag: 5_activation=relu,height=2.4385,steps=100,width=0.0790\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 15.856283719721294\n",
" neg_mean_loss: -15.856283719721294\n",
" node_ip: 127.0.0.1\n",
" pid: 45909\n",
" time_since_restore: 10.727468013763428\n",
" time_this_iter_s: 0.10604405403137207\n",
" time_total_s: 10.727468013763428\n",
" timestamp: 1658499562\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 402f9534\n",
" warmup_time: 0.0026721954345703125\n",
" \n",
"Result for objective_4208c6aa:\n",
" date: 2022-07-22_15-19-24\n",
" done: false\n",
" experiment_id: ce292c59f7c74e1ab42de6004eb3d846\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: 5.911460436218253\n",
" neg_mean_loss: -5.911460436218253\n",
" node_ip: 127.0.0.1\n",
" pid: 45914\n",
" time_since_restore: 10.168487071990967\n",
" time_this_iter_s: 0.10619020462036133\n",
" time_total_s: 10.168487071990967\n",
" timestamp: 1658499564\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 4208c6aa\n",
" warmup_time: 0.002707958221435547\n",
" \n",
"Result for objective_481b0de6:\n",
" date: 2022-07-22_15-19-24\n",
" done: false\n",
" experiment_id: ceafc8f530054af6bc676fd081f84c46\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 11.865986796947288\n",
" neg_mean_loss: -11.865986796947288\n",
" node_ip: 127.0.0.1\n",
" pid: 45933\n",
" time_since_restore: 0.10329604148864746\n",
" time_this_iter_s: 0.10329604148864746\n",
" time_total_s: 0.10329604148864746\n",
" timestamp: 1658499564\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 481b0de6\n",
" warmup_time: 0.0028421878814697266\n",
" \n",
"Result for objective_421aeb64:\n",
" date: 2022-07-22_15-19-24\n",
" done: false\n",
" experiment_id: a4b264c05660423ea645cef7a9cd38bb\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: 0.5627408539120644\n",
" neg_mean_loss: -0.5627408539120644\n",
" node_ip: 127.0.0.1\n",
" pid: 45917\n",
" time_since_restore: 10.207238912582397\n",
" time_this_iter_s: 0.1081080436706543\n",
" time_total_s: 10.207238912582397\n",
" timestamp: 1658499564\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 421aeb64\n",
" warmup_time: 0.0031020641326904297\n",
" \n",
"Result for objective_421d81ee:\n",
" date: 2022-07-22_15-19-24\n",
" done: false\n",
" experiment_id: 836a62c82d6c467fb232414586872fd0\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: -8.591242584097142\n",
" neg_mean_loss: 8.591242584097142\n",
" node_ip: 127.0.0.1\n",
" pid: 45918\n",
" time_since_restore: 10.2047119140625\n",
" time_this_iter_s: 0.1085958480834961\n",
" time_total_s: 10.2047119140625\n",
" timestamp: 1658499564\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 421d81ee\n",
" warmup_time: 0.002989053726196289\n",
" \n",
"Result for objective_4208c6aa:\n",
" date: 2022-07-22_15-19-25\n",
" done: true\n",
" experiment_id: ce292c59f7c74e1ab42de6004eb3d846\n",
" experiment_tag: 6_activation=relu,height=-41.6248,steps=100,width=14.4351\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 5.907010419420164\n",
" neg_mean_loss: -5.907010419420164\n",
" node_ip: 127.0.0.1\n",
" pid: 45914\n",
" time_since_restore: 10.811041831970215\n",
" time_this_iter_s: 0.10398101806640625\n",
" time_total_s: 10.811041831970215\n",
" timestamp: 1658499565\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 4208c6aa\n",
" warmup_time: 0.002707958221435547\n",
" \n",
"Result for objective_421aeb64:\n",
" date: 2022-07-22_15-19-25\n",
" done: true\n",
" experiment_id: a4b264c05660423ea645cef7a9cd38bb\n",
" experiment_tag: 7_activation=relu,height=-94.9771,steps=100,width=17.6810\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 0.559098189196817\n",
" neg_mean_loss: -0.559098189196817\n",
" node_ip: 127.0.0.1\n",
" pid: 45917\n",
" time_since_restore: 10.850210905075073\n",
" time_this_iter_s: 0.10677385330200195\n",
" time_total_s: 10.850210905075073\n",
" timestamp: 1658499565\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 421aeb64\n",
" warmup_time: 0.0031020641326904297\n",
" \n",
"Result for objective_421d81ee:\n",
" date: 2022-07-22_15-19-25\n",
" done: true\n",
" experiment_id: 836a62c82d6c467fb232414586872fd0\n",
" experiment_tag: 8_activation=tanh,height=-96.7222,steps=100,width=13.1718\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -8.596112689018389\n",
" neg_mean_loss: 8.596112689018389\n",
" node_ip: 127.0.0.1\n",
" pid: 45918\n",
" time_since_restore: 10.845621109008789\n",
" time_this_iter_s: 0.10427713394165039\n",
" time_total_s: 10.845621109008789\n",
" timestamp: 1658499565\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 421d81ee\n",
" warmup_time: 0.002989053726196289\n",
" \n"
]
}
],
2022-04-25 11:10:58 -07:00
"source": [
2022-07-24 18:53:57 +01:00
"tuner = tune.Tuner(\n",
2022-04-25 11:10:58 -07:00
" objective,\n",
2022-07-24 18:53:57 +01:00
" tune_config=tune.TuneConfig(\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" search_alg=algo,\n",
" num_samples=-1,\n",
" time_budget_s=time_budget_s,\n",
" ),\n",
" param_space={\"steps\": 100},\n",
")\n",
"results = tuner.fit()"
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 14,
2022-04-25 11:10:58 -07:00
"id": "48f06c5f",
"metadata": {},
2022-07-24 18:53:57 +01:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Best hyperparameters found were: {'steps': 100, 'width': 13.171830039895717, 'height': -96.72215542618497, 'activation': 'tanh'}\n"
]
}
],
2022-04-25 11:10:58 -07:00
"source": [
2022-07-24 18:53:57 +01:00
"print(\"Best hyperparameters found were: \", results.get_best_result().config)"
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "markdown",
"id": "47b906b7",
"metadata": {},
"source": [
"## Running Tune experiments with CFO\n",
"\n",
2022-05-13 15:39:18 +02:00
"This example demonstrates the usage of Frugal Optimization for Cost-related\n",
"Hyperparameters (CFO) with Ray Tune.\n",
2022-04-25 11:10:58 -07:00
"\n",
2022-05-13 15:39:18 +02:00
"We now define the search algorithm as built from `CFO`, constrained to a maximum of `4`\n",
"concurrent trials with a `ConcurrencyLimiter`."
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 15,
2022-04-25 11:10:58 -07:00
"id": "29f2c0be",
"metadata": {},
"outputs": [],
"source": [
"algo = CFO()\n",
"algo = ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "a89cf9bb",
"metadata": {},
"source": [
2022-05-13 15:39:18 +02:00
"The number of samples is the number of hyperparameter combinations that will be\n",
"tried out. This Tune run is set to `1000` samples.\n",
2022-04-25 11:10:58 -07:00
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 16,
2022-04-25 11:10:58 -07:00
"id": "3d15f10d",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 1000"
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 17,
2022-04-25 11:10:58 -07:00
"id": "137f3ec0",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# If 1000 samples take too long, you can reduce this number.\n",
"# We override this number here for our smoke tests.\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "cfa2e413",
"metadata": {},
"source": [
2022-05-13 15:39:18 +02:00
"Next we define a search space. The critical assumption is that the optimal\n",
"hyperparameters live within this space. Yet, if the space is very large, then\n",
"those hyperparameters may be difficult to find in a short amount of time."
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 18,
2022-04-25 11:10:58 -07:00
"id": "ab4fbcbe",
"metadata": {},
"outputs": [],
"source": [
"search_config = {\n",
" \"steps\": 100,\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
" \"activation\": tune.choice([\"relu, tanh\"])\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "b19390a2",
"metadata": {},
"source": [
2022-05-13 15:39:18 +02:00
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective`\n",
"by searching `search_config` via `algo`, `num_samples` times. This previous sentence\n",
"is fully characterizes the search problem we aim to solve. With this in mind,\n",
2022-07-24 18:53:57 +01:00
"notice how efficient it is to execute `tuner.fit()`."
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 19,
2022-04-25 11:10:58 -07:00
"id": "2acf19f5",
"metadata": {},
2022-07-24 18:53:57 +01:00
"outputs": [
{
"data": {
"text/html": [
"== Status ==<br>Current time: 2022-07-22 15:20:11 (running for 00:00:43.01)<br>Memory usage on this node: 10.0/16.0 GiB<br>Using FIFO scheduling algorithm.<br>Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/4.69 GiB heap, 0.0/2.0 GiB objects<br>Current best trial: 4d51b38c with mean_loss=-8.661424748129757 and parameters={'steps': 100, 'width': 15.124213652112319, 'height': -97.27768667042203, 'activation': 'relu, tanh'}<br>Result logdir: /Users/kai/ray_results/cfo_exp<br>Number of trials: 10/10 (10 TERMINATED)<br><table>\n",
"<thead>\n",
"<tr><th>Trial name </th><th>status </th><th>loc </th><th>activation </th><th style=\"text-align: right;\"> height</th><th style=\"text-align: right;\"> steps</th><th style=\"text-align: right;\"> width</th><th style=\"text-align: right;\"> loss</th><th style=\"text-align: right;\"> iter</th><th style=\"text-align: right;\"> total time (s)</th><th style=\"text-align: right;\"> iterations</th><th style=\"text-align: right;\"> neg_mean_loss</th></tr>\n",
"</thead>\n",
"<tbody>\n",
"<tr><td>objective_4bc281fe</td><td>TERMINATED</td><td>127.0.0.1:45958</td><td>relu, tanh </td><td style=\"text-align: right;\"> 29.4532</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 1.94864 </td><td style=\"text-align: right;\"> 4.43814</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7193</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -4.43814</td></tr>\n",
"<tr><td>objective_4d4ed536</td><td>TERMINATED</td><td>127.0.0.1:45963</td><td>relu, tanh </td><td style=\"text-align: right;\">-26.2681</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 7.25618 </td><td style=\"text-align: right;\">-1.48952</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.1932</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 1.48952</td></tr>\n",
"<tr><td>objective_4d5064aa</td><td>TERMINATED</td><td>127.0.0.1:45964</td><td>relu, tanh </td><td style=\"text-align: right;\">-46.7215</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">15.1097 </td><td style=\"text-align: right;\">-3.60574</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.1983</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 3.60574</td></tr>\n",
"<tr><td>objective_4d51b38c</td><td>TERMINATED</td><td>127.0.0.1:45965</td><td>relu, tanh </td><td style=\"text-align: right;\">-97.2777</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\">15.1242 </td><td style=\"text-align: right;\">-8.66142</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.1977</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 8.66142</td></tr>\n",
"<tr><td>objective_53b39e84</td><td>TERMINATED</td><td>127.0.0.1:45983</td><td>relu, tanh </td><td style=\"text-align: right;\"> 25.7812</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 1.77704 </td><td style=\"text-align: right;\"> 4.11597</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7964</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -4.11597</td></tr>\n",
"<tr><td>objective_558c6132</td><td>TERMINATED</td><td>127.0.0.1:45990</td><td>relu, tanh </td><td style=\"text-align: right;\"> 33.1252</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 2.12024 </td><td style=\"text-align: right;\"> 4.76727</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7659</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -4.76727</td></tr>\n",
"<tr><td>objective_558f27aa</td><td>TERMINATED</td><td>127.0.0.1:45991</td><td>relu, tanh </td><td style=\"text-align: right;\"> 40.929 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 0.0138032</td><td style=\"text-align: right;\">13.8907 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.785 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -13.8907 </td></tr>\n",
"<tr><td>objective_5592ca68</td><td>TERMINATED</td><td>127.0.0.1:45992</td><td>relu, tanh </td><td style=\"text-align: right;\"> 17.9775</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 3.88348 </td><td style=\"text-align: right;\"> 3.05125</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7802</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -3.05125</td></tr>\n",
"<tr><td>objective_5b999414</td><td>TERMINATED</td><td>127.0.0.1:46011</td><td>relu, tanh </td><td style=\"text-align: right;\"> 11.0354</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 3.27423 </td><td style=\"text-align: right;\"> 2.40281</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 12.6262</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -2.40281</td></tr>\n",
"<tr><td>objective_5d8462c2</td><td>TERMINATED</td><td>127.0.0.1:46018</td><td>relu, tanh </td><td style=\"text-align: right;\"> 24.9195</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 4.49273 </td><td style=\"text-align: right;\"> 3.71184</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7339</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -3.71184</td></tr>\n",
"</tbody>\n",
"</table><br><br>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_4bc281fe:\n",
" date: 2022-07-22_15-19-30\n",
" done: false\n",
" experiment_id: 129b24a058ee40fd991fb40844988057\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 13.945321626643842\n",
" neg_mean_loss: -13.945321626643842\n",
" node_ip: 127.0.0.1\n",
" pid: 45958\n",
" time_since_restore: 0.10440444946289062\n",
" time_this_iter_s: 0.10440444946289062\n",
" time_total_s: 0.10440444946289062\n",
" timestamp: 1658499570\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 4bc281fe\n",
" warmup_time: 0.003987789154052734\n",
" \n",
"Result for objective_4d4ed536:\n",
" date: 2022-07-22_15-19-33\n",
" done: false\n",
" experiment_id: 6c18c359d1574792ac99ed37265d5d87\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 8.373189005362395\n",
" neg_mean_loss: -8.373189005362395\n",
" node_ip: 127.0.0.1\n",
" pid: 45963\n",
" time_since_restore: 0.10463309288024902\n",
" time_this_iter_s: 0.10463309288024902\n",
" time_total_s: 0.10463309288024902\n",
" timestamp: 1658499573\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 4d4ed536\n",
" warmup_time: 0.0028281211853027344\n",
" \n",
"Result for objective_4d51b38c:\n",
" date: 2022-07-22_15-19-33\n",
" done: false\n",
" experiment_id: 5c9cf6fc35c4411b8ec13e0b90b791a4\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 1.2722313329577961\n",
" neg_mean_loss: -1.2722313329577961\n",
" node_ip: 127.0.0.1\n",
" pid: 45965\n",
" time_since_restore: 0.10346817970275879\n",
" time_this_iter_s: 0.10346817970275879\n",
" time_total_s: 0.10346817970275879\n",
" timestamp: 1658499573\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 4d51b38c\n",
" warmup_time: 0.0027761459350585938\n",
" \n",
"Result for objective_4d5064aa:\n",
" date: 2022-07-22_15-19-33\n",
" done: false\n",
" experiment_id: 93ed3aa2df8c4f71b0b1e9437c062ccf\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 6.327848408616047\n",
" neg_mean_loss: -6.327848408616047\n",
" node_ip: 127.0.0.1\n",
" pid: 45964\n",
" time_since_restore: 0.10395598411560059\n",
" time_this_iter_s: 0.10395598411560059\n",
" time_total_s: 0.10395598411560059\n",
" timestamp: 1658499573\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 4d5064aa\n",
" warmup_time: 0.002546072006225586\n",
" \n",
"Result for objective_4bc281fe:\n",
" date: 2022-07-22_15-19-35\n",
" done: false\n",
" experiment_id: 129b24a058ee40fd991fb40844988057\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 4.9297078000588614\n",
" neg_mean_loss: -4.9297078000588614\n",
" node_ip: 127.0.0.1\n",
" pid: 45958\n",
" time_since_restore: 5.122281312942505\n",
" time_this_iter_s: 0.10893416404724121\n",
" time_total_s: 5.122281312942505\n",
" timestamp: 1658499575\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 4bc281fe\n",
" warmup_time: 0.003987789154052734\n",
" \n",
"Result for objective_4d4ed536:\n",
" date: 2022-07-22_15-19-38\n",
" done: false\n",
" experiment_id: 6c18c359d1574792ac99ed37265d5d87\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -1.3419437208788234\n",
" neg_mean_loss: 1.3419437208788234\n",
" node_ip: 127.0.0.1\n",
" pid: 45963\n",
" time_since_restore: 5.132510185241699\n",
" time_this_iter_s: 0.10718894004821777\n",
" time_total_s: 5.132510185241699\n",
" timestamp: 1658499578\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 4d4ed536\n",
" warmup_time: 0.0028281211853027344\n",
" \n",
"Result for objective_4d51b38c:\n",
" date: 2022-07-22_15-19-38\n",
" done: false\n",
" experiment_id: 5c9cf6fc35c4411b8ec13e0b90b791a4\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -8.58904124947806\n",
" neg_mean_loss: 8.58904124947806\n",
" node_ip: 127.0.0.1\n",
" pid: 45965\n",
" time_since_restore: 5.1296000480651855\n",
" time_this_iter_s: 0.10642719268798828\n",
" time_total_s: 5.1296000480651855\n",
" timestamp: 1658499578\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 4d51b38c\n",
" warmup_time: 0.0027761459350585938\n",
" \n",
"Result for objective_4d5064aa:\n",
" date: 2022-07-22_15-19-38\n",
" done: false\n",
" experiment_id: 93ed3aa2df8c4f71b0b1e9437c062ccf\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -3.5332931133029772\n",
" neg_mean_loss: 3.5332931133029772\n",
" node_ip: 127.0.0.1\n",
" pid: 45964\n",
" time_since_restore: 5.1527419090271\n",
" time_this_iter_s: 0.10794281959533691\n",
" time_total_s: 5.1527419090271\n",
" timestamp: 1658499578\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 4d5064aa\n",
" warmup_time: 0.002546072006225586\n",
" \n",
"Result for objective_4bc281fe:\n",
" date: 2022-07-22_15-19-41\n",
" done: false\n",
" experiment_id: 129b24a058ee40fd991fb40844988057\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 4.462994199505408\n",
" neg_mean_loss: -4.462994199505408\n",
" node_ip: 127.0.0.1\n",
" pid: 45958\n",
" time_since_restore: 10.180925369262695\n",
" time_this_iter_s: 0.10581827163696289\n",
" time_total_s: 10.180925369262695\n",
" timestamp: 1658499581\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 4bc281fe\n",
" warmup_time: 0.003987789154052734\n",
" \n",
"Result for objective_4bc281fe:\n",
" date: 2022-07-22_15-19-41\n",
" done: true\n",
" experiment_id: 129b24a058ee40fd991fb40844988057\n",
" experiment_tag: 1_activation=relu_tanh,height=29.4532,steps=100,width=1.9486\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 4.438137591322411\n",
" neg_mean_loss: -4.438137591322411\n",
" node_ip: 127.0.0.1\n",
" pid: 45958\n",
" time_since_restore: 10.71930718421936\n",
" time_this_iter_s: 0.10692596435546875\n",
" time_total_s: 10.71930718421936\n",
" timestamp: 1658499581\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 4bc281fe\n",
" warmup_time: 0.003987789154052734\n",
" \n",
"Result for objective_4d4ed536:\n",
" date: 2022-07-22_15-19-43\n",
" done: false\n",
" experiment_id: 6c18c359d1574792ac99ed37265d5d87\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: -1.48078832328251\n",
" neg_mean_loss: 1.48078832328251\n",
" node_ip: 127.0.0.1\n",
" pid: 45963\n",
" time_since_restore: 10.033550262451172\n",
" time_this_iter_s: 0.10465312004089355\n",
" time_total_s: 10.033550262451172\n",
" timestamp: 1658499583\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 4d4ed536\n",
" warmup_time: 0.0028281211853027344\n",
" \n",
"Result for objective_4d51b38c:\n",
" date: 2022-07-22_15-19-43\n",
" done: false\n",
" experiment_id: 5c9cf6fc35c4411b8ec13e0b90b791a4\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: -8.657174711294605\n",
" neg_mean_loss: 8.657174711294605\n",
" node_ip: 127.0.0.1\n",
" pid: 45965\n",
" time_since_restore: 10.026285171508789\n",
" time_this_iter_s: 0.10726714134216309\n",
" time_total_s: 10.026285171508789\n",
" timestamp: 1658499583\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 4d51b38c\n",
" warmup_time: 0.0027761459350585938\n",
" \n",
"Result for objective_4d5064aa:\n",
" date: 2022-07-22_15-19-43\n",
" done: false\n",
" experiment_id: 93ed3aa2df8c4f71b0b1e9437c062ccf\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: -3.601490481891755\n",
" neg_mean_loss: 3.601490481891755\n",
" node_ip: 127.0.0.1\n",
" pid: 45964\n",
" time_since_restore: 10.049538135528564\n",
" time_this_iter_s: 0.10654830932617188\n",
" time_total_s: 10.049538135528564\n",
" timestamp: 1658499583\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 4d5064aa\n",
" warmup_time: 0.002546072006225586\n",
" \n",
"Result for objective_53b39e84:\n",
" date: 2022-07-22_15-19-44\n",
" done: false\n",
" experiment_id: abfeeaaa7eae4ad798c655bed2d1e223\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 13.57812082128205\n",
" neg_mean_loss: -13.57812082128205\n",
" node_ip: 127.0.0.1\n",
" pid: 45983\n",
" time_since_restore: 0.10458230972290039\n",
" time_this_iter_s: 0.10458230972290039\n",
" time_total_s: 0.10458230972290039\n",
" timestamp: 1658499584\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 53b39e84\n",
" warmup_time: 0.002878904342651367\n",
" \n",
"Result for objective_4d4ed536:\n",
" date: 2022-07-22_15-19-44\n",
" done: true\n",
" experiment_id: 6c18c359d1574792ac99ed37265d5d87\n",
" experiment_tag: 2_activation=relu_tanh,height=-26.2681,steps=100,width=7.2562\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -1.489516678620475\n",
" neg_mean_loss: 1.489516678620475\n",
" node_ip: 127.0.0.1\n",
" pid: 45963\n",
" time_since_restore: 11.193175077438354\n",
" time_this_iter_s: 0.11231780052185059\n",
" time_total_s: 11.193175077438354\n",
" timestamp: 1658499584\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 4d4ed536\n",
" warmup_time: 0.0028281211853027344\n",
" \n",
"Result for objective_4d51b38c:\n",
" date: 2022-07-22_15-19-44\n",
" done: true\n",
" experiment_id: 5c9cf6fc35c4411b8ec13e0b90b791a4\n",
" experiment_tag: 4_activation=relu_tanh,height=-97.2777,steps=100,width=15.1242\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -8.661424748129757\n",
" neg_mean_loss: 8.661424748129757\n",
" node_ip: 127.0.0.1\n",
" pid: 45965\n",
" time_since_restore: 11.19774603843689\n",
" time_this_iter_s: 0.10802578926086426\n",
" time_total_s: 11.19774603843689\n",
" timestamp: 1658499584\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 4d51b38c\n",
" warmup_time: 0.0027761459350585938\n",
" \n",
"Result for objective_4d5064aa:\n",
" date: 2022-07-22_15-19-44\n",
" done: true\n",
" experiment_id: 93ed3aa2df8c4f71b0b1e9437c062ccf\n",
" experiment_tag: 3_activation=relu_tanh,height=-46.7215,steps=100,width=15.1097\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -3.6057445346106167\n",
" neg_mean_loss: 3.6057445346106167\n",
" node_ip: 127.0.0.1\n",
" pid: 45964\n",
" time_since_restore: 11.198285102844238\n",
" time_this_iter_s: 0.10615801811218262\n",
" time_total_s: 11.198285102844238\n",
" timestamp: 1658499584\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 4d5064aa\n",
" warmup_time: 0.002546072006225586\n",
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_558c6132:\n",
" date: 2022-07-22_15-19-47\n",
" done: false\n",
" experiment_id: 98df2ca60c5f4a33a3da6870c3d8db4c\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 14.31252243200564\n",
" neg_mean_loss: -14.31252243200564\n",
" node_ip: 127.0.0.1\n",
" pid: 45990\n",
" time_since_restore: 0.10317206382751465\n",
" time_this_iter_s: 0.10317206382751465\n",
" time_total_s: 0.10317206382751465\n",
" timestamp: 1658499587\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 558c6132\n",
" warmup_time: 0.003049135208129883\n",
" \n",
"Result for objective_558f27aa:\n",
" date: 2022-07-22_15-19-47\n",
" done: false\n",
" experiment_id: 73def1ffc7f443669328b88695453248\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 15.092898207127158\n",
" neg_mean_loss: -15.092898207127158\n",
" node_ip: 127.0.0.1\n",
" pid: 45991\n",
" time_since_restore: 0.10495114326477051\n",
" time_this_iter_s: 0.10495114326477051\n",
" time_total_s: 0.10495114326477051\n",
" timestamp: 1658499587\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 558f27aa\n",
" warmup_time: 0.0029478073120117188\n",
" \n",
"Result for objective_5592ca68:\n",
" date: 2022-07-22_15-19-47\n",
" done: false\n",
" experiment_id: 1de409fb58a84c3bba522620ec7e1fe4\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 12.79774504616053\n",
" neg_mean_loss: -12.79774504616053\n",
" node_ip: 127.0.0.1\n",
" pid: 45992\n",
" time_since_restore: 0.10448837280273438\n",
" time_this_iter_s: 0.10448837280273438\n",
" time_total_s: 0.10448837280273438\n",
" timestamp: 1658499587\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 5592ca68\n",
" warmup_time: 0.0027618408203125\n",
" \n",
"Result for objective_53b39e84:\n",
" date: 2022-07-22_15-19-49\n",
" done: false\n",
" experiment_id: abfeeaaa7eae4ad798c655bed2d1e223\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 46\n",
" iterations_since_restore: 47\n",
" mean_loss: 4.668111815725785\n",
" neg_mean_loss: -4.668111815725785\n",
" node_ip: 127.0.0.1\n",
" pid: 45983\n",
" time_since_restore: 5.113715171813965\n",
" time_this_iter_s: 0.10550904273986816\n",
" time_total_s: 5.113715171813965\n",
" timestamp: 1658499589\n",
" timesteps_since_restore: 0\n",
" training_iteration: 47\n",
" trial_id: 53b39e84\n",
" warmup_time: 0.002878904342651367\n",
" \n",
"Result for objective_558c6132:\n",
" date: 2022-07-22_15-19-52\n",
" done: false\n",
" experiment_id: 98df2ca60c5f4a33a3da6870c3d8db4c\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 5.2245036670526375\n",
" neg_mean_loss: -5.2245036670526375\n",
" node_ip: 127.0.0.1\n",
" pid: 45990\n",
" time_since_restore: 5.1552488803863525\n",
" time_this_iter_s: 0.10927176475524902\n",
" time_total_s: 5.1552488803863525\n",
" timestamp: 1658499592\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 558c6132\n",
" warmup_time: 0.003049135208129883\n",
" \n",
"Result for objective_558f27aa:\n",
" date: 2022-07-22_15-19-52\n",
" done: false\n",
" experiment_id: 73def1ffc7f443669328b88695453248\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 14.483670237907232\n",
" neg_mean_loss: -14.483670237907232\n",
" node_ip: 127.0.0.1\n",
" pid: 45991\n",
" time_since_restore: 5.174212217330933\n",
" time_this_iter_s: 0.10701107978820801\n",
" time_total_s: 5.174212217330933\n",
" timestamp: 1658499592\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 558f27aa\n",
" warmup_time: 0.0029478073120117188\n",
" \n",
"Result for objective_5592ca68:\n",
" date: 2022-07-22_15-19-52\n",
" done: false\n",
" experiment_id: 1de409fb58a84c3bba522620ec7e1fe4\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 3.3171620341364583\n",
" neg_mean_loss: -3.3171620341364583\n",
" node_ip: 127.0.0.1\n",
" pid: 45992\n",
" time_since_restore: 5.171269178390503\n",
" time_this_iter_s: 0.10331320762634277\n",
" time_total_s: 5.171269178390503\n",
" timestamp: 1658499592\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 5592ca68\n",
" warmup_time: 0.0027618408203125\n",
" \n",
"Result for objective_53b39e84:\n",
" date: 2022-07-22_15-19-54\n",
" done: false\n",
" experiment_id: abfeeaaa7eae4ad798c655bed2d1e223\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: 4.148686061272061\n",
" neg_mean_loss: -4.148686061272061\n",
" node_ip: 127.0.0.1\n",
" pid: 45983\n",
" time_since_restore: 10.137071132659912\n",
" time_this_iter_s: 0.10901093482971191\n",
" time_total_s: 10.137071132659912\n",
" timestamp: 1658499594\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 53b39e84\n",
" warmup_time: 0.002878904342651367\n",
" \n",
"Result for objective_53b39e84:\n",
" date: 2022-07-22_15-19-54\n",
" done: true\n",
" experiment_id: abfeeaaa7eae4ad798c655bed2d1e223\n",
" experiment_tag: 5_activation=relu_tanh,height=25.7812,steps=100,width=1.7770\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 4.1159662035734215\n",
" neg_mean_loss: -4.1159662035734215\n",
" node_ip: 127.0.0.1\n",
" pid: 45983\n",
" time_since_restore: 10.796404361724854\n",
" time_this_iter_s: 0.10543107986450195\n",
" time_total_s: 10.796404361724854\n",
" timestamp: 1658499594\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 53b39e84\n",
" warmup_time: 0.002878904342651367\n",
" \n",
"Result for objective_5b999414:\n",
" date: 2022-07-22_15-19-57\n",
" done: false\n",
" experiment_id: 5548713c4f5046c0a44e1946db840d73\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 12.103540718376786\n",
" neg_mean_loss: -12.103540718376786\n",
" node_ip: 127.0.0.1\n",
" pid: 46011\n",
" time_since_restore: 0.10435795783996582\n",
" time_this_iter_s: 0.10435795783996582\n",
" time_total_s: 0.10435795783996582\n",
" timestamp: 1658499597\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 5b999414\n",
" warmup_time: 0.002862691879272461\n",
" \n",
"Result for objective_558c6132:\n",
" date: 2022-07-22_15-19-57\n",
" done: false\n",
" experiment_id: 98df2ca60c5f4a33a3da6870c3d8db4c\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 4.790299222921972\n",
" neg_mean_loss: -4.790299222921972\n",
" node_ip: 127.0.0.1\n",
" pid: 45990\n",
" time_since_restore: 10.228116989135742\n",
" time_this_iter_s: 0.146165132522583\n",
" time_total_s: 10.228116989135742\n",
" timestamp: 1658499597\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 558c6132\n",
" warmup_time: 0.003049135208129883\n",
" \n",
"Result for objective_558f27aa:\n",
" date: 2022-07-22_15-19-57\n",
" done: false\n",
" experiment_id: 73def1ffc7f443669328b88695453248\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 13.944411303108408\n",
" neg_mean_loss: -13.944411303108408\n",
" node_ip: 127.0.0.1\n",
" pid: 45991\n",
" time_since_restore: 10.248456001281738\n",
" time_this_iter_s: 0.13899707794189453\n",
" time_total_s: 10.248456001281738\n",
" timestamp: 1658499597\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 558f27aa\n",
" warmup_time: 0.0029478073120117188\n",
" \n",
"Result for objective_5592ca68:\n",
" date: 2022-07-22_15-19-57\n",
" done: false\n",
" experiment_id: 1de409fb58a84c3bba522620ec7e1fe4\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 3.064378230421662\n",
" neg_mean_loss: -3.064378230421662\n",
" node_ip: 127.0.0.1\n",
" pid: 45992\n",
" time_since_restore: 10.244185209274292\n",
" time_this_iter_s: 0.16223502159118652\n",
" time_total_s: 10.244185209274292\n",
" timestamp: 1658499597\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 5592ca68\n",
" warmup_time: 0.0027618408203125\n",
" \n",
"Result for objective_558c6132:\n",
" date: 2022-07-22_15-19-58\n",
" done: true\n",
" experiment_id: 98df2ca60c5f4a33a3da6870c3d8db4c\n",
" experiment_tag: 6_activation=relu_tanh,height=33.1252,steps=100,width=2.1202\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 4.767266385536331\n",
" neg_mean_loss: -4.767266385536331\n",
" node_ip: 127.0.0.1\n",
" pid: 45990\n",
" time_since_restore: 10.765881776809692\n",
" time_this_iter_s: 0.10750794410705566\n",
" time_total_s: 10.765881776809692\n",
" timestamp: 1658499598\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 558c6132\n",
" warmup_time: 0.003049135208129883\n",
" \n",
"Result for objective_558f27aa:\n",
" date: 2022-07-22_15-19-58\n",
" done: true\n",
" experiment_id: 73def1ffc7f443669328b88695453248\n",
" experiment_tag: 7_activation=relu_tanh,height=40.9290,steps=100,width=0.0138\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 13.890665978269285\n",
" neg_mean_loss: -13.890665978269285\n",
" node_ip: 127.0.0.1\n",
" pid: 45991\n",
" time_since_restore: 10.784975051879883\n",
" time_this_iter_s: 0.10729503631591797\n",
" time_total_s: 10.784975051879883\n",
" timestamp: 1658499598\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 558f27aa\n",
" warmup_time: 0.0029478073120117188\n",
" \n",
"Result for objective_5592ca68:\n",
" date: 2022-07-22_15-19-58\n",
" done: true\n",
" experiment_id: 1de409fb58a84c3bba522620ec7e1fe4\n",
" experiment_tag: 8_activation=relu_tanh,height=17.9775,steps=100,width=3.8835\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 3.0512532903599245\n",
" neg_mean_loss: -3.0512532903599245\n",
" node_ip: 127.0.0.1\n",
" pid: 45992\n",
" time_since_restore: 10.780152082443237\n",
" time_this_iter_s: 0.10820770263671875\n",
" time_total_s: 10.780152082443237\n",
" timestamp: 1658499598\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 5592ca68\n",
" warmup_time: 0.0027618408203125\n",
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_5d8462c2:\n",
" date: 2022-07-22_15-20-00\n",
" done: false\n",
" experiment_id: 37975742a6894daeb580d222effadf3c\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 13.491949373944276\n",
" neg_mean_loss: -13.491949373944276\n",
" node_ip: 127.0.0.1\n",
" pid: 46018\n",
" time_since_restore: 0.1021728515625\n",
" time_this_iter_s: 0.1021728515625\n",
" time_total_s: 0.1021728515625\n",
" timestamp: 1658499600\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 5d8462c2\n",
" warmup_time: 0.0030527114868164062\n",
" \n",
"Result for objective_5b999414:\n",
" date: 2022-07-22_15-20-02\n",
" done: false\n",
" experiment_id: 5548713c4f5046c0a44e1946db840d73\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 29\n",
" iterations_since_restore: 30\n",
" mean_loss: 3.0563505542773735\n",
" neg_mean_loss: -3.0563505542773735\n",
" node_ip: 127.0.0.1\n",
" pid: 46011\n",
" time_since_restore: 5.116016149520874\n",
" time_this_iter_s: 0.1060340404510498\n",
" time_total_s: 5.116016149520874\n",
" timestamp: 1658499602\n",
" timesteps_since_restore: 0\n",
" training_iteration: 30\n",
" trial_id: 5b999414\n",
" warmup_time: 0.002862691879272461\n",
" \n",
"Result for objective_5d8462c2:\n",
" date: 2022-07-22_15-20-05\n",
" done: false\n",
" experiment_id: 37975742a6894daeb580d222effadf3c\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 3.9441144421537513\n",
" neg_mean_loss: -3.9441144421537513\n",
" node_ip: 127.0.0.1\n",
" pid: 46018\n",
" time_since_restore: 5.143904685974121\n",
" time_this_iter_s: 0.1046457290649414\n",
" time_total_s: 5.143904685974121\n",
" timestamp: 1658499605\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 5d8462c2\n",
" warmup_time: 0.0030527114868164062\n",
" \n",
"Result for objective_5b999414:\n",
" date: 2022-07-22_15-20-07\n",
" done: false\n",
" experiment_id: 5548713c4f5046c0a44e1946db840d73\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 76\n",
" iterations_since_restore: 77\n",
" mean_loss: 2.4898772989369\n",
" neg_mean_loss: -2.4898772989369\n",
" node_ip: 127.0.0.1\n",
" pid: 46011\n",
" time_since_restore: 10.162328004837036\n",
" time_this_iter_s: 0.10597825050354004\n",
" time_total_s: 10.162328004837036\n",
" timestamp: 1658499607\n",
" timesteps_since_restore: 0\n",
" training_iteration: 77\n",
" trial_id: 5b999414\n",
" warmup_time: 0.002862691879272461\n",
" \n",
"Result for objective_5b999414:\n",
" date: 2022-07-22_15-20-09\n",
" done: true\n",
" experiment_id: 5548713c4f5046c0a44e1946db840d73\n",
" experiment_tag: 9_activation=relu_tanh,height=11.0354,steps=100,width=3.2742\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 2.4028084118568103\n",
" neg_mean_loss: -2.4028084118568103\n",
" node_ip: 127.0.0.1\n",
" pid: 46011\n",
" time_since_restore: 12.626188039779663\n",
" time_this_iter_s: 0.10697293281555176\n",
" time_total_s: 12.626188039779663\n",
" timestamp: 1658499609\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 5b999414\n",
" warmup_time: 0.002862691879272461\n",
" \n",
"Result for objective_5d8462c2:\n",
" date: 2022-07-22_15-20-10\n",
" done: false\n",
" experiment_id: 37975742a6894daeb580d222effadf3c\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 3.723261470545891\n",
" neg_mean_loss: -3.723261470545891\n",
" node_ip: 127.0.0.1\n",
" pid: 46018\n",
" time_since_restore: 10.198927879333496\n",
" time_this_iter_s: 0.10776281356811523\n",
" time_total_s: 10.198927879333496\n",
" timestamp: 1658499610\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 5d8462c2\n",
" warmup_time: 0.0030527114868164062\n",
" \n",
"Result for objective_5d8462c2:\n",
" date: 2022-07-22_15-20-11\n",
" done: true\n",
" experiment_id: 37975742a6894daeb580d222effadf3c\n",
" experiment_tag: 10_activation=relu_tanh,height=24.9195,steps=100,width=4.4927\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 3.711835922326217\n",
" neg_mean_loss: -3.711835922326217\n",
" node_ip: 127.0.0.1\n",
" pid: 46018\n",
" time_since_restore: 10.733852863311768\n",
" time_this_iter_s: 0.10374593734741211\n",
" time_total_s: 10.733852863311768\n",
" timestamp: 1658499611\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 5d8462c2\n",
" warmup_time: 0.0030527114868164062\n",
" \n"
]
}
],
2022-04-25 11:10:58 -07:00
"source": [
2022-07-24 18:53:57 +01:00
"tuner = tune.Tuner(\n",
2022-04-25 11:10:58 -07:00
" objective,\n",
2022-07-24 18:53:57 +01:00
" tune_config=tune.TuneConfig(\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" search_alg=algo,\n",
" num_samples=num_samples,\n",
" ),\n",
" param_space=search_config,\n",
")\n",
"results = tuner.fit()"
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "markdown",
"id": "71e6f6ec",
"metadata": {},
"source": [
"Here are the hyperparameters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 20,
2022-04-25 11:10:58 -07:00
"id": "b657dccf",
"metadata": {},
2022-07-24 18:53:57 +01:00
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Best hyperparameters found were: {'steps': 100, 'width': 15.124213652112319, 'height': -97.27768667042203, 'activation': 'relu, tanh'}\n"
]
}
],
2022-04-25 11:10:58 -07:00
"source": [
2022-07-24 18:53:57 +01:00
"print(\"Best hyperparameters found were: \", results.get_best_result().config)"
2022-04-25 11:10:58 -07:00
]
},
{
"cell_type": "code",
2022-07-24 18:53:57 +01:00
"execution_count": 21,
2022-04-25 11:10:58 -07:00
"id": "505995af",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.shutdown()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
2022-07-24 18:53:57 +01:00
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
},
2022-04-25 11:10:58 -07:00
"orphan": true
},
"nbformat": 4,
"nbformat_minor": 5
2022-07-09 19:47:21 -07:00
}
