hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-10 05:16:49 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions
ray/doc/source/tune/examples/nevergrad_example.ipynb
scv119 3edfc78ee2 update version number to 2.0.0rc0
2022-07-27 18:43:27 +00:00

2233 lines
93 KiB
Text
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"cells": [
{
"cell_type": "markdown",
"id": "5a1d28f3",
"metadata": {},
"source": [
"# Running Tune experiments with Nevergrad\n",
"\n",
"In this tutorial we introduce Nevergrad, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with Nevergrad and, as a result, allow you to seamlessly scale up a Nevergrad optimization process - without sacrificing performance.\n",
"\n",
"Nevergrad provides gradient/derivative-free optimization able to handle noise over the objective landscape, including evolutionary, bandit, and Bayesian optimization algorithms. Nevergrad internally supports search spaces which are continuous, discrete or a mixture of thereof. It also provides a library of functions on which to test the optimization algorithms and compare with other benchmarks.\n",
"\n",
"In this example we minimize a simple objective to briefly demonstrate the usage of Nevergrad with Ray Tune via `NevergradSearch`. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `nevergrad==0.4.3.post7` library is installed. To learn more, please refer to [Nevergrad website](https://github.com/facebookresearch/nevergrad)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "5ab54f85",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting nevergrad==0.4.3.post7\n",
" Using cached nevergrad-0.4.3.post7-py3-none-any.whl (400 kB)\n",
"Collecting cma>=2.6.0\n",
" Downloading cma-3.2.2-py2.py3-none-any.whl (249 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m249.1/249.1 kB\u001b[0m \u001b[31m3.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: bayesian-optimization>=1.2.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from nevergrad==0.4.3.post7) (1.2.0)\n",
"Requirement already satisfied: numpy>=1.15.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from nevergrad==0.4.3.post7) (1.21.6)\n",
"Requirement already satisfied: typing-extensions>=3.6.6 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from nevergrad==0.4.3.post7) (4.1.1)\n",
"Requirement already satisfied: scipy>=0.14.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from bayesian-optimization>=1.2.0->nevergrad==0.4.3.post7) (1.4.1)\n",
"Requirement already satisfied: scikit-learn>=0.18.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from bayesian-optimization>=1.2.0->nevergrad==0.4.3.post7) (0.24.2)\n",
"Requirement already satisfied: threadpoolctl>=2.0.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from scikit-learn>=0.18.0->bayesian-optimization>=1.2.0->nevergrad==0.4.3.post7) (3.0.0)\n",
"Requirement already satisfied: joblib>=0.11 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from scikit-learn>=0.18.0->bayesian-optimization>=1.2.0->nevergrad==0.4.3.post7) (1.1.0)\n",
"Installing collected packages: cma, nevergrad\n",
"Successfully installed cma-3.2.2 nevergrad-0.4.3.post7\n",
"\u001b[33mWARNING: There was an error checking the latest version of pip.\u001b[0m\u001b[33m\n",
"\u001b[0m"
]
}
],
"source": [
"# !pip install ray[tune]\n",
"!pip install nevergrad==0.4.3.post7 "
]
},
{
"cell_type": "markdown",
"id": "66cb8206",
"metadata": {},
"source": [
"Click below to see all the imports we need for this example.\n",
"You can also launch directly into a Binder instance to run this notebook yourself.\n",
"Just click on the rocket symbol at the top of the navigation."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "1f6d7a31",
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/nevergrad/optimization/differentialevolution.py:107: InefficientSettingsWarning: DE algorithms are inefficient with budget < 60\n",
" \"DE algorithms are inefficient with budget < 60\", base.errors.InefficientSettingsWarning\n"
]
}
],
"source": [
"import time\n",
"\n",
"import ray\n",
"import nevergrad as ng\n",
"from ray import tune\n",
"from ray.air import session\n",
"from ray.tune.search import ConcurrencyLimiter\n",
"from ray.tune.search.nevergrad import NevergradSearch"
]
},
{
"cell_type": "markdown",
"id": "41f2c881",
"metadata": {},
"source": [
"Let's start by defining a simple evaluation function.\n",
"We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
"This setup assumes that we're running multiple `step`s of an experiment and try to tune two hyperparameters,\n",
"namely `width` and `height`, and `activation`."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "271bd5c5",
"metadata": {},
"outputs": [],
"source": [
"def evaluate(step, width, height, activation):\n",
" time.sleep(0.1)\n",
" activation_boost = 10 if activation==\"relu\" else 1\n",
" return (0.1 + width * step / 100) ** (-1) + height * 0.1 + activation_boost"
]
},
{
"cell_type": "markdown",
"id": "f060ea83",
"metadata": {},
"source": [
"Next, our `objective` function takes a Tune `config`, evaluates the `score` of your experiment in a training loop,\n",
"and uses `session.report` to report the `score` back to Tune."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "c71fc423",
"metadata": {},
"outputs": [],
"source": [
"def objective(config):\n",
" for step in range(config[\"steps\"]):\n",
" score = evaluate(step, config[\"width\"], config[\"height\"], config[\"activation\"])\n",
" session.report({\"iterations\": step, \"mean_loss\": score})"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "619263ee",
"metadata": {
"lines_to_next_cell": 0,
"tags": [
"remove-cell"
]
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:ray._private.services:View the Ray dashboard at \u001b[1m\u001b[32mhttp://127.0.0.1:8266\u001b[39m\u001b[22m\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
" <div style=\"margin-left: 50px;display: flex;flex-direction: row;align-items: center\">\n",
" <h3 style=\"color: var(--jp-ui-font-color0)\">Ray</h3>\n",
" <svg version=\"1.1\" id=\"ray\" width=\"3em\" viewBox=\"0 0 144.5 144.6\" style=\"margin-left: 3em;margin-right: 3em\">\n",
" <g id=\"layer-1\">\n",
" <path fill=\"#00a2e9\" class=\"st0\" d=\"M97.3,77.2c-3.8-1.1-6.2,0.9-8.3,5.1c-3.5,6.8-9.9,9.9-17.4,9.6S58,88.1,54.8,81.2c-1.4-3-3-4-6.3-4.1\n",
" c-5.6-0.1-9.9,0.1-13.1,6.4c-3.8,7.6-13.6,10.2-21.8,7.6C5.2,88.4-0.4,80.5,0,71.7c0.1-8.4,5.7-15.8,13.8-18.2\n",
" c8.4-2.6,17.5,0.7,22.3,8c1.3,1.9,1.3,5.2,3.6,5.6c3.9,0.6,8,0.2,12,0.2c1.8,0,1.9-1.6,2.4-2.8c3.5-7.8,9.7-11.8,18-11.9\n",
" c8.2-0.1,14.4,3.9,17.8,11.4c1.3,2.8,2.9,3.6,5.7,3.3c1-0.1,2,0.1,3,0c2.8-0.5,6.4,1.7,8.1-2.7s-2.3-5.5-4.1-7.5\n",
" c-5.1-5.7-10.9-10.8-16.1-16.3C84,38,81.9,37.1,78,38.3C66.7,42,56.2,35.7,53,24.1C50.3,14,57.3,2.8,67.7,0.5\n",
" C78.4-2,89,4.7,91.5,15.3c0.1,0.3,0.1,0.5,0.2,0.8c0.7,3.4,0.7,6.9-0.8,9.8c-1.7,3.2-0.8,5,1.5,7.2c6.7,6.5,13.3,13,19.8,19.7\n",
" c1.8,1.8,3,2.1,5.5,1.2c9.1-3.4,17.9-0.6,23.4,7c4.8,6.9,4.6,16.1-0.4,22.9c-5.4,7.2-14.2,9.9-23.1,6.5c-2.3-0.9-3.5-0.6-5.1,1.1\n",
" c-6.7,6.9-13.6,13.7-20.5,20.4c-1.8,1.8-2.5,3.2-1.4,5.9c3.5,8.7,0.3,18.6-7.7,23.6c-7.9,5-18.2,3.8-24.8-2.9\n",
" c-6.4-6.4-7.4-16.2-2.5-24.3c4.9-7.8,14.5-11,23.1-7.8c3,1.1,4.7,0.5,6.9-1.7C91.7,98.4,98,92.3,104.2,86c1.6-1.6,4.1-2.7,2.6-6.2\n",
" c-1.4-3.3-3.8-2.5-6.2-2.6C99.8,77.2,98.9,77.2,97.3,77.2z M72.1,29.7c5.5,0.1,9.9-4.3,10-9.8c0-0.1,0-0.2,0-0.3\n",
" C81.8,14,77,9.8,71.5,10.2c-5,0.3-9,4.2-9.3,9.2c-0.2,5.5,4,10.1,9.5,10.3C71.8,29.7,72,29.7,72.1,29.7z M72.3,62.3\n",
" c-5.4-0.1-9.9,4.2-10.1,9.7c0,0.2,0,0.3,0,0.5c0.2,5.4,4.5,9.7,9.9,10c5.1,0.1,9.9-4.7,10.1-9.8c0.2-5.5-4-10-9.5-10.3\n",
" C72.6,62.3,72.4,62.3,72.3,62.3z M115,72.5c0.1,5.4,4.5,9.7,9.8,9.9c5.6-0.2,10-4.8,10-10.4c-0.2-5.4-4.6-9.7-10-9.7\n",
" c-5.3-0.1-9.8,4.2-9.9,9.5C115,72.1,115,72.3,115,72.5z M19.5,62.3c-5.4,0.1-9.8,4.4-10,9.8c-0.1,5.1,5.2,10.4,10.2,10.3\n",
" c5.6-0.2,10-4.9,9.8-10.5c-0.1-5.4-4.5-9.7-9.9-9.6C19.6,62.3,19.5,62.3,19.5,62.3z M71.8,134.6c5.9,0.2,10.3-3.9,10.4-9.6\n",
" c0.5-5.5-3.6-10.4-9.1-10.8c-5.5-0.5-10.4,3.6-10.8,9.1c0,0.5,0,0.9,0,1.4c-0.2,5.3,4,9.8,9.3,10\n",
" C71.6,134.6,71.7,134.6,71.8,134.6z\"/>\n",
" </g>\n",
" </svg>\n",
" <table>\n",
" <tr>\n",
" <td style=\"text-align: left\"><b>Python version:</b></td>\n",
" <td style=\"text-align: left\"><b>3.7.7</b></td>\n",
" </tr>\n",
" <tr>\n",
" <td style=\"text-align: left\"><b>Ray version:</b></td>\n",
" <td style=\"text-align: left\"><b> 2.0.0rc0</b></td>\n",
" </tr>\n",
" <tr>\n",
" <td style=\"text-align: left\"><b>Dashboard:</b></td>\n",
" <td style=\"text-align: left\"><b><a href=\"http://127.0.0.1:8266\" target=\"_blank\">http://127.0.0.1:8266</a></b></td>\n",
"</tr>\n",
"\n",
" </table>\n",
" </div>\n",
"</div>\n"
],
"text/plain": [
"RayContext(dashboard_url='127.0.0.1:8266', python_version='3.7.7', ray_version='2.0.0rc0', ray_commit='{{RAY_COMMIT_SHA}}', address_info={'node_ip_address': '127.0.0.1', 'raylet_ip_address': '127.0.0.1', 'redis_address': None, 'object_store_address': '/tmp/ray/session_2022-07-22_15-23-54_250484_46348/sockets/plasma_store', 'raylet_socket_name': '/tmp/ray/session_2022-07-22_15-23-54_250484_46348/sockets/raylet', 'webui_url': '127.0.0.1:8266', 'session_dir': '/tmp/ray/session_2022-07-22_15-23-54_250484_46348', 'metrics_export_port': 60625, 'gcs_address': '127.0.0.1:63562', 'address': '127.0.0.1:63562', 'dashboard_agent_listen_port': 52365, 'node_id': '833f68610f3299cf23cacacc02fe1c7dce7e9ea49bb948b17af93bf0'})"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ray.init(configure_logging=False)"
]
},
{
"cell_type": "markdown",
"id": "5b7a4b94",
"metadata": {},
"source": [
"Now we construct the hyperparameter search space using `ConfigSpace`"
]
},
{
"cell_type": "markdown",
"id": "e2405373",
"metadata": {},
"source": [
"Next we define the search algorithm built from `NevergradSearch`, constrained to a maximum of `4` concurrent trials with a `ConcurrencyLimiter`. Here we use `ng.optimizers.OnePlusOne`, a simple evolutionary algorithm."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "f099b674",
"metadata": {},
"outputs": [],
"source": [
"algo = NevergradSearch(\n",
" optimizer=ng.optimizers.OnePlusOne,\n",
")\n",
"algo = tune.search.ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "4bddc4e5",
"metadata": {},
"source": [
"The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
"(you can decrease this if it takes too long on your machine)."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "adb807bc",
"metadata": {},
"outputs": [],
"source": [
"num_samples = 1000"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "191c7f89",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"# If 1000 samples take too long, you can reduce this number.\n",
"# We override this number here for our smoke tests.\n",
"num_samples = 10"
]
},
{
"cell_type": "markdown",
"id": "a3956381",
"metadata": {},
"source": [
"Finally, all that's left is to define a search space."
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "8829ffc5",
"metadata": {},
"outputs": [],
"source": [
"search_config = {\n",
" \"steps\": 100,\n",
" \"width\": tune.uniform(0, 20),\n",
" \"height\": tune.uniform(-100, 100),\n",
" \"activation\": tune.choice([\"relu, tanh\"])\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "0b9d051f",
"metadata": {},
"source": [
"Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_space` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, observe how efficient it is to execute `tuner.fit()`."
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "769f4368",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING:ray.tune.trainable.function_trainable:Function checkpointing is disabled. This may result in unexpected behavior when using checkpointing features or certain schedulers. To enable, set the train function arguments to be `func(config, checkpoint_dir=None)`.\n"
]
},
{
"data": {
"text/html": [
"== Status ==<br>Current time: 2022-07-22 15:24:44 (running for 00:00:43.69)<br>Memory usage on this node: 10.4/16.0 GiB<br>Using FIFO scheduling algorithm.<br>Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/4.61 GiB heap, 0.0/2.0 GiB objects<br>Current best trial: 004f499a with mean_loss=-7.595329711238255 and parameters={'steps': 100, 'width': 2.1174116156230918, 'height': -90.50653873694615, 'activation': 'relu, tanh'}<br>Result logdir: /Users/kai/ray_results/objective_2022-07-22_15-23-59<br>Number of trials: 10/10 (10 TERMINATED)<br><table>\n",
"<thead>\n",
"<tr><th>Trial name </th><th>status </th><th>loc </th><th>activation </th><th style=\"text-align: right;\"> height</th><th style=\"text-align: right;\"> width</th><th style=\"text-align: right;\"> loss</th><th style=\"text-align: right;\"> iter</th><th style=\"text-align: right;\"> total time (s)</th><th style=\"text-align: right;\"> iterations</th><th style=\"text-align: right;\"> neg_mean_loss</th></tr>\n",
"</thead>\n",
"<tbody>\n",
"<tr><td>objective_ee2ca136</td><td>TERMINATED</td><td>127.0.0.1:46434</td><td>relu, tanh </td><td style=\"text-align: right;\"> 0 </td><td style=\"text-align: right;\">10 </td><td style=\"text-align: right;\"> 1.1 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.942 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -1.1 </td></tr>\n",
"<tr><td>objective_efe1626e</td><td>TERMINATED</td><td>127.0.0.1:46441</td><td>relu, tanh </td><td style=\"text-align: right;\">-31.0013 </td><td style=\"text-align: right;\"> 9.28761 </td><td style=\"text-align: right;\">-1.99254</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.5354</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 1.99254</td></tr>\n",
"<tr><td>objective_efe34e4e</td><td>TERMINATED</td><td>127.0.0.1:46442</td><td>relu, tanh </td><td style=\"text-align: right;\"> 5.21403</td><td style=\"text-align: right;\"> 9.48974 </td><td style=\"text-align: right;\"> 1.62672</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.6606</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -1.62672</td></tr>\n",
"<tr><td>objective_efe55c2a</td><td>TERMINATED</td><td>127.0.0.1:46443</td><td>relu, tanh </td><td style=\"text-align: right;\">-20.8721 </td><td style=\"text-align: right;\">11.3958 </td><td style=\"text-align: right;\">-0.99935</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.6083</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 0.99935</td></tr>\n",
"<tr><td>objective_f6688086</td><td>TERMINATED</td><td>127.0.0.1:46467</td><td>relu, tanh </td><td style=\"text-align: right;\"> 57.2829 </td><td style=\"text-align: right;\">17.7296 </td><td style=\"text-align: right;\"> 6.78493</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.716 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -6.78493</td></tr>\n",
"<tr><td>objective_f85ed926</td><td>TERMINATED</td><td>127.0.0.1:46478</td><td>relu, tanh </td><td style=\"text-align: right;\"> 40.5543 </td><td style=\"text-align: right;\">19.0813 </td><td style=\"text-align: right;\"> 5.10809</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7158</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -5.10809</td></tr>\n",
"<tr><td>objective_f86ee276</td><td>TERMINATED</td><td>127.0.0.1:46481</td><td>relu, tanh </td><td style=\"text-align: right;\"> 93.8686 </td><td style=\"text-align: right;\"> 1.60757 </td><td style=\"text-align: right;\">10.9781 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7415</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -10.9781 </td></tr>\n",
"<tr><td>objective_f880a02e</td><td>TERMINATED</td><td>127.0.0.1:46484</td><td>relu, tanh </td><td style=\"text-align: right;\">-80.5769 </td><td style=\"text-align: right;\"> 5.84852 </td><td style=\"text-align: right;\">-6.88791</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7335</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 6.88791</td></tr>\n",
"<tr><td>objective_fe4e7a44</td><td>TERMINATED</td><td>127.0.0.1:46499</td><td>relu, tanh </td><td style=\"text-align: right;\"> 9.62911</td><td style=\"text-align: right;\"> 0.622909</td><td style=\"text-align: right;\"> 3.35823</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 13.1428</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -3.35823</td></tr>\n",
"<tr><td>objective_004f499a</td><td>TERMINATED</td><td>127.0.0.1:46504</td><td>relu, tanh </td><td style=\"text-align: right;\">-90.5065 </td><td style=\"text-align: right;\"> 2.11741 </td><td style=\"text-align: right;\">-7.59533</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7688</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 7.59533</td></tr>\n",
"</tbody>\n",
"</table><br><br>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:ray._private.runtime_env.plugin_schema_manager:Loading the default runtime env schemas: ['/Users/kai/coding/ray/python/ray/_private/runtime_env/../../runtime_env/schemas/working_dir_schema.json', '/Users/kai/coding/ray/python/ray/_private/runtime_env/../../runtime_env/schemas/pip_schema.json'].\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_ee2ca136:\n",
" date: 2022-07-22_15-24-03\n",
" done: false\n",
" experiment_id: c0ad5ddb78cc4cc88e8195f5bde34e20\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 11.0\n",
" neg_mean_loss: -11.0\n",
" node_ip: 127.0.0.1\n",
" pid: 46434\n",
" time_since_restore: 0.10390329360961914\n",
" time_this_iter_s: 0.10390329360961914\n",
" time_total_s: 0.10390329360961914\n",
" timestamp: 1658499843\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: ee2ca136\n",
" warmup_time: 0.003058910369873047\n",
" \n",
"Result for objective_efe1626e:\n",
" date: 2022-07-22_15-24-06\n",
" done: false\n",
" experiment_id: c6d33a9a30c040c0929b657f1d3e1557\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 7.899868000941147\n",
" neg_mean_loss: -7.899868000941147\n",
" node_ip: 127.0.0.1\n",
" pid: 46441\n",
" time_since_restore: 0.10202908515930176\n",
" time_this_iter_s: 0.10202908515930176\n",
" time_total_s: 0.10202908515930176\n",
" timestamp: 1658499846\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: efe1626e\n",
" warmup_time: 0.003210783004760742\n",
" \n",
"Result for objective_efe55c2a:\n",
" date: 2022-07-22_15-24-06\n",
" done: false\n",
" experiment_id: 903e9605ba894aa0bc55297229b8a77b\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 8.91279068144465\n",
" neg_mean_loss: -8.91279068144465\n",
" node_ip: 127.0.0.1\n",
" pid: 46443\n",
" time_since_restore: 0.1029670238494873\n",
" time_this_iter_s: 0.1029670238494873\n",
" time_total_s: 0.1029670238494873\n",
" timestamp: 1658499846\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: efe55c2a\n",
" warmup_time: 0.0031630992889404297\n",
" \n",
"Result for objective_efe34e4e:\n",
" date: 2022-07-22_15-24-06\n",
" done: false\n",
" experiment_id: 1efbb9c1becb436e8f304b61e9edd61b\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 11.521403250268252\n",
" neg_mean_loss: -11.521403250268252\n",
" node_ip: 127.0.0.1\n",
" pid: 46442\n",
" time_since_restore: 0.10443997383117676\n",
" time_this_iter_s: 0.10443997383117676\n",
" time_total_s: 0.10443997383117676\n",
" timestamp: 1658499846\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: efe34e4e\n",
" warmup_time: 0.002650022506713867\n",
" \n",
"Result for objective_ee2ca136:\n",
" date: 2022-07-22_15-24-08\n",
" done: false\n",
" experiment_id: c0ad5ddb78cc4cc88e8195f5bde34e20\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 45\n",
" iterations_since_restore: 46\n",
" mean_loss: 1.2173913043478262\n",
" neg_mean_loss: -1.2173913043478262\n",
" node_ip: 127.0.0.1\n",
" pid: 46434\n",
" time_since_restore: 5.145823955535889\n",
" time_this_iter_s: 0.10791492462158203\n",
" time_total_s: 5.145823955535889\n",
" timestamp: 1658499848\n",
" timesteps_since_restore: 0\n",
" training_iteration: 46\n",
" trial_id: ee2ca136\n",
" warmup_time: 0.003058910369873047\n",
" \n",
"Result for objective_efe1626e:\n",
" date: 2022-07-22_15-24-11\n",
" done: false\n",
" experiment_id: c6d33a9a30c040c0929b657f1d3e1557\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -1.8761766831351419\n",
" neg_mean_loss: 1.8761766831351419\n",
" node_ip: 127.0.0.1\n",
" pid: 46441\n",
" time_since_restore: 5.138395071029663\n",
" time_this_iter_s: 0.10561490058898926\n",
" time_total_s: 5.138395071029663\n",
" timestamp: 1658499851\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: efe1626e\n",
" warmup_time: 0.003210783004760742\n",
" \n",
"Result for objective_efe55c2a:\n",
" date: 2022-07-22_15-24-11\n",
" done: false\n",
" experiment_id: 903e9605ba894aa0bc55297229b8a77b\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -0.9039251143764186\n",
" neg_mean_loss: 0.9039251143764186\n",
" node_ip: 127.0.0.1\n",
" pid: 46443\n",
" time_since_restore: 5.145689249038696\n",
" time_this_iter_s: 0.10677504539489746\n",
" time_total_s: 5.145689249038696\n",
" timestamp: 1658499851\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: efe55c2a\n",
" warmup_time: 0.0031630992889404297\n",
" \n",
"Result for objective_efe34e4e:\n",
" date: 2022-07-22_15-24-11\n",
" done: false\n",
" experiment_id: 1efbb9c1becb436e8f304b61e9edd61b\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 1.7406930109818743\n",
" neg_mean_loss: -1.7406930109818743\n",
" node_ip: 127.0.0.1\n",
" pid: 46442\n",
" time_since_restore: 5.151263952255249\n",
" time_this_iter_s: 0.10529589653015137\n",
" time_total_s: 5.151263952255249\n",
" timestamp: 1658499851\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: efe34e4e\n",
" warmup_time: 0.002650022506713867\n",
" \n",
"Result for objective_ee2ca136:\n",
" date: 2022-07-22_15-24-13\n",
" done: false\n",
" experiment_id: c0ad5ddb78cc4cc88e8195f5bde34e20\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 92\n",
" iterations_since_restore: 93\n",
" mean_loss: 1.10752688172043\n",
" neg_mean_loss: -1.10752688172043\n",
" node_ip: 127.0.0.1\n",
" pid: 46434\n",
" time_since_restore: 10.185918092727661\n",
" time_this_iter_s: 0.10853385925292969\n",
" time_total_s: 10.185918092727661\n",
" timestamp: 1658499853\n",
" timesteps_since_restore: 0\n",
" training_iteration: 93\n",
" trial_id: ee2ca136\n",
" warmup_time: 0.003058910369873047\n",
" \n",
"Result for objective_ee2ca136:\n",
" date: 2022-07-22_15-24-14\n",
" done: true\n",
" experiment_id: c0ad5ddb78cc4cc88e8195f5bde34e20\n",
" experiment_tag: 1_activation=relu_tanh,height=0.0000,steps=100,width=10.0000\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 1.1\n",
" neg_mean_loss: -1.1\n",
" node_ip: 127.0.0.1\n",
" pid: 46434\n",
" time_since_restore: 10.941971063613892\n",
" time_this_iter_s: 0.10557413101196289\n",
" time_total_s: 10.941971063613892\n",
" timestamp: 1658499854\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: ee2ca136\n",
" warmup_time: 0.003058910369873047\n",
" \n",
"Result for objective_efe34e4e:\n",
" date: 2022-07-22_15-24-16\n",
" done: false\n",
" experiment_id: 1efbb9c1becb436e8f304b61e9edd61b\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 90\n",
" iterations_since_restore: 91\n",
" mean_loss: 1.63713376556457\n",
" neg_mean_loss: -1.63713376556457\n",
" node_ip: 127.0.0.1\n",
" pid: 46442\n",
" time_since_restore: 9.770322799682617\n",
" time_this_iter_s: 0.1040806770324707\n",
" time_total_s: 9.770322799682617\n",
" timestamp: 1658499856\n",
" timesteps_since_restore: 0\n",
" training_iteration: 91\n",
" trial_id: efe34e4e\n",
" warmup_time: 0.002650022506713867\n",
" \n",
"Result for objective_efe1626e:\n",
" date: 2022-07-22_15-24-16\n",
" done: false\n",
" experiment_id: c6d33a9a30c040c0929b657f1d3e1557\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 92\n",
" iterations_since_restore: 93\n",
" mean_loss: -1.9844528559710652\n",
" neg_mean_loss: 1.9844528559710652\n",
" node_ip: 127.0.0.1\n",
" pid: 46441\n",
" time_since_restore: 9.955276012420654\n",
" time_this_iter_s: 0.10721087455749512\n",
" time_total_s: 9.955276012420654\n",
" timestamp: 1658499856\n",
" timesteps_since_restore: 0\n",
" training_iteration: 93\n",
" trial_id: efe1626e\n",
" warmup_time: 0.003210783004760742\n",
" \n",
"Result for objective_efe55c2a:\n",
" date: 2022-07-22_15-24-16\n",
" done: false\n",
" experiment_id: 903e9605ba894aa0bc55297229b8a77b\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 91\n",
" iterations_since_restore: 92\n",
" mean_loss: -0.991699632507838\n",
" neg_mean_loss: 0.991699632507838\n",
" node_ip: 127.0.0.1\n",
" pid: 46443\n",
" time_since_restore: 9.83866286277771\n",
" time_this_iter_s: 0.10593676567077637\n",
" time_total_s: 9.83866286277771\n",
" timestamp: 1658499856\n",
" timesteps_since_restore: 0\n",
" training_iteration: 92\n",
" trial_id: efe55c2a\n",
" warmup_time: 0.0031630992889404297\n",
" \n",
"Result for objective_f6688086:\n",
" date: 2022-07-22_15-24-17\n",
" done: false\n",
" experiment_id: aa090556819b41c5b1e93d761dc9311a\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 16.728285222315503\n",
" neg_mean_loss: -16.728285222315503\n",
" node_ip: 127.0.0.1\n",
" pid: 46467\n",
" time_since_restore: 0.10288572311401367\n",
" time_this_iter_s: 0.10288572311401367\n",
" time_total_s: 0.10288572311401367\n",
" timestamp: 1658499857\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: f6688086\n",
" warmup_time: 0.002875089645385742\n",
" \n",
"Result for objective_efe1626e:\n",
" date: 2022-07-22_15-24-17\n",
" done: true\n",
" experiment_id: c6d33a9a30c040c0929b657f1d3e1557\n",
" experiment_tag: 2_activation=relu_tanh,height=-31.0013,steps=100,width=9.2876\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -1.9925441896649678\n",
" neg_mean_loss: 1.9925441896649678\n",
" node_ip: 127.0.0.1\n",
" pid: 46441\n",
" time_since_restore: 11.535439252853394\n",
" time_this_iter_s: 0.10611414909362793\n",
" time_total_s: 11.535439252853394\n",
" timestamp: 1658499857\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: efe1626e\n",
" warmup_time: 0.003210783004760742\n",
" \n",
"Result for objective_efe55c2a:\n",
" date: 2022-07-22_15-24-17\n",
" done: true\n",
" experiment_id: 903e9605ba894aa0bc55297229b8a77b\n",
" experiment_tag: 4_activation=relu_tanh,height=-20.8721,steps=100,width=11.3958\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -0.9993497773424045\n",
" neg_mean_loss: 0.9993497773424045\n",
" node_ip: 127.0.0.1\n",
" pid: 46443\n",
" time_since_restore: 11.608299970626831\n",
" time_this_iter_s: 0.10889291763305664\n",
" time_total_s: 11.608299970626831\n",
" timestamp: 1658499857\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: efe55c2a\n",
" warmup_time: 0.0031630992889404297\n",
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_efe34e4e:\n",
" date: 2022-07-22_15-24-18\n",
" done: true\n",
" experiment_id: 1efbb9c1becb436e8f304b61e9edd61b\n",
" experiment_tag: 3_activation=relu_tanh,height=5.2140,steps=100,width=9.4897\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 1.6267236167220034\n",
" neg_mean_loss: -1.6267236167220034\n",
" node_ip: 127.0.0.1\n",
" pid: 46442\n",
" time_since_restore: 11.660571098327637\n",
" time_this_iter_s: 0.1082301139831543\n",
" time_total_s: 11.660571098327637\n",
" timestamp: 1658499858\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: efe34e4e\n",
" warmup_time: 0.002650022506713867\n",
" \n",
"Result for objective_f85ed926:\n",
" date: 2022-07-22_15-24-20\n",
" done: false\n",
" experiment_id: 402684968ef24377ae7787c9af50d3ae\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 15.055429995999226\n",
" neg_mean_loss: -15.055429995999226\n",
" node_ip: 127.0.0.1\n",
" pid: 46478\n",
" time_since_restore: 0.10394287109375\n",
" time_this_iter_s: 0.10394287109375\n",
" time_total_s: 0.10394287109375\n",
" timestamp: 1658499860\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: f85ed926\n",
" warmup_time: 0.0031621456146240234\n",
" \n",
"Result for objective_f86ee276:\n",
" date: 2022-07-22_15-24-20\n",
" done: false\n",
" experiment_id: cbdc4d94c3d64e59aa53e4465349dbe1\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 20.386860838650087\n",
" neg_mean_loss: -20.386860838650087\n",
" node_ip: 127.0.0.1\n",
" pid: 46481\n",
" time_since_restore: 0.10312676429748535\n",
" time_this_iter_s: 0.10312676429748535\n",
" time_total_s: 0.10312676429748535\n",
" timestamp: 1658499860\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: f86ee276\n",
" warmup_time: 0.002810955047607422\n",
" \n",
"Result for objective_f880a02e:\n",
" date: 2022-07-22_15-24-20\n",
" done: false\n",
" experiment_id: fb5af0209b2b45eda67ab40960a48a99\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 2.94231225430282\n",
" neg_mean_loss: -2.94231225430282\n",
" node_ip: 127.0.0.1\n",
" pid: 46484\n",
" time_since_restore: 0.10325503349304199\n",
" time_this_iter_s: 0.10325503349304199\n",
" time_total_s: 0.10325503349304199\n",
" timestamp: 1658499860\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: f880a02e\n",
" warmup_time: 0.0027141571044921875\n",
" \n",
"Result for objective_f6688086:\n",
" date: 2022-07-22_15-24-22\n",
" done: false\n",
" experiment_id: aa090556819b41c5b1e93d761dc9311a\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 6.846867893893624\n",
" neg_mean_loss: -6.846867893893624\n",
" node_ip: 127.0.0.1\n",
" pid: 46467\n",
" time_since_restore: 5.122231721878052\n",
" time_this_iter_s: 0.10641169548034668\n",
" time_total_s: 5.122231721878052\n",
" timestamp: 1658499862\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: f6688086\n",
" warmup_time: 0.002875089645385742\n",
" \n",
"Result for objective_f85ed926:\n",
" date: 2022-07-22_15-24-25\n",
" done: false\n",
" experiment_id: 402684968ef24377ae7787c9af50d3ae\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 5.1657053480548045\n",
" neg_mean_loss: -5.1657053480548045\n",
" node_ip: 127.0.0.1\n",
" pid: 46478\n",
" time_since_restore: 5.151860952377319\n",
" time_this_iter_s: 0.10704493522644043\n",
" time_total_s: 5.151860952377319\n",
" timestamp: 1658499865\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: f85ed926\n",
" warmup_time: 0.0031621456146240234\n",
" \n",
"Result for objective_f86ee276:\n",
" date: 2022-07-22_15-24-25\n",
" done: false\n",
" experiment_id: cbdc4d94c3d64e59aa53e4465349dbe1\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 11.55568804118157\n",
" neg_mean_loss: -11.55568804118157\n",
" node_ip: 127.0.0.1\n",
" pid: 46481\n",
" time_since_restore: 5.140729665756226\n",
" time_this_iter_s: 0.1076810359954834\n",
" time_total_s: 5.140729665756226\n",
" timestamp: 1658499865\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: f86ee276\n",
" warmup_time: 0.002810955047607422\n",
" \n",
"Result for objective_f880a02e:\n",
" date: 2022-07-22_15-24-25\n",
" done: false\n",
" experiment_id: fb5af0209b2b45eda67ab40960a48a99\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -6.706663109525936\n",
" neg_mean_loss: 6.706663109525936\n",
" node_ip: 127.0.0.1\n",
" pid: 46484\n",
" time_since_restore: 5.157264947891235\n",
" time_this_iter_s: 0.1107029914855957\n",
" time_total_s: 5.157264947891235\n",
" timestamp: 1658499865\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: f880a02e\n",
" warmup_time: 0.0027141571044921875\n",
" \n",
"Result for objective_f6688086:\n",
" date: 2022-07-22_15-24-27\n",
" done: false\n",
" experiment_id: aa090556819b41c5b1e93d761dc9311a\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 6.787930201151392\n",
" neg_mean_loss: -6.787930201151392\n",
" node_ip: 127.0.0.1\n",
" pid: 46467\n",
" time_since_restore: 10.178911924362183\n",
" time_this_iter_s: 0.10588788986206055\n",
" time_total_s: 10.178911924362183\n",
" timestamp: 1658499867\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: f6688086\n",
" warmup_time: 0.002875089645385742\n",
" \n",
"Result for objective_f6688086:\n",
" date: 2022-07-22_15-24-27\n",
" done: true\n",
" experiment_id: aa090556819b41c5b1e93d761dc9311a\n",
" experiment_tag: 5_activation=relu_tanh,height=57.2829,steps=100,width=17.7296\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 6.784934893475021\n",
" neg_mean_loss: -6.784934893475021\n",
" node_ip: 127.0.0.1\n",
" pid: 46467\n",
" time_since_restore: 10.71599006652832\n",
" time_this_iter_s: 0.10760498046875\n",
" time_total_s: 10.71599006652832\n",
" timestamp: 1658499867\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: f6688086\n",
" warmup_time: 0.002875089645385742\n",
" \n",
"Result for objective_fe4e7a44:\n",
" date: 2022-07-22_15-24-30\n",
" done: false\n",
" experiment_id: 95711fcb897a4f9fae3ebd811619fba9\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 11.962911259228687\n",
" neg_mean_loss: -11.962911259228687\n",
" node_ip: 127.0.0.1\n",
" pid: 46499\n",
" time_since_restore: 0.10338783264160156\n",
" time_this_iter_s: 0.10338783264160156\n",
" time_total_s: 0.10338783264160156\n",
" timestamp: 1658499870\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: fe4e7a44\n",
" warmup_time: 0.002811908721923828\n",
" \n",
"Result for objective_f85ed926:\n",
" date: 2022-07-22_15-24-30\n",
" done: false\n",
" experiment_id: 402684968ef24377ae7787c9af50d3ae\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 5.110873373928027\n",
" neg_mean_loss: -5.110873373928027\n",
" node_ip: 127.0.0.1\n",
" pid: 46478\n",
" time_since_restore: 10.178997039794922\n",
" time_this_iter_s: 0.10618400573730469\n",
" time_total_s: 10.178997039794922\n",
" timestamp: 1658499870\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: f85ed926\n",
" warmup_time: 0.0031621456146240234\n",
" \n",
"Result for objective_f86ee276:\n",
" date: 2022-07-22_15-24-30\n",
" done: false\n",
" experiment_id: cbdc4d94c3d64e59aa53e4465349dbe1\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 11.007548256848812\n",
" neg_mean_loss: -11.007548256848812\n",
" node_ip: 127.0.0.1\n",
" pid: 46481\n",
" time_since_restore: 10.165759801864624\n",
" time_this_iter_s: 0.10771489143371582\n",
" time_total_s: 10.165759801864624\n",
" timestamp: 1658499870\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: f86ee276\n",
" warmup_time: 0.002810955047607422\n",
" \n",
"Result for objective_f880a02e:\n",
" date: 2022-07-22_15-24-30\n",
" done: false\n",
" experiment_id: fb5af0209b2b45eda67ab40960a48a99\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: -6.87903993853598\n",
" neg_mean_loss: 6.87903993853598\n",
" node_ip: 127.0.0.1\n",
" pid: 46484\n",
" time_since_restore: 10.188904047012329\n",
" time_this_iter_s: 0.10939908027648926\n",
" time_total_s: 10.188904047012329\n",
" timestamp: 1658499870\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: f880a02e\n",
" warmup_time: 0.0027141571044921875\n",
" \n",
"Result for objective_f85ed926:\n",
" date: 2022-07-22_15-24-31\n",
" done: true\n",
" experiment_id: 402684968ef24377ae7787c9af50d3ae\n",
" experiment_tag: 6_activation=relu_tanh,height=40.5543,steps=100,width=19.0813\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 5.108087948450606\n",
" neg_mean_loss: -5.108087948450606\n",
" node_ip: 127.0.0.1\n",
" pid: 46478\n",
" time_since_restore: 10.71581220626831\n",
" time_this_iter_s: 0.10749602317810059\n",
" time_total_s: 10.71581220626831\n",
" timestamp: 1658499871\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: f85ed926\n",
" warmup_time: 0.0031621456146240234\n",
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_f86ee276:\n",
" date: 2022-07-22_15-24-31\n",
" done: true\n",
" experiment_id: cbdc4d94c3d64e59aa53e4465349dbe1\n",
" experiment_tag: 7_activation=relu_tanh,height=93.8686,steps=100,width=1.6076\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 10.978053669828887\n",
" neg_mean_loss: -10.978053669828887\n",
" node_ip: 127.0.0.1\n",
" pid: 46481\n",
" time_since_restore: 10.741472005844116\n",
" time_this_iter_s: 0.1456291675567627\n",
" time_total_s: 10.741472005844116\n",
" timestamp: 1658499871\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: f86ee276\n",
" warmup_time: 0.002810955047607422\n",
" \n",
"Result for objective_f880a02e:\n",
" date: 2022-07-22_15-24-31\n",
" done: true\n",
" experiment_id: fb5af0209b2b45eda67ab40960a48a99\n",
" experiment_tag: 8_activation=relu_tanh,height=-80.5769,steps=100,width=5.8485\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -6.887909370541976\n",
" neg_mean_loss: 6.887909370541976\n",
" node_ip: 127.0.0.1\n",
" pid: 46484\n",
" time_since_restore: 10.733515977859497\n",
" time_this_iter_s: 0.1060791015625\n",
" time_total_s: 10.733515977859497\n",
" timestamp: 1658499871\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: f880a02e\n",
" warmup_time: 0.0027141571044921875\n",
" \n",
"Result for objective_004f499a:\n",
" date: 2022-07-22_15-24-33\n",
" done: false\n",
" experiment_id: c3015ffd206444cd9c2eb1152aae5dd0\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 1.9493461263053842\n",
" neg_mean_loss: -1.9493461263053842\n",
" node_ip: 127.0.0.1\n",
" pid: 46504\n",
" time_since_restore: 0.1044008731842041\n",
" time_this_iter_s: 0.1044008731842041\n",
" time_total_s: 0.1044008731842041\n",
" timestamp: 1658499873\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 004f499a\n",
" warmup_time: 0.0026237964630126953\n",
" \n",
"Result for objective_fe4e7a44:\n",
" date: 2022-07-22_15-24-35\n",
" done: false\n",
" experiment_id: 95711fcb897a4f9fae3ebd811619fba9\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 25\n",
" iterations_since_restore: 26\n",
" mean_loss: 5.873327389463485\n",
" neg_mean_loss: -5.873327389463485\n",
" node_ip: 127.0.0.1\n",
" pid: 46499\n",
" time_since_restore: 5.199802875518799\n",
" time_this_iter_s: 0.1075749397277832\n",
" time_total_s: 5.199802875518799\n",
" timestamp: 1658499875\n",
" timesteps_since_restore: 0\n",
" training_iteration: 26\n",
" trial_id: fe4e7a44\n",
" warmup_time: 0.002811908721923828\n",
" \n",
"Result for objective_004f499a:\n",
" date: 2022-07-22_15-24-38\n",
" done: false\n",
" experiment_id: c3015ffd206444cd9c2eb1152aae5dd0\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -7.137564846035238\n",
" neg_mean_loss: 7.137564846035238\n",
" node_ip: 127.0.0.1\n",
" pid: 46504\n",
" time_since_restore: 5.15981912612915\n",
" time_this_iter_s: 0.10737729072570801\n",
" time_total_s: 5.15981912612915\n",
" timestamp: 1658499878\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 004f499a\n",
" warmup_time: 0.0026237964630126953\n",
" \n",
"Result for objective_fe4e7a44:\n",
" date: 2022-07-22_15-24-40\n",
" done: false\n",
" experiment_id: 95711fcb897a4f9fae3ebd811619fba9\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 72\n",
" iterations_since_restore: 73\n",
" mean_loss: 3.7860835740628094\n",
" neg_mean_loss: -3.7860835740628094\n",
" node_ip: 127.0.0.1\n",
" pid: 46499\n",
" time_since_restore: 10.234857082366943\n",
" time_this_iter_s: 0.10819101333618164\n",
" time_total_s: 10.234857082366943\n",
" timestamp: 1658499880\n",
" timesteps_since_restore: 0\n",
" training_iteration: 73\n",
" trial_id: fe4e7a44\n",
" warmup_time: 0.002811908721923828\n",
" \n",
"Result for objective_fe4e7a44:\n",
" date: 2022-07-22_15-24-43\n",
" done: true\n",
" experiment_id: 95711fcb897a4f9fae3ebd811619fba9\n",
" experiment_tag: 9_activation=relu_tanh,height=9.6291,steps=100,width=0.6229\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 3.3582342398877607\n",
" neg_mean_loss: -3.3582342398877607\n",
" node_ip: 127.0.0.1\n",
" pid: 46499\n",
" time_since_restore: 13.142819166183472\n",
" time_this_iter_s: 0.10675215721130371\n",
" time_total_s: 13.142819166183472\n",
" timestamp: 1658499883\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: fe4e7a44\n",
" warmup_time: 0.002811908721923828\n",
" \n",
"Result for objective_004f499a:\n",
" date: 2022-07-22_15-24-43\n",
" done: false\n",
" experiment_id: c3015ffd206444cd9c2eb1152aae5dd0\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: -7.572268959036313\n",
" neg_mean_loss: 7.572268959036313\n",
" node_ip: 127.0.0.1\n",
" pid: 46504\n",
" time_since_restore: 10.228418827056885\n",
" time_this_iter_s: 0.10715484619140625\n",
" time_total_s: 10.228418827056885\n",
" timestamp: 1658499883\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 004f499a\n",
" warmup_time: 0.0026237964630126953\n",
" \n",
"Result for objective_004f499a:\n",
" date: 2022-07-22_15-24-44\n",
" done: true\n",
" experiment_id: c3015ffd206444cd9c2eb1152aae5dd0\n",
" experiment_tag: 10_activation=relu_tanh,height=-90.5065,steps=100,width=2.1174\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -7.595329711238255\n",
" neg_mean_loss: 7.595329711238255\n",
" node_ip: 127.0.0.1\n",
" pid: 46504\n",
" time_since_restore: 10.768790006637573\n",
" time_this_iter_s: 0.10774111747741699\n",
" time_total_s: 10.768790006637573\n",
" timestamp: 1658499884\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 004f499a\n",
" warmup_time: 0.0026237964630126953\n",
" \n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:ray.tune.tune:Total run time: 44.70 seconds (43.68 seconds for the tuning loop).\n"
]
}
],
"source": [
"tuner = tune.Tuner(\n",
" objective,\n",
" tune_config=tune.TuneConfig(\n",
" metric=\"mean_loss\",\n",
" mode=\"min\",\n",
" search_alg=algo,\n",
" num_samples=num_samples,\n",
" ),\n",
" param_space=search_config,\n",
")\n",
"results = tuner.fit()"
]
},
{
"cell_type": "markdown",
"id": "950003be",
"metadata": {},
"source": [
"Here are the hyperparamters found to minimize the mean loss of the defined objective."
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "0f021674",
"metadata": {
"lines_to_next_cell": 0
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Best hyperparameters found were: {'steps': 100, 'width': 2.1174116156230918, 'height': -90.50653873694615, 'activation': 'relu, tanh'}\n"
]
}
],
"source": [
"print(\"Best hyperparameters found were: \", results.get_best_result().config)"
]
},
{
"cell_type": "markdown",
"id": "0d3824ae",
"metadata": {},
"source": [
"## Optional: passing the (hyper)parameter space into the search algorithm\n",
"\n",
"We can also pass the search space into `NevergradSearch` using their designed format."
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "89ae7455",
"metadata": {},
"outputs": [],
"source": [
"space = ng.p.Dict(\n",
" width=ng.p.Scalar(lower=0, upper=20),\n",
" height=ng.p.Scalar(lower=-100, upper=100),\n",
" activation=ng.p.Choice(choices=[\"relu\", \"tanh\"])\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "e52eeab1",
"metadata": {},
"outputs": [],
"source": [
"algo = NevergradSearch(\n",
" optimizer=ng.optimizers.OnePlusOne,\n",
" space=space,\n",
" metric=\"mean_loss\",\n",
" mode=\"min\"\n",
")\n",
"algo = tune.search.ConcurrencyLimiter(algo, max_concurrent=4)"
]
},
{
"cell_type": "markdown",
"id": "f8926177",
"metadata": {},
"source": [
"Again we run the experiment, this time with a less passed via the `config` and instead passed through `search_alg`."
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "64f39800",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"== Status ==<br>Current time: 2022-07-22 15:25:27 (running for 00:00:43.22)<br>Memory usage on this node: 10.8/16.0 GiB<br>Using FIFO scheduling algorithm.<br>Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/4.61 GiB heap, 0.0/2.0 GiB objects<br>Result logdir: /Users/kai/ray_results/objective_2022-07-22_15-24-44<br>Number of trials: 10/10 (10 TERMINATED)<br><table>\n",
"<thead>\n",
"<tr><th>Trial name </th><th>status </th><th>loc </th><th>activation </th><th style=\"text-align: right;\"> height</th><th style=\"text-align: right;\"> width</th><th style=\"text-align: right;\"> loss</th><th style=\"text-align: right;\"> iter</th><th style=\"text-align: right;\"> total time (s)</th><th style=\"text-align: right;\"> iterations</th><th style=\"text-align: right;\"> neg_mean_loss</th></tr>\n",
"</thead>\n",
"<tbody>\n",
"<tr><td>objective_085274be</td><td>TERMINATED</td><td>127.0.0.1:46516</td><td>tanh </td><td style=\"text-align: right;\"> 0 </td><td style=\"text-align: right;\">10 </td><td style=\"text-align: right;\"> 1.1 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7324</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -1.1 </td></tr>\n",
"<tr><td>objective_09dee4f2</td><td>TERMINATED</td><td>127.0.0.1:46524</td><td>tanh </td><td style=\"text-align: right;\">-44.4216 </td><td style=\"text-align: right;\">12.9653 </td><td style=\"text-align: right;\">-3.36485 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.3476</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 3.36485 </td></tr>\n",
"<tr><td>objective_09e0846a</td><td>TERMINATED</td><td>127.0.0.1:46525</td><td>relu </td><td style=\"text-align: right;\">-38.0638 </td><td style=\"text-align: right;\">11.1574 </td><td style=\"text-align: right;\"> 6.28334 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.3103</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -6.28334 </td></tr>\n",
"<tr><td>objective_09e21122</td><td>TERMINATED</td><td>127.0.0.1:46526</td><td>relu </td><td style=\"text-align: right;\"> 41.2509 </td><td style=\"text-align: right;\"> 9.75585</td><td style=\"text-align: right;\">14.2276 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 11.3512</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -14.2276 </td></tr>\n",
"<tr><td>objective_1045958e</td><td>TERMINATED</td><td>127.0.0.1:46544</td><td>relu </td><td style=\"text-align: right;\">-73.2818 </td><td style=\"text-align: right;\"> 5.78832</td><td style=\"text-align: right;\"> 2.84334 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7372</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -2.84334 </td></tr>\n",
"<tr><td>objective_12309db2</td><td>TERMINATED</td><td>127.0.0.1:46549</td><td>relu </td><td style=\"text-align: right;\">-94.9666 </td><td style=\"text-align: right;\">16.9764 </td><td style=\"text-align: right;\"> 0.562486</td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7329</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -0.562486</td></tr>\n",
"<tr><td>objective_12342770</td><td>TERMINATED</td><td>127.0.0.1:46550</td><td>tanh </td><td style=\"text-align: right;\">-98.0775 </td><td style=\"text-align: right;\">17.2252 </td><td style=\"text-align: right;\">-8.74945 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7455</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 8.74945 </td></tr>\n",
"<tr><td>objective_12374d7e</td><td>TERMINATED</td><td>127.0.0.1:46551</td><td>relu </td><td style=\"text-align: right;\"> -1.60759</td><td style=\"text-align: right;\">18.0841 </td><td style=\"text-align: right;\"> 9.89479 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.7348</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -9.89479 </td></tr>\n",
"<tr><td>objective_18344524</td><td>TERMINATED</td><td>127.0.0.1:46569</td><td>tanh </td><td style=\"text-align: right;\">-41.1284 </td><td style=\"text-align: right;\">12.2952 </td><td style=\"text-align: right;\">-3.03135 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 12.622 </td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> 3.03135 </td></tr>\n",
"<tr><td>objective_1a1e29b8</td><td>TERMINATED</td><td>127.0.0.1:46576</td><td>tanh </td><td style=\"text-align: right;\"> 64.0289 </td><td style=\"text-align: right;\">10.0482 </td><td style=\"text-align: right;\"> 7.50242 </td><td style=\"text-align: right;\"> 100</td><td style=\"text-align: right;\"> 10.8237</td><td style=\"text-align: right;\"> 99</td><td style=\"text-align: right;\"> -7.50242 </td></tr>\n",
"</tbody>\n",
"</table><br><br>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_085274be:\n",
" date: 2022-07-22_15-24-47\n",
" done: false\n",
" experiment_id: be25590fb790400ca28b548b8359a120\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 11.0\n",
" neg_mean_loss: -11.0\n",
" node_ip: 127.0.0.1\n",
" pid: 46516\n",
" time_since_restore: 0.10224103927612305\n",
" time_this_iter_s: 0.10224103927612305\n",
" time_total_s: 0.10224103927612305\n",
" timestamp: 1658499887\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 085274be\n",
" warmup_time: 0.0029327869415283203\n",
" \n",
"Result for objective_09dee4f2:\n",
" date: 2022-07-22_15-24-49\n",
" done: false\n",
" experiment_id: 9469ffc8df11476db00e329acd2ae8b6\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 6.557843114006006\n",
" neg_mean_loss: -6.557843114006006\n",
" node_ip: 127.0.0.1\n",
" pid: 46524\n",
" time_since_restore: 0.10509586334228516\n",
" time_this_iter_s: 0.10509586334228516\n",
" time_total_s: 0.10509586334228516\n",
" timestamp: 1658499889\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 09dee4f2\n",
" warmup_time: 0.002938985824584961\n",
" \n",
"Result for objective_09e21122:\n",
" date: 2022-07-22_15-24-49\n",
" done: false\n",
" experiment_id: 0a825198482b4bd4aded6c318dff0dcf\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 24.125094234750687\n",
" neg_mean_loss: -24.125094234750687\n",
" node_ip: 127.0.0.1\n",
" pid: 46526\n",
" time_since_restore: 0.10476112365722656\n",
" time_this_iter_s: 0.10476112365722656\n",
" time_total_s: 0.10476112365722656\n",
" timestamp: 1658499889\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 09e21122\n",
" warmup_time: 0.003228902816772461\n",
" \n",
"Result for objective_09e0846a:\n",
" date: 2022-07-22_15-24-49\n",
" done: false\n",
" experiment_id: 260410fc1fb24f78af507cd38a8361a9\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 16.193619847512025\n",
" neg_mean_loss: -16.193619847512025\n",
" node_ip: 127.0.0.1\n",
" pid: 46525\n",
" time_since_restore: 0.10422873497009277\n",
" time_this_iter_s: 0.10422873497009277\n",
" time_total_s: 0.10422873497009277\n",
" timestamp: 1658499889\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 09e0846a\n",
" warmup_time: 0.002719879150390625\n",
" \n",
"Result for objective_085274be:\n",
" date: 2022-07-22_15-24-52\n",
" done: false\n",
" experiment_id: be25590fb790400ca28b548b8359a120\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 1.2083333333333333\n",
" neg_mean_loss: -1.2083333333333333\n",
" node_ip: 127.0.0.1\n",
" pid: 46516\n",
" time_since_restore: 5.132040977478027\n",
" time_this_iter_s: 0.1084139347076416\n",
" time_total_s: 5.132040977478027\n",
" timestamp: 1658499892\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 085274be\n",
" warmup_time: 0.0029327869415283203\n",
" \n",
"Result for objective_09dee4f2:\n",
" date: 2022-07-22_15-24-54\n",
" done: false\n",
" experiment_id: 9469ffc8df11476db00e329acd2ae8b6\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -3.280701838145472\n",
" neg_mean_loss: 3.280701838145472\n",
" node_ip: 127.0.0.1\n",
" pid: 46524\n",
" time_since_restore: 5.134023189544678\n",
" time_this_iter_s: 0.10428118705749512\n",
" time_total_s: 5.134023189544678\n",
" timestamp: 1658499894\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 09dee4f2\n",
" warmup_time: 0.002938985824584961\n",
" \n",
"Result for objective_09e21122:\n",
" date: 2022-07-22_15-24-54\n",
" done: false\n",
" experiment_id: 0a825198482b4bd4aded6c318dff0dcf\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 14.33852997141215\n",
" neg_mean_loss: -14.33852997141215\n",
" node_ip: 127.0.0.1\n",
" pid: 46526\n",
" time_since_restore: 5.139041185379028\n",
" time_this_iter_s: 0.10721492767333984\n",
" time_total_s: 5.139041185379028\n",
" timestamp: 1658499894\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 09e21122\n",
" warmup_time: 0.003228902816772461\n",
" \n",
"Result for objective_09e0846a:\n",
" date: 2022-07-22_15-24-54\n",
" done: false\n",
" experiment_id: 260410fc1fb24f78af507cd38a8361a9\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 6.3807469650477024\n",
" neg_mean_loss: -6.3807469650477024\n",
" node_ip: 127.0.0.1\n",
" pid: 46525\n",
" time_since_restore: 5.149268865585327\n",
" time_this_iter_s: 0.10705304145812988\n",
" time_total_s: 5.149268865585327\n",
" timestamp: 1658499894\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 09e0846a\n",
" warmup_time: 0.002719879150390625\n",
" \n",
"Result for objective_085274be:\n",
" date: 2022-07-22_15-24-57\n",
" done: false\n",
" experiment_id: be25590fb790400ca28b548b8359a120\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 1.1052631578947367\n",
" neg_mean_loss: -1.1052631578947367\n",
" node_ip: 127.0.0.1\n",
" pid: 46516\n",
" time_since_restore: 10.194181203842163\n",
" time_this_iter_s: 0.10754203796386719\n",
" time_total_s: 10.194181203842163\n",
" timestamp: 1658499897\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 085274be\n",
" warmup_time: 0.0029327869415283203\n",
" \n",
"Result for objective_085274be:\n",
" date: 2022-07-22_15-24-57\n",
" done: true\n",
" experiment_id: be25590fb790400ca28b548b8359a120\n",
" experiment_tag: 1_activation=tanh,height=0.0000,steps=100,width=10.0000\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 1.1\n",
" neg_mean_loss: -1.1\n",
" node_ip: 127.0.0.1\n",
" pid: 46516\n",
" time_since_restore: 10.732417106628418\n",
" time_this_iter_s: 0.10759997367858887\n",
" time_total_s: 10.732417106628418\n",
" timestamp: 1658499897\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 085274be\n",
" warmup_time: 0.0029327869415283203\n",
" \n",
"Result for objective_09dee4f2:\n",
" date: 2022-07-22_15-24-59\n",
" done: false\n",
" experiment_id: 9469ffc8df11476db00e329acd2ae8b6\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: -3.3599044605358728\n",
" neg_mean_loss: 3.3599044605358728\n",
" node_ip: 127.0.0.1\n",
" pid: 46524\n",
" time_since_restore: 10.055138111114502\n",
" time_this_iter_s: 0.10467100143432617\n",
" time_total_s: 10.055138111114502\n",
" timestamp: 1658499899\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 09dee4f2\n",
" warmup_time: 0.002938985824584961\n",
" \n",
"Result for objective_09e21122:\n",
" date: 2022-07-22_15-24-59\n",
" done: false\n",
" experiment_id: 0a825198482b4bd4aded6c318dff0dcf\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: 14.23411049568083\n",
" neg_mean_loss: -14.23411049568083\n",
" node_ip: 127.0.0.1\n",
" pid: 46526\n",
" time_since_restore: 10.066343307495117\n",
" time_this_iter_s: 0.1061861515045166\n",
" time_total_s: 10.066343307495117\n",
" timestamp: 1658499899\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 09e21122\n",
" warmup_time: 0.003228902816772461\n",
" \n",
"Result for objective_09e0846a:\n",
" date: 2022-07-22_15-24-59\n",
" done: false\n",
" experiment_id: 260410fc1fb24f78af507cd38a8361a9\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: 6.2890729561522765\n",
" neg_mean_loss: -6.2890729561522765\n",
" node_ip: 127.0.0.1\n",
" pid: 46525\n",
" time_since_restore: 10.100499868392944\n",
" time_this_iter_s: 0.10625505447387695\n",
" time_total_s: 10.100499868392944\n",
" timestamp: 1658499899\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 09e0846a\n",
" warmup_time: 0.002719879150390625\n",
" \n",
"Result for objective_1045958e:\n",
" date: 2022-07-22_15-25-00\n",
" done: false\n",
" experiment_id: 3579bfc2b346424b833f82f23d459807\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 12.671822501738296\n",
" neg_mean_loss: -12.671822501738296\n",
" node_ip: 127.0.0.1\n",
" pid: 46544\n",
" time_since_restore: 0.10491013526916504\n",
" time_this_iter_s: 0.10491013526916504\n",
" time_total_s: 0.10491013526916504\n",
" timestamp: 1658499900\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 1045958e\n",
" warmup_time: 0.002847909927368164\n",
" \n",
"Result for objective_09dee4f2:\n",
" date: 2022-07-22_15-25-01\n",
" done: true\n",
" experiment_id: 9469ffc8df11476db00e329acd2ae8b6\n",
" experiment_tag: 2_activation=tanh,height=-44.4216,steps=100,width=12.9653\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -3.3648509190285747\n",
" neg_mean_loss: 3.3648509190285747\n",
" node_ip: 127.0.0.1\n",
" pid: 46524\n",
" time_since_restore: 11.347625017166138\n",
" time_this_iter_s: 0.10793185234069824\n",
" time_total_s: 11.347625017166138\n",
" timestamp: 1658499901\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 09dee4f2\n",
" warmup_time: 0.002938985824584961\n",
" \n",
"Result for objective_09e0846a:\n",
" date: 2022-07-22_15-25-01\n",
" done: true\n",
" experiment_id: 260410fc1fb24f78af507cd38a8361a9\n",
" experiment_tag: 3_activation=relu,height=-38.0638,steps=100,width=11.1574\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 6.28333982260153\n",
" neg_mean_loss: -6.28333982260153\n",
" node_ip: 127.0.0.1\n",
" pid: 46525\n",
" time_since_restore: 11.310342788696289\n",
" time_this_iter_s: 0.10942316055297852\n",
" time_total_s: 11.310342788696289\n",
" timestamp: 1658499901\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 09e0846a\n",
" warmup_time: 0.002719879150390625\n",
" \n",
"Result for objective_09e21122:\n",
" date: 2022-07-22_15-25-01\n",
" done: true\n",
" experiment_id: 0a825198482b4bd4aded6c318dff0dcf\n",
" experiment_tag: 4_activation=relu,height=41.2509,steps=100,width=9.7559\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 14.22757115653867\n",
" neg_mean_loss: -14.22757115653867\n",
" node_ip: 127.0.0.1\n",
" pid: 46526\n",
" time_since_restore: 11.351194143295288\n",
" time_this_iter_s: 0.10791015625\n",
" time_total_s: 11.351194143295288\n",
" timestamp: 1658499901\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 09e21122\n",
" warmup_time: 0.003228902816772461\n",
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_12309db2:\n",
" date: 2022-07-22_15-25-03\n",
" done: false\n",
" experiment_id: 3915d19b775c4ee1843b5dcf79560a93\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 10.503337773185708\n",
" neg_mean_loss: -10.503337773185708\n",
" node_ip: 127.0.0.1\n",
" pid: 46549\n",
" time_since_restore: 0.10407328605651855\n",
" time_this_iter_s: 0.10407328605651855\n",
" time_total_s: 0.10407328605651855\n",
" timestamp: 1658499903\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 12309db2\n",
" warmup_time: 0.0030128955841064453\n",
" \n",
"Result for objective_12342770:\n",
" date: 2022-07-22_15-25-03\n",
" done: false\n",
" experiment_id: 1da16f0c20d7438b93927280dc40e5a1\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 1.1922491879354844\n",
" neg_mean_loss: -1.1922491879354844\n",
" node_ip: 127.0.0.1\n",
" pid: 46550\n",
" time_since_restore: 0.1050269603729248\n",
" time_this_iter_s: 0.1050269603729248\n",
" time_total_s: 0.1050269603729248\n",
" timestamp: 1658499903\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: '12342770'\n",
" warmup_time: 0.0032460689544677734\n",
" \n",
"Result for objective_12374d7e:\n",
" date: 2022-07-22_15-25-03\n",
" done: false\n",
" experiment_id: 5788d010ee194eeeabfc3592d37fb2cc\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 19.839240921278268\n",
" neg_mean_loss: -19.839240921278268\n",
" node_ip: 127.0.0.1\n",
" pid: 46551\n",
" time_since_restore: 0.10313534736633301\n",
" time_this_iter_s: 0.10313534736633301\n",
" time_total_s: 0.10313534736633301\n",
" timestamp: 1658499903\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 12374d7e\n",
" warmup_time: 0.002891063690185547\n",
" \n",
"Result for objective_1045958e:\n",
" date: 2022-07-22_15-25-05\n",
" done: false\n",
" experiment_id: 3579bfc2b346424b833f82f23d459807\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 3.0263684682219036\n",
" neg_mean_loss: -3.0263684682219036\n",
" node_ip: 127.0.0.1\n",
" pid: 46544\n",
" time_since_restore: 5.180821180343628\n",
" time_this_iter_s: 0.10701394081115723\n",
" time_total_s: 5.180821180343628\n",
" timestamp: 1658499905\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 1045958e\n",
" warmup_time: 0.002847909927368164\n",
" \n",
"Result for objective_12309db2:\n",
" date: 2022-07-22_15-25-08\n",
" done: false\n",
" experiment_id: 3915d19b775c4ee1843b5dcf79560a93\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 0.6271166903395393\n",
" neg_mean_loss: -0.6271166903395393\n",
" node_ip: 127.0.0.1\n",
" pid: 46549\n",
" time_since_restore: 5.172047138214111\n",
" time_this_iter_s: 0.11187624931335449\n",
" time_total_s: 5.172047138214111\n",
" timestamp: 1658499908\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 12309db2\n",
" warmup_time: 0.0030128955841064453\n",
" \n",
"Result for objective_12342770:\n",
" date: 2022-07-22_15-25-08\n",
" done: false\n",
" experiment_id: 1da16f0c20d7438b93927280dc40e5a1\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: -8.685737519988487\n",
" neg_mean_loss: 8.685737519988487\n",
" node_ip: 127.0.0.1\n",
" pid: 46550\n",
" time_since_restore: 5.172597885131836\n",
" time_this_iter_s: 0.10711097717285156\n",
" time_total_s: 5.172597885131836\n",
" timestamp: 1658499908\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: '12342770'\n",
" warmup_time: 0.0032460689544677734\n",
" \n",
"Result for objective_12374d7e:\n",
" date: 2022-07-22_15-25-08\n",
" done: false\n",
" experiment_id: 5788d010ee194eeeabfc3592d37fb2cc\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 9.955526689542863\n",
" neg_mean_loss: -9.955526689542863\n",
" node_ip: 127.0.0.1\n",
" pid: 46551\n",
" time_since_restore: 5.162422180175781\n",
" time_this_iter_s: 0.10872411727905273\n",
" time_total_s: 5.162422180175781\n",
" timestamp: 1658499908\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 12374d7e\n",
" warmup_time: 0.002891063690185547\n",
" \n",
"Result for objective_1045958e:\n",
" date: 2022-07-22_15-25-10\n",
" done: false\n",
" experiment_id: 3579bfc2b346424b833f82f23d459807\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 2.852294770731789\n",
" neg_mean_loss: -2.852294770731789\n",
" node_ip: 127.0.0.1\n",
" pid: 46544\n",
" time_since_restore: 10.196197271347046\n",
" time_this_iter_s: 0.10780715942382812\n",
" time_total_s: 10.196197271347046\n",
" timestamp: 1658499910\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 1045958e\n",
" warmup_time: 0.002847909927368164\n",
" \n",
"Result for objective_1045958e:\n",
" date: 2022-07-22_15-25-11\n",
" done: true\n",
" experiment_id: 3579bfc2b346424b833f82f23d459807\n",
" experiment_tag: 5_activation=relu,height=-73.2818,steps=100,width=5.7883\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 2.8433363404373386\n",
" neg_mean_loss: -2.8433363404373386\n",
" node_ip: 127.0.0.1\n",
" pid: 46544\n",
" time_since_restore: 10.737220287322998\n",
" time_this_iter_s: 0.10835909843444824\n",
" time_total_s: 10.737220287322998\n",
" timestamp: 1658499911\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 1045958e\n",
" warmup_time: 0.002847909927368164\n",
" \n",
"Result for objective_18344524:\n",
" date: 2022-07-22_15-25-13\n",
" done: false\n",
" experiment_id: 10740ef080664baaa8cca7ba5cb899ac\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 6.887162536491742\n",
" neg_mean_loss: -6.887162536491742\n",
" node_ip: 127.0.0.1\n",
" pid: 46569\n",
" time_since_restore: 0.10450887680053711\n",
" time_this_iter_s: 0.10450887680053711\n",
" time_total_s: 0.10450887680053711\n",
" timestamp: 1658499913\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: '18344524'\n",
" warmup_time: 0.002858877182006836\n",
" \n",
"Result for objective_12309db2:\n",
" date: 2022-07-22_15-25-13\n",
" done: false\n",
" experiment_id: 3915d19b775c4ee1843b5dcf79560a93\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 0.5656126475885976\n",
" neg_mean_loss: -0.5656126475885976\n",
" node_ip: 127.0.0.1\n",
" pid: 46549\n",
" time_since_restore: 10.193128108978271\n",
" time_this_iter_s: 0.10602211952209473\n",
" time_total_s: 10.193128108978271\n",
" timestamp: 1658499913\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 12309db2\n",
" warmup_time: 0.0030128955841064453\n",
" \n",
"Result for objective_12342770:\n",
" date: 2022-07-22_15-25-13\n",
" done: false\n",
" experiment_id: 1da16f0c20d7438b93927280dc40e5a1\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: -8.746369700451542\n",
" neg_mean_loss: 8.746369700451542\n",
" node_ip: 127.0.0.1\n",
" pid: 46550\n",
" time_since_restore: 10.210996866226196\n",
" time_this_iter_s: 0.10719585418701172\n",
" time_total_s: 10.210996866226196\n",
" timestamp: 1658499913\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: '12342770'\n",
" warmup_time: 0.0032460689544677734\n",
" \n",
"Result for objective_12374d7e:\n",
" date: 2022-07-22_15-25-13\n",
" done: false\n",
" experiment_id: 5788d010ee194eeeabfc3592d37fb2cc\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 94\n",
" iterations_since_restore: 95\n",
" mean_loss: 9.897723841978765\n",
" neg_mean_loss: -9.897723841978765\n",
" node_ip: 127.0.0.1\n",
" pid: 46551\n",
" time_since_restore: 10.19912338256836\n",
" time_this_iter_s: 0.10725522041320801\n",
" time_total_s: 10.19912338256836\n",
" timestamp: 1658499913\n",
" timesteps_since_restore: 0\n",
" training_iteration: 95\n",
" trial_id: 12374d7e\n",
" warmup_time: 0.002891063690185547\n",
" \n",
"Result for objective_12309db2:\n",
" date: 2022-07-22_15-25-14\n",
" done: true\n",
" experiment_id: 3915d19b775c4ee1843b5dcf79560a93\n",
" experiment_tag: 6_activation=relu,height=-94.9666,steps=100,width=16.9764\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 0.5624860552037738\n",
" neg_mean_loss: -0.5624860552037738\n",
" node_ip: 127.0.0.1\n",
" pid: 46549\n",
" time_since_restore: 10.73293399810791\n",
" time_this_iter_s: 0.1079857349395752\n",
" time_total_s: 10.73293399810791\n",
" timestamp: 1658499914\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 12309db2\n",
" warmup_time: 0.0030128955841064453\n",
" \n",
"Result for objective_12342770:\n",
" date: 2022-07-22_15-25-14\n",
" done: true\n",
" experiment_id: 1da16f0c20d7438b93927280dc40e5a1\n",
" experiment_tag: 7_activation=tanh,height=-98.0775,steps=100,width=17.2252\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -8.749451683536464\n",
" neg_mean_loss: 8.749451683536464\n",
" node_ip: 127.0.0.1\n",
" pid: 46550\n",
" time_since_restore: 10.745534181594849\n",
" time_this_iter_s: 0.10716795921325684\n",
" time_total_s: 10.745534181594849\n",
" timestamp: 1658499914\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: '12342770'\n",
" warmup_time: 0.0032460689544677734\n",
" \n",
"Result for objective_12374d7e:\n",
" date: 2022-07-22_15-25-14\n",
" done: true\n",
" experiment_id: 5788d010ee194eeeabfc3592d37fb2cc\n",
" experiment_tag: 8_activation=relu,height=-1.6076,steps=100,width=18.0841\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 9.894786565536863\n",
" neg_mean_loss: -9.894786565536863\n",
" node_ip: 127.0.0.1\n",
" pid: 46551\n",
" time_since_restore: 10.734816074371338\n",
" time_this_iter_s: 0.10638594627380371\n",
" time_total_s: 10.734816074371338\n",
" timestamp: 1658499914\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 12374d7e\n",
" warmup_time: 0.002891063690185547\n",
" \n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result for objective_1a1e29b8:\n",
" date: 2022-07-22_15-25-17\n",
" done: false\n",
" experiment_id: 26fc8c5f05a6407690dc8cada561576e\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 0\n",
" iterations_since_restore: 1\n",
" mean_loss: 17.40289288319415\n",
" neg_mean_loss: -17.40289288319415\n",
" node_ip: 127.0.0.1\n",
" pid: 46576\n",
" time_since_restore: 0.10475707054138184\n",
" time_this_iter_s: 0.10475707054138184\n",
" time_total_s: 0.10475707054138184\n",
" timestamp: 1658499917\n",
" timesteps_since_restore: 0\n",
" training_iteration: 1\n",
" trial_id: 1a1e29b8\n",
" warmup_time: 0.0028810501098632812\n",
" \n",
"Result for objective_18344524:\n",
" date: 2022-07-22_15-25-19\n",
" done: false\n",
" experiment_id: 10740ef080664baaa8cca7ba5cb899ac\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 30\n",
" iterations_since_restore: 31\n",
" mean_loss: -2.8488858516331668\n",
" neg_mean_loss: 2.8488858516331668\n",
" node_ip: 127.0.0.1\n",
" pid: 46569\n",
" time_since_restore: 5.212157964706421\n",
" time_this_iter_s: 0.10646200180053711\n",
" time_total_s: 5.212157964706421\n",
" timestamp: 1658499919\n",
" timesteps_since_restore: 0\n",
" training_iteration: 31\n",
" trial_id: '18344524'\n",
" warmup_time: 0.002858877182006836\n",
" \n",
"Result for objective_1a1e29b8:\n",
" date: 2022-07-22_15-25-22\n",
" done: false\n",
" experiment_id: 26fc8c5f05a6407690dc8cada561576e\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 47\n",
" iterations_since_restore: 48\n",
" mean_loss: 7.610248309424407\n",
" neg_mean_loss: -7.610248309424407\n",
" node_ip: 127.0.0.1\n",
" pid: 46576\n",
" time_since_restore: 5.170850038528442\n",
" time_this_iter_s: 0.1072230339050293\n",
" time_total_s: 5.170850038528442\n",
" timestamp: 1658499922\n",
" timesteps_since_restore: 0\n",
" training_iteration: 48\n",
" trial_id: 1a1e29b8\n",
" warmup_time: 0.0028810501098632812\n",
" \n",
"Result for objective_18344524:\n",
" date: 2022-07-22_15-25-24\n",
" done: false\n",
" experiment_id: 10740ef080664baaa8cca7ba5cb899ac\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 77\n",
" iterations_since_restore: 78\n",
" mean_loss: -3.0083151799393004\n",
" neg_mean_loss: 3.0083151799393004\n",
" node_ip: 127.0.0.1\n",
" pid: 46569\n",
" time_since_restore: 10.263540983200073\n",
" time_this_iter_s: 0.10780215263366699\n",
" time_total_s: 10.263540983200073\n",
" timestamp: 1658499924\n",
" timesteps_since_restore: 0\n",
" training_iteration: 78\n",
" trial_id: '18344524'\n",
" warmup_time: 0.002858877182006836\n",
" \n",
"Result for objective_18344524:\n",
" date: 2022-07-22_15-25-26\n",
" done: true\n",
" experiment_id: 10740ef080664baaa8cca7ba5cb899ac\n",
" experiment_tag: 9_activation=tanh,height=-41.1284,steps=100,width=12.2952\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: -3.0313530888899516\n",
" neg_mean_loss: 3.0313530888899516\n",
" node_ip: 127.0.0.1\n",
" pid: 46569\n",
" time_since_restore: 12.62198805809021\n",
" time_this_iter_s: 0.1080331802368164\n",
" time_total_s: 12.62198805809021\n",
" timestamp: 1658499926\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: '18344524'\n",
" warmup_time: 0.002858877182006836\n",
" \n",
"Result for objective_1a1e29b8:\n",
" date: 2022-07-22_15-25-27\n",
" done: false\n",
" experiment_id: 26fc8c5f05a6407690dc8cada561576e\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 93\n",
" iterations_since_restore: 94\n",
" mean_loss: 7.5087713304782575\n",
" neg_mean_loss: -7.5087713304782575\n",
" node_ip: 127.0.0.1\n",
" pid: 46576\n",
" time_since_restore: 10.177540063858032\n",
" time_this_iter_s: 0.1076350212097168\n",
" time_total_s: 10.177540063858032\n",
" timestamp: 1658499927\n",
" timesteps_since_restore: 0\n",
" training_iteration: 94\n",
" trial_id: 1a1e29b8\n",
" warmup_time: 0.0028810501098632812\n",
" \n",
"Result for objective_1a1e29b8:\n",
" date: 2022-07-22_15-25-27\n",
" done: true\n",
" experiment_id: 26fc8c5f05a6407690dc8cada561576e\n",
" experiment_tag: 10_activation=tanh,height=64.0289,steps=100,width=10.0482\n",
" hostname: Kais-MacBook-Pro.local\n",
" iterations: 99\n",
" iterations_since_restore: 100\n",
" mean_loss: 7.502418319119348\n",
" neg_mean_loss: -7.502418319119348\n",
" node_ip: 127.0.0.1\n",
" pid: 46576\n",
" time_since_restore: 10.823745012283325\n",
" time_this_iter_s: 0.10851097106933594\n",
" time_total_s: 10.823745012283325\n",
" timestamp: 1658499927\n",
" timesteps_since_restore: 0\n",
" training_iteration: 100\n",
" trial_id: 1a1e29b8\n",
" warmup_time: 0.0028810501098632812\n",
" \n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:ray.tune.tune:Total run time: 43.33 seconds (43.21 seconds for the tuning loop).\n"
]
}
],
"source": [
"tuner = tune.Tuner(\n",
" objective,\n",
" tune_config=tune.TuneConfig(\n",
"# metric=\"mean_loss\",\n",
"# mode=\"min\",\n",
" search_alg=algo,\n",
" num_samples=num_samples,\n",
" ),\n",
" param_space={\"steps\": 100},\n",
")\n",
"results = tuner.fit()"
]
},
{
"cell_type": "markdown",
"id": "64a17648",
"metadata": {},
"source": [
"Here are the hyperparamters found to minimize the mean loss of the defined objective. Note that we have to pass the metric and mode here because we don't set it in the TuneConfig."
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "aac3e88b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Best hyperparameters found were: {'steps': 100, 'width': 17.225166732233465, 'height': -98.07750812064515, 'activation': 'tanh'}\n"
]
}
],
"source": [
"print(\"Best hyperparameters found were: \", results.get_best_result(\"mean_loss\", \"min\").config)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "0478c1ea",
"metadata": {
"tags": [
"remove-cell"
]
},
"outputs": [],
"source": [
"ray.shutdown()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
},
"orphan": true
},
"nbformat": 4,
"nbformat_minor": 5
}
Reference in a new issue View git blame Copy permalink