ray/doc/source/tune/examples/hebo_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "83323340",
   "metadata": {},
   "source": [
    "# Running Tune experiments with HEBOSearch\n",
    "\n",
    "In this tutorial we introduce HEBO, while running a simple Ray Tune experiment. Tune’s Search Algorithms integrate with ZOOpt and, as a result, allow you to seamlessly scale up a HEBO optimization process - without sacrificing performance.\n",
    "\n",
    "Heteroscadastic Evolutionary Bayesian Optimization (HEBO) does not rely on the gradient of the objective function, but instead, learns from samples of the search space. It is suitable for optimizing functions that are nondifferentiable, with many local minima, or even unknown but only testable. This necessarily makes the algorithm belong to the domain of \"derivative-free optimization\" and \"black-box optimization\".\n",
    "\n",
    "In this example we minimize a simple objective to briefly demonstrate the usage of HEBO with Ray Tune via `HEBOSearch`. It's useful to keep in mind that despite the emphasis on machine learning experiments, Ray Tune optimizes any implicit or explicit objective. Here we assume `zoopt==0.4.1` library is installed. To learn more, please refer to the [HEBO website](https://github.com/huawei-noah/HEBO/tree/master/HEBO)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b8de7864",
   "metadata": {
    "tags": [
     "remove-cell"
    ],
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting HEBO==0.3.2\n",
      "  Downloading HEBO-0.3.2-py3-none-any.whl (106 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m106.3/106.3 kB\u001b[0m \u001b[31m1.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: gpytorch>=1.4.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from HEBO==0.3.2) (1.6.0)\n",
      "Requirement already satisfied: scikit-learn>=0.22 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from HEBO==0.3.2) (0.24.2)\n",
      "Collecting GPy>=1.9.9\n",
      "  Using cached GPy-1.10.0-cp37-cp37m-macosx_10_9_x86_64.whl (1.5 MB)\n",
      "Requirement already satisfied: torch>=1.9.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from HEBO==0.3.2) (1.9.0)\n",
      "Requirement already satisfied: numpy>=1.16 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from HEBO==0.3.2) (1.21.6)\n",
      "Collecting catboost>=0.24.4\n",
      "  Downloading catboost-1.0.6-cp37-none-macosx_10_6_universal2.whl (21.9 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.9/21.9 MB\u001b[0m \u001b[31m5.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
      "\u001b[?25hCollecting pymoo>=0.5.0\n",
      "  Downloading pymoo-0.5.0-cp37-cp37m-macosx_10_14_x86_64.whl (1.1 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.1/1.1 MB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: pandas>=1.0.1 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from HEBO==0.3.2) (1.3.5)\n",
      "Requirement already satisfied: matplotlib in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from catboost>=0.24.4->HEBO==0.3.2) (3.5.0)\n",
      "Collecting graphviz\n",
      "  Downloading graphviz-0.20-py3-none-any.whl (46 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m47.0/47.0 kB\u001b[0m \u001b[31m1.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: plotly in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from catboost>=0.24.4->HEBO==0.3.2) (5.6.0)\n",
      "Requirement already satisfied: scipy in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from catboost>=0.24.4->HEBO==0.3.2) (1.4.1)\n",
      "Requirement already satisfied: six in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from catboost>=0.24.4->HEBO==0.3.2) (1.16.0)\n",
      "Requirement already satisfied: cython>=0.29 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from GPy>=1.9.9->HEBO==0.3.2) (0.29.30)\n",
      "Collecting paramz>=0.9.0\n",
      "  Using cached paramz-0.9.5-py3-none-any.whl\n",
      "Requirement already satisfied: python-dateutil>=2.7.3 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from pandas>=1.0.1->HEBO==0.3.2) (2.8.2)\n",
      "Requirement already satisfied: pytz>=2017.3 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from pandas>=1.0.1->HEBO==0.3.2) (2022.1)\n",
      "Collecting cma==2.7\n",
      "  Using cached cma-2.7.0-py2.py3-none-any.whl (239 kB)\n",
      "Collecting autograd>=1.3\n",
      "  Downloading autograd-1.4-py3-none-any.whl (48 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m48.8/48.8 kB\u001b[0m \u001b[31m1.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: joblib>=0.11 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from scikit-learn>=0.22->HEBO==0.3.2) (1.1.0)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from scikit-learn>=0.22->HEBO==0.3.2) (3.0.0)\n",
      "Requirement already satisfied: typing-extensions in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from torch>=1.9.0->HEBO==0.3.2) (4.1.1)\n",
      "Requirement already satisfied: future>=0.15.2 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from autograd>=1.3->pymoo>=0.5.0->HEBO==0.3.2) (0.18.2)\n",
      "Requirement already satisfied: pillow>=6.2.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from matplotlib->catboost>=0.24.4->HEBO==0.3.2) (9.1.0)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from matplotlib->catboost>=0.24.4->HEBO==0.3.2) (1.3.2)\n",
      "Requirement already satisfied: packaging>=20.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from matplotlib->catboost>=0.24.4->HEBO==0.3.2) (21.3)\n",
      "Requirement already satisfied: setuptools-scm>=4 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from matplotlib->catboost>=0.24.4->HEBO==0.3.2) (6.3.2)\n",
      "Requirement already satisfied: pyparsing>=2.2.1 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from matplotlib->catboost>=0.24.4->HEBO==0.3.2) (2.4.7)\n",
      "Requirement already satisfied: cycler>=0.10 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from matplotlib->catboost>=0.24.4->HEBO==0.3.2) (0.11.0)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from matplotlib->catboost>=0.24.4->HEBO==0.3.2) (4.28.2)\n",
      "Requirement already satisfied: decorator>=4.0.10 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from paramz>=0.9.0->GPy>=1.9.9->HEBO==0.3.2) (5.1.1)\n",
      "Requirement already satisfied: tenacity>=6.2.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from plotly->catboost>=0.24.4->HEBO==0.3.2) (8.0.1)\n",
      "Requirement already satisfied: setuptools in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from setuptools-scm>=4->matplotlib->catboost>=0.24.4->HEBO==0.3.2) (59.5.0)\n",
      "Requirement already satisfied: tomli>=1.0.0 in /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages (from setuptools-scm>=4->matplotlib->catboost>=0.24.4->HEBO==0.3.2) (1.2.3)\n",
      "Installing collected packages: cma, graphviz, autograd, paramz, GPy, pymoo, catboost, HEBO\n",
      "  Attempting uninstall: cma\n",
      "    Found existing installation: cma 3.2.2\n",
      "    Uninstalling cma-3.2.2:\n",
      "      Successfully uninstalled cma-3.2.2\n",
      "Successfully installed GPy-1.10.0 HEBO-0.3.2 autograd-1.4 catboost-1.0.6 cma-2.7.0 graphviz-0.20 paramz-0.9.5 pymoo-0.5.0\n",
      "\u001b[33mWARNING: There was an error checking the latest version of pip.\u001b[0m\u001b[33m\n",
      "\u001b[0m"
     ]
    }
   ],
   "source": [
    "# !pip install ray[tune]\n",
    "!pip install HEBO==0.3.2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37141309",
   "metadata": {},
   "source": [
    "Click below to see all the imports we need for this example.\n",
    "You can also launch directly into a Binder instance to run this notebook yourself.\n",
    "Just click on the rocket symbol at the top of the navigation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9dae789a",
   "metadata": {
    "tags": [
     "hide-input"
    ],
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "import ray\n",
    "from ray import tune\n",
    "from ray.air import session\n",
    "from ray.tune.search.hebo import HEBOSearch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5747767",
   "metadata": {},
   "source": [
    "Let's start by defining a simple evaluation function.\n",
    "We artificially sleep for a bit (`0.1` seconds) to simulate a long-running ML experiment.\n",
    "This setup assumes that we're running multiple `step`s of an experiment and try to tune two hyperparameters,\n",
    "namely `width` and `height`, and `activation`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d654948a",
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "def evaluate(step, width, height, activation):\n",
    "    time.sleep(0.1)\n",
    "    activation_boost = 10 if activation==\"relu\" else 1\n",
    "    return (0.1 + width * step / 100) ** (-1) + height * 0.1 + activation_boost"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e61acd5f",
   "metadata": {},
   "source": [
    "Next, our ``objective`` function takes a Tune ``config``, evaluates the `score` of your experiment in a training loop,\n",
    "and uses `session.report` to report the `score` back to Tune."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b7e5f38b",
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "def objective(config):\n",
    "    for step in range(config[\"steps\"]):\n",
    "        score = evaluate(step, config[\"width\"], config[\"height\"], config[\"activation\"])\n",
    "        session.report({\"iterations\": step, \"mean_loss\": score})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "49966a2e",
   "metadata": {
    "tags": [
     "remove-cell"
    ],
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "    <div style=\"margin-left: 50px;display: flex;flex-direction: row;align-items: center\">\n",
       "        <h3 style=\"color: var(--jp-ui-font-color0)\">Ray</h3>\n",
       "        <svg version=\"1.1\" id=\"ray\" width=\"3em\" viewBox=\"0 0 144.5 144.6\" style=\"margin-left: 3em;margin-right: 3em\">\n",
       "            <g id=\"layer-1\">\n",
       "                <path fill=\"#00a2e9\" class=\"st0\" d=\"M97.3,77.2c-3.8-1.1-6.2,0.9-8.3,5.1c-3.5,6.8-9.9,9.9-17.4,9.6S58,88.1,54.8,81.2c-1.4-3-3-4-6.3-4.1\n",
       "                    c-5.6-0.1-9.9,0.1-13.1,6.4c-3.8,7.6-13.6,10.2-21.8,7.6C5.2,88.4-0.4,80.5,0,71.7c0.1-8.4,5.7-15.8,13.8-18.2\n",
       "                    c8.4-2.6,17.5,0.7,22.3,8c1.3,1.9,1.3,5.2,3.6,5.6c3.9,0.6,8,0.2,12,0.2c1.8,0,1.9-1.6,2.4-2.8c3.5-7.8,9.7-11.8,18-11.9\n",
       "                    c8.2-0.1,14.4,3.9,17.8,11.4c1.3,2.8,2.9,3.6,5.7,3.3c1-0.1,2,0.1,3,0c2.8-0.5,6.4,1.7,8.1-2.7s-2.3-5.5-4.1-7.5\n",
       "                    c-5.1-5.7-10.9-10.8-16.1-16.3C84,38,81.9,37.1,78,38.3C66.7,42,56.2,35.7,53,24.1C50.3,14,57.3,2.8,67.7,0.5\n",
       "                    C78.4-2,89,4.7,91.5,15.3c0.1,0.3,0.1,0.5,0.2,0.8c0.7,3.4,0.7,6.9-0.8,9.8c-1.7,3.2-0.8,5,1.5,7.2c6.7,6.5,13.3,13,19.8,19.7\n",
       "                    c1.8,1.8,3,2.1,5.5,1.2c9.1-3.4,17.9-0.6,23.4,7c4.8,6.9,4.6,16.1-0.4,22.9c-5.4,7.2-14.2,9.9-23.1,6.5c-2.3-0.9-3.5-0.6-5.1,1.1\n",
       "                    c-6.7,6.9-13.6,13.7-20.5,20.4c-1.8,1.8-2.5,3.2-1.4,5.9c3.5,8.7,0.3,18.6-7.7,23.6c-7.9,5-18.2,3.8-24.8-2.9\n",
       "                    c-6.4-6.4-7.4-16.2-2.5-24.3c4.9-7.8,14.5-11,23.1-7.8c3,1.1,4.7,0.5,6.9-1.7C91.7,98.4,98,92.3,104.2,86c1.6-1.6,4.1-2.7,2.6-6.2\n",
       "                    c-1.4-3.3-3.8-2.5-6.2-2.6C99.8,77.2,98.9,77.2,97.3,77.2z M72.1,29.7c5.5,0.1,9.9-4.3,10-9.8c0-0.1,0-0.2,0-0.3\n",
       "                    C81.8,14,77,9.8,71.5,10.2c-5,0.3-9,4.2-9.3,9.2c-0.2,5.5,4,10.1,9.5,10.3C71.8,29.7,72,29.7,72.1,29.7z M72.3,62.3\n",
       "                    c-5.4-0.1-9.9,4.2-10.1,9.7c0,0.2,0,0.3,0,0.5c0.2,5.4,4.5,9.7,9.9,10c5.1,0.1,9.9-4.7,10.1-9.8c0.2-5.5-4-10-9.5-10.3\n",
       "                    C72.6,62.3,72.4,62.3,72.3,62.3z M115,72.5c0.1,5.4,4.5,9.7,9.8,9.9c5.6-0.2,10-4.8,10-10.4c-0.2-5.4-4.6-9.7-10-9.7\n",
       "                    c-5.3-0.1-9.8,4.2-9.9,9.5C115,72.1,115,72.3,115,72.5z M19.5,62.3c-5.4,0.1-9.8,4.4-10,9.8c-0.1,5.1,5.2,10.4,10.2,10.3\n",
       "                    c5.6-0.2,10-4.9,9.8-10.5c-0.1-5.4-4.5-9.7-9.9-9.6C19.6,62.3,19.5,62.3,19.5,62.3z M71.8,134.6c5.9,0.2,10.3-3.9,10.4-9.6\n",
       "                    c0.5-5.5-3.6-10.4-9.1-10.8c-5.5-0.5-10.4,3.6-10.8,9.1c0,0.5,0,0.9,0,1.4c-0.2,5.3,4,9.8,9.3,10\n",
       "                    C71.6,134.6,71.7,134.6,71.8,134.6z\"/>\n",
       "            </g>\n",
       "        </svg>\n",
       "        <table>\n",
       "            <tr>\n",
       "                <td style=\"text-align: left\"><b>Python version:</b></td>\n",
       "                <td style=\"text-align: left\"><b>3.7.7</b></td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                <td style=\"text-align: left\"><b>Ray version:</b></td>\n",
       "                <td style=\"text-align: left\"><b> 3.0.0.dev0</b></td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "    <td style=\"text-align: left\"><b>Dashboard:</b></td>\n",
       "    <td style=\"text-align: left\"><b><a href=\"http://127.0.0.1:8266\" target=\"_blank\">http://127.0.0.1:8266</a></b></td>\n",
       "</tr>\n",
       "\n",
       "        </table>\n",
       "    </div>\n",
       "</div>\n"
      ],
      "text/plain": [
       "RayContext(dashboard_url='127.0.0.1:8266', python_version='3.7.7', ray_version='3.0.0.dev0', ray_commit='{{RAY_COMMIT_SHA}}', address_info={'node_ip_address': '127.0.0.1', 'raylet_ip_address': '127.0.0.1', 'redis_address': None, 'object_store_address': '/tmp/ray/session_2022-07-22_15-34-28_908798_47358/sockets/plasma_store', 'raylet_socket_name': '/tmp/ray/session_2022-07-22_15-34-28_908798_47358/sockets/raylet', 'webui_url': '127.0.0.1:8266', 'session_dir': '/tmp/ray/session_2022-07-22_15-34-28_908798_47358', 'metrics_export_port': 65083, 'gcs_address': '127.0.0.1:61944', 'address': '127.0.0.1:61944', 'dashboard_agent_listen_port': 52365, 'node_id': '6fb8a71e3e7e8ba1b689be3f4802c238368cfef20015b5abd0aa742f'})"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ray.init(configure_logging=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79567a1b",
   "metadata": {},
   "source": [
    "While defining the search algorithm, we may choose to provide an initial set of hyperparameters that we believe are especially promising or informative, and\n",
    "pass this information as a helpful starting point for the `HyperOptSearch` object.\n",
    "\n",
    "We also set the maximum concurrent trials to `8`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "9aaf222e",
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "\n",
    "previously_run_params = [\n",
    "    {\"width\": 10, \"height\": 0, \"activation\": \"relu\"},\n",
    "    {\"width\": 15, \"height\": -20, \"activation\": \"tanh\"},\n",
    "]\n",
    "\n",
    "known_rewards = [-189, -1144]\n",
    "\n",
    "max_concurrent = 8\n",
    "\n",
    "algo = HEBOSearch(\n",
    "    metric=\"mean_loss\",\n",
    "    mode=\"min\",\n",
    "    points_to_evaluate=previously_run_params,\n",
    "    evaluated_rewards=known_rewards,\n",
    "    random_state_seed=123,\n",
    "    max_concurrent=max_concurrent,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19942c67",
   "metadata": {},
   "source": [
    "The number of samples is the number of hyperparameter combinations that will be tried out. This Tune run is set to `1000` samples.\n",
    "(you can decrease this if it takes too long on your machine)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ea2a405c",
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "num_samples = 1000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "cdc1b707",
   "metadata": {
    "tags": [
     "remove-cell"
    ],
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "# If 1000 samples take too long, you can reduce this number.\n",
    "# We override this number here for our smoke tests.\n",
    "num_samples = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40fe2a91",
   "metadata": {},
   "source": [
    "Next we define a search space. The critical assumption is that the optimal hyperparamters live within this space. Yet, if the space is very large, then those hyperparameters may be difficult to find in a short amount of time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "0a530e21",
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "search_config = {\n",
    "    \"steps\": 100,\n",
    "    \"width\": tune.uniform(0, 20),\n",
    "    \"height\": tune.uniform(-100, 100),\n",
    "    \"activation\": tune.choice([\"relu\", \"tanh\"])\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e38301f",
   "metadata": {},
   "source": [
    "Finally, we run the experiment to `\"min\"`imize the \"mean_loss\" of the `objective` by searching `search_config` via `algo`, `num_samples` times. This previous sentence is fully characterizes the search problem we aim to solve. With this in mind, notice how efficient it is to execute `tuner.fit()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "3fb63ef3",
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "== Status ==<br>Current time: 2022-07-22 15:35:11 (running for 00:00:36.78)<br>Memory usage on this node: 10.2/16.0 GiB<br>Using FIFO scheduling algorithm.<br>Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/5.3 GiB heap, 0.0/2.0 GiB objects<br>Current best trial: 72267d26 with mean_loss=-8.280721582416527 and parameters={'steps': 100, 'width': 16.267813332265522, 'height': -93.42430416543701, 'activation': 'tanh'}<br>Result logdir: /Users/kai/ray_results/objective_2022-07-22_15-34-34<br>Number of trials: 10/10 (10 TERMINATED)<br><table>\n",
       "<thead>\n",
       "<tr><th>Trial name        </th><th>status    </th><th>loc            </th><th>activation  </th><th style=\"text-align: right;\">   height</th><th style=\"text-align: right;\">  width</th><th style=\"text-align: right;\">    loss</th><th style=\"text-align: right;\">  iter</th><th style=\"text-align: right;\">  total time (s)</th><th style=\"text-align: right;\">  iterations</th><th style=\"text-align: right;\">  neg_mean_loss</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr><td>objective_67ec1a0a</td><td>TERMINATED</td><td>127.0.0.1:47498</td><td>relu        </td><td style=\"text-align: right;\">-100     </td><td style=\"text-align: right;\"> 0     </td><td style=\"text-align: right;\">10      </td><td style=\"text-align: right;\">   100</td><td style=\"text-align: right;\">         11.4386</td><td style=\"text-align: right;\">          99</td><td style=\"text-align: right;\">      -10      </td></tr>\n",
       "<tr><td>objective_69ac3226</td><td>TERMINATED</td><td>127.0.0.1:47512</td><td>relu        </td><td style=\"text-align: right;\">   0     </td><td style=\"text-align: right;\">10     </td><td style=\"text-align: right;\">10.1    </td><td style=\"text-align: right;\">   100</td><td style=\"text-align: right;\">         10.9018</td><td style=\"text-align: right;\">          99</td><td style=\"text-align: right;\">      -10.1    </td></tr>\n",
       "<tr><td>objective_69ada8d6</td><td>TERMINATED</td><td>127.0.0.1:47513</td><td>relu        </td><td style=\"text-align: right;\"> -50     </td><td style=\"text-align: right;\">15     </td><td style=\"text-align: right;\"> 5.06689</td><td style=\"text-align: right;\">   100</td><td style=\"text-align: right;\">         10.7768</td><td style=\"text-align: right;\">          99</td><td style=\"text-align: right;\">       -5.06689</td></tr>\n",
       "<tr><td>objective_69af2530</td><td>TERMINATED</td><td>127.0.0.1:47514</td><td>tanh        </td><td style=\"text-align: right;\">  50     </td><td style=\"text-align: right;\"> 5     </td><td style=\"text-align: right;\"> 6.19802</td><td style=\"text-align: right;\">   100</td><td style=\"text-align: right;\">         10.9312</td><td style=\"text-align: right;\">          99</td><td style=\"text-align: right;\">       -6.19802</td></tr>\n",
       "<tr><td>objective_69b0a8a6</td><td>TERMINATED</td><td>127.0.0.1:47515</td><td>tanh        </td><td style=\"text-align: right;\"> -25     </td><td style=\"text-align: right;\"> 7.5   </td><td style=\"text-align: right;\">-1.36711</td><td style=\"text-align: right;\">   100</td><td style=\"text-align: right;\">         10.7948</td><td style=\"text-align: right;\">          99</td><td style=\"text-align: right;\">        1.36711</td></tr>\n",
       "<tr><td>objective_69b2375c</td><td>TERMINATED</td><td>127.0.0.1:47516</td><td>relu        </td><td style=\"text-align: right;\">  75     </td><td style=\"text-align: right;\">17.5   </td><td style=\"text-align: right;\">17.5574 </td><td style=\"text-align: right;\">   100</td><td style=\"text-align: right;\">         10.8966</td><td style=\"text-align: right;\">          99</td><td style=\"text-align: right;\">      -17.5574 </td></tr>\n",
       "<tr><td>objective_69b3bb9a</td><td>TERMINATED</td><td>127.0.0.1:47517</td><td>tanh        </td><td style=\"text-align: right;\"> -75     </td><td style=\"text-align: right;\">12.5   </td><td style=\"text-align: right;\">-6.41984</td><td style=\"text-align: right;\">   100</td><td style=\"text-align: right;\">         10.9022</td><td style=\"text-align: right;\">          99</td><td style=\"text-align: right;\">        6.41984</td></tr>\n",
       "<tr><td>objective_69b58f60</td><td>TERMINATED</td><td>127.0.0.1:47519</td><td>relu        </td><td style=\"text-align: right;\">  25     </td><td style=\"text-align: right;\"> 2.5   </td><td style=\"text-align: right;\">12.8883 </td><td style=\"text-align: right;\">   100</td><td style=\"text-align: right;\">         10.8995</td><td style=\"text-align: right;\">          99</td><td style=\"text-align: right;\">      -12.8883 </td></tr>\n",
       "<tr><td>objective_72267d26</td><td>TERMINATED</td><td>127.0.0.1:47563</td><td>tanh        </td><td style=\"text-align: right;\"> -93.4243</td><td style=\"text-align: right;\">16.2678</td><td style=\"text-align: right;\">-8.28072</td><td style=\"text-align: right;\">   100</td><td style=\"text-align: right;\">         10.7101</td><td style=\"text-align: right;\">          99</td><td style=\"text-align: right;\">        8.28072</td></tr>\n",
       "<tr><td>objective_75ed3e0e</td><td>TERMINATED</td><td>127.0.0.1:47568</td><td>tanh        </td><td style=\"text-align: right;\">  28.8058</td><td style=\"text-align: right;\">15.0428</td><td style=\"text-align: right;\"> 3.94728</td><td style=\"text-align: right;\">   100</td><td style=\"text-align: right;\">         10.7472</td><td style=\"text-align: right;\">          99</td><td style=\"text-align: right;\">       -3.94728</td></tr>\n",
       "</tbody>\n",
       "</table><br><br>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result for objective_67ec1a0a:\n",
      "  date: 2022-07-22_15-34-37\n",
      "  done: false\n",
      "  experiment_id: b2cc3485f1024cbbbb5947a9acd341e9\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 0\n",
      "  iterations_since_restore: 1\n",
      "  mean_loss: 10.0\n",
      "  neg_mean_loss: -10.0\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47498\n",
      "  time_since_restore: 0.10423088073730469\n",
      "  time_this_iter_s: 0.10423088073730469\n",
      "  time_total_s: 0.10423088073730469\n",
      "  timestamp: 1658500477\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 1\n",
      "  trial_id: 67ec1a0a\n",
      "  warmup_time: 0.0028820037841796875\n",
      "  \n",
      "Result for objective_69ada8d6:\n",
      "  date: 2022-07-22_15-34-40\n",
      "  done: false\n",
      "  experiment_id: 9ca732d0f466455cbaa1da6f553a17ab\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 0\n",
      "  iterations_since_restore: 1\n",
      "  mean_loss: 15.0\n",
      "  neg_mean_loss: -15.0\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47513\n",
      "  time_since_restore: 0.10410189628601074\n",
      "  time_this_iter_s: 0.10410189628601074\n",
      "  time_total_s: 0.10410189628601074\n",
      "  timestamp: 1658500480\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 1\n",
      "  trial_id: 69ada8d6\n",
      "  warmup_time: 0.00498199462890625\n",
      "  \n",
      "Result for objective_69ac3226:\n",
      "  date: 2022-07-22_15-34-40\n",
      "  done: false\n",
      "  experiment_id: 5fa0de7eaf624b22bf76f0407a5dc3cd\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 0\n",
      "  iterations_since_restore: 1\n",
      "  mean_loss: 20.0\n",
      "  neg_mean_loss: -20.0\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47512\n",
      "  time_since_restore: 0.10359311103820801\n",
      "  time_this_iter_s: 0.10359311103820801\n",
      "  time_total_s: 0.10359311103820801\n",
      "  timestamp: 1658500480\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 1\n",
      "  trial_id: 69ac3226\n",
      "  warmup_time: 0.007561922073364258\n",
      "  \n",
      "Result for objective_69af2530:\n",
      "  date: 2022-07-22_15-34-40\n",
      "  done: false\n",
      "  experiment_id: 4c5fc14d64b04ec2b071fb751a9c6bde\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 0\n",
      "  iterations_since_restore: 1\n",
      "  mean_loss: 16.0\n",
      "  neg_mean_loss: -16.0\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47514\n",
      "  time_since_restore: 0.1039130687713623\n",
      "  time_this_iter_s: 0.1039130687713623\n",
      "  time_total_s: 0.1039130687713623\n",
      "  timestamp: 1658500480\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 1\n",
      "  trial_id: 69af2530\n",
      "  warmup_time: 0.002995014190673828\n",
      "  \n",
      "Result for objective_69b0a8a6:\n",
      "  date: 2022-07-22_15-34-40\n",
      "  done: false\n",
      "  experiment_id: 013499ab54ed4a4f92666a27945d673e\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 0\n",
      "  iterations_since_restore: 1\n",
      "  mean_loss: 8.5\n",
      "  neg_mean_loss: -8.5\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47515\n",
      "  time_since_restore: 0.10396409034729004\n",
      "  time_this_iter_s: 0.10396409034729004\n",
      "  time_total_s: 0.10396409034729004\n",
      "  timestamp: 1658500480\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 1\n",
      "  trial_id: 69b0a8a6\n",
      "  warmup_time: 0.00310516357421875\n",
      "  \n",
      "Result for objective_69b2375c:\n",
      "  date: 2022-07-22_15-34-40\n",
      "  done: false\n",
      "  experiment_id: 7e5aeb8fba3a42e6ae6471eedfc75fd2\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 0\n",
      "  iterations_since_restore: 1\n",
      "  mean_loss: 27.5\n",
      "  neg_mean_loss: -27.5\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47516\n",
      "  time_since_restore: 0.10422396659851074\n",
      "  time_this_iter_s: 0.10422396659851074\n",
      "  time_total_s: 0.10422396659851074\n",
      "  timestamp: 1658500480\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 1\n",
      "  trial_id: 69b2375c\n",
      "  warmup_time: 0.0030279159545898438\n",
      "  \n",
      "Result for objective_69b58f60:\n",
      "  date: 2022-07-22_15-34-40\n",
      "  done: false\n",
      "  experiment_id: 02a11d6943f04adaaf8a7c50cbcee0dd\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 0\n",
      "  iterations_since_restore: 1\n",
      "  mean_loss: 22.5\n",
      "  neg_mean_loss: -22.5\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47519\n",
      "  time_since_restore: 0.1043708324432373\n",
      "  time_this_iter_s: 0.1043708324432373\n",
      "  time_total_s: 0.1043708324432373\n",
      "  timestamp: 1658500480\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 1\n",
      "  trial_id: 69b58f60\n",
      "  warmup_time: 0.0027899742126464844\n",
      "  \n",
      "Result for objective_69b3bb9a:\n",
      "  date: 2022-07-22_15-34-40\n",
      "  done: false\n",
      "  experiment_id: a2d184f9ca934a768b78fbb438dbf28f\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 0\n",
      "  iterations_since_restore: 1\n",
      "  mean_loss: 3.5\n",
      "  neg_mean_loss: -3.5\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47517\n",
      "  time_since_restore: 0.10400700569152832\n",
      "  time_this_iter_s: 0.10400700569152832\n",
      "  time_total_s: 0.10400700569152832\n",
      "  timestamp: 1658500480\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 1\n",
      "  trial_id: 69b3bb9a\n",
      "  warmup_time: 0.002424955368041992\n",
      "  \n",
      "Result for objective_67ec1a0a:\n",
      "  date: 2022-07-22_15-34-42\n",
      "  done: false\n",
      "  experiment_id: b2cc3485f1024cbbbb5947a9acd341e9\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 41\n",
      "  iterations_since_restore: 42\n",
      "  mean_loss: 10.0\n",
      "  neg_mean_loss: -10.0\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47498\n",
      "  time_since_restore: 5.111451864242554\n",
      "  time_this_iter_s: 0.10680818557739258\n",
      "  time_total_s: 5.111451864242554\n",
      "  timestamp: 1658500482\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 42\n",
      "  trial_id: 67ec1a0a\n",
      "  warmup_time: 0.0028820037841796875\n",
      "  \n",
      "Result for objective_69ac3226:\n",
      "  date: 2022-07-22_15-34-45\n",
      "  done: false\n",
      "  experiment_id: 5fa0de7eaf624b22bf76f0407a5dc3cd\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 46\n",
      "  iterations_since_restore: 47\n",
      "  mean_loss: 10.212765957446809\n",
      "  neg_mean_loss: -10.212765957446809\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47512\n",
      "  time_since_restore: 5.153754234313965\n",
      "  time_this_iter_s: 0.10711407661437988\n",
      "  time_total_s: 5.153754234313965\n",
      "  timestamp: 1658500485\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 47\n",
      "  trial_id: 69ac3226\n",
      "  warmup_time: 0.007561922073364258\n",
      "  \n",
      "Result for objective_69af2530:\n",
      "  date: 2022-07-22_15-34-45\n",
      "  done: false\n",
      "  experiment_id: 4c5fc14d64b04ec2b071fb751a9c6bde\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 46\n",
      "  iterations_since_restore: 47\n",
      "  mean_loss: 6.416666666666667\n",
      "  neg_mean_loss: -6.416666666666667\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47514\n",
      "  time_since_restore: 5.158767938613892\n",
      "  time_this_iter_s: 0.10651683807373047\n",
      "  time_total_s: 5.158767938613892\n",
      "  timestamp: 1658500485\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 47\n",
      "  trial_id: 69af2530\n",
      "  warmup_time: 0.002995014190673828\n",
      "  \n",
      "Result for objective_69b3bb9a:\n",
      "  date: 2022-07-22_15-34-45\n",
      "  done: false\n",
      "  experiment_id: a2d184f9ca934a768b78fbb438dbf28f\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 46\n",
      "  iterations_since_restore: 47\n",
      "  mean_loss: -6.329059829059829\n",
      "  neg_mean_loss: 6.329059829059829\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47517\n",
      "  time_since_restore: 5.1304240226745605\n",
      "  time_this_iter_s: 0.10833311080932617\n",
      "  time_total_s: 5.1304240226745605\n",
      "  timestamp: 1658500485\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 47\n",
      "  trial_id: 69b3bb9a\n",
      "  warmup_time: 0.002424955368041992\n",
      "  \n",
      "Result for objective_69b58f60:\n",
      "  date: 2022-07-22_15-34-45\n",
      "  done: false\n",
      "  experiment_id: 02a11d6943f04adaaf8a7c50cbcee0dd\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 46\n",
      "  iterations_since_restore: 47\n",
      "  mean_loss: 13.3\n",
      "  neg_mean_loss: -13.3\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47519\n",
      "  time_since_restore: 5.138491868972778\n",
      "  time_this_iter_s: 0.10869002342224121\n",
      "  time_total_s: 5.138491868972778\n",
      "  timestamp: 1658500485\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 47\n",
      "  trial_id: 69b58f60\n",
      "  warmup_time: 0.0027899742126464844\n",
      "  \n",
      "Result for objective_69b2375c:\n",
      "  date: 2022-07-22_15-34-45\n",
      "  done: false\n",
      "  experiment_id: 7e5aeb8fba3a42e6ae6471eedfc75fd2\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 46\n",
      "  iterations_since_restore: 47\n",
      "  mean_loss: 17.62269938650307\n",
      "  neg_mean_loss: -17.62269938650307\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47516\n",
      "  time_since_restore: 5.13613486289978\n",
      "  time_this_iter_s: 0.10693097114562988\n",
      "  time_total_s: 5.13613486289978\n",
      "  timestamp: 1658500485\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 47\n",
      "  trial_id: 69b2375c\n",
      "  warmup_time: 0.0030279159545898438\n",
      "  \n",
      "Result for objective_69ada8d6:\n",
      "  date: 2022-07-22_15-34-45\n",
      "  done: false\n",
      "  experiment_id: 9ca732d0f466455cbaa1da6f553a17ab\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 47\n",
      "  iterations_since_restore: 48\n",
      "  mean_loss: 5.13986013986014\n",
      "  neg_mean_loss: -5.13986013986014\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47513\n",
      "  time_since_restore: 5.1575539112091064\n",
      "  time_this_iter_s: 0.10637593269348145\n",
      "  time_total_s: 5.1575539112091064\n",
      "  timestamp: 1658500485\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 48\n",
      "  trial_id: 69ada8d6\n",
      "  warmup_time: 0.00498199462890625\n",
      "  \n",
      "Result for objective_69b0a8a6:\n",
      "  date: 2022-07-22_15-34-45\n",
      "  done: false\n",
      "  experiment_id: 013499ab54ed4a4f92666a27945d673e\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 47\n",
      "  iterations_since_restore: 48\n",
      "  mean_loss: -1.2241379310344827\n",
      "  neg_mean_loss: 1.2241379310344827\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47515\n",
      "  time_since_restore: 5.211113929748535\n",
      "  time_this_iter_s: 0.10501360893249512\n",
      "  time_total_s: 5.211113929748535\n",
      "  timestamp: 1658500485\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 48\n",
      "  trial_id: 69b0a8a6\n",
      "  warmup_time: 0.00310516357421875\n",
      "  \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result for objective_67ec1a0a:\n",
      "  date: 2022-07-22_15-34-47\n",
      "  done: false\n",
      "  experiment_id: b2cc3485f1024cbbbb5947a9acd341e9\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 87\n",
      "  iterations_since_restore: 88\n",
      "  mean_loss: 10.0\n",
      "  neg_mean_loss: -10.0\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47498\n",
      "  time_since_restore: 10.140707731246948\n",
      "  time_this_iter_s: 0.10805296897888184\n",
      "  time_total_s: 10.140707731246948\n",
      "  timestamp: 1658500487\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 88\n",
      "  trial_id: 67ec1a0a\n",
      "  warmup_time: 0.0028820037841796875\n",
      "  \n",
      "Result for objective_67ec1a0a:\n",
      "  date: 2022-07-22_15-34-48\n",
      "  done: true\n",
      "  experiment_id: b2cc3485f1024cbbbb5947a9acd341e9\n",
      "  experiment_tag: 1_activation=relu,height=-100.0000,steps=100,width=0.0000\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 99\n",
      "  iterations_since_restore: 100\n",
      "  mean_loss: 10.0\n",
      "  neg_mean_loss: -10.0\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47498\n",
      "  time_since_restore: 11.438636064529419\n",
      "  time_this_iter_s: 0.1079721450805664\n",
      "  time_total_s: 11.438636064529419\n",
      "  timestamp: 1658500488\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 100\n",
      "  trial_id: 67ec1a0a\n",
      "  warmup_time: 0.0028820037841796875\n",
      "  \n",
      "Result for objective_69b3bb9a:\n",
      "  date: 2022-07-22_15-34-50\n",
      "  done: false\n",
      "  experiment_id: a2d184f9ca934a768b78fbb438dbf28f\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 92\n",
      "  iterations_since_restore: 93\n",
      "  mean_loss: -6.413793103448276\n",
      "  neg_mean_loss: 6.413793103448276\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47517\n",
      "  time_since_restore: 10.136809825897217\n",
      "  time_this_iter_s: 0.10945367813110352\n",
      "  time_total_s: 10.136809825897217\n",
      "  timestamp: 1658500490\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 93\n",
      "  trial_id: 69b3bb9a\n",
      "  warmup_time: 0.002424955368041992\n",
      "  \n",
      "Result for objective_69ac3226:\n",
      "  date: 2022-07-22_15-34-50\n",
      "  done: false\n",
      "  experiment_id: 5fa0de7eaf624b22bf76f0407a5dc3cd\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 93\n",
      "  iterations_since_restore: 94\n",
      "  mean_loss: 10.106382978723405\n",
      "  neg_mean_loss: -10.106382978723405\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47512\n",
      "  time_since_restore: 10.222928285598755\n",
      "  time_this_iter_s: 0.10617327690124512\n",
      "  time_total_s: 10.222928285598755\n",
      "  timestamp: 1658500490\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 94\n",
      "  trial_id: 69ac3226\n",
      "  warmup_time: 0.007561922073364258\n",
      "  \n",
      "Result for objective_69ada8d6:\n",
      "  date: 2022-07-22_15-34-50\n",
      "  done: false\n",
      "  experiment_id: 9ca732d0f466455cbaa1da6f553a17ab\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 94\n",
      "  iterations_since_restore: 95\n",
      "  mean_loss: 5.070422535211268\n",
      "  neg_mean_loss: -5.070422535211268\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47513\n",
      "  time_since_restore: 10.23993706703186\n",
      "  time_this_iter_s: 0.10660290718078613\n",
      "  time_total_s: 10.23993706703186\n",
      "  timestamp: 1658500490\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 95\n",
      "  trial_id: 69ada8d6\n",
      "  warmup_time: 0.00498199462890625\n",
      "  \n",
      "Result for objective_69b2375c:\n",
      "  date: 2022-07-22_15-34-50\n",
      "  done: false\n",
      "  experiment_id: 7e5aeb8fba3a42e6ae6471eedfc75fd2\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 93\n",
      "  iterations_since_restore: 94\n",
      "  mean_loss: 17.561068702290076\n",
      "  neg_mean_loss: -17.561068702290076\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47516\n",
      "  time_since_restore: 10.21057415008545\n",
      "  time_this_iter_s: 0.10399723052978516\n",
      "  time_total_s: 10.21057415008545\n",
      "  timestamp: 1658500490\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 94\n",
      "  trial_id: 69b2375c\n",
      "  warmup_time: 0.0030279159545898438\n",
      "  \n",
      "Result for objective_69b58f60:\n",
      "  date: 2022-07-22_15-34-50\n",
      "  done: false\n",
      "  experiment_id: 02a11d6943f04adaaf8a7c50cbcee0dd\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 93\n",
      "  iterations_since_restore: 94\n",
      "  mean_loss: 12.912371134020619\n",
      "  neg_mean_loss: -12.912371134020619\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47519\n",
      "  time_since_restore: 10.214950799942017\n",
      "  time_this_iter_s: 0.10687804222106934\n",
      "  time_total_s: 10.214950799942017\n",
      "  timestamp: 1658500490\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 94\n",
      "  trial_id: 69b58f60\n",
      "  warmup_time: 0.0027899742126464844\n",
      "  \n",
      "Result for objective_69af2530:\n",
      "  date: 2022-07-22_15-34-50\n",
      "  done: false\n",
      "  experiment_id: 4c5fc14d64b04ec2b071fb751a9c6bde\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 93\n",
      "  iterations_since_restore: 94\n",
      "  mean_loss: 6.2105263157894735\n",
      "  neg_mean_loss: -6.2105263157894735\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47514\n",
      "  time_since_restore: 10.267423152923584\n",
      "  time_this_iter_s: 0.10761213302612305\n",
      "  time_total_s: 10.267423152923584\n",
      "  timestamp: 1658500490\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 94\n",
      "  trial_id: 69af2530\n",
      "  warmup_time: 0.002995014190673828\n",
      "  \n",
      "Result for objective_69b0a8a6:\n",
      "  date: 2022-07-22_15-34-50\n",
      "  done: false\n",
      "  experiment_id: 013499ab54ed4a4f92666a27945d673e\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 94\n",
      "  iterations_since_restore: 95\n",
      "  mean_loss: -1.36013986013986\n",
      "  neg_mean_loss: 1.36013986013986\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47515\n",
      "  time_since_restore: 10.256262063980103\n",
      "  time_this_iter_s: 0.10606575012207031\n",
      "  time_total_s: 10.256262063980103\n",
      "  timestamp: 1658500490\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 95\n",
      "  trial_id: 69b0a8a6\n",
      "  warmup_time: 0.00310516357421875\n",
      "  \n",
      "Result for objective_69ada8d6:\n",
      "  date: 2022-07-22_15-34-51\n",
      "  done: true\n",
      "  experiment_id: 9ca732d0f466455cbaa1da6f553a17ab\n",
      "  experiment_tag: 3_activation=relu,height=-50.0000,steps=100,width=15.0000\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 99\n",
      "  iterations_since_restore: 100\n",
      "  mean_loss: 5.066889632107023\n",
      "  neg_mean_loss: -5.066889632107023\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47513\n",
      "  time_since_restore: 10.77684497833252\n",
      "  time_this_iter_s: 0.10641121864318848\n",
      "  time_total_s: 10.77684497833252\n",
      "  timestamp: 1658500491\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 100\n",
      "  trial_id: 69ada8d6\n",
      "  warmup_time: 0.00498199462890625\n",
      "  \n",
      "Result for objective_69b0a8a6:\n",
      "  date: 2022-07-22_15-34-51\n",
      "  done: true\n",
      "  experiment_id: 013499ab54ed4a4f92666a27945d673e\n",
      "  experiment_tag: 5_activation=tanh,height=-25.0000,steps=100,width=7.5000\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 99\n",
      "  iterations_since_restore: 100\n",
      "  mean_loss: -1.367109634551495\n",
      "  neg_mean_loss: 1.367109634551495\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47515\n",
      "  time_since_restore: 10.794761180877686\n",
      "  time_this_iter_s: 0.10670304298400879\n",
      "  time_total_s: 10.794761180877686\n",
      "  timestamp: 1658500491\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 100\n",
      "  trial_id: 69b0a8a6\n",
      "  warmup_time: 0.00310516357421875\n",
      "  \n",
      "Result for objective_69ac3226:\n",
      "  date: 2022-07-22_15-34-51\n",
      "  done: true\n",
      "  experiment_id: 5fa0de7eaf624b22bf76f0407a5dc3cd\n",
      "  experiment_tag: 2_activation=relu,height=0.0000,steps=100,width=10.0000\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 99\n",
      "  iterations_since_restore: 100\n",
      "  mean_loss: 10.1\n",
      "  neg_mean_loss: -10.1\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47512\n",
      "  time_since_restore: 10.901827096939087\n",
      "  time_this_iter_s: 0.13848495483398438\n",
      "  time_total_s: 10.901827096939087\n",
      "  timestamp: 1658500491\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 100\n",
      "  trial_id: 69ac3226\n",
      "  warmup_time: 0.007561922073364258\n",
      "  \n",
      "Result for objective_69b2375c:\n",
      "  date: 2022-07-22_15-34-51\n",
      "  done: true\n",
      "  experiment_id: 7e5aeb8fba3a42e6ae6471eedfc75fd2\n",
      "  experiment_tag: 6_activation=relu,height=75.0000,steps=100,width=17.5000\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 99\n",
      "  iterations_since_restore: 100\n",
      "  mean_loss: 17.55738880918221\n",
      "  neg_mean_loss: -17.55738880918221\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47516\n",
      "  time_since_restore: 10.896636962890625\n",
      "  time_this_iter_s: 0.14625000953674316\n",
      "  time_total_s: 10.896636962890625\n",
      "  timestamp: 1658500491\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 100\n",
      "  trial_id: 69b2375c\n",
      "  warmup_time: 0.0030279159545898438\n",
      "  \n",
      "Result for objective_69b3bb9a:\n",
      "  date: 2022-07-22_15-34-51\n",
      "  done: true\n",
      "  experiment_id: a2d184f9ca934a768b78fbb438dbf28f\n",
      "  experiment_tag: 7_activation=tanh,height=-75.0000,steps=100,width=12.5000\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 99\n",
      "  iterations_since_restore: 100\n",
      "  mean_loss: -6.419839679358717\n",
      "  neg_mean_loss: 6.419839679358717\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47517\n",
      "  time_since_restore: 10.902234077453613\n",
      "  time_this_iter_s: 0.12049722671508789\n",
      "  time_total_s: 10.902234077453613\n",
      "  timestamp: 1658500491\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 100\n",
      "  trial_id: 69b3bb9a\n",
      "  warmup_time: 0.002424955368041992\n",
      "  \n",
      "Result for objective_69b58f60:\n",
      "  date: 2022-07-22_15-34-51\n",
      "  done: true\n",
      "  experiment_id: 02a11d6943f04adaaf8a7c50cbcee0dd\n",
      "  experiment_tag: 8_activation=relu,height=25.0000,steps=100,width=2.5000\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 99\n",
      "  iterations_since_restore: 100\n",
      "  mean_loss: 12.888349514563107\n",
      "  neg_mean_loss: -12.888349514563107\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47519\n",
      "  time_since_restore: 10.899547815322876\n",
      "  time_this_iter_s: 0.1467878818511963\n",
      "  time_total_s: 10.899547815322876\n",
      "  timestamp: 1658500491\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 100\n",
      "  trial_id: 69b58f60\n",
      "  warmup_time: 0.0027899742126464844\n",
      "  \n",
      "Result for objective_69af2530:\n",
      "  date: 2022-07-22_15-34-51\n",
      "  done: true\n",
      "  experiment_id: 4c5fc14d64b04ec2b071fb751a9c6bde\n",
      "  experiment_tag: 4_activation=tanh,height=50.0000,steps=100,width=5.0000\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 99\n",
      "  iterations_since_restore: 100\n",
      "  mean_loss: 6.198019801980198\n",
      "  neg_mean_loss: -6.198019801980198\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47514\n",
      "  time_since_restore: 10.931232929229736\n",
      "  time_this_iter_s: 0.12574982643127441\n",
      "  time_total_s: 10.931232929229736\n",
      "  timestamp: 1658500491\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 100\n",
      "  trial_id: 69af2530\n",
      "  warmup_time: 0.002995014190673828\n",
      "  \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result for objective_72267d26:\n",
      "  date: 2022-07-22_15-34-58\n",
      "  done: false\n",
      "  experiment_id: 05d0fd74bba34c209c3fb167e5aabb6e\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 0\n",
      "  iterations_since_restore: 1\n",
      "  mean_loss: 1.657569583456299\n",
      "  neg_mean_loss: -1.657569583456299\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47563\n",
      "  time_since_restore: 0.10434603691101074\n",
      "  time_this_iter_s: 0.10434603691101074\n",
      "  time_total_s: 0.10434603691101074\n",
      "  timestamp: 1658500498\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 1\n",
      "  trial_id: 72267d26\n",
      "  warmup_time: 0.0029430389404296875\n",
      "  \n",
      "Result for objective_75ed3e0e:\n",
      "  date: 2022-07-22_15-35-00\n",
      "  done: false\n",
      "  experiment_id: 57d23b5e98454d9eb68f3dee5b5f2642\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 0\n",
      "  iterations_since_restore: 1\n",
      "  mean_loss: 13.88058437447561\n",
      "  neg_mean_loss: -13.88058437447561\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47568\n",
      "  time_since_restore: 0.10134601593017578\n",
      "  time_this_iter_s: 0.10134601593017578\n",
      "  time_total_s: 0.10134601593017578\n",
      "  timestamp: 1658500500\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 1\n",
      "  trial_id: 75ed3e0e\n",
      "  warmup_time: 0.002665996551513672\n",
      "  \n",
      "Result for objective_72267d26:\n",
      "  date: 2022-07-22_15-35-03\n",
      "  done: false\n",
      "  experiment_id: 05d0fd74bba34c209c3fb167e5aabb6e\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 47\n",
      "  iterations_since_restore: 48\n",
      "  mean_loss: -8.213329397880102\n",
      "  neg_mean_loss: 8.213329397880102\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47563\n",
      "  time_since_restore: 5.1267828941345215\n",
      "  time_this_iter_s: 0.10927891731262207\n",
      "  time_total_s: 5.1267828941345215\n",
      "  timestamp: 1658500503\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 48\n",
      "  trial_id: 72267d26\n",
      "  warmup_time: 0.0029430389404296875\n",
      "  \n",
      "Result for objective_75ed3e0e:\n",
      "  date: 2022-07-22_15-35-05\n",
      "  done: false\n",
      "  experiment_id: 57d23b5e98454d9eb68f3dee5b5f2642\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 47\n",
      "  iterations_since_restore: 48\n",
      "  mean_loss: 4.020052046405574\n",
      "  neg_mean_loss: -4.020052046405574\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47568\n",
      "  time_since_restore: 5.158367156982422\n",
      "  time_this_iter_s: 0.10702204704284668\n",
      "  time_total_s: 5.158367156982422\n",
      "  timestamp: 1658500505\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 48\n",
      "  trial_id: 75ed3e0e\n",
      "  warmup_time: 0.002665996551513672\n",
      "  \n",
      "Result for objective_72267d26:\n",
      "  date: 2022-07-22_15-35-08\n",
      "  done: false\n",
      "  experiment_id: 05d0fd74bba34c209c3fb167e5aabb6e\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 94\n",
      "  iterations_since_restore: 95\n",
      "  mean_loss: -8.277460523241512\n",
      "  neg_mean_loss: 8.277460523241512\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47563\n",
      "  time_since_restore: 10.168545961380005\n",
      "  time_this_iter_s: 0.10625672340393066\n",
      "  time_total_s: 10.168545961380005\n",
      "  timestamp: 1658500508\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 95\n",
      "  trial_id: 72267d26\n",
      "  warmup_time: 0.0029430389404296875\n",
      "  \n",
      "Result for objective_72267d26:\n",
      "  date: 2022-07-22_15-35-08\n",
      "  done: true\n",
      "  experiment_id: 05d0fd74bba34c209c3fb167e5aabb6e\n",
      "  experiment_tag: 9_activation=tanh,height=-93.4243,steps=100,width=16.2678\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 99\n",
      "  iterations_since_restore: 100\n",
      "  mean_loss: -8.280721582416527\n",
      "  neg_mean_loss: 8.280721582416527\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47563\n",
      "  time_since_restore: 10.71009612083435\n",
      "  time_this_iter_s: 0.10849308967590332\n",
      "  time_total_s: 10.71009612083435\n",
      "  timestamp: 1658500508\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 100\n",
      "  trial_id: 72267d26\n",
      "  warmup_time: 0.0029430389404296875\n",
      "  \n",
      "Result for objective_75ed3e0e:\n",
      "  date: 2022-07-22_15-35-10\n",
      "  done: false\n",
      "  experiment_id: 57d23b5e98454d9eb68f3dee5b5f2642\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 94\n",
      "  iterations_since_restore: 95\n",
      "  mean_loss: 3.950807906063858\n",
      "  neg_mean_loss: -3.950807906063858\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47568\n",
      "  time_since_restore: 10.20597505569458\n",
      "  time_this_iter_s: 0.10656380653381348\n",
      "  time_total_s: 10.20597505569458\n",
      "  timestamp: 1658500510\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 95\n",
      "  trial_id: 75ed3e0e\n",
      "  warmup_time: 0.002665996551513672\n",
      "  \n",
      "Result for objective_75ed3e0e:\n",
      "  date: 2022-07-22_15-35-11\n",
      "  done: true\n",
      "  experiment_id: 57d23b5e98454d9eb68f3dee5b5f2642\n",
      "  experiment_tag: 10_activation=tanh,height=28.8058,steps=100,width=15.0428\n",
      "  hostname: Kais-MacBook-Pro.local\n",
      "  iterations: 99\n",
      "  iterations_since_restore: 100\n",
      "  mean_loss: 3.947284919356474\n",
      "  neg_mean_loss: -3.947284919356474\n",
      "  node_ip: 127.0.0.1\n",
      "  pid: 47568\n",
      "  time_since_restore: 10.74724817276001\n",
      "  time_this_iter_s: 0.10794186592102051\n",
      "  time_total_s: 10.74724817276001\n",
      "  timestamp: 1658500511\n",
      "  timesteps_since_restore: 0\n",
      "  training_iteration: 100\n",
      "  trial_id: 75ed3e0e\n",
      "  warmup_time: 0.002665996551513672\n",
      "  \n"
     ]
    }
   ],
   "source": [
    "tuner = tune.Tuner(\n",
    "    objective,\n",
    "    tune_config=tune.TuneConfig(\n",
    "        metric=\"mean_loss\",\n",
    "        mode=\"min\",\n",
    "        search_alg=algo,\n",
    "        num_samples=num_samples,\n",
    "    ),\n",
    "    param_space=search_config,\n",
    ")\n",
    "results = tuner.fit()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "963e547b",
   "metadata": {},
   "source": [
    "Here are the hyperparamters found to minimize the mean loss of the defined objective."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8b8a23e4",
   "metadata": {
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best hyperparameters found were:  {'steps': 100, 'width': 16.267813332265522, 'height': -93.42430416543701, 'activation': 'tanh'}\n"
     ]
    }
   ],
   "source": [
    "print(\"Best hyperparameters found were: \", results.get_best_result().config)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "d0882e22",
   "metadata": {
    "tags": [
     "remove-cell"
    ],
    "vscode": {
     "languageId": "python"
    }
   },
   "outputs": [],
   "source": [
    "ray.shutdown()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "orphan": true
 },
 "nbformat": 4,
 "nbformat_minor": 5
}