hiro/ray
1
0
Fork 0
mirror of https://github.com/vale981/ray synced 2025-03-07 02:51:39 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions
ray/doc/source/ray-air/examples/tfx_tabular_train_to_serve.ipynb
xwjiang2010 29a063afdf
[air] add feast example (#25417)
2022-06-07 14:55:42 -07:00

1860 lines
110 KiB
Text
Raw Blame History

{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "VaFMt6AIhYbK"
},
"source": [
"# Tabular data training and serving with Keras and Ray AIR\n",
"\n",
"This notebook is adapted from [a Keras tutorial](https://www.tensorflow.org/tfx/tutorials/tfx/components_keras).\n",
"It uses [Chicago Taxi dataset](https://data.cityofchicago.org/Transportation/Taxi-Trips/wrvz-psew) and a DNN Keras model to predict whether a trip may generate a big tip.\n",
"\n",
"In this example, we showcase how to achieve the same tasks as the Keras Tutorial using [Ray AIR](https://docs.ray.io/en/latest/ray-air/getting-started.html), covering\n",
"every step from data ingestion to pushing a model to serving.\n",
"\n",
"1. Read a CSV into [Ray Dataset](https://docs.ray.io/en/latest/data/dataset.html).\n",
"2. Process the dataset by chaining [Ray AIR preprocessors](https://docs.ray.io/en/latest/ray-air/getting-started.html#preprocessors).\n",
"3. Train the model using the TensorflowTrainer from AIR.\n",
"4. Serve the model using Ray Serve and the above preprocessors."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sQbdfyWQhYbO"
},
"source": [
"Uncomment and run the following line in order to install all the necessary dependencies:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "YajFzmkthYbO",
"outputId": "cd4f1959-4ef4-465e-9e9d-71dfc3de28ff"
},
"outputs": [],
"source": [
"# ! pip install \"tensorflow>=2.8.0\" \"ray[tune, data, serve]>=1.12.1\"\n",
"# ! pip install fastapi"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pvSRaEHChYbP"
},
"source": [
"## Set up Ray"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LRdL3kWBhYbQ"
},
"source": [
"We will use `ray.init()` to initialize a local cluster. By default, this cluster will be composed of only the machine you are running this notebook on. If you wish to attach to an existing Ray cluster, you can do so through `ray.init(address=\"auto\")`."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "MOsHUjgdIrIW",
"outputId": "8a21ead5-bb2d-4a3d-ae41-17a313688b24"
},
"outputs": [
{
"data": {
"text/plain": [
"RayContext(dashboard_url='127.0.0.1:8265', python_version='3.7.7', ray_version='3.0.0.dev0', ray_commit='67cd984b92b50a028fe2b672cf713c4205361354', address_info={'node_ip_address': '172.31.70.13', 'raylet_ip_address': '172.31.70.13', 'redis_address': None, 'object_store_address': '/tmp/ray/session_2022-05-25_13-39-26_703288_182/sockets/plasma_store', 'raylet_socket_name': '/tmp/ray/session_2022-05-25_13-39-26_703288_182/sockets/raylet', 'webui_url': '127.0.0.1:8265', 'session_dir': '/tmp/ray/session_2022-05-25_13-39-26_703288_182', 'metrics_export_port': 57149, 'gcs_address': '172.31.70.13:9031', 'address': '172.31.70.13:9031', 'node_id': '18f82d1c3eaf9dccb9a1813246ace59f0505ea9128ecc969f4aa7ffa'})"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from pprint import pprint\n",
"import ray\n",
"\n",
"ray.shutdown()\n",
"ray.init()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "oJiSdWy2hYbR"
},
"source": [
"We can check the resources our cluster is composed of. If you are running this notebook on your local machine or Google Colab, you should see the number of CPU cores and GPUs available on the said machine."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "KlMz0dt9hYbS",
"outputId": "e7234b52-08b4-49fc-e14c-72f283b893f2"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'CPU': 8.0,\n",
" 'memory': 19643840103.0,\n",
" 'node:172.31.70.13': 1.0,\n",
" 'object_store_memory': 9821920051.0}\n"
]
}
],
"source": [
"pprint(ray.cluster_resources())"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "jAgvLbhT8nB0"
},
"source": [
"## Getting the data"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "IXQb4--97_Cf"
},
"source": [
"Let's start with defining a helper function to get the data to work with. Some columns are dropped for simplicity."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"id": "gAbhv9OqhYbT"
},
"outputs": [],
"source": [
"import pandas as pd\n",
"\n",
"LABEL = \"is_big_tip\"\n",
"\n",
"def get_data() -> pd.DataFrame:\n",
" \"\"\"Fetch the taxi fare data to work on.\"\"\"\n",
" _data = pd.read_csv(\n",
" \"https://raw.githubusercontent.com/tensorflow/tfx/master/\"\n",
" \"tfx/examples/chicago_taxi_pipeline/data/simple/data.csv\"\n",
" )\n",
" _data[LABEL] = _data[\"tips\"] / _data[\"fare\"] > 0.2\n",
" # We drop some columns here for the sake of simplicity.\n",
" return _data.drop(\n",
" [\n",
" \"tips\",\n",
" \"fare\",\n",
" \"dropoff_latitude\",\n",
" \"dropoff_longitude\",\n",
" \"pickup_latitude\",\n",
" \"pickup_longitude\",\n",
" \"pickup_census_tract\",\n",
" ],\n",
" axis=1,\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "FbeYf1aF8ISK"
},
"outputs": [],
"source": [
"data = get_data()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1WALC3kT8WgL"
},
"source": [
"Now let's take a look at the data. Notice that some values are missing. This is exactly where preprocessing comes into the picture. We will come back to this in the preprocessing session below."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 206
},
"id": "8tugpr5S8gPq",
"outputId": "3c57a348-12a7-4b6c-f9b2-fabdcb7a7c88"
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>pickup_community_area</th>\n",
" <th>trip_start_month</th>\n",
" <th>trip_start_hour</th>\n",
" <th>trip_start_day</th>\n",
" <th>trip_start_timestamp</th>\n",
" <th>trip_miles</th>\n",
" <th>dropoff_census_tract</th>\n",
" <th>payment_type</th>\n",
" <th>company</th>\n",
" <th>trip_seconds</th>\n",
" <th>dropoff_community_area</th>\n",
" <th>is_big_tip</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>NaN</td>\n",
" <td>5</td>\n",
" <td>19</td>\n",
" <td>6</td>\n",
" <td>1400269500</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>Credit Card</td>\n",
" <td>Chicago Elite Cab Corp. (Chicago Carriag</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>NaN</td>\n",
" <td>3</td>\n",
" <td>19</td>\n",
" <td>5</td>\n",
" <td>1362683700</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>Unknown</td>\n",
" <td>Chicago Elite Cab Corp.</td>\n",
" <td>300.0</td>\n",
" <td>NaN</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>60.0</td>\n",
" <td>10</td>\n",
" <td>2</td>\n",
" <td>3</td>\n",
" <td>1380593700</td>\n",
" <td>12.6</td>\n",
" <td>NaN</td>\n",
" <td>Cash</td>\n",
" <td>Taxi Affiliation Services</td>\n",
" <td>1380.0</td>\n",
" <td>NaN</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>10.0</td>\n",
" <td>10</td>\n",
" <td>1</td>\n",
" <td>2</td>\n",
" <td>1382319000</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>Cash</td>\n",
" <td>Taxi Affiliation Services</td>\n",
" <td>180.0</td>\n",
" <td>NaN</td>\n",
" <td>False</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>14.0</td>\n",
" <td>5</td>\n",
" <td>7</td>\n",
" <td>5</td>\n",
" <td>1369897200</td>\n",
" <td>0.0</td>\n",
" <td>NaN</td>\n",
" <td>Cash</td>\n",
" <td>Dispatch Taxi Affiliation</td>\n",
" <td>1080.0</td>\n",
" <td>NaN</td>\n",
" <td>False</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" pickup_community_area trip_start_month trip_start_hour trip_start_day \\\n",
"0 NaN 5 19 6 \n",
"1 NaN 3 19 5 \n",
"2 60.0 10 2 3 \n",
"3 10.0 10 1 2 \n",
"4 14.0 5 7 5 \n",
"\n",
" trip_start_timestamp trip_miles dropoff_census_tract payment_type \\\n",
"0 1400269500 0.0 NaN Credit Card \n",
"1 1362683700 0.0 NaN Unknown \n",
"2 1380593700 12.6 NaN Cash \n",
"3 1382319000 0.0 NaN Cash \n",
"4 1369897200 0.0 NaN Cash \n",
"\n",
" company trip_seconds \\\n",
"0 Chicago Elite Cab Corp. (Chicago Carriag 0.0 \n",
"1 Chicago Elite Cab Corp. 300.0 \n",
"2 Taxi Affiliation Services 1380.0 \n",
"3 Taxi Affiliation Services 180.0 \n",
"4 Dispatch Taxi Affiliation 1080.0 \n",
"\n",
" dropoff_community_area is_big_tip \n",
"0 NaN False \n",
"1 NaN False \n",
"2 NaN False \n",
"3 NaN False \n",
"4 NaN False "
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"data.head(5)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xzNQKJMA9YV-"
},
"source": [
"We continue to split the data into training and test data.\n",
"For the test data, we separate out the features to run serving on as well as labels to compare serving results with."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"id": "YSLvrBMC9aRv"
},
"outputs": [],
"source": [
"import numpy as np\n",
"from sklearn.model_selection import train_test_split\n",
"from typing import Tuple\n",
"\n",
"\n",
"def split_data(data: pd.DataFrame) -> Tuple[ray.data.Dataset, pd.DataFrame, np.array]:\n",
" \"\"\"Split the data in a stratified way.\n",
"\n",
" Returns:\n",
" A tuple containing train dataset, test data and test label.\n",
" \"\"\"\n",
" # There is a native offering in Ray Dataset for split as well.\n",
" # However, supporting stratification is a TODO there. So use\n",
" # scikit-learn equivalent here.\n",
" train_data, test_data = train_test_split(\n",
" data, stratify=data[[LABEL]], random_state=1113\n",
" )\n",
" _train_ds = ray.data.from_pandas(train_data)\n",
" _test_label = test_data[LABEL].values\n",
" _test_df = test_data.drop([LABEL], axis=1)\n",
" return _train_ds, _test_df, _test_label\n",
"\n",
"train_ds, test_df, test_label = split_data(data)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "xfhRl7eO981w",
"outputId": "f80d90ff-fc8a-4a7d-b544-31633823d596"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"There are 11251 samples for training and 3751 samples for testing.\n"
]
}
],
"source": [
"print(f\"There are {train_ds.count()} samples for training and {test_df.shape[0]} samples for testing.\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "N7tiwqdP-zVS"
},
"source": [
"## Preprocessing"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4RRkXuteIrIh"
},
"source": [
"Let's focus on preprocessing first.\n",
"Usually, input data needs to go through some preprocessing before being\n",
"fed into model. It is a good idea to package preprocessing logic into\n",
"a modularized component so that the same logic can be applied to both\n",
"training data as well as data for online serving or offline batch prediction.\n",
"\n",
"In AIR, this component is a [`Preprocessor`](https://docs.ray.io/en/latest/ray-air/getting-started.html#preprocessors).\n",
"It is constructed in a way that allows easy composition.\n",
"\n",
"Now let's construct a chained preprocessor composed of simple preprocessors, including\n",
"1. Imputer for filling missing features;\n",
"2. OneHotEncoder for encoding categorical features;\n",
"3. BatchMapper where arbitrary user-defined function can be applied to batches of records;\n",
"and so on. Take a look at [`Preprocessor`](https://docs.ray.io/en/latest/ray-air/getting-started.html#preprocessors).\n",
"The output of the preprocessing step goes into model for training."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"id": "zVvslsfMIrIh"
},
"outputs": [],
"source": [
"from ray.air.preprocessors import (\n",
" BatchMapper,\n",
" Chain,\n",
" OneHotEncoder,\n",
" SimpleImputer,\n",
")\n",
"\n",
"def get_preprocessor():\n",
" \"\"\"Construct a chain of preprocessors.\"\"\"\n",
" imputer1 = SimpleImputer(\n",
" [\"dropoff_census_tract\"], strategy=\"most_frequent\"\n",
" )\n",
" imputer2 = SimpleImputer(\n",
" [\"pickup_community_area\", \"dropoff_community_area\"],\n",
" strategy=\"most_frequent\",\n",
" )\n",
" imputer3 = SimpleImputer([\"payment_type\"], strategy=\"most_frequent\")\n",
" imputer4 = SimpleImputer(\n",
" [\"company\"], strategy=\"most_frequent\")\n",
" imputer5 = SimpleImputer(\n",
" [\"trip_start_timestamp\", \"trip_miles\", \"trip_seconds\"], strategy=\"mean\"\n",
" )\n",
"\n",
" ohe = OneHotEncoder(\n",
" columns=[\n",
" \"trip_start_hour\",\n",
" \"trip_start_day\",\n",
" \"trip_start_month\",\n",
" \"dropoff_census_tract\",\n",
" \"pickup_community_area\",\n",
" \"dropoff_community_area\",\n",
" \"payment_type\",\n",
" \"company\",\n",
" ],\n",
" limit={\n",
" \"dropoff_census_tract\": 25,\n",
" \"pickup_community_area\": 20,\n",
" \"dropoff_community_area\": 20,\n",
" \"payment_type\": 2,\n",
" \"company\": 7,\n",
" },\n",
" )\n",
"\n",
" def batch_mapper_fn(df):\n",
" df[\"trip_start_year\"] = pd.to_datetime(df[\"trip_start_timestamp\"], unit=\"s\").dt.year\n",
" df = df.drop([\"trip_start_timestamp\"], axis=1)\n",
" return df\n",
"\n",
" chained_pp = Chain(\n",
" imputer1,\n",
" imputer2,\n",
" imputer3,\n",
" imputer4,\n",
" imputer5,\n",
" ohe,\n",
" BatchMapper(batch_mapper_fn),\n",
" )\n",
" return chained_pp\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "V2BIiegi_brE"
},
"source": [
"Now let's define some constants for clarity."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"id": "ejGVU-uN_dVP"
},
"outputs": [],
"source": [
"# Note that `INPUT_SIZE` here is corresponding to the output dimension\n",
"# of the previously defined processing steps.\n",
"# This is used to specify the input shape of Keras model as well as\n",
"# when converting from training data from `ray.data.Dataset` to `tf.Tensor`.\n",
"INPUT_SIZE = 120\n",
"# The training batch size. Based on `NUM_WORKERS`, each worker\n",
"# will get its own share of this batch size. For example, if\n",
"# `NUM_WORKERS = 2`, each worker will work on 4 samples per batch.\n",
"BATCH_SIZE = 8\n",
"# Number of epoch. Adjust it based on how quickly you want the run to be.\n",
"EPOCH = 1\n",
"# Number of training workers.\n",
"# Adjust this accordingly based on the resources you have!\n",
"NUM_WORKERS = 2"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "whPRbBNbIrIl"
},
"source": [
"## Training"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "W7QYTpxXIrIl"
},
"source": [
"Let's starting with defining a simple Keras model for the classification task."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"id": "MwhAeEOuhYbV"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"def build_model():\n",
" model = tf.keras.models.Sequential()\n",
" model.add(tf.keras.Input(shape=(INPUT_SIZE,)))\n",
" model.add(tf.keras.layers.Dense(50, activation=\"relu\"))\n",
" model.add(tf.keras.layers.Dense(1, activation=\"sigmoid\"))\n",
" return model"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UVVji2YKADrh"
},
"source": [
"Now let's define the training loop. This code will be run on each training\n",
"worker in a distributed fashion. See more details [here](https://docs.ray.io/en/latest/train/train.html)."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"id": "U5pdjIzoAGRd"
},
"outputs": [],
"source": [
"from ray import train\n",
"from ray.train.tensorflow import prepare_dataset_shard\n",
"from ray.tune.integration.keras import TuneReportCallback\n",
"\n",
"def train_loop_per_worker():\n",
" dataset_shard = train.get_dataset_shard(\"train\")\n",
"\n",
" strategy = tf.distribute.experimental.MultiWorkerMirroredStrategy()\n",
" with strategy.scope():\n",
" model = build_model()\n",
" model.compile(\n",
" loss=\"binary_crossentropy\",\n",
" optimizer=\"adam\",\n",
" metrics=[\"accuracy\"],\n",
" )\n",
"\n",
" for epoch in range(EPOCH):\n",
" # This will make sure that the training workers will get their own\n",
" # share of batch to work on.\n",
" # See `ray.train.tensorflow.prepare_dataset_shard` for more information.\n",
" tf_dataset = prepare_dataset_shard(\n",
" dataset_shard.to_tf(\n",
" label_column=LABEL,\n",
" output_signature=(\n",
" tf.TensorSpec(shape=(BATCH_SIZE, INPUT_SIZE), dtype=tf.float32),\n",
" tf.TensorSpec(shape=(BATCH_SIZE,), dtype=tf.int64),\n",
" ),\n",
" batch_size=BATCH_SIZE,\n",
" drop_last=True,\n",
" )\n",
" )\n",
"\n",
" model.fit(tf_dataset, verbose=0)\n",
" # This saves checkpoint in a way that can be used by Ray Serve coherently.\n",
" train.save_checkpoint(epoch=epoch, model=model.get_weights())"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RzfPtOMoIrIu"
},
"source": [
"Now let's define a trainer that takes in the training loop,\n",
"the training dataset as well the preprocessor that we just defined.\n",
"\n",
"And run it!\n",
"\n",
"Notice that you can tune how long you want the run to be by changing ``EPOCH``."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 357
},
"id": "fzpWK7nuTJmN",
"outputId": "10020cb8-35ec-4f81-a528-0c99f7bdffea"
},
"outputs": [
{
"data": {
"text/html": [
"== Status ==<br>Current time: 2022-05-25 13:41:39 (running for 00:00:23.53)<br>Memory usage on this node: 3.4/30.9 GiB<br>Using FIFO scheduling algorithm.<br>Resources requested: 0/8 CPUs, 0/0 GPUs, 0.0/18.29 GiB heap, 0.0/9.15 GiB objects<br>Result logdir: /home/ray/ray_results/TensorflowTrainer_2022-05-25_13-41-15<br>Number of trials: 1/1 (1 TERMINATED)<br><table>\n",
"<thead>\n",
"<tr><th>Trial name </th><th>status </th><th>loc </th></tr>\n",
"</thead>\n",
"<tbody>\n",
"<tr><td>TensorflowTrainer_05521_00000</td><td>TERMINATED</td><td>172.31.70.13:961</td></tr>\n",
"</tbody>\n",
"</table><br><br>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1066)\u001b[0m WARNING:tensorflow:From /tmp/ipykernel_823/2334437326.py:8: _CollectiveAllReduceStrategyExperimental.__init__ (from tensorflow.python.distribute.collective_all_reduce_strategy) is deprecated and will be removed in a future version.\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1066)\u001b[0m Instructions for updating:\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1066)\u001b[0m use distribute.MultiWorkerMirroredStrategy instead\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1066)\u001b[0m 2022-05-25 13:41:25.540330: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1066)\u001b[0m 2022-05-25 13:41:25.540362: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1066)\u001b[0m 2022-05-25 13:41:25.540389: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (ip-172-31-70-13): /proc/driver/nvidia/version does not exist\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1066)\u001b[0m 2022-05-25 13:41:25.540926: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1066)\u001b[0m To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1066)\u001b[0m 2022-05-25 13:41:25.546818: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job worker -> {0 -> 172.31.70.13:40435, 1 -> 172.31.70.13:36503}\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1066)\u001b[0m 2022-05-25 13:41:25.547396: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:427] Started server with target: grpc://172.31.70.13:40435\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1067)\u001b[0m WARNING:tensorflow:From /tmp/ipykernel_823/2334437326.py:8: _CollectiveAllReduceStrategyExperimental.__init__ (from tensorflow.python.distribute.collective_all_reduce_strategy) is deprecated and will be removed in a future version.\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1067)\u001b[0m Instructions for updating:\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1067)\u001b[0m use distribute.MultiWorkerMirroredStrategy instead\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1067)\u001b[0m 2022-05-25 13:41:25.540330: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1067)\u001b[0m 2022-05-25 13:41:25.540362: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1067)\u001b[0m 2022-05-25 13:41:25.540389: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (ip-172-31-70-13): /proc/driver/nvidia/version does not exist\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1067)\u001b[0m 2022-05-25 13:41:25.540926: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1067)\u001b[0m To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1067)\u001b[0m 2022-05-25 13:41:25.546743: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job worker -> {0 -> 172.31.70.13:40435, 1 -> 172.31.70.13:36503}\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1067)\u001b[0m 2022-05-25 13:41:25.547263: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:427] Started server with target: grpc://172.31.70.13:36503\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1066)\u001b[0m 2022-05-25 13:41:25.699052: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1066)\u001b[0m 2022-05-25 13:41:25.755886: W tensorflow/core/framework/dataset.cc:679] Input of GeneratorDatasetOp::Dataset will not be optimized because the dataset does not implement the AsGraphDefInternal() method needed to apply optimizations.\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1067)\u001b[0m 2022-05-25 13:41:25.699183: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=1067)\u001b[0m 2022-05-25 13:41:25.734795: W tensorflow/core/framework/dataset.cc:679] Input of GeneratorDatasetOp::Dataset will not be optimized because the dataset does not implement the AsGraphDefInternal() method needed to apply optimizations.\n",
"2022-05-25 13:41:39,405\tERROR checkpoint_manager.py:193 -- Result dict has no key: training_iteration. checkpoint_score_attr must be set to a key of the result dict. Valid keys are ['trial_id', 'experiment_id', 'date', 'timestamp', 'pid', 'hostname', 'node_ip', 'config', 'done']\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Trial TensorflowTrainer_05521_00000 completed. Last result: \n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2022-05-25 13:41:39,516\tINFO tune.py:753 -- Total run time: 24.49 seconds (23.53 seconds for the tuning loop).\n"
]
}
],
"source": [
"from ray.air.train.integrations.tensorflow import TensorflowTrainer\n",
"\n",
"trainer = TensorflowTrainer(\n",
" train_loop_per_worker=train_loop_per_worker,\n",
" scaling_config={\"num_workers\": NUM_WORKERS},\n",
" datasets={\"train\": train_ds},\n",
" preprocessor=get_preprocessor(),\n",
")\n",
"result = trainer.fit()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Nb0HkOV2R4uL"
},
"source": [
"## Moving on to Serve"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "OlzjlW8QR_q6"
},
"source": [
"We will use Ray Serve to serve the trained model. A core concept of Ray Serve is [Deployment](https://docs.ray.io/en/latest/serve/core-apis.html). It allows you to define and update your business logic or models that will handle incoming requests as well as how this is exposed over HTTP or in Python.\n",
"\n",
"In the case of serving model, `ray.serve.model_wrappers.ModelWrapper` and `ray.serve.model_wrappers.ModelWrapperDeployment` wrap a `ray.air.checkpoint.Checkpoint` into a Ray Serve deployment that can readily serve HTTP requests.\n",
"Note, ``Checkpoint`` captures both model and preprocessing steps in a way compatible with Ray Serve and ensures that ml workload can transition seamlessly between training and\n",
"serving.\n",
"\n",
"This removes the boilerplate code and minimizes the effort to bring your model to production!\n",
"\n",
"Generally speaking the http request can either send in json or data.\n",
"Upon receiving this payload, Ray Serve would need some \"adapter\" to convert the request payload into some shape or form that can be accepted as input to the preprocessing steps. In this case, we send in a json request and converts it to a pandas DataFrame through `dataframe_adapter`, defined below."
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"id": "BBbcMwc9Rz66"
},
"outputs": [],
"source": [
"from fastapi import Request\n",
"\n",
"async def dataframe_adapter(request: Request):\n",
" \"\"\"Serve HTTP Adapter that reads JSON and converts to pandas DataFrame.\"\"\"\n",
" content = await request.json()\n",
" return pd.DataFrame.from_dict(content)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SOnl90IuRywD"
},
"source": [
"Now let's wrap everything in a serve endpoint that exposes a URL to where requests can be sent to."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"id": "ujmwT8ZhScq1"
},
"outputs": [],
"source": [
"from ray import serve\n",
"from ray.air.checkpoint import Checkpoint\n",
"from ray.air.predictors.integrations.tensorflow import TensorflowPredictor\n",
"from ray.serve.model_wrappers import ModelWrapperDeployment\n",
"\n",
"\n",
"def serve_model(checkpoint: Checkpoint, model_definition, adapter, name=\"Model\") -> str:\n",
" \"\"\"Expose a serve endpoint.\n",
"\n",
" Returns:\n",
" serve URL.\n",
" \"\"\"\n",
" serve.start(detached=True)\n",
" deployment = ModelWrapperDeployment.options(name=name)\n",
" deployment.deploy(\n",
" TensorflowPredictor,\n",
" checkpoint,\n",
" # This is due to a current limitation on Serve that's\n",
" # being addressed.\n",
" # TODO(xwjiang): Change to True.\n",
" batching_params=False,\n",
" model_definition=model_definition,\n",
" http_adapter=adapter,\n",
" )\n",
" return deployment.url\n"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"id": "uRe9a8947pl9"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2022-05-25 13:41:39,979\tINFO worker.py:863 -- Using address localhost:9031 set in the environment variable RAY_ADDRESS\n",
"2022-05-25 13:41:39,981\tINFO worker.py:965 -- Connecting to existing Ray cluster at address: 172.31.70.13:9031\n",
"\u001b[2m\u001b[36m(ServeController pid=1582)\u001b[0m INFO 2022-05-25 13:41:41,026 controller 1582 checkpoint_path.py:17 - Using RayInternalKVStore for controller checkpoint and recovery.\n",
"\u001b[2m\u001b[36m(ServeController pid=1582)\u001b[0m INFO 2022-05-25 13:41:41,028 controller 1582 http_state.py:115 - Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:SERVE_PROXY_ACTOR-node:172.31.70.13-0' on node 'node:172.31.70.13-0' listening on '127.0.0.1:8000'\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO: Started server process [1616]\n",
"\u001b[2m\u001b[36m(ServeController pid=1582)\u001b[0m INFO 2022-05-25 13:41:43,907 controller 1582 deployment_state.py:1219 - Adding 1 replicas to deployment 'Model'.\n"
]
}
],
"source": [
"# Generally speaking, training and serving are done in totally different ray clusters.\n",
"# To simulate that, let's shutdown the old ray cluster in preparation for serving.\n",
"ray.shutdown()\n",
"\n",
"endpoint_uri = serve_model(result.checkpoint, build_model, dataframe_adapter)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rzHSwa2bSyee"
},
"source": [
"Let's write a helper function to send requests to this endpoint and compare the results with labels."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"id": "E9m80HDmSz66"
},
"outputs": [],
"source": [
"import requests\n",
"\n",
"NUM_SERVE_REQUESTS = 100\n",
"\n",
"def send_requests(df: pd.DataFrame, label: np.array):\n",
" for i in range(NUM_SERVE_REQUESTS):\n",
" one_row = df.iloc[[i]].to_dict()\n",
" serve_result = requests.post(endpoint_uri, json=one_row).json()\n",
" print(\n",
" f\"request[{i}] prediction: {serve_result['predictions']['0']} \"\n",
" f\"- label: {str(label[i])}\"\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"id": "GFPwKc5JTgnI"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:46,962 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 4.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:46,961 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m 2022-05-25 13:41:47.030073: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m 2022-05-25 13:41:47.030109: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m 2022-05-25 13:41:47.030127: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (ip-172-31-70-13): /proc/driver/nvidia/version does not exist\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m 2022-05-25 13:41:47.030420: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,062 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 97.2ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,069 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,061 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 94.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,068 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.2ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[0] prediction: [0.06145152449607849] - label: True\n",
"request[1] prediction: [0.035036444664001465] - label: False\n",
"request[2] prediction: [0.00014954805374145508] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,152 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 81.5ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,159 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,244 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,151 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 79.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,158 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,242 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 79.9ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,251 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,340 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,250 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,339 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 84.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,347 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[3] prediction: [0.00040149688720703125] - label: False\n",
"request[4] prediction: [0.0006460845470428467] - label: False\n",
"request[5] prediction: [5.145270915818401e-05] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,348 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.1ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,432 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,439 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,431 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 79.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,438 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,524 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,531 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,522 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,529 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[6] prediction: [0.00012576580047607422] - label: False\n",
"request[7] prediction: [0.11168283224105835] - label: False\n",
"request[8] prediction: [0.001226186752319336] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,618 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.2ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,625 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,616 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,624 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,716 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 88.8ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,723 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,715 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 86.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,722 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,810 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.4ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,817 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,808 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,816 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,905 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 85.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:47,912 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,903 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 83.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:47,911 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[9] prediction: [0.34951621294021606] - label: True\n",
"request[10] prediction: [0.0005253255367279053] - label: False\n",
"request[11] prediction: [0.15207794308662415] - label: True\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,007 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 92.7ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,014 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,006 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 90.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,013 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,104 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 87.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,111 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,102 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 85.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,110 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.4ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[12] prediction: [0.18404856324195862] - label: False\n",
"request[13] prediction: [0.0002694427967071533] - label: False\n",
"request[14] prediction: [0.4514728784561157] - label: True\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,197 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,204 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,195 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,203 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,291 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.2ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,298 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,289 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,297 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,383 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.9ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,390 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,382 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,389 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,476 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.8ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,477 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.9ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,486 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,576 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 88.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,485 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,575 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 85.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[15] prediction: [0.27676916122436523] - label: True\n",
"request[16] prediction: [0.0002836883068084717] - label: False\n",
"request[17] prediction: [0.00015559792518615723] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,583 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,670 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,676 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,582 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,668 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,675 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,770 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 89.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,777 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,768 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 86.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,775 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[18] prediction: [0.00046330690383911133] - label: False\n",
"request[19] prediction: [0.0004684329032897949] - label: False\n",
"request[20] prediction: [3.3787964639486745e-06] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,867 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 88.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,875 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.2ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,865 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 85.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,874 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,961 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.5ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:48,968 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,960 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:48,967 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[21] prediction: [7.349652878474444e-05] - label: False\n",
"request[22] prediction: [0.9666146039962769] - label: False\n",
"request[23] prediction: [0.648256778717041] - label: True\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,058 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.9ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,065 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,056 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 83.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,064 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,153 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 85.7ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,160 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,152 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 83.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,159 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,247 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.7ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,254 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,246 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,253 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,339 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,346 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,338 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,345 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[24] prediction: [0.0005317330360412598] - label: False\n",
"request[25] prediction: [0.0009829699993133545] - label: False\n",
"request[26] prediction: [4.699190685641952e-05] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,431 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.1ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,437 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,429 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 79.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,436 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,523 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.7ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,530 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,522 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,529 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[27] prediction: [0.00036773085594177246] - label: False\n",
"request[28] prediction: [0.0009016096591949463] - label: False\n",
"request[29] prediction: [0.0010350942611694336] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,619 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,626 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,617 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 84.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,625 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,715 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 87.2ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,722 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,806 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 81.5ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,812 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,714 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 84.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,721 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,804 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 79.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,811 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.2ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[30] prediction: [0.00015658140182495117] - label: False\n",
"request[31] prediction: [0.0002771615982055664] - label: False\n",
"request[32] prediction: [0.00018978118896484375] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,896 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 81.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,903 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,894 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 78.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,902 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,987 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:49,993 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,986 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 79.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:49,992 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.2ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,083 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 87.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,090 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,082 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 85.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,089 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,176 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.5ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,182 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,174 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,181 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[33] prediction: [3.473066271908465e-06] - label: False\n",
"request[34] prediction: [0.6746773719787598] - label: True\n",
"request[35] prediction: [9.975024295272306e-05] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,269 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.1ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,276 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,267 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,274 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,367 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 88.5ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,373 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,365 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 85.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,372 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[36] prediction: [0.6957459449768066] - label: True\n",
"request[37] prediction: [9.089633385883644e-05] - label: False\n",
"request[38] prediction: [0.0011962354183197021] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,459 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.2ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,466 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,458 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,465 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,553 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,560 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,551 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,559 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,648 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 85.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,655 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,646 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,654 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,742 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.8ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,749 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,835 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,741 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.2ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,748 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,834 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,841 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,842 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[39] prediction: [0.17760178446769714] - label: True\n",
"request[40] prediction: [0.0002015531063079834] - label: False\n",
"request[41] prediction: [0.1619948446750641] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,930 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:50,937 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,928 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:50,936 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,028 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 88.7ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,035 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,027 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 85.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,034 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[42] prediction: [0.33818382024765015] - label: False\n",
"request[43] prediction: [0.0015011429786682129] - label: False\n",
"request[44] prediction: [0.25155213475227356] - label: True\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,123 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 85.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,131 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,122 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,129 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,222 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 88.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,229 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,220 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 85.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,227 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[45] prediction: [0.00032654404640197754] - label: False\n",
"request[46] prediction: [0.17674636840820312] - label: True\n",
"request[47] prediction: [0.0005522370338439941] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,317 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,325 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,316 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 83.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,323 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,413 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,420 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,412 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 83.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,419 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,508 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 85.4ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,515 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,507 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,514 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,603 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 85.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,611 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,602 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,610 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[48] prediction: [0.4599609673023224] - label: True\n",
"request[49] prediction: [0.00020781159400939941] - label: False\n",
"request[50] prediction: [0.904892086982727] - label: True\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,707 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 93.9ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,715 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,706 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 91.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,713 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,804 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.9ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,812 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,803 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 84.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,811 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[51] prediction: [0.00017628073692321777] - label: False\n",
"request[52] prediction: [0.000532686710357666] - label: False\n",
"request[53] prediction: [0.9158856868743896] - label: True\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,899 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.8ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,906 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,898 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,905 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:51,998 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 89.4ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,006 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:51,997 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 86.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,005 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,093 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 85.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,101 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,092 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.2ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,100 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,190 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 85.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,197 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,282 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,189 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,196 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,281 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 79.9ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[54] prediction: [5.0893559091491625e-06] - label: False\n",
"request[55] prediction: [0.00286141037940979] - label: False\n",
"request[56] prediction: [0.011083006858825684] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,290 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,374 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,381 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,289 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,373 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 79.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,380 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,469 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,476 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,468 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 83.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,475 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[57] prediction: [0.26305174827575684] - label: True\n",
"request[58] prediction: [0.00010508932609809563] - label: False\n",
"request[59] prediction: [0.00021386146545410156] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,562 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.1ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,568 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,560 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,567 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,653 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,660 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,652 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 79.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,659 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[60] prediction: [0.003042280673980713] - label: False\n",
"request[61] prediction: [0.00024968385696411133] - label: False\n",
"request[62] prediction: [0.0008098483085632324] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,750 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 87.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,757 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,748 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 84.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,756 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,844 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 85.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,851 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,843 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,850 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,936 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:52,942 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,934 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:52,941 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,028 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,035 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,026 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,034 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[63] prediction: [0.33183401823043823] - label: True\n",
"request[64] prediction: [8.115687523968518e-05] - label: False\n",
"request[65] prediction: [0.00014576315879821777] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,120 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.5ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,126 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,118 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 79.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,125 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,212 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.8ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,219 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,309 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,210 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.2ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,218 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,307 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 84.1ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[66] prediction: [0.00024360418319702148] - label: False\n",
"request[67] prediction: [0.0001226141903316602] - label: False\n",
"request[68] prediction: [1.9553499441826716e-06] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,317 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,407 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 87.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,316 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,405 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 84.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,413 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,414 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,498 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 81.7ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,505 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,497 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 79.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,504 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[69] prediction: [0.30919885635375977] - label: True\n",
"request[70] prediction: [0.0012976527214050293] - label: False\n",
"request[71] prediction: [0.00011369130515959114] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,590 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.7ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,600 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,588 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,598 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,685 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.8ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,692 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,684 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,690 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,779 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 85.2ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,786 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,778 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,785 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,871 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,878 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,870 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,877 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[72] prediction: [0.007035255432128906] - label: False\n",
"request[73] prediction: [0.0002562999725341797] - label: False\n",
"request[74] prediction: [0.00039577484130859375] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,963 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.8ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:53,970 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,962 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.2ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:53,969 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,054 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,061 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,053 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 79.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,060 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[75] prediction: [0.0020570456981658936] - label: False\n",
"request[76] prediction: [0.07112029194831848] - label: True\n",
"request[77] prediction: [0.0006128251552581787] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,149 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.1ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,156 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,148 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 83.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,155 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,241 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.8ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,248 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.1ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,334 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.2ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,240 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,247 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,333 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.6ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[78] prediction: [0.20891618728637695] - label: False\n",
"request[79] prediction: [3.706395727931522e-05] - label: False\n",
"request[80] prediction: [0.0033214688301086426] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,341 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,430 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,437 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,340 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,428 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 83.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,436 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,522 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,529 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,521 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.2ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,528 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,615 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,622 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,614 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,621 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,707 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.2ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,715 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,706 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,714 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[81] prediction: [0.35949671268463135] - label: False\n",
"request[82] prediction: [0.000265657901763916] - label: False\n",
"request[83] prediction: [0.0003078281879425049] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,807 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 89.8ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,814 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,806 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 87.2ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,813 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,901 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 84.2ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,907 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,899 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,906 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[84] prediction: [4.058940976392478e-05] - label: False\n",
"request[85] prediction: [0.0028511881828308105] - label: False\n",
"request[86] prediction: [0.8699249029159546] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:54,995 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 85.4ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,003 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:54,994 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,002 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.4ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,089 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.9ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,097 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.2ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,088 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,096 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,185 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,192 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,184 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 83.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,191 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,277 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.9ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,284 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,276 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,283 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[87] prediction: [0.0020498335361480713] - label: False\n",
"request[88] prediction: [0.0005712807178497314] - label: False\n",
"request[89] prediction: [0.0022663772106170654] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,369 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.4ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,376 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,462 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,368 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,375 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,460 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,468 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,469 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.0ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,560 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 87.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,567 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,559 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 84.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,566 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[90] prediction: [0.2362249493598938] - label: False\n",
"request[91] prediction: [0.8337559700012207] - label: False\n",
"request[92] prediction: [0.0002936720848083496] - label: False\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,652 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.4ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,659 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,651 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,658 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,744 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.7ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,750 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,742 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.1ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,749 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[93] prediction: [0.0001958608627319336] - label: False\n",
"request[94] prediction: [0.00015294551849365234] - label: False\n",
"request[95] prediction: [0.15767309069633484] - label: True\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,839 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.2ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,846 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.6ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,838 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 83.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,845 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,935 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.1ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:55,942 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,933 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 83.5ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:55,940 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:56,028 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 83.9ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:56,035 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:56,027 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 81.2ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:56,034 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:56,124 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 86.5ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:56,131 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 3.0ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:56,122 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 83.8ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:56,130 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"request[96] prediction: [0.17295515537261963] - label: True\n",
"request[97] prediction: [0.00022396445274353027] - label: False\n",
"request[98] prediction: [0.0002429187297821045] - label: False\n",
"request[99] prediction: [0.7572120428085327] - label: True\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:56,221 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 87.6ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:56,228 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:56,220 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 84.7ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:56,227 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:56,316 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 85.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:56,323 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 307 2.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:56,315 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 82.4ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:56,322 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 0.3ms\n",
"\u001b[2m\u001b[36m(HTTPProxyActor pid=1616)\u001b[0m INFO 2022-05-25 13:41:56,408 http_proxy 172.31.70.13 http_proxy.py:320 - POST /Model 200 82.9ms\n",
"\u001b[2m\u001b[36m(Model pid=1661)\u001b[0m INFO 2022-05-25 13:41:56,407 Model Model#ryhdIZ replica.py:483 - HANDLE __call__ OK 80.1ms\n"
]
}
],
"source": [
"send_requests(test_df, test_label)"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [
"jAgvLbhT8nB0"
],
"name": "tfx (1) (1) (1).ipynb",
"provenance": []
},
"interpreter": {
"hash": "3c0d54d489a08ae47a06eae2fd00ff032d6cddb527c382959b7b2575f6a8167f"
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Reference in a new issue View git blame Copy permalink