"*This example is adapted from Continual AI Avalanche quick start https://avalanche.continualai.org/*"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1VsUrzVm1W-h"
},
"source": [
"# Incremental Learning with Ray AIR\n",
"\n",
"In this example, we show how to use Ray AIR to incrementally train a simple image classification PyTorch model\n",
"on a stream of incoming tasks.\n",
"\n",
"Each task is a random permutation of the MNIST Dataset, which is a common benchmark\n",
"used for continual training. After training on all the\n",
"tasks, the model is expected to be able to make predictions on data from any task.\n",
"\n",
"In this example, we use just a naive finetuning strategy, where the model is trained\n",
"on each task, without any special methods to prevent [catastrophic forgetting](\n",
"https://en.wikipedia.org/wiki/Catastrophic_interference). Model performance is\n",
"expected to be poor.\n",
"\n",
"More precisely, this example showcases domain incremental training, in which during\n",
"prediction/testing\n",
"time, the model is asked to predict on data from tasks trained on so far with the\n",
"task ID not provided. This is opposed to task incremental training, where the task ID is\n",
"provided during prediction/testing time.\n",
"\n",
"For more information on the 3 different categories for incremental/continual\n",
"learning, please see [\"Three scenarios for continual learning\" by van de Ven and Tolias](https://arxiv.org/pdf/1904.07734.pdf)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Q3oGiuqYfj9_"
},
"source": [
"This example will cover the following:\n",
"1. Loading a PyTorch Dataset to Ray Datasets\n",
"2. Create an `Iterator[ray.data.Datasets]` abstraction to represent a stream of data to train on for incremental training.\n",
"3. Implement a custom Ray AIR preprocessor to preprocess the Dataset.\n",
"4. Incrementally train a model using data parallel training.\n",
"5. Use our trained model to perform batch prediction on test data.\n",
"6. Incrementally deploying our trained model with Ray Serve and performing online prediction queries."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "z52Y8O4q1bIk"
},
"source": [
"# Step 1: Installations and Initializing Ray\n",
"\n",
"To get started, let's first install the necessary packages: Ray AIR, torch, and torchvision. Uncomment the below lines and run the cell to install the necessary packages."
"\u001b[?25hRequirement already satisfied: msgpack<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (1.0.3)\n",
"Requirement already satisfied: virtualenv in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (20.14.1)\n",
"Requirement already satisfied: protobuf>=3.15.3 in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (3.17.3)\n",
"Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (3.13)\n",
"Requirement already satisfied: click<=8.0.4,>=7.0 in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (7.1.2)\n",
"Requirement already satisfied: attrs in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (21.4.0)\n",
"Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (2.23.0)\n",
"Requirement already satisfied: frozenlist in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (1.3.0)\n",
"Requirement already satisfied: aiosignal in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (1.2.0)\n",
"Requirement already satisfied: numpy>=1.16 in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (1.21.6)\n",
"Requirement already satisfied: jsonschema in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (4.3.3)\n",
"Requirement already satisfied: grpcio<=1.43.0,>=1.28.1 in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (1.43.0)\n",
"Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (3.7.0)\n",
"Requirement already satisfied: six>=1.5.2 in /usr/local/lib/python3.7/dist-packages (from grpcio<=1.43.0,>=1.28.1->ray==3.0.0.dev0) (1.15.0)\n",
"Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray==3.0.0.dev0) (0.18.1)\n",
"Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray==3.0.0.dev0) (4.2.0)\n",
"Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray==3.0.0.dev0) (4.11.3)\n",
"Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray==3.0.0.dev0) (5.7.1)\n",
"Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.7/dist-packages (from importlib-resources>=1.4.0->jsonschema->ray==3.0.0.dev0) (3.8.0)\n",
"Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->ray==3.0.0.dev0) (2.10)\n",
"Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->ray==3.0.0.dev0) (3.0.4)\n",
"Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->ray==3.0.0.dev0) (1.24.3)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->ray==3.0.0.dev0) (2022.5.18.1)\n",
"Requirement already satisfied: platformdirs<3,>=2 in /usr/local/lib/python3.7/dist-packages (from virtualenv->ray==3.0.0.dev0) (2.5.2)\n",
"Requirement already satisfied: distlib<1,>=0.3.1 in /usr/local/lib/python3.7/dist-packages (from virtualenv->ray==3.0.0.dev0) (0.3.4)\n",
"Installing collected packages: ray\n",
"Successfully installed ray-3.0.0.dev0\n",
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Requirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (1.11.0+cu113)\n",
"Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch) (4.2.0)\n",
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Requirement already satisfied: torchvision in /usr/local/lib/python3.7/dist-packages (0.12.0+cu113)\n",
"Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from torchvision) (2.23.0)\n",
"Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.7/dist-packages (from torchvision) (7.1.2)\n",
"Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from torchvision) (1.21.6)\n",
"Requirement already satisfied: torch==1.11.0 in /usr/local/lib/python3.7/dist-packages (from torchvision) (1.11.0+cu113)\n",
"Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torchvision) (4.2.0)\n",
"Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->torchvision) (2.10)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->torchvision) (2022.5.18.1)\n",
"Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->torchvision) (3.0.4)\n",
"Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->torchvision) (1.24.3)\n"
]
}
],
"source": [
"# !pip install -q \"ray[air]\"\n",
"# !pip install -q torch\n",
"# !pip install -q torchvision"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RpD4STX3g1dq"
},
"source": [
"Then, let's initialize Ray! We can just import and call `ray.init()`. If you are running on a Ray cluster, then you can do `ray.init(\"auto\")` to connect to the cluster instead of initiailzing a new local Ray instance."
"# If runnning on a cluster, use the below line instead.\n",
"# ray.init(\"auto\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AedcxD_FClQL"
},
"source": [
"# Step 2: Define our PyTorch Model\n",
"\n",
"Now that we have the necessary installations, let's define our PyTorch model. For this example to classify MNIST images, we will use a simple multi-layer perceptron."
"We can now create a stream of tasks (where each task contains a dataset to train on). For this example, we will create an artificial stream of tasks consisting of\n",
"permuted variations of MNIST, which is a classic benchmark in continual learning\n",
"research.\n",
"\n",
"For real-world scenarios, this step is not necessary as fresh data will already be\n",
"arriving as a stream of tasks. It does not need to be artificially created."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3SVSrkqrDJuc"
},
"source": [
"## 3a: Load MNIST Dataset to a Ray Dataset\n",
"\n",
"Let's first define a simple function that will return the original MNIST Dataset as a distributed Ray Dataset. Ray Datasets are the standard way to load and exchange data in Ray libraries and applications, read more about them [here](https://docs.ray.io/en/latest/data/dataset.html)!\n",
"\n",
"The function in the below code snippet does the following:\n",
"1. Downloads the MNIST Dataset from torchvision in-memory\n",
"2. Loads the in-memory Torch Dataset into a Ray Dataset\n",
"3. Converts the Ray Dataset into a Pandas format. Instead of the Ray Dataset iterating over tuples, it will have 2 columns: \"image\" & \"label\". \n",
"<!-- TODO: Figure out when and how to use TensorArray extension -->\n",
"<!-- The image will be stored as a multi-dimensional tensor (via the [TensorArray format](https://docs.ray.io/en/latest/data/dataset-tensor-support.html) instead of a PIL image). -->\n",
"This will allow us to apply built-in preprocessors to the Ray Dataset and allow Ray Datasets to be used with Ray AIR Predictors.\n",
" <!-- and also means that any transformations done to the images can be done in a zero-copy fashion. -->\n",
"\n",
"For this example, since we are just working with MNIST dataset, which is small, we use the [`SimpleTorchDataSource`](https://docs.ray.io/en/master/data/package-ref.html?highlight=SimpleTorchDatasource#ray.data.datasource.SimpleTorchDatasource) which just loads the full MNIST dataset into memory.\n",
"\n",
"For loading larger datasets in a parallel fashion, you should use [Ray Dataset's additional read APIs](https://docs.ray.io/en/master/data/dataset.html#supported-input-formats) to load data from parquet, csv, image files, and more!"
"Now we can create our \"stream\" abstraction. This abstraction provides two\n",
"methods (`generate_train_stream` and `generate_test_stream`) that each returns an Iterator\n",
"over Ray Datasets. Each item in this iterator contains a unique permutation of\n",
"MNIST, and is one task that we want to train on.\n",
"\n",
"In this example, \"the stream of tasks\" is contrived since all the data for all tasks exist already in an offline setting. For true online continual learning, you would want to implement a custom dataset iterator that reads from some stream datasource to produce new tasks. The only abstraction that's needed is `Iterator[ray.data.Dataset]`.\n",
"\n",
"Note that the test dataset stream has the same permutations that are used for the training dataset stream. In general for continual learning, it is expected that the data distribution of the test/prediction data follows what the model was trained on. If you notice that the distribution of new prediction queries is changing compared to the distribution of the training data, then you should probably trigger training of a new task."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "f2EagMWCN3he"
},
"outputs": [],
"source": [
"from typing import Iterator, List\n",
"import random\n",
"import numpy as np\n",
"\n",
"from ray.data import ActorPoolStrategy\n",
"\n",
"\n",
"class PermutedMNISTStream:\n",
" \"\"\"Generates streams of permuted MNIST Datasets.\n",
"# Step 4: Define the logic for Training and Inference/Prediction\n",
"\n",
"Now that we can get an Iterator over Ray Datasets, we can incrementally train our model in a data parallel fashion via Ray Train, while incrementally deploying our model via Ray Serve. Let's define some helper functions to allow us to do this!\n",
"\n",
"If you are not familiar with data parallel training, it is a form of distributed training strategies, where we have multiple model replicas, and each replica trains on a different batch of data. After each batch, the gradients are synchronized across the replicas. This effecitively allows us to train on more data in a shorter amount of time."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SBWxP1sP-G-o"
},
"source": [
"## 4a: Define our training logic for each Data Parallel worker\n",
"\n",
"The first thing we need to do is to define the training loop that will be run on each training worker. \n",
"\n",
"The training loop takes in a `config` Dict as an argument that we can use to pass in any configurations for training.\n",
"\n",
"This is just standard PyTorch training, with the difference being that we can leverage [Ray Train's utility functions](https://docs.ray.io/en/master/train/api.html#training-function-utilities):\n",
"- `ray.train.torch.prepare_model(...)`: This will prepare the model for distributed training by wrapping it in PyTorch `DistributedDataParallel` and moving it to the correct accelerator device.\n",
"- `ray.train.get_dataset_shard(...)`: This will get the Ray Dataset shard for this particular Data Parallel worker.\n",
"- `ray.train.save_checkpoint(...)`: This will tell Ray Train to save the provided arguments as a checkpoint. Checkpoints will be written to disk under the `~/ray_results` directory.\n",
"- `ray.train.load_checkpoint()`: Returns a checkpoint to resume from. This is useful for either fault tolerance purposes, or for our purposes, to continue training the same model on a new incoming dataset."
"Next, we define our `Preprocessor` to preprocess our data before training and prediction. Our preprocessor will normalize the MNIST Images by the mean and standard deviation of the MNIST training dataset. This is a common operation to do on MNIST to improve training: https://discuss.pytorch.org/t/normalization-in-the-mnist-example/457"
"To do this, we leverage the built-in `ray.air.BatchPredictor`. We define a `batch_predict` function that will take in a Checkpoint and a Test Dataset and outputs the accuracy our model achieves on the test dataset."
"## 4d: Define logic for Deploying and Querying our model\n",
"\n",
"In addition to batch inference, we also want to deploy our model so that we can submit live queries to it for online inference. We use Ray Serve's `ModelWrapperDeployment` utility to deploy our trained model. \n",
"\n",
"Once we deploy the model, we can send HTTP requests to our deployment."
" # TODO: Figure out how Serve deals with Pandas DataFrame returned by Predictors.\n",
" return results"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-NQDj0rFVUX3"
},
"source": [
"# Step 5: Putting it all together\n",
"\n",
"Once we have defined our training logic and our preprocessor, we can put everything together!\n",
"\n",
"For each dataset in our stream, we do the following:\n",
"1. Train on the dataset in Data Parallel fashion. We create a `TorchTrainer`, specify the config for the training loop we defined above, the dataset to train on, and how much we want to scale. `TorchTrainer` also accepts a `checkpoint` arg to continue training from a previously saved checkpoint.\n",
"2. Get the saved checkpoint from the training run.\n",
"3. Test our trained model on a test set containing test data from all the tasks trained on so far.\n",
"3. After training on each task, we deploy our model so we can query it for predictions.\n",
"\n",
"In this example, the training and test data for each task is well-defined beforehand by the benchmark. For real-world scenarios, this probably will not be the case. It is very likely that the prediction requests after training on one task will become the training data for the next task. \n"
"\u001b[2m\u001b[36m(_prepare_read pid=1772)\u001b[0m 2022-05-25 22:25:35,236\tWARNING torch_datasource.py:56 -- `SimpleTorchDatasource` doesn't support parallel reads. The `parallelism` argument will be ignored.\n",
"\u001b[2m\u001b[36m(_prepare_read pid=1772)\u001b[0m 2022-05-25 22:25:53,593\tWARNING torch_datasource.py:56 -- `SimpleTorchDatasource` doesn't support parallel reads. The `parallelism` argument will be ignored.\n",
"\u001b[2m\u001b[36m(_prepare_read pid=1978)\u001b[0m 2022-05-25 22:25:58,761\tWARNING torch_datasource.py:56 -- `SimpleTorchDatasource` doesn't support parallel reads. The `parallelism` argument will be ignored.\n",
"\u001b[2m\u001b[36m(_map_block_nosplit pid=2159)\u001b[0m /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:178.)\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=2197)\u001b[0m 2022-05-25 22:26:19,944\tINFO torch.py:347 -- Setting up process group for: env:// [rank=0, world_size=1]\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=2197)\u001b[0m 2022-05-25 22:26:20,033\tINFO torch.py:98 -- Moving model to device: cuda:0\n"
"2022-05-25 22:27:16,013\tERROR checkpoint_manager.py:193 -- Result dict has no key: training_iteration. checkpoint_score_attr must be set to a key of the result dict. Valid keys are ['trial_id', 'experiment_id', 'date', 'timestamp', 'pid', 'hostname', 'node_ip', 'config', 'done']\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Trial TorchTrainer_a8585_00000 completed. Last result: \n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2022-05-25 22:27:16,138\tINFO tune.py:753 -- Total run time: 74.68 seconds (74.45 seconds for the tuning loop).\n",
"Map Progress (1 actors 1 pending): 0%| | 0/1 [00:01<?, ?it/s]\u001b[2m\u001b[36m(BlockWorker pid=2267)\u001b[0m /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:178.)\n",
"\u001b[2m\u001b[36m(ServeController pid=2382)\u001b[0m INFO 2022-05-25 22:27:23,467 controller 2382 checkpoint_path.py:17 - Using RayInternalKVStore for controller checkpoint and recovery.\n",
"\u001b[2m\u001b[36m(ServeController pid=2382)\u001b[0m INFO 2022-05-25 22:27:23,470 controller 2382 http_state.py:115 - Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:XnXlnS:SERVE_PROXY_ACTOR-node:172.28.0.2-0' on node 'node:172.28.0.2-0' listening on '127.0.0.1:8000'\n",
"Shuffle Map: 0%| | 0/1 [00:00<?, ?it/s]\u001b[2m\u001b[36m(HTTPProxyActor pid=2415)\u001b[0m INFO: Started server process [2415]\n",
"\u001b[2m\u001b[36m(_prepare_read pid=2726)\u001b[0m 2022-05-25 22:27:40,353\tWARNING torch_datasource.py:56 -- `SimpleTorchDatasource` doesn't support parallel reads. The `parallelism` argument will be ignored.\n",
"\u001b[2m\u001b[36m(_map_block_nosplit pid=2909)\u001b[0m /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:178.)\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=2948)\u001b[0m 2022-05-25 22:28:01,917\tINFO torch.py:347 -- Setting up process group for: env:// [rank=0, world_size=1]\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=2948)\u001b[0m 2022-05-25 22:28:02,063\tINFO torch.py:98 -- Moving model to device: cuda:0\n"
"2022-05-25 22:28:52,221\tERROR checkpoint_manager.py:193 -- Result dict has no key: training_iteration. checkpoint_score_attr must be set to a key of the result dict. Valid keys are ['trial_id', 'experiment_id', 'date', 'timestamp', 'pid', 'hostname', 'node_ip', 'config', 'done']\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Trial TorchTrainer_e4f66_00000 completed. Last result: \n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2022-05-25 22:28:52,344\tINFO tune.py:753 -- Total run time: 69.20 seconds (68.99 seconds for the tuning loop).\n",
"Map Progress (1 actors 1 pending): 0%| | 0/2 [00:01<?, ?it/s]\u001b[2m\u001b[36m(BlockWorker pid=3027)\u001b[0m /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:178.)\n",
"\u001b[2m\u001b[36m(ServeController pid=3209)\u001b[0m INFO 2022-05-25 22:29:02,797 controller 3209 checkpoint_path.py:17 - Using RayInternalKVStore for controller checkpoint and recovery.\n",
"\u001b[2m\u001b[36m(ServeController pid=3209)\u001b[0m INFO 2022-05-25 22:29:02,802 controller 3209 http_state.py:115 - Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:lsPTvu:SERVE_PROXY_ACTOR-node:172.28.0.2-0' on node 'node:172.28.0.2-0' listening on '127.0.0.1:8000'\n",
"Shuffle Map: 0%| | 0/1 [00:00<?, ?it/s]\u001b[2m\u001b[36m(HTTPProxyActor pid=3241)\u001b[0m INFO: Started server process [3241]\n",
"\u001b[2m\u001b[36m(_prepare_read pid=3556)\u001b[0m 2022-05-25 22:29:19,825\tWARNING torch_datasource.py:56 -- `SimpleTorchDatasource` doesn't support parallel reads. The `parallelism` argument will be ignored.\n",
"\u001b[2m\u001b[36m(_map_block_nosplit pid=3738)\u001b[0m /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:178.)\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=3778)\u001b[0m 2022-05-25 22:29:41,392\tINFO torch.py:347 -- Setting up process group for: env:// [rank=0, world_size=1]\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=3778)\u001b[0m 2022-05-25 22:29:41,549\tINFO torch.py:98 -- Moving model to device: cuda:0\n"
"2022-05-25 22:30:31,831\tERROR checkpoint_manager.py:193 -- Result dict has no key: training_iteration. checkpoint_score_attr must be set to a key of the result dict. Valid keys are ['trial_id', 'experiment_id', 'date', 'timestamp', 'pid', 'hostname', 'node_ip', 'config', 'done']\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Trial TorchTrainer_2040e_00000 completed. Last result: \n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2022-05-25 22:30:31,953\tINFO tune.py:753 -- Total run time: 69.33 seconds (69.12 seconds for the tuning loop).\n",
"Map Progress (1 actors 1 pending): 0%| | 0/3 [00:01<?, ?it/s]\u001b[2m\u001b[36m(BlockWorker pid=3857)\u001b[0m /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:178.)\n",
"Map Progress (2 actors 1 pending): 33%|███▎ | 1/3 [00:04<00:08, 4.24s/it]\u001b[2m\u001b[36m(BlockWorker pid=3886)\u001b[0m /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:178.)\n",
"\u001b[2m\u001b[36m(ServeController pid=4011)\u001b[0m INFO 2022-05-25 22:30:43,081 controller 4011 checkpoint_path.py:17 - Using RayInternalKVStore for controller checkpoint and recovery.\n",
"\u001b[2m\u001b[36m(ServeController pid=4011)\u001b[0m INFO 2022-05-25 22:30:43,084 controller 4011 http_state.py:115 - Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:viEsyL:SERVE_PROXY_ACTOR-node:172.28.0.2-0' on node 'node:172.28.0.2-0' listening on '127.0.0.1:8000'\n",
"Shuffle Map: 0%| | 0/1 [00:00<?, ?it/s]\u001b[2m\u001b[36m(HTTPProxyActor pid=4043)\u001b[0m INFO: Started server process [4043]\n",
" if ray.available_resources().get(\"CPU\", 0) < num_workers+1:\n",
" # If there are no more CPUs left, then shutdown the Serve replicas so we can continue training on the next task.\n",
" serve.shutdown()\n",
"\n",
" \n",
"serve.shutdown()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ORWpRkPjcPbD"
},
"source": [
"Now that we have finished all of our training, let's see the accuracy of our model after training on each task. \n",
"\n",
"We should see the accuracy decrease over time. This is to be expected since we are using just a naive fine-tuning strategy so our model is prone to catastrophic forgetting.\n",
"\n",
"As we increase the number of tasks, the model performance on all the tasks trained on so far should decrease."
"# [Optional] Step 6: Compare against full training.\n",
"\n",
"We have now incrementally trained our simple multi-layer perceptron. Let's compare the incrementally trained model via fine tuning against a model that is trained on all the tasks up front.\n",
"\n",
"Since we are using a naive fine-tuning strategy, we should expect that our incrementally trained model will perform worse than the the one that is fully trained! However, there's various other strategies that have been developed and are actively being researched to improve accuracy for incremental training. And overall, incremental/continual learning allows you to train in many real world settings where the entire dataset is not available up front, but new data is arriving at a relatively high rate."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RNHsEVBHc0p2"
},
"source": [
"Let's first combine all of our datasets for each task into a single, unified Dataset"
"\u001b[2m\u001b[1m\u001b[36m(scheduler +8m58s)\u001b[0m Tip: use `ray status` to view detailed cluster status. To disable these messages, set RAY_SCHEDULER_EVENTS=0.\n",
"\u001b[2m\u001b[1m\u001b[33m(scheduler +8m58s)\u001b[0m Warning: The following resource request cannot be scheduled right now: {'CPU': 1.0}. This is likely due to all cluster resources being claimed by actors. Consider creating fewer actors or adding more nodes to this Ray cluster.\n"
"\u001b[2m\u001b[36m(_map_block_nosplit pid=4666)\u001b[0m /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:178.)\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=4709)\u001b[0m 2022-05-18 23:50:06,950\tINFO torch.py:347 -- Setting up process group for: env:// [rank=0, world_size=1]\n",
"\u001b[2m\u001b[36m(BaseWorkerMixin pid=4709)\u001b[0m 2022-05-18 23:50:07,011\tINFO torch.py:98 -- Moving model to device: cuda:0\n"
"2022-05-18 23:52:49,915\tERROR checkpoint_manager.py:193 -- Result dict has no key: training_iteration. checkpoint_score_attr must be set to a key of the result dict. Valid keys are ['trial_id', 'experiment_id', 'date', 'timestamp', 'pid', 'hostname', 'node_ip', 'config', 'done']\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Trial TorchTrainer_24496_00000 completed. Last result: \n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2022-05-18 23:52:50,042\tINFO tune.py:753 -- Total run time: 207.53 seconds (207.39 seconds for the tuning loop).\n"
]
}
],
"source": [
"# Now we do training with the same configurations as before\n",
"Map Progress (1 actors 1 pending): 0%| | 0/3 [00:01<?, ?it/s]\u001b[2m\u001b[36m(BlockWorker pid=4840)\u001b[0m /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:178.)\n",
"Finally, let's compare the accuracies between the incrementally trained model and the fully trained model. We should see that the fully trained model performs better."
"Incrementally trained model accuracy: 0.9207666666666666\n"
]
}
],
"source": [
"print(\"Fully trained model accuracy: \", fully_trained_accuracy)\n",
"print(\"Incrementally trained model accuracy: \", accuracy_for_all_tasks[-1])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FuqKePrYe-Fz"
},
"source": [
"# Next Steps\n",
"\n",
"Once you've completed this notebook, you should be set to play around with scalable incremental training using Ray, either by trying more fancy algorithms for incremental learning other than naive fine-tuning, or attempting to scale out to larger datasets!\n",
"\n",
"If you run into any issues, or have any feature requests, please file an issue on the [Ray Github](https://github.com/ray-project/ray/issues).\n",
