09_pytorch_model_deployment.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o8Ztt63FQaQA"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mrdbourke/pytorch-deep-learning/blob/main/09_pytorch_model_deployment.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
        "\n",
        "[View Source Code](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/09_pytorch_model_deployment.ipynb) | [View Slides](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/slides/09_pytorch_model_deployment.pdf)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PEMkPE-lQaQC"
      },
      "source": [
        "# 09. PyTorch Model Deployment\n",
        "\n",
        "Welcome to Milestone Project 3: PyTorch Model Deployment!\n",
        "\n",
        "We've come a long way with our FoodVision Mini project.\n",
        "\n",
        "But so far our PyTorch models have only been accessible to us.\n",
        "\n",
        "How about we bring FoodVision Mini to life and make it publically accessible?\n",
        "\n",
        "In other words, **we're going to deploy our FoodVision Mini model to the internet as a usable app!**\n",
        "\n",
        "<img src=\"https://github.com/mrdbourke/pytorch-deep-learning/raw/main/images/09-model-deployment-what-were-doing-demo-trimmed-cropped-small.gif\" alt=\"demo of foodvision mini computer vision model being used on a mobile device to predict on an image of sushi and getting it right\" width=900/>\n",
        "\n",
        "*Trying out the [deployed version of FoodVision Mini](https://huggingface.co/spaces/mrdbourke/foodvision_mini) (what we're going to build) on my lunch. The model got it right too 🍣!*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xGPs4HoMQaQC"
      },
      "source": [
        "## What is machine learning model deployment?\n",
        "\n",
        "**Machine learning model deployment** is the process of making your machine learning model accessible to someone or something else.\n",
        "\n",
        "Someone else being a person who can interact with your model in some way.\n",
        "\n",
        "For example, someone taking a photo on their smartphone of food and then having our FoodVision Mini model classify it into pizza, steak or sushi.\n",
        "\n",
        "Something else might be another program, app or even another model that interacts with your machine learning model(s).\n",
        "\n",
        "For example, a banking database might rely on a machine learning model making predictions as to whether a transaction is fraudulent or not before transferring funds.\n",
        "\n",
        "Or an operating system may lower its resource consumption based on a machine learning model making predictions on how much power someone generally uses at specific times of day.\n",
        "\n",
        "These use cases can be mixed and matched as well.\n",
        "\n",
        "For example, a Tesla car's computer vision system will interact with the car's route planning program (something else) and then the route planning program will get inputs and feedback from the driver (someone else).\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-what-is-model-deployment-someone-or-something-else.png\" width=900 alt=\"two use cases for model deployment, making your model available to someone else, for example, someone using it in an app, or making it available to something else such as another program or model\"/>\n",
        "\n",
        "*Machine learning model deployment involves making your model available to someone or something else. For example, someone might use your model as part of a food recognition app (such as FoodVision Mini or [Nutrify](https://nutrify.app)). And something else might be another model or program using your model such as a banking system using a machine learning model to detect if a transaction is fraud or not.*\n",
        "          \n",
        "          \n",
        "          "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O1ZRpruJQaQD"
      },
      "source": [
        "## Why deploy a machine learning model?\n",
        "\n",
        "One of the most important philosophical questions in machine learning is:\n",
        "\n",
        "<div align=\"center\">\n",
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-does-it-exist.jpeg\" alt=\"curious dinosaur often referred to as philosoraptor asking the question if a machine learning model never leaves a notebook, does it exist?\" width=300/>\n",
        "</div>\n",
        "\n",
        "Deploying a model is as important as training one.\n",
        "\n",
        "Because although you can get a pretty good idea of how your model's going to function by evaluting it on a well crafted test set or visualizing its results, you never really know how it's going to perform until you release it to the wild.\n",
        "\n",
        "Having people who've never used your model interact with it will often reveal edge cases you never thought of during training.\n",
        "\n",
        "For example, what happens if someone was to upload a photo that *wasn't* of food to our FoodVision Mini model?\n",
        "\n",
        "One solution would be to create another model that first classifies images as \"food\" or \"not food\" and passing the target image through that model first (this is what [Nutrify](https://nutrify.app) does).\n",
        "\n",
        "Then if the image is of \"food\" it goes to our FoodVision Mini model and gets classified into pizza, steak or sushi.\n",
        "\n",
        "And if it's \"not food\", a message is displayed.\n",
        "\n",
        "But what if these predictions were wrong?\n",
        "\n",
        "What happens then?\n",
        "\n",
        "You can see how these questions could keep going.\n",
        "\n",
        "Thus this highlights the importance of model deployment: it helps you figure out errors in your model that aren't obvious during training/testing.\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-pytorch-workflow-with-deployment.png\" alt=\"A PyTorch workflow with added model deployment and monitoring step\" width=900/>\n",
        "\n",
        "*We covered a PyTorch workflow back in [01. PyTorch Workflow](https://www.learnpytorch.io/01_pytorch_workflow/). But once you've got a good model, deployment is a good next step. Monitoring involves seeing how your model goes on the most important data split: data from the real world. For more resources on deployment and monitoring see [PyTorch Extra Resources](https://www.learnpytorch.io/pytorch_extra_resources/#resources-for-machine-learning-and-deep-learning-engineering).*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "95f7MBruQaQD"
      },
      "source": [
        "## Different types of machine learning model deployment\n",
        "\n",
        "Whole books could be written on the different types of machine learning model deployment (and many good ones are listed in [PyTorch Extra Resources](https://www.learnpytorch.io/pytorch_extra_resources/#resources-for-machine-learning-and-deep-learning-engineering)).\n",
        "\n",
        "And the field is still developing in terms of best practices.\n",
        "\n",
        "But I like to start with the question:\n",
        "\n",
        "> \"What is the most ideal scenario for my machine learning model to be used?\"\n",
        "\n",
        "And then work backwards from there.\n",
        "\n",
        "Of course, you may not know this ahead of time. But you're smart enough to imagine such things.\n",
        "\n",
        "In the case of FoodVision Mini, our ideal scenario might be:\n",
        "\n",
        "* Someone takes a photo on a mobile device (through an app or web broswer).\n",
        "* The prediction comes back fast.\n",
        "\n",
        "Easy.\n",
        "\n",
        "So we've got two main criteria:\n",
        "\n",
        "1. The model should work on a mobile device (this means there will be some compute constraints).\n",
        "2. The model should make predictions *fast* (because a slow app is a boring app).\n",
        "\n",
        "And of course, depending on your use case, your requirements may vary.\n",
        "\n",
        "You may notice the above two points break down into another two questions:\n",
        "\n",
        "1. **Where's it going to go?** - As in, where is it going to be stored?\n",
        "2. **How's it going to function?** - As in, does it return predictions immediately? Or do they come later?\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-deployment-questions-to-ask.png\" alt=\"some questions to ask when starting to deploy machine learning models, what's the model ideal use case, then work backwards and ask where's my model going to go and how's my model going to function\" width=900/>\n",
        "\n",
        "*When starting to deploy machine learning models, it's helpful to start by asking what's the most ideal use case and then work backwards from there, asking where the model's going to go and then how it's going to function.*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "nk7w6mRxQaQD"
      },
      "source": [
        "### Where's it going to go?\n",
        "\n",
        "When you deploy your machine learning model, where does it live?\n",
        "\n",
        "The main debate here is usually on-device (also called edge/in the browser) or on the cloud (a computer/server that isn't the *actual* device someone/something calls the model from).\n",
        "\n",
        "Both have their pros and cons.\n",
        "\n",
        "| **Deployment location** | **Pros** | **Cons** |\n",
        "| ----- | ----- | ----- |\n",
        "| **On-device (edge/in the browser)** | Can be very fast (since no data leaves the device) | Limited compute power (larger models take longer to run) |\n",
        "| | Privacy preserving (again no data has to leave the device) | Limited storage space (smaller model size required) |\n",
        "| | No internet connection required (sometimes) | Device-specific skills often required |\n",
        "| | | |\n",
        "| **On cloud** | Near unlimited compute power (can scale up when needed) | Costs can get out of hand (if proper scaling limits aren't enforced) |\n",
        "| | Can deploy one model and use everywhere (via API) | Predictions can be slower due to data having to leave device and predictions having to come back (network latency) |\n",
        "| | Links into existing cloud ecosystem | Data has to leave device (this may cause privacy concerns) |\n",
        "\n",
        "There are more details to these but I've left resources in the [extra-curriculum](https://www.learnpytorch.io/09_pytorch_model_deployment/#extra-curriculum) to learn more.\n",
        "\n",
        "Let's give an example.\n",
        "\n",
        "If we're deploying FoodVision Mini as an app, we want it to perform well and fast.\n",
        "\n",
        "So which model would we prefer?\n",
        "\n",
        "1. A model on-device that performs at 95% accuracy with an inference time (latency) of one second per prediction.\n",
        "2. A model on the cloud that performs at 98% accuracy with an inference time of 10 seconds per per prediction (bigger, better model but takes longer to compute).\n",
        "\n",
        "I've made these numbers up but they showcase a potential difference between on-device and on the cloud.\n",
        "\n",
        "Option 1 could potentially be a smaller less performant model that runs fast because its able to fit on a mobile device.\n",
        "\n",
        "Option 2 could potentially a larger more performant model that requires more compute and storage but it takes a bit longer to run because we have to send data off the device and get it back (so even though the actual prediction might be fast, the network time and data transfer has to factored in).\n",
        "\n",
        "For FoodVision Mini, we'd likely prefer option 1, because the small hit in performance is far outweighed by the faster inference speed.\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-model-deployment-on-device-vs-cloud.png\" width=900 alt=\"tesla computer vision system on device vs on the cloud\"/>\n",
        "\n",
        "*In the case of a Tesla car's computer vision system, which would be better? A smaller model that performs well on device (model is on the car) or a larger model that performs better that's on the cloud? In this case, you'd much prefer the model being on the car. The extra network time it would take for data to go from the car to the cloud and then back to the car just wouldn't be worth it (or potentially even impossible with poor signal areas).*\n",
        "\n",
        "> **Note:** For a full example of seeing what it's like to deploy a PyTorch model to an edge device, see the [PyTorch tutorial on achieving real-time inference (30fps+)](https://pytorch.org/tutorials/intermediate/realtime_rpi.html) with a computer vision model on a Raspberry Pi."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K4COD719QaQD"
      },
      "source": [
        "### How's it going to function?\n",
        "\n",
        "Back to the ideal use case, when you deploy your machine learning model, how should it work?\n",
        "\n",
        "As in, would you like predictions returned immediately?\n",
        "\n",
        "Or is it okay for them to happen later?\n",
        "\n",
        "These two scenarios are generally referred to as:\n",
        "\n",
        "* **Online (real-time)** - Predicitions/inference happen **immediately**. For example, someone uploads an image, the image gets transformed and predictions are returned or someone makes a purchase and the transaction is verified to be non-fradulent by a model so the purchase can go through.\n",
        "* **Offline (batch)** - Predictions/inference happen **periodically**. For example, a photos application sorts your images into different categories (such as beach, mealtime, family, friends) whilst your mobile device is plugged into charge.\n",
        "\n",
        "> **Note:** \"Batch\" refers to inference being performed on multiple samples at a time. However, to add a little confusion, batch processing can happen immediately/online (multiple images being classified at once) and/or offline (mutliple images being predicted/trained on at once).  \n",
        "\n",
        "The main difference between each being: predictions being made immediately or periodically.\n",
        "\n",
        "Periodically can have a varying timescale too, from every few seconds to every few hours or days.\n",
        "\n",
        "And you can mix and match the two.\n",
        "\n",
        "In the case of FoodVision Mini, we'd want our inference pipeline to happen online (real-time), so when someone uploads an image of pizza, steak or sushi, the prediction results are returned immediately (any slower than real-time would make a boring experience).\n",
        "\n",
        "But for our training pipeline, it's okay for it to happen in a batch (offline) fashion, which is what we've been doing throughout the previous chapters."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YsxH-k2XQaQD"
      },
      "source": [
        "### Ways to deploy a machine learning model\n",
        "\n",
        "We've discussed a couple of options for deploying machine learning models (on-device and cloud).\n",
        "\n",
        "And each of these will have their specific requirements:\n",
        "\n",
        "| **Tool/resource** | **Deployment type** |\n",
        "| ----- | ----- |\n",
        "| [Google's ML Kit](https://developers.google.com/ml-kit) | On-device (Android and iOS) |\n",
        "| [Apple's Core ML](https://developer.apple.com/documentation/coreml) and [`coremltools` Python package](https://coremltools.readme.io/docs) | On-device (all Apple devices) |\n",
        "| [Amazon Web Service's (AWS) Sagemaker](https://aws.amazon.com/sagemaker/) | Cloud |\n",
        "| [Google Cloud's Vertex AI](https://cloud.google.com/vertex-ai) | Cloud |\n",
        "| [Microsoft's Azure Machine Learning](https://azure.microsoft.com/en-au/services/machine-learning/) | Cloud |\n",
        "| [Hugging Face Spaces](https://huggingface.co/spaces) | Cloud |\n",
        "| API with [FastAPI](https://fastapi.tiangolo.com) | Cloud/self-hosted server |\n",
        "| API with [TorchServe](https://pytorch.org/serve/) | Cloud/self-hosted server |\n",
        "| [ONNX (Open Neural Network Exchange)](https://onnx.ai/index.html) | Many/general |\n",
        "| Many more... |\n",
        "\n",
        "> **Note:** An [application programming interface (API)](https://en.wikipedia.org/wiki/API) is a way for two (or more) computer programs to interact with each other. For example, if your model was deployed as API, you would be able to write a program that could send data to it and then receive predictions back.\n",
        "\n",
        "Which option you choose will be highly dependent on what you're building/who you're working with.\n",
        "\n",
        "But with so many options, it can be very intimidating.\n",
        "\n",
        "So best to start small and keep it simple.\n",
        "\n",
        "And one of the best ways to do so is by turning your machine learning model into a demo app with [Gradio](https://gradio.app) and then deploying it on Hugging Face Spaces.\n",
        "\n",
        "We'll be doing just that with FoodVision Mini later on.\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-tools-and-places-to-deploy-ml-models.png\" alt=\"tools and places to deploy machine learning models\" width=900/>\n",
        "\n",
        "*A handful of places and tools to host and deploy machine learning models. There are plenty I've missed so if you'd like to add more, please leave a [discussion on GitHub](https://github.com/mrdbourke/pytorch-deep-learning/discussions).*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k9C_10LEQaQE"
      },
      "source": [
        "## What we're going to cover\n",
        "\n",
        "Enough talking about deploying a machine learning model.\n",
        "\n",
        "Let's become machine learning engineers and actually deploy one.\n",
        "\n",
        "Our goal is to deploy our FoodVision Model via a demo Gradio app with the following metrics:\n",
        "1. **Performance:** 95%+ accuracy.\n",
        "2. **Speed:** real-time inference of 30FPS+ (each prediction has a latency of lower than ~0.03s).\n",
        "\n",
        "We'll start by running an experiment to compare our best two models so far: EffNetB2 and ViT feature extractors.\n",
        "\n",
        "Then we'll deploy the one which performs closest to our goal metrics.\n",
        "\n",
        "Finally, we'll finish with a (BIG) surprise bonus.\n",
        "\n",
        "| **Topic** | **Contents** |\n",
        "| ----- | ----- |\n",
        "| **0. Getting setup** | We've written a fair bit of useful code over the past few sections, let's download it and make sure we can use it again. |\n",
        "| **1. Get data** | Let's download the [`pizza_steak_sushi_20_percent.zip`](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/data/pizza_steak_sushi_20_percent.zip) dataset so we can train our previously best performing models on the same dataset. |\n",
        "| **2. FoodVision Mini model deployment experiment outline** | Even on the third milestone project, we're still going to be running multiple experiments to see which model (EffNetB2 or ViT) achieves closest to our goal metrics. |\n",
        "| **3. Creating an EffNetB2 feature extractor** | An EfficientNetB2 feature extractor performed the best on our pizza, steak, sushi dataset in [07. PyTorch Experiment Tracking](https://www.learnpytorch.io/07_pytorch_experiment_tracking/), let's recreate it as a candidate for deployment. |\n",
        "| **4. Creating a ViT feature extractor** | A ViT feature extractor has been the best performing model yet on our pizza, steak, sushi dataset in [08. PyTorch Paper Replicating](https://www.learnpytorch.io/08_pytorch_paper_replicating/), let's recreate it as a candidate for deployment alongside EffNetB2. |\n",
        "| **5. Making predictions with our trained models and timing them** | We've built two of the best performing models yet, let's make predictions with them and track their results. |\n",
        "| **6. Comparing model results, prediction times and size** | Let's compare our models to see which performs best with our goals. |\n",
        "| **7. Bringing FoodVision Mini to life by creating a Gradio demo** | One of our models performs better than the other (in terms of our goals), so let's turn it into a working app demo! |\n",
        "| **8. Turning our FoodVision Mini Gradio demo into a deployable app** | Our Gradio app demo works locally, let's prepare it for deployment! |\n",
        "| **9. Deploying our Gradio demo to HuggingFace Spaces** | Let's take FoodVision Mini to the web and make it pubically accessible for all! |\n",
        "| **10. Creating a BIG surprise** | We've built FoodVision Mini, time to step things up a notch. |\n",
        "| **11. Deploying our BIG surprise** | Deploying one app was fun, how about we make it two? |"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iKzNMCGDQaQE"
      },
      "source": [
        "## Where can you get help?\n",
        "\n",
        "All of the materials for this course [are available on GitHub](https://github.com/mrdbourke/pytorch-deep-learning).\n",
        "\n",
        "If you run into trouble, you can ask a question on the course [GitHub Discussions page](https://github.com/mrdbourke/pytorch-deep-learning/discussions).\n",
        "\n",
        "And of course, there's the [PyTorch documentation](https://pytorch.org/docs/stable/index.html) and [PyTorch developer forums](https://discuss.pytorch.org/), a very helpful place for all things PyTorch."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uugvtvhHQaQE"
      },
      "source": [
        "## 0. Getting setup\n",
        "\n",
        "As we've done previously, let's make sure we've got all of the modules we'll need for this section.\n",
        "\n",
        "We'll import the Python scripts (such as `data_setup.py` and `engine.py`) we created in [05. PyTorch Going Modular](https://www.learnpytorch.io/05_pytorch_going_modular/).\n",
        "\n",
        "To do so, we'll download [`going_modular`](https://github.com/mrdbourke/pytorch-deep-learning/tree/main/going_modular) directory from the [`pytorch-deep-learning` repository](https://github.com/mrdbourke/pytorch-deep-learning) (if we don't already have it).\n",
        "\n",
        "We'll also get the [`torchinfo`](https://github.com/TylerYep/torchinfo) package if it's not available.\n",
        "\n",
        "`torchinfo` will help later on to give us a visual representation of our model.\n",
        "\n",
        "And since later on we'll be using `torchvision` v0.13 package (available as of July 2022), we'll make sure we've got the latest versions.\n",
        "\n",
        "> **Note:** If you're using Google Colab, and you don't have a GPU turned on yet, it's now time to turn one on via `Runtime -> Change runtime type -> Hardware accelerator -> GPU`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YGoNW45jQaQE",
        "outputId": "32186fa7-795c-4139-c3e6-084e522f19f7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "torch version: 1.13.0.dev20220824+cu113\n",
            "torchvision version: 0.14.0.dev20220824+cu113\n"
          ]
        }
      ],
      "source": [
        "# For this notebook to run with updated APIs, we need torch 1.12+ and torchvision 0.13+\n",
        "try:\n",
        "    import torch\n",
        "    import torchvision\n",
        "    assert int(torch.__version__.split(\".\")[1]) >= 12, \"torch version should be 1.12+\"\n",
        "    assert int(torchvision.__version__.split(\".\")[1]) >= 13, \"torchvision version should be 0.13+\"\n",
        "    print(f\"torch version: {torch.__version__}\")\n",
        "    print(f\"torchvision version: {torchvision.__version__}\")\n",
        "except:\n",
        "    print(f\"[INFO] torch/torchvision versions not as required, installing nightly versions.\")\n",
        "    !pip3 install -U torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113\n",
        "    import torch\n",
        "    import torchvision\n",
        "    print(f\"torch version: {torch.__version__}\")\n",
        "    print(f\"torchvision version: {torchvision.__version__}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U47e9I5AQaQF"
      },
      "source": [
        "> **Note:** If you're using Google Colab and the cell above starts to install various software packages, you may have to restart your runtime after running the above cell. After restarting, you can run the cell again and verify you've got the right versions of `torch` and `torchvision`.\n",
        "\n",
        "Now we'll continue with the regular imports, setting up device agnostic code and this time we'll also get the [`helper_functions.py`](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/helper_functions.py) script from GitHub.\n",
        "\n",
        "The `helper_functions.py` script contains several functions we created in previous sections:\n",
        "* `set_seeds()` to set the random seeds (created in [07. PyTorch Experiment Tracking section 0](https://www.learnpytorch.io/07_pytorch_experiment_tracking/#create-a-helper-function-to-set-seeds)).\n",
        "* `download_data()` to download a data source given a link (created in [07. PyTorch Experiment Tracking section 1](https://www.learnpytorch.io/07_pytorch_experiment_tracking/#1-get-data)).\n",
        "* `plot_loss_curves()` to inspect our model's training results (created in [04. PyTorch Custom Datasets section 7.8](https://www.learnpytorch.io/04_pytorch_custom_datasets/#78-plot-the-loss-curves-of-model-0))\n",
        "\n",
        "> **Note:** It may be a better idea for many of the functions in the `helper_functions.py` script to be merged into `going_modular/going_modular/utils.py`, perhaps that's an extension you'd like to try.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "daffmIB8QaQF"
      },
      "outputs": [],
      "source": [
        "# Continue with regular imports\n",
        "import matplotlib.pyplot as plt\n",
        "import torch\n",
        "import torchvision\n",
        "\n",
        "from torch import nn\n",
        "from torchvision import transforms\n",
        "\n",
        "# Try to get torchinfo, install it if it doesn't work\n",
        "try:\n",
        "    from torchinfo import summary\n",
        "except:\n",
        "    print(\"[INFO] Couldn't find torchinfo... installing it.\")\n",
        "    !pip install -q torchinfo\n",
        "    from torchinfo import summary\n",
        "\n",
        "# Try to import the going_modular directory, download it from GitHub if it doesn't work\n",
        "try:\n",
        "    from going_modular.going_modular import data_setup, engine\n",
        "    from helper_functions import download_data, set_seeds, plot_loss_curves\n",
        "except:\n",
        "    # Get the going_modular scripts\n",
        "    print(\"[INFO] Couldn't find going_modular or helper_functions scripts... downloading them from GitHub.\")\n",
        "    !git clone https://github.com/mrdbourke/pytorch-deep-learning\n",
        "    !mv pytorch-deep-learning/going_modular .\n",
        "    !mv pytorch-deep-learning/helper_functions.py . # get the helper_functions.py script\n",
        "    !rm -rf pytorch-deep-learning\n",
        "    from going_modular.going_modular import data_setup, engine\n",
        "    from helper_functions import download_data, set_seeds, plot_loss_curves"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "epBBEPitQaQH"
      },
      "source": [
        "Finally, we'll setup device-agnostic code to make sure our models run on the GPU."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gsV0RgknQaQH",
        "outputId": "113b2640-4697-4cce-d12a-fcf106cb3226"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'cuda'"
            ]
          },
          "execution_count": 3,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
        "device"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J2oUrGH1QaQH"
      },
      "source": [
        "## 1. Getting data\n",
        "\n",
        "We left off in [08. PyTorch Paper Replicating](https://www.learnpytorch.io/08_pytorch_paper_replicating/#106-save-feature-extractor-vit-model-and-check-file-size) comparing our own Vision Transformer (ViT) feature extractor model to the EfficientNetB2 (EffNetB2) feature extractor model we created in [07. PyTorch Experiment Tracking](https://www.learnpytorch.io/07_pytorch_experiment_tracking/#9-load-in-the-best-model-and-make-predictions-with-it).\n",
        "\n",
        "And we found that there was a slight difference in the comparison.\n",
        "\n",
        "The EffNetB2 model was trained on 20% of the pizza, steak and sushi data from Food101 where as the ViT model was trained on 10%.\n",
        "\n",
        "Since our goal is to deploy the best model for our FoodVision Mini problem, let's start by downloading the [20% pizza, steak and sushi dataset](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/data/pizza_steak_sushi_20_percent.zip) and train an EffNetB2 feature extractor and ViT feature extractor on it and then compare the two models.\n",
        "\n",
        "This way we'll be comparing apples to apples (one model trained on a dataset to another model trained on the same dataset).\n",
        "\n",
        "> **Note:** The dataset we're downloading is a sample of the entire [Food101 dataset](https://pytorch.org/vision/main/generated/torchvision.datasets.Food101.html#food101) (101 food classes with 1,000 images each). More specifically, 20% refers to 20% of images from the pizza, steak and sushi classes selected at random. You can see how this dataset was created in [`extras/04_custom_data_creation.ipynb`](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/extras/04_custom_data_creation.ipynb) and more details in [04. PyTorch Custom Datasets section 1](https://www.learnpytorch.io/04_pytorch_custom_datasets/#1-get-data).\n",
        "\n",
        "We can download the data using the `download_data()` function we created in [07. PyTorch Experiment Tracking section 1](https://www.learnpytorch.io/07_pytorch_experiment_tracking/#1-get-data) from [`helper_functions.py`](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/helper_functions.py)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3kNwVrdqQaQH",
        "outputId": "a438999e-ee53-499d-acda-080652c44a5c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[INFO] data/pizza_steak_sushi_20_percent directory exists, skipping download.\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "PosixPath('data/pizza_steak_sushi_20_percent')"
            ]
          },
          "execution_count": 4,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Download pizza, steak, sushi images from GitHub\n",
        "data_20_percent_path = download_data(source=\"https://github.com/mrdbourke/pytorch-deep-learning/raw/main/data/pizza_steak_sushi_20_percent.zip\",\n",
        "                                     destination=\"pizza_steak_sushi_20_percent\")\n",
        "\n",
        "data_20_percent_path"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WC84IB5tQaQH"
      },
      "source": [
        "Wonderful!\n",
        "\n",
        "Now we've got a dataset, let's create training and test paths."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RDZXeJ1pQaQH"
      },
      "outputs": [],
      "source": [
        "# Setup directory paths to train and test images\n",
        "train_dir = data_20_percent_path / \"train\"\n",
        "test_dir = data_20_percent_path / \"test\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ymZXhWa8QaQH"
      },
      "source": [
        "## 2. FoodVision Mini model deployment experiment outline\n",
        "\n",
        "The ideal deployed model FoodVision Mini performs well and fast.\n",
        "\n",
        "We'd like our model to perform as close to real-time as possible.\n",
        "\n",
        "Real-time in this case being ~30FPS (frames per second) because that's [about how fast the human eye can see](https://www.healthline.com/health/human-eye-fps) (there is debate on this but let's just use ~30FPS as our benchmark).\n",
        "\n",
        "And for classifying three different classes (pizza, steak and sushi), we'd like a model that performs at 95%+ accuracy.\n",
        "\n",
        "Of course, higher accuracy would be nice but this might sacrifice speed.\n",
        "\n",
        "So our goals are:\n",
        "\n",
        "1. **Performance** - A model that performs at 95%+ accuracy.\n",
        "2. **Speed** - A model that can classify an image at ~30FPS (0.03 seconds inference time per image, also known as latency).\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-model-deployments-speed-vs-inference.png\" alt=\"foodvision mini goals in terms of performance and inference time.\" width=750/>\n",
        "\n",
        "*FoodVision Mini deployment goals. We'd like a fast predicting well-performing model (because a slow app is boring).*\n",
        "\n",
        "We'll put an emphasis on speed, meaning, we'd prefer a model performing at 90%+ accuracy at ~30FPS than a model performing 95%+ accuracy at 10FPS.\n",
        "\n",
        "To try and achieve these results, let's bring in our best performing models from the previous sections:\n",
        "\n",
        "1. **EffNetB2 feature extractor** (EffNetB2 for short) - originally created in [07. PyTorch Experiment Tracking section 7.5](https://www.learnpytorch.io/07_pytorch_experiment_tracking/#75-create-feature-extractor-models) using [`torchvision.models.efficientnet_b2()`](https://pytorch.org/vision/stable/models/generated/torchvision.models.efficientnet_b2.html#efficientnet-b2) with adjusted `classifier` layers.\n",
        "2. **ViT-B/16 feature extractor** (ViT for short) - originally created in [08. PyTorch Paper Replicating section 10](https://www.learnpytorch.io/08_pytorch_paper_replicating/#10-using-a-pretrained-vit-from-torchvisionmodels-on-the-same-dataset) using [`torchvision.models.vit_b_16()`](https://pytorch.org/vision/stable/models/generated/torchvision.models.vit_b_16.html#vit-b-16) with adjusted `head` layers.\n",
        "    * **Note** ViT-B/16 stands for \"Vision Transformer Base, patch size 16\".\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-model-deployment-two-experiments.png\" alt=\"modelling experiments for foodvision mini deployments, one effnetb2 feature extractor model and a vision transformer feature extractor model\" width=750 />\n",
        "\n",
        "> **Note:** A \"feature extractor model\" often starts with a model that has been pretrained on a dataset similar to your own problem. The pretrained model's base layers are often left frozen (the pretrained patterns/weights stay the same) whilst some of the top (or classifier/classification head) layers get customized to your own problem by training on your own data. We covered the concept of a feature extractor model in [06. PyTorch Transfer Learning section 3.4](https://www.learnpytorch.io/06_pytorch_transfer_learning/#34-freezing-the-base-model-and-changing-the-output-layer-to-suit-our-needs)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GusL8A_IQaQH"
      },
      "source": [
        "## 3. Creating an EffNetB2 feature extractor\n",
        "\n",
        "We first created an EffNetB2 feature extractor model in [07. PyTorch Experiment Tracking section 7.5](https://www.learnpytorch.io/07_pytorch_experiment_tracking/#75-create-feature-extractor-models).\n",
        "\n",
        "And by the end of that section we saw it performed very well.\n",
        "\n",
        "So let's now recreate it here so we can compare its results to a ViT feature extractor trained on the same data.\n",
        "\n",
        "To do so we can:\n",
        "1. Setup the pretrained weights as [`weights=torchvision.models.EfficientNet_B2_Weights.DEFAULT`](https://pytorch.org/vision/stable/models/generated/torchvision.models.efficientnet_b2.html#torchvision.models.EfficientNet_B2_Weights), where \"`DEFAULT`\" means \"best currently available\" (or could use `weights=\"DEFAULT\"`).\n",
        "2. Get the pretrained model image transforms from the weights with the `transforms()` method (we need these so we can convert our images into the same format as the pretrained EffNetB2 was trained on).\n",
        "3. Create a pretrained model instance by passing the weights to an instance of [`torchvision.models.efficientnet_b2`](https://pytorch.org/vision/stable/models/generated/torchvision.models.efficientnet_b2.html#efficientnet-b2).\n",
        "4. Freeze the base layers in the model.\n",
        "5. Update the classifier head to suit our own data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WqktLnWjQaQH"
      },
      "outputs": [],
      "source": [
        "# 1. Setup pretrained EffNetB2 weights\n",
        "effnetb2_weights = torchvision.models.EfficientNet_B2_Weights.DEFAULT\n",
        "\n",
        "# 2. Get EffNetB2 transforms\n",
        "effnetb2_transforms = effnetb2_weights.transforms()\n",
        "\n",
        "# 3. Setup pretrained model\n",
        "effnetb2 = torchvision.models.efficientnet_b2(weights=effnetb2_weights) # could also use weights=\"DEFAULT\"\n",
        "\n",
        "# 4. Freeze the base layers in the model (this will freeze all layers to begin with)\n",
        "for param in effnetb2.parameters():\n",
        "    param.requires_grad = False"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CTX5ATH9QaQH"
      },
      "source": [
        "Now to change the classifier head, let's first inspect it using the `classifier` attribute of our model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "w92SQBXEQaQI",
        "outputId": "e2540ac5-b039-417e-b8d7-51aa9a0179f8"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Sequential(\n",
              "  (0): Dropout(p=0.3, inplace=True)\n",
              "  (1): Linear(in_features=1408, out_features=1000, bias=True)\n",
              ")"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Check out EffNetB2 classifier head\n",
        "effnetb2.classifier"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GG58AIExQaQI"
      },
      "source": [
        "Excellent! To change the classifier head to suit our own problem, let's replace the `out_features` variable with the same number of classes we have (in our case, `out_features=3`, one for pizza, steak, sushi).\n",
        "\n",
        "> **Note:** This process of changing the output layers/classifier head will be dependent on the problem you're working on. For example, if you wanted a different *number* of outputs or a different *kind* of ouput, you would have to change the output layers accordingly."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qw4Za-5lQaQI"
      },
      "outputs": [],
      "source": [
        "# 5. Update the classifier head\n",
        "effnetb2.classifier = nn.Sequential(\n",
        "    nn.Dropout(p=0.3, inplace=True), # keep dropout layer same\n",
        "    nn.Linear(in_features=1408, # keep in_features same\n",
        "              out_features=3)) # change out_features to suit our number of classes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cvRmDArnQaQI"
      },
      "source": [
        "Beautiful!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a73io3iCQaQI"
      },
      "source": [
        "### 3.1 Creating a function to make an EffNetB2 feature extractor\n",
        "Looks like our EffNetB2 feature extractor is ready to go, however, since there's quite a few steps involved here, how about we turn the code above into a function we can re-use later?\n",
        "\n",
        "We'll call it `create_effnetb2_model()` and it'll take a customizable number of classes and a random seed parameter for reproducibility.\n",
        "\n",
        "Ideally, it will return an EffNetB2 feature extractor along with its assosciated transforms."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qlOsNZ54QaQI"
      },
      "outputs": [],
      "source": [
        "def create_effnetb2_model(num_classes:int=3,\n",
        "                          seed:int=42):\n",
        "    \"\"\"Creates an EfficientNetB2 feature extractor model and transforms.\n",
        "\n",
        "    Args:\n",
        "        num_classes (int, optional): number of classes in the classifier head.\n",
        "            Defaults to 3.\n",
        "        seed (int, optional): random seed value. Defaults to 42.\n",
        "\n",
        "    Returns:\n",
        "        model (torch.nn.Module): EffNetB2 feature extractor model.\n",
        "        transforms (torchvision.transforms): EffNetB2 image transforms.\n",
        "    \"\"\"\n",
        "    # 1, 2, 3. Create EffNetB2 pretrained weights, transforms and model\n",
        "    weights = torchvision.models.EfficientNet_B2_Weights.DEFAULT\n",
        "    transforms = weights.transforms()\n",
        "    model = torchvision.models.efficientnet_b2(weights=weights)\n",
        "\n",
        "    # 4. Freeze all layers in base model\n",
        "    for param in model.parameters():\n",
        "        param.requires_grad = False\n",
        "\n",
        "    # 5. Change classifier head with random seed for reproducibility\n",
        "    torch.manual_seed(seed)\n",
        "    model.classifier = nn.Sequential(\n",
        "        nn.Dropout(p=0.3, inplace=True),\n",
        "        nn.Linear(in_features=1408, out_features=num_classes),\n",
        "    )\n",
        "\n",
        "    return model, transforms"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jo7RuxvoQaQI"
      },
      "source": [
        "Woohoo! That's a nice looking function, let's try it out."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Nf-Xb48SQaQI"
      },
      "outputs": [],
      "source": [
        "effnetb2, effnetb2_transforms = create_effnetb2_model(num_classes=3,\n",
        "                                                      seed=42)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b1Afc_G7QaQI"
      },
      "source": [
        "No errors, nice, now to really try it out, let's get a summary with `torchinfo.summary()`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vi9gK-R5QaQI"
      },
      "outputs": [],
      "source": [
        "from torchinfo import summary\n",
        "\n",
        "# # Print EffNetB2 model summary (uncomment for full output)\n",
        "# summary(effnetb2,\n",
        "#         input_size=(1, 3, 224, 224),\n",
        "#         col_names=[\"input_size\", \"output_size\", \"num_params\", \"trainable\"],\n",
        "#         col_width=20,\n",
        "#         row_settings=[\"var_names\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dpDjEntJQaQI"
      },
      "source": [
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-effnetb2-feature-extractor.png\" alt=\"effnetb2 feature extractor model summary\" width=900/>\n",
        "\n",
        "Base layers frozen, top layers trainable and customized!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DchSqokqQaQJ"
      },
      "source": [
        "### 3.2 Creating DataLoaders for EffNetB2\n",
        "\n",
        "Our EffNetB2 feature extractor is ready, time to create some `DataLoader`s.\n",
        "\n",
        "We can do this by using the [`data_setup.create_dataloaders()`](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/going_modular/going_modular/data_setup.py) function we created in [05. PyTorch Going Modular section 2](https://www.learnpytorch.io/05_pytorch_going_modular/#2-create-datasets-and-dataloaders-data_setuppy).\n",
        "\n",
        "We'll use a `batch_size` of 32 and transform our images using the `effnetb2_transforms` so they're in the same format that our `effnetb2` model was trained on."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8tTASGSnQaQJ"
      },
      "outputs": [],
      "source": [
        "# Setup DataLoaders\n",
        "from going_modular.going_modular import data_setup\n",
        "train_dataloader_effnetb2, test_dataloader_effnetb2, class_names = data_setup.create_dataloaders(train_dir=train_dir,\n",
        "                                                                                                 test_dir=test_dir,\n",
        "                                                                                                 transform=effnetb2_transforms,\n",
        "                                                                                                 batch_size=32)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OORIsSJjQaQJ"
      },
      "source": [
        "### 3.3 Training EffNetB2 feature extractor\n",
        "\n",
        "Model ready, `DataLoader`s ready, let's train!\n",
        "\n",
        "Just like in [07. PyTorch Experiment Tracking section 7.6](https://www.learnpytorch.io/07_pytorch_experiment_tracking/#76-create-experiments-and-set-up-training-code), ten epochs should be enough to get good results.\n",
        "\n",
        "We can do so by creating an optimizer (we'll use [`torch.optim.Adam()`](https://pytorch.org/docs/stable/generated/torch.optim.Adam.html#torch.optim.Adam) with a learning rate of `1e-3`), a loss function (we'll use [`torch.nn.CrossEntropyLoss()`](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html) for multi-class classification) and then passing these as well as our `DataLoader`s to the [`engine.train()`](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/going_modular/going_modular/engine.py) function we created in [05. PyTorch Going Modular section 4](https://www.learnpytorch.io/05_pytorch_going_modular/#4-creating-train_step-and-test_step-functions-and-train-to-combine-them)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6zX5ZxJ_QaQJ",
        "outputId": "83e02f0a-a4d0-4d2f-e3f7-23ab38818c8c",
        "colab": {
          "referenced_widgets": [
            "128923b0faf94e0591a606c01403fbb5"
          ]
        }
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "128923b0faf94e0591a606c01403fbb5",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/10 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch: 1 | train_loss: 0.9856 | train_acc: 0.5604 | test_loss: 0.7408 | test_acc: 0.9347\n",
            "Epoch: 2 | train_loss: 0.7175 | train_acc: 0.8438 | test_loss: 0.5869 | test_acc: 0.9409\n",
            "Epoch: 3 | train_loss: 0.5876 | train_acc: 0.8917 | test_loss: 0.4909 | test_acc: 0.9500\n",
            "Epoch: 4 | train_loss: 0.4474 | train_acc: 0.9062 | test_loss: 0.4355 | test_acc: 0.9409\n",
            "Epoch: 5 | train_loss: 0.4290 | train_acc: 0.9104 | test_loss: 0.3915 | test_acc: 0.9443\n",
            "Epoch: 6 | train_loss: 0.4381 | train_acc: 0.8896 | test_loss: 0.3512 | test_acc: 0.9688\n",
            "Epoch: 7 | train_loss: 0.4245 | train_acc: 0.8771 | test_loss: 0.3268 | test_acc: 0.9563\n",
            "Epoch: 8 | train_loss: 0.3897 | train_acc: 0.8958 | test_loss: 0.3457 | test_acc: 0.9381\n",
            "Epoch: 9 | train_loss: 0.3749 | train_acc: 0.8812 | test_loss: 0.3129 | test_acc: 0.9131\n",
            "Epoch: 10 | train_loss: 0.3757 | train_acc: 0.8604 | test_loss: 0.2813 | test_acc: 0.9688\n"
          ]
        }
      ],
      "source": [
        "from going_modular.going_modular import engine\n",
        "\n",
        "# Setup optimizer\n",
        "optimizer = torch.optim.Adam(params=effnetb2.parameters(),\n",
        "                             lr=1e-3)\n",
        "# Setup loss function\n",
        "loss_fn = torch.nn.CrossEntropyLoss()\n",
        "\n",
        "# Set seeds for reproducibility and train the model\n",
        "set_seeds()\n",
        "effnetb2_results = engine.train(model=effnetb2,\n",
        "                                train_dataloader=train_dataloader_effnetb2,\n",
        "                                test_dataloader=test_dataloader_effnetb2,\n",
        "                                epochs=10,\n",
        "                                optimizer=optimizer,\n",
        "                                loss_fn=loss_fn,\n",
        "                                device=device)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "goOFdLROQaQM"
      },
      "source": [
        "### 3.4 Inspecting EffNetB2 loss curves\n",
        "\n",
        "Nice!\n",
        "\n",
        "As we saw in 07. PyTorch Experiment Tracking, the EffNetB2 feature extractor model works quite well on our data.\n",
        "\n",
        "Let's turn its results into loss curves to inspect them further.\n",
        "\n",
        "> **Note:** Loss curves are one of the best ways to visualize how your model's performing. For more on loss curves, check out [04. PyTorch Custom Datasets section 8: What should an ideal loss curve look like?](https://www.learnpytorch.io/04_pytorch_custom_datasets/#8-what-should-an-ideal-loss-curve-look-like)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_hqgEk-iQaQM",
        "outputId": "5d84af89-2457-4721-fa39-99838a957591"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2oAAAG5CAYAAAD/HsejAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAACMo0lEQVR4nOzdd3hUVf7H8fdJrySk0SGBAKGDhCYdLGBX7GJXdC27rq4/cXfdXV1ddXV31V0Vsa6VtXdRkaYgJSgonQk1tCSEkkJIO78/7hACUhJIcmeSz+t55iEzc+/cbyIy+cw553uMtRYRERERERHxHQFuFyAiIiIiIiIHU1ATERERERHxMQpqIiIiIiIiPkZBTURERERExMcoqImIiIiIiPgYBTUREREREREfo6AmIiIiIiLiYxTURE6AMWa9MeYUt+sQERGpa8aYmcaYncaYULdrEWkMFNRERERE5KiMMcnAUMAC59TjdYPq61oivkZBTaSWGWNCjTFPGGO2eG9P7P/00RiTYIz51BizyxiTZ4z51hgT4H3uHmPMZmNMvjFmlTFmtLvfiYiISKWrgHnAK8DV+x80xrQxxrxvjMkxxuwwxvynynM3GmNWeN/XlhtjTvI+bo0xqVWOe8UY86D36xHGmCzve+I24GVjTFPve2eOd0TvU2NM6yrnxxljXva+5+40xnzofXypMebsKscFG2NyjTG96+hnJFKrFNREat8fgIFAb6AX0B/4o/e5u4AsIBFoBvwesMaYzsBtQD9rbTRwOrC+XqsWERE5squAN7y3040xzYwxgcCnwAYgGWgFTAEwxlwE/MV7XhOcUbgd1bxWcyAOaAdMwPl99WXv/bbAXuA/VY5/DYgAugFJwL+8j78KjK9y3BnAVmvt4mrWIeIqDSeL1L4rgNuttdkAxpj7geeA+4BSoAXQzlrrAb71HlMOhAJdjTE51tr1bhQuIiJyKGPMEJyQ9La1NtcYkwlcjjPC1hK421pb5j38O++fNwB/t9Yu9N731OCSFcCfrbX7vPf3Au9VqechYIb36xbAWCDeWrvTe8gs75+vA/cZY5pYa/cAV+KEOhG/oBE1kdrXEufTxf02eB8DeAznzeorY8xaY8xEAG9ouwPn08dsY8wUY0xLRERE3Hc18JW1Ntd7/03vY22ADVVCWlVtgMzjvF6OtbZ4/x1jTIQx5jljzAZjzB5gNhDrHdFrA+RVCWmVrLVbgDnAOGNMLE6ge+M4axKpdwpqIrVvC84nj/u19T6GtTbfWnuXtbY9cDZw5/61aNbaN621+z+1tMCj9Vu2iIjIwYwx4cDFwHBjzDbvurHf4kzt3w60PULDj01AhyO8bBHOVMX9mh/yvD3k/l1AZ2CAtbYJMGx/ed7rxHmD2OH8F2f640XA99bazUc4TsTnKKiJnLhgY0zY/hvwFvBHY0yiMSYB+BPO9AuMMWcZY1KNMQbYA5QD5caYzsaYUd6mI8U40zzK3fl2REREKp2H837UFWftdW+gC87U/fOArcAjxphI7/vgYO95LwC/M8b0NY5UY8z+DzEXA5cbYwKNMWOA4ceoIRrnfXGXMSYO+PP+J6y1W4EvgGe8TUeCjTHDqpz7IXAS8BucNWsifkNBTeTEfY7zBrL/FgZkAD8BPwM/AA96j+0ITAMKgO+BZ6y1M3HWpz0C5ALbcBZD/77evgMREZHDuxp42Vq70Vq7bf8Np5nHZTizQ1KBjTjNsi4BsNa+AzyEM00yHycwxXlf8zfe83bhrOv+8Bg1PAGE47xHzgOmHvL8lThrwFcC2ThLCfDWsX99WwrwfvW/bRH3GWsPHV0WEREREWkYjDF/AjpZa8cf82ARH6KujyIiIiLSIHmnSl6PM+om4lc09VFEREREGhxjzI04zUa+sNbOdrsekZrS1EcREREREREfoxE1ERERERERH+PaGrWEhASbnJzs1uVFRKQeLVq0KNdam+h2Hf5C75EiIo3D0d4fXQtqycnJZGRkuHV5ERGpR8aYDW7X4E/0Hiki0jgc7f1RUx9FRERERER8jIKaiIiIiIiIj1FQExERERER8THa8FpEGr3S0lKysrIoLi52uxS/FxYWRuvWrQkODna7FBEREb+moCYijV5WVhbR0dEkJydjjHG7HL9lrWXHjh1kZWWRkpLidjkiIiJ+7ZhTH40xLxljso0xS4/wvDHGPGWM8RhjfjLGnFT7ZYqI1J3i4mLi4+MV0k6QMYb4+HiNTIqIiNSC6qxRewUYc5TnxwIdvbcJwLMnXpaISP1SSKsd+jmKiIjUjmMGNWvtbCDvKIecC7xqHfOAWGNMi9oqUEREREREpLGpja6PrYBNVe5neR/7BWPMBGNMhjEmIycnpxYuLSIiIiIi0vDURlA73DwXe7gDrbWTrbXp1tr0xMTEWri0iIj/27VrF88880yNzzvjjDPYtWtXjc+75pprePfdd2t8noiIiNSf2ghqWUCbKvdbA1tq4XVFRBqFIwW18vLyo573+eefExsbW0dViYiIiJtqoz3/x8BtxpgpwABgt7V2ay28rohIvbv/k2Us37KnVl+za8sm/Pnsbkd8fuLEiWRmZtK7d2+Cg4OJioqiRYsWLF68mOXLl3PeeeexadMmiouL+c1vfsOECRMASE5OJiMjg4KCAsaOHcuQIUOYO3curVq14qOPPiI8PPyYtX3zzTf87ne/o6ysjH79+vHss88SGhrKxIkT+fjjjwkKCuK0007j8ccf55133uH+++8nMDCQmJgYZs+eXWs/IxERETnYMYOaMeYtYASQYIzJAv4MBANYaycBnwNnAB6gCLi2rooVEWmIHnnkEZYuXcrixYuZOXMmZ555JkuXLq3ci+yll14iLi6OvXv30q9fP8aNG0d8fPxBr7FmzRreeustnn/+eS6++GLee+89xo8ff9TrFhcXc8011/DNN9/QqVMnrrrqKp599lmuuuoqPvjgA1auXIkxpnJ65QMPPMCXX35Jq1atjmvKpYiIiFTfMYOatfayYzxvgVtrrSIRERcdbeSrvvTv3/+gDaOfeuopPvjgAwA2bdrEmjVrfhHUUlJS6N27NwB9+/Zl/fr1x7zOqlWrSElJoVOnTgBcffXVPP3009x2222EhYVxww03cOaZZ3LWWWcBMHjwYK655houvvhiLrjgglr4TkVERORIamONmisqKiyLN+2itLzC7VJERGpVZGRk5dczZ85k2rRpfP/99yxZsoQ+ffocdkPp0NDQyq8DAwMpKys75nWcz9l+KSgoiAULFjBu3Dg+/PBDxoxxttKcNGkSDz74IJs2baJ3797s2LGjpt+aiEjdKyuBgmy3q5CGbvsy2LurTi/ht0Htm5XZnPf0HBauP9oWbyIivi86Opr8/PzDPrd7926aNm1KREQEK1euZN68ebV23bS0NNavX4/H4wHgtddeY/jw4RQUFLB7927OOOMMnnjiCRYvXgxAZmYmAwYM4IEHHiAhIYFNmzYd5dVFROpZRTksfgv+fRI83hH+nQ5f3AOrv4SSQrerk4Zkz1Z47QJ47/o6vUxtNBNxxaAO8QQHGmauyuHkDglulyMictzi4+MZPHgw3bt3Jzw8nGbNmlU+N2bMGCZNmkTPnj3p3LkzAwcOrLXrhoWF8fLLL3PRRRdVNhO5+eabycvL49xzz6W4uBhrLf/6178AuPvuu1mzZg3WWkaPHk2vXr1qrRYRkeNmLaz5Gqb9BbKXQcs+kH4tbJgLi/4L8ydBQDC0HQipo6HDKGjWAwL8drxC3FRaDP+7AkoK4NS/1umlzJGmvtS19PR0m5GRcUKvMf6F+WzbU8y0O4fXUlUi0hitWLGCLl26uF1Gg3G4n6cxZpG1Nt2lkvxObbxHijQKmxbCtD/DhjkQ1x5G/wm6ngfGu81vaTFs/B4yv4HMGbB9qfN4ZCK0H+kEt/YjIbrZES8hUsla+OBm+GkKXPIGdDnrhF/yaO+PfjuiBjAyLYm/frqcTXlFtImLcLscEREREakPuWvgm/thxScQmQRn/gNOuhoCgw8+LjgMOox0bgD52yBzuvf2Dfz8tvN4sx6QOsoZbWs7CIJCEfmF7592QtqI39dKSDsW/w5qnRP566cwfWU2V5+c7HY5IiI+5dZbb2XOnDkHPfab3/yGa6/VLioi4qf2bIVZj8APr0FwOIz8Awy8BUKjqnd+dHPofblzq6iAbT8dGG37/hmY8yQEhUPykAPTJBM6HRihk8bLMw2+vg+6nAPD7q6XS/p1UGufGEVyfAQzVimoiYgc6umnn3a7BBGR2lG82wlR3z8DFWXQfwIM+x1EnkCfgoAAaNnbuQ29C/blw/o53uA2HaZOdI5r0toZkUsdDSnDISKuNr4j8Sc7MuHd6yCpK5z3bL2tb/TroAbO9Mc3529kb0k54SGBbpcjIiIiIrWltBgWvgDfPg57d0KPi2Hk7yEu5djn1lRoNHQe49wAdq4/ME1y+cfw42tgAqDlSQdG21qlQ6Df/zotR1O8B966DEwgXPpG9Udva4Hf/80alZbEy3PWMzczl9FdtBBURERExO9VlMNPb8OMh2D3JugwGk75M7Sox26zTZMh/TrnVl4GmxcdGG2b/RjMehRCm0DKsAPBrWly/dUnda+iAt6/EXZ44KqP6v2/r98Htf4pcUSEBDJjVbaCmoiIiIg/O7TVfovecO5/oP0Id+sKDIK2A5zbyN9DUR6sm+0EN890WPmpc1xcByewpY521rmFRrtbt5yYGQ/C6qlwxuOQMrTeL+/3QS00KJDBqQnMWJmDtRajxZ4iIiIi/icrA77+M2z4DpqmwIUvO632fXG/s4g46Haec7PW6UK5f7Rt8Ruw8Hln77Y2Aw50k2zeyze/Fzm8pe/Bt95uov1ucKWEBvG3ZVRaEpt37WX19gK3SxERqbFdu3bxzDPPHNe5TzzxBEVFRUc9Jjk5mdzc3ON6fRGROpe7Bv53JbwwGnJXO6MXty2E7hf4R7AxBhI7wcBfwRXvwD3r4aqPYdAtsG83fPMATB4Bj6fCu9fD6i+dqZ3iu7YugQ9vhTYDnb+PLg0E+cHf/mMb0TkRcNr0i4j4m7oOaiIiPmnPVvjkN/D0AGckauQf4Nc/Qv8bf7kfmj8JCoX2w+HUB+Dm7+Cu1XD+ZEg9BdbOgDcvhid7way/w54tblcrhyrIgbcud0ZNL3kNgkJcK8Xvpz4CtIgJp0uLJsxYlc2vRnRwuxwR8WdfTIRtP9fuazbvAWMfOeLTEydOJDMzk969e3PqqaeSlJTE22+/zb59+zj//PO5//77KSws5OKLLyYrK4vy8nLuu+8+tm/fzpYtWxg5ciQJCQnMmDHjmKX885//5KWXXgLghhtu4I477jjsa19yySVMnDiRjz/+mKCgIE477TQef/zxWvuRiEgj9otW+zfC0N9BVKLbldWN6GbQ6xLnVlYCqz6HRa84jVJmPgKdxkD6tc70yAB1MHdVWQm8fRUU5cJ1UyEqydVyGkRQAxiVlsikWWvZXVRKTIQffwojIo3OI488wtKlS1m8eDFfffUV7777LgsWLMBayznnnMPs2bPJycmhZcuWfPbZZwDs3r2bmJgY/vnPfzJjxgwSEo69l9CiRYt4+eWXmT9/PtZaBgwYwPDhw1m7du0vXjsvL48PPviAlStXYoxh165ddfkj8HnGmDHAk0Ag8IK19pFDnm8KvAR0AIqB66y1S73PrQfygXKgzFqbXo+li/iOsn1Oq/3Zj3lb7V/kjKLVRat9XxUUcmBtW95aWPRfZ03bqs8gpi2cdBX0GQ9NWrhdaeP0xf/Bxrkw7kVo2cftahpSUEvi6RmZzF6Tw9m9Wrpdjoj4q6OMfNWHr776iq+++oo+fZw3iIKCAtasWcPQoUP53e9+xz333MNZZ53F0KE17z713Xffcf755xMZGQnABRdcwLfffsuYMWN+8dplZWWEhYVxww03cOaZZ3LWWWfV6vfpT4wxgcDTwKlAFrDQGPOxtXZ5lcN+Dyy21p5vjEnzHj+6yvMjrbVaKCiNU2Wr/b/B7o3utNr3RXHt4dT7nbC66jPIeNnpMjjzYeg8Fvpe62y0rVG2+rHwRVj0Mgy+A3pc6HY1QANZowbQu01TYiOCmbFK69RExH9Za7n33ntZvHgxixcvxuPxcP3119OpUycWLVpEjx49uPfee3nggQeO67UP53CvHRQUxIIFCxg3bhwffvghY8aMOdFvzZ/1BzzW2rXW2hJgCnDuIcd0Bb4BsNauBJKNMdozxl+UFMGmhbD1JygvdbuahsNaWP0VTBoKH97srPm56iO48n2FtKqCQqDb+XD1x3D7D3DybbBxHrwxDp7s7YxA5m9zu8qGbf13zmhax9Nh9J/crqZSgxlRCwwwDO+UyKxVOVRUWAIC1KZfRPxDdHQ0+fn5AJx++uncd999XHHFFURFRbF582aCg4MpKysjLi6O8ePHExUVxSuvvHLQudWZ+jhs2DCuueYaJk6ciLWWDz74gNdee40tW7b84rULCgooKirijDPOYODAgaSmptblj8DXtQI2VbmfBQw45JglwAXAd8aY/kA7oDWwHbDAV8YYCzxnrZ18uIsYYyYAEwDatm1bq9+AVFFS5KxD3boYtiyGLT9C7iqwFc7zgaHQvLuzf1fLPtCyNySm+XdzCzf4U6t9XxLfwWlCMvKPzt5si16G6Q/CDO8oW/q10H6Ufo61addGZ11a0xQY97xPjWA2mKAGzvTHjxZvYUnWLvq0bep2OSIi1RIfH8/gwYPp3r07Y8eO5fLLL2fQoEEAREVF8frrr+PxeLj77rsJCAggODiYZ599FoAJEyYwduxYWrRoccxmIieddBLXXHMN/fv3B5xmIn369OHLL7/8xWvn5+dz7rnnUlxcjLWWf/3rX3X7Q/Bth/vk79DhyUeAJ40xi4GfgR+BMu9zg621W4wxScDXxpiV1trZv3hBJ8BNBkhPTz/88KfUTEkRbF96IJBtXQw5Kw+EssgkJ4h1Odv5s3Sv97glzlS9jBed4/aHt5Z9vAGut8LbkeSucdrRr/gYIhOd1uYnXe1q5zy/FBTibE/Q/QLYkek0H1n8hhPeYts6P9M+4yG6uduV+reSQqfDY3kZXDYFwmLcrugg5khTYepaenq6zcjIqNXX3FlYQt8Hv+a2kanceVrnWn1tEWm4VqxYQZcuXdwuo8E43M/TGLPIX5toGGMGAX+x1p7uvX8vgLX24SMcb4B1QE9r7Z5DnvsLUGCtPWoLzbp4j2zwqoayrYudwJWzCqx3v6rIxIODVss+EN3iyPsjVVQ4zR72v9aWxU6AK3FGvwkKg2bdD7xWi97e8NagPgOvvvxtTgfDH16F4HA4+dcw6FYIjXK7soajbJ8T1DJehvXfQkDQgbVs7UdqlK2mrIV3rnE+VLj8Heh4iitlHO39sUH9a9I0MoQ+bZsyY1WOgpqIiNSWhUBHY0wKsBm4FLi86gHGmFigyLuG7QZgtrV2jzEmEgiw1uZ7vz4NqPkCQzlY6V7YtvTAKNmWxd6RsiqhrEVvSDvLCVItekOTljXbtDYgABJSndv+xgIVFZCXWSUMLoYlU5xOhlAlvPU5cN2GHt4ObbXf7wYYdnfDbbXvpqBQ6D7OueV64IdXYPGbsOITiG0Hfa+G3uOd7QDk2L59HJZ/6Ew1dSmkHUuD+5djVFoSj325iuw9xSQ1CXO7HBGRejNgwAD27dt30GOvvfYaPXr0cKmihsFaW2aMuQ34Eqc9/0vW2mXGmJu9z08CugCvGmPKgeXA9d7TmwEfOINsBAFvWmun1vf34Nf2h7Kqa8qqhrKIBCcYpZ1xYF1ZTUNZdQUEQEJH59bzIuexquFtf3Bc8hYsfN55Pij8l2veEjr7V3grL3Pa6e/Ng6I8KNrhfL1rkxNS9+Y1zlb7bkpIhdMehFH3OUFt0SvOlNMZf4POZzhr2VJGaJTtSFZ+7qz963GxM/rroxrU1EeA5Vv2cMZT3/L3cT25uF+bWn99EWl4VqxYQVpaGqYufrFrZKy1rFy5skFNfXRDo536WLoXti+rMtVwMWSvOCSU9T54CmOTVnUTyk5ERQXs8BwIl1sXe6dNFjjP7w9vVb+P+gpvpXudsHVo6CraH8R2HPz83jxn1OxIOoyCU/6iLo6+IHeNdy3bm85/t6bJB9ayubxxs0/JXgEvnOJ84HLtF85UXRcd7f2xwQU1ay2DHp5On7axPDu+b62/vog0POvWrSM6Opr4+HiFtRNgrWXHjh3k5+eTknLwp+oKajXTKIJaabF3TVmV6YsHhbL4X64p88VQVl0HhTdvEN320yHhrcfBQTSh05HDm7Wwb0+VULWzSujKO0zo8gax0qIj1xgS5bTQD4+r8mf8IY81PfgxrUHzPaXFB9aybfjOWcuWdqazli1leOMeZSvKg+dHOU1EJsyEmFZuV9R41qgBGGMYmZbIJ0u2UlJWQUhQI/7LKCLV0rp1a7KyssjJyXG7FL8XFhZG69at3S5DfEnxHti5DvLWOX/mepzRpZwVzpomcH7xb9EbOo05sLYrprX/hrLDCQiAxE7OrefFzmMV5U54q9oA5cc3YIF3B4f94S0+9ZBQ5v1z/8/vFwyExzo/1/A4J+A26+GEqyMGsabOGijxf8FhzrrKHhdCzmr44b9Ox8jlHzkt6PteDb2vaHyjbOVl8O51sGczXP2pT4S0Y2lwI2oAXy3bxoTXFvHmDQM4OfXYewuJiEjd0ohazfjViJq1UJjrdEjcH8iqfl2Ue/DxkYnQvOeBQNayT8MLZSeianjbP9q4c4MTvMLjIKLpEUa6qjwWFuNTe0GJDygt9q5lexk2zIGAYGeULf1aSB7WOEbZpv4e5j0N5/wHTrrS7WoqNaoRNYDBqQmEBAYwfWW2gpqIiMiJqih3PoXOW3tgZCxvLeStd77eP30PAOMEr6bJzi+CcSnOp/j7/wxr4tI34ScCAiGxs3PrdYnb1UhDERzmNMDpeZGzbcWi/8KSN52uh3HtD6xli2ygvzcvftMJaf1v8qmQdiwNMqhFhgYxoH0cM1Zl88ezurpdjoiIiO8rLYZdG6oEsSojYzs3QEXpgWMDQ5x24HHtIXnwgSAW197ZjFdT6ER8V2JnGPM3GP0nZzrkoldg2p9h9mMw+DfO/nchkW5XWXuyMuCT30DKMDj9IberqZEGGdQARnZO4oFPl7NxRxFt4yPcLkdERMR9xbsPDmKVgWydM2JGleUQIdEQlwzNujn7kVWOjLV32t9rap2IfwsOc0Zte13iNPKZ8ZBzy3gJRv7eWcfm7/+f79kKU65wNre/6L8QGOx2RTXSYIPaqDQnqE1fuZ1rBmtPDxERaUS2LoHty385Mla04+DjIhO9o2JDDg5icSnOeietGxNpHJK6wCWvw8Z58NUf4ePbYd6zzmbQqaf4578FpcXwvytgXz5c+b6zftPPNNiglpwQSUpCJDNW5SioiYhI4/LNA+CZBibAu14sBbqcfXAQa5oModFuVyoivqTtQLj+a2ft2rT74Y0LnZb+p/3Vv/bKsxY+vQM2L3ICaLNubld0XBpsUANn+uPr8zdQVFJGREiD/lZFREQOOO0hGPOod71YiNvViIg/MQa6nQ+dz4SMF2HWo/DccOh1KYz6o/Phj6+b9wwseQtG3Ot8SOWnGnQvzlFpSZSUVTDXs+PYB4uIiDQUSWmQkKqQJiLHLygEBv4Kfr0YBv8alr4P/+4L0/7irHf1VZ5vnOmbXc6GYf/ndjUnpEEHtX4pTYkICWT6qmy3SxERERER8T/hsc5atdszoOu58N2/4Kk+MP85KCtxu7qD7ciEd6+FxC5w3iS/3x/Ov6s/htCgQIakJjBzZTZubewtIiIiIuL3YtvCBZNhwixnzdcX/wfPDHBa/PvC79nFe+Cty5y1uZe9CaFRbld0whp0UANn+uOW3cWs2p7vdikiIiIiIv6tZW+46mO4/B1nT8W3r4KXTodNC9yrqaIC3p8AOzxOG/6mye7VUosafFAbmZYEwPSVmv4oIiIiInLCjIFOp8HNc+DsJ2HnenjxVPjflc70w/o24yFY/QWMeQTaD6//69eRBh/UmjUJo2uLJsxcmeN2KSIiIiIiDUdgEPS9Bm7/wemw6PkGnu4PX9wDhfXUzG/p+/Dt49DnSuh/Y/1cs540+KAGzvTHRRt3sruo1O1SREREREQaltAoGDERfv0D9BkPCybDU72dxiOle+vuult/gg9vgTYD4Mx/+OfG3EfRKILayLQkyisss9ZoVE1EREREpE5EN3emQv7qe2h3stPK/9/psGSKs46sNhXkwJTLISLO2dQ6KLR2X98HNIqg1rtNLE0jgpmhdWoiIiIiInUrKQ0u/x9c/QlEJsAHN8Hk4bB2Zu28flmJ08SkMAcufQOikmrndX1MtYKaMWaMMWaVMcZjjJl4mOebGmM+MMb8ZIxZYIzpXvulHr/AAMPwTonMWp1DeYUPtA8VEREREWnoUobBjTPgghdg70549Vx4/ULYvvzEXnfqPbBxLpzzH2jZp3Zq9UHHDGrGmEDgaWAs0BW4zBjT9ZDDfg8sttb2BK4CnqztQk/UyLQk8gpLWJK1y+1SREREREQah4AA6HkR3JbhbJy9aQFMGgwf3QZ7ttb89Ra+CBkvweDfOK/bgFVnRK0/4LHWrrXWlgBTgHMPOaYr8A2AtXYlkGyMaVarlZ6g4Z0SCTBo+qOIiIiISH0LDnPC1W8Ww4CbnXVr/z4JZvwN9hVU7zXWz3E22k49FUb/uU7L9QXVCWqtgE1V7md5H6tqCXABgDGmP9AOaH3oCxljJhhjMowxGTk59dvYIzYihJPaNmXGKgU1ERERERFXRMTBmIfhtgXQ6XSY9Sg81ccZJSsvO/J5uzbC21dC0xQY9wIEBNZfzS6pTlA7XJ/LQxd6PQI0NcYsBm4HfgR+8ZO21k621qZba9MTExNrWusJG5mWxNLNe8jeU1zv1xYREREREa+49nDRK3DDNxDfAT79LTx7Mqz6AuwhUaOk0OnwWF4Kl70F4bFuVFzvqhPUsoA2Ve63BrZUPcBau8dae621tjfOGrVEYF1tFVlbRqU5HWE0qiYiIiIi4gNap8O1X8Alb4Ath7cuhVfOgs0/OM9b6+yVtm0pjHsREjq6W289qk5QWwh0NMakGGNCgEuBj6seYIyJ9T4HcAMw21q7p3ZLPXFpzaNpERPGdK1TExERERHxDcZAl7PglnlwxuOQsxKeHwnvXg9f3wfLP4RT/gKdTnO70noVdKwDrLVlxpjbgC+BQOAla+0yY8zN3ucnAV2AV40x5cBy4Po6rPm4GWMY0TmJjxdvpqSsgpCgRrGNnIiIiIiI7wsMhv43Qs9LYM4T8P3TUFYMPS5yGpE0MscMagDW2s+Bzw95bFKVr78H/GIcclRaEm8t2MjC9XkMTk1wuxwREREREakqrAmM/hOkXw+rv4DeVzijbo1MtYJaQzI4NZ6QwACmr8xWUBMRERE5grLyCvIKS8gp2EdO/j5yC0q8fzr3dxaVkN4ujqsGtaNpZMixX1CkpmJaQb8b3K7CNY0uqEWEBDGgfRwzVmVz31mH7tstIiIi0nCVV1h2FpVUhq2qwevQIJZXVPKL5nsAESGBJEaHEhkSxL+mrWbSrEwuTm/NDUPb0yYuov6/KZEGqtEFNXCmP97/yXI27CikXXyk2+WIiIiIHDdrLbuKSskp2Edu/r7KETDnfslBj+cVllBe8cv0FRoUQGJ0KAlRobSJi6BP26YkRoc6t6iQyucSokKJDD3w6+Oa7flMnr2WNxds5LV5GzijRwtuGtaBHq1j6vNHINIgNcqgNrKzE9Smr8zm2sEpbpcjIiIiclhFJWX8nLWb7CojXZWjXt4glluwj7LDhK/gQENiVCgJ0aG0iAmjR6uYyvCVELX/TyeERYUGYY5jDVDHZtE8dlEv7jqtMy/PWceb8zfy6U9bGZwaz4RhHRjWMeG4XldEGmlQS06IpH1CJDNW5SioiYiIiE/ZvbeU6Su3M3XpNmatzqG4tKLyucAAQ0JUSGXQ6tK8CQnRoZWBLDEqlMToEBKjwmgSfnzh63g0jwnj3jO6cOuoVN6av5GX5qzj6pcWkNY8mpuGt+esni0JDlS3bZGaaJRBDWBkWhKvzdtAUUkZESGN9scgIiIiPiC3YB9fL3fC2dzMXErLLUnRoVyc3oaRnZNoGRtOYnQoseHBBAT47ghVk7BgbhregWsHp/DR4s1Mnr2W3/5vCY9NXcV1Q1K4tH9bokL1e5dIdTTa/1NGpSXx4nfrmOPZwaldm7ldjoiIiDQyW3bt5ctl25i6dBsL1+dRYaFNXDjXDk7h9G7N6dMm1qdD2dGEBAVwUXobxp3Umpmrs3lu1loe/GwFT32zhisHtePqk5NJig5zu0wRn9Zog1q/5DgiQwKZvjJbQU1ERI7KGDMGeBIIBF6w1j5yyPNNgZeADkAxcJ21dml1zpXGZV1uIVOXbmPq0q0sydoNQKdmUdw2MpXTuzena4smDWpNV0CAYVRaM0alNePHjTuZPHstz8zM5PnZ6xjXtxU3DG1Ph8Qot8sU8UmNNqiFBAUwpGMCM1dlY61tUP8oiohI7THGBAJPA6cCWcBCY8zH1trlVQ77PbDYWnu+MSbNe/zoap4rDZi1lpXb8r3hbBurtucD0LN1DHef3pkx3Zs3mqDSp21Tnh3fl3W5hbzw7VreWZTFlIWbOLVLM24a3p6+7eLcLlHEpzTaoAbO9Mcvl21n5bZ8urRo4nY5IiLim/oDHmvtWgBjzBTgXKBq2OoKPAxgrV1pjEk2xjQD2lfjXGlgKiosS7J2MXXZNr5cuo31O4owBvq1i+O+s7pyerdmtG7aePcbS0mI5KHze/DbUzvx6tz1vDpvA18t3056u6bcNLwDo9OS/HbKp0htatRBbUTnJACmr8xWUBMRkSNpBWyqcj8LGHDIMUuAC4DvjDH9gXZA62qeC4AxZgIwAaBt27a1UrjUn7LyChau31m55mzbnmKCAgyDOsRz47D2nNa1OYnRoW6X6VMSokK587TO3DyiA28v3MTz367jxlcz6JAYyYRh7TmvTytCgwLdLlPENY06qDVrEka3lk2YuSqbW0emul2OiIj4psN9tH/oplWPAE8aYxYDPwM/AmXVPNd50NrJwGSA9PT0wx4jvmVfWTlzM3cw9edtfL1iO3mFJYQGBTCsUyL/170zo9OaERMR7HaZPi8iJIhrBqcwfmA7Pl+6jedmZXLPez/z+FeruXZwMlcMaEdMuH6O0vg06qAGzvTHp2d42FVUQmxEiNvliIiI78kC2lS53xrYUvUAa+0e4FoA4yx6Xue9RRzrXPEvRSVlzF6dwxdLtzF9RTb5+8qIDAlkVJdmjO3enOGdEolU+/njEhQYwDm9WnJ2zxbMzdzBpFmZ/H3qKp6e7uGy/m25bkgKLWPD3S6zwbHW8sPGneQXlzGoQ7xGMX1Io/+XZGRaEv+e7mHW6hzO7d3K7XJERMT3LAQ6GmNSgM3ApcDlVQ8wxsQCRdbaEuAGYLa1do8x5pjniu/bvbeUGSuz+WLp1soNqGMjghnTvTljezTn5A4JhAXrl9vaYoxhcGoCg1MTWLZlN8/PXsvLc9fzytz1nNOrJROGtyetuZasnKj84lI+/HEzr8/bWNnkJio0iJFpSfrQwUc0+p9+r9axxEWGMGNltoKaiIj8grW2zBhzG/AlTov9l6y1y4wxN3ufnwR0AV41xpTjNAq5/mjnuvF9SM0cbQPqMd2a0z8ljqDAALfLbPC6tYzhiUv78LvTO/PSd+uZsnAj7/+4meGdErlpeHsGtY9X5+4aWrltD6/P28AHP2ymsKSc7q2a8Oi4HiQ1CePLpdv4avl2PlmypXIa79juzTWN1yXGWnemwaenp9uMjAxXrn2o3/5vMTNXZZPxx1MJVJchEZFaZ4xZZK1Nd7sOf+FL75GNyZE2oB7bvYXfb0DdUOwqKuGN+Rt5ec46cgtK6NEqhpuGt2dMt+YKzkdRUlbBF0u38sa8jSxYn0dIUABn92zJlYPa0at1zEFht6y8gowNOyu3lKjaGGdM9+ZqjFPLjvb+qKAGfLxkC79+60fe+9XJ9G3X1O1yREQaHAW1mvGl98jGoLi0nNvf+pGvl28HoGNSFGO7N2+QG1A3FMWl5bz/w2Ze+HYta3MLaRsXwQ1DU7iobxvCQzQNdb/Nu/by5vwN/G/hJnILSmgXH8EVA9pyUd82NI08dm+GigrLT5t3V27SXnWridO7N2/0W03UBgW1Y9hdVEqfv37FLSNS+d3pnd0uR0SkwVFQqxlfeo9s6ErKKrj59UXMWJXNbSNTObd3K1KTGscG1A1BeYXl6+XbeW52Jj9u3EXTiGCuGpTMVYPaER/VOEd9Kios33pyee37DUxf6Xz4MCqtGVcOasfQ1ITjHhW21rJq+4HN21duO7B5++ndmjeqzdtrk4JaNVw0aS5FJeV89uuhbpciItLgKKjVjK+9RzZU5RWWX0/5kc9+2srfzu/B5QO0f52/staSsWEnz81ay7QV2wkw0LtNLEM7JjKsUyK9Wsc0+KmROwtLeGfRJt6Yv5ENO4pIiArhkn5tuKx/2zoZ9VqXW1g5VXjxpl0AdGoWxZhuGo2uCQW1anhmpoe/T13F/N+PplmTMLfLERFpUBTUasbX3iMboooKyz3v/cQ7i7L4wxlduHFYe7dLklriyc7n48VbmLUml5+ydmEtNAkLYnBqgje4JTSY6XrWWpZk7ea17zfwyU9bKCmroH9yHOMHtWNMt+aEBNVPON2yay9fLdvG1GXbWLDOWd/ZNi6CMd2ba33nMSioVcPKbXsY88S3PHJBDy7tr0/URERqk4Jazfjae2RDY63l/k+W88rc9fx6dEfuPLWT2yVJHdlVVMJ3nly+XZ3L7DU5bN1dDED7hEiGdXJC24CUeL9rQ7+3pJyPlzit9X/evJvIkEDOP6kV4we2c33rgtyCfUxbvp2py7Yxx+N0TG3WJNSZHqmOqb+goFYN1loGPzKd7q1imHyVfpcQEalNCmo142vvkQ3NP75axb+ne7hucAr3ndVF07MaCWstmTkFzFqdy7drcpi3dgfFpRUEBxrS28UxtFMCwzom0rVFE58d/cnMKeCNeRt5d9Em9hSX0blZNOMHteP8Pq2I8sGwuae4lOkrspm6dBszV2dTXFpB04hgTunSjLE9mjM4NaHRb7B9tPdH3/sv6hJjDCPSkvjox83sKytv9H9pREREGqJJszL593QPl/Zro5DWyBhjSE2KJjUpmuuHpFBcWk7G+p18uyaHWatz+PvUVfx96ioSokIY4p0mObRTAknR7i6JKSuvYNqK7bw+byPfeXIJDjSM6d6CKwe2o19yU5/+O9wkLJjz+rTivD6t2FtSzqzV2ZXNSN5ZlKUNto9BP40qRnVO4s35G1m4bidDOia4XY6IiIjUotfnbeCRL1ZyVs8WPHR+D5/+BVfqXlhwIEM6JjCkYwL3ntGF7D3FfLvGGW37dk0uHy7eAkBa82iGd0pkaMdE0pObEhZcPx/mb99TzJQFm3hrwUa27SmmZUwYd5/emYvT2/jlPmbhIYGM6d6CMd1bUFJWwdzMXKYeZoPtMd2ac0oXbbANmvp4kKKSMno/8DXjB7TjT2d3dbscEZEGQ1Mfa8YX3yP93Qc/ZnHn20sY1TmJSVf2JVhrZOQoKiosy7fuYfaaHL5dnUvGhjxKyy1hwQEMSIl31rd1TCA1KapWA7+1lu/X7uD1eRv4atl2yioswzolcuXAdoxKSyLQR6dknoiqG2x/uWwbW3cfvMH2qV2buT6qWZe0Rq0GrnppAVl5RUz/3Qi3SxERaTAU1GrGV98j/dXUpdu49c0fGJASx0vX9Ku3ERFpOAr3lTFv7Q6+XZPL7NU5rM0tBKBFTBjDvFMkh6QmEBtx7E2kD2dPcSnvL8ri9fkb8WQXEBsRzMXpbbi8f1uSEyJr81vxaYfbYDvAwJk9W3LTsPZ0bxXjdom1TmvUamBU50T+8sly1ucWNqr/MURERBqi2atz+PVbP9KzdQzPX5WukCbHJTI0iNFdmjG6SzMANuUVVU6T/HzpVv6XsQljoGfrWIZ3TGBop0R6t4k95sjtsi27eX3eRj78cTN7S8vp1SaWxy/qxVk9WzTKv6sBAYbebWLp3SaWe8Z0ZtX2fN7/YTNvzd/IJ0u2MDg1npuGdWBox4RGMXVZI2qH2LijiGGPzeBPZ3XluiEpbpcjItIgaEStZnz1PdLfLFyfx5UvziclIYopNw7UmhepE2XlFSzJ2s3s1Tl8uyaHxZt2UWEhOjSIQR32T5NMpG28s3dbcWk5Xyzdymvfb+CHjbsICw7g3F5Oa/0erRveiFFt2FNcylvzN/LSnHVs37OPLi2acNOw9pzZs4XfT2PW1McaGv2PmbSMDee16we4XYqISIOgoFYzvvwe6S9+ztrN5c/PI7FJKG/fNIiEKP9rviD+aXdRKXMyndG22atz2bxrLwDJ8RH0ahPLt2tyySssoX1CJFcMbMeFJ7XWhwjVVFJWwUeLNzN59lrWZBfQKjac64akcGm/Nn7bMVJTH2toZOckXv1+A4X7yvz2P7qIiEhjtWZ7Ple9NJ8m4cG8fv0AhTSpVzERwZzRowVn9GiBtZa1uYXe0TZnfduAlHiuHNSOkzvEN4rpe7UpJCiAi9LbMO6k1sxcnc2kWWv566fLeeqbNYwf2JarT05uUI1HlEIOY1RaEi98t445nlxO69bc7XJERESkmjbsKOSKF+YTHBjAmzcOoGVsuNslSSNmjKFDYhQdEqO4drCW1NSWgADDqLRmjEprxo8bdzJ59lqemZnJ89+uY9xJrbhhaHs6JEa5XeYJ8+9JnXUkPTmOqNAgZqzKdrsUERERqaatu/dyxQvzKSmv4PUbBtAuXk3BRBq6Pm2b8uz4vky/awQX9W3Nez9s5pR/zmLCqxks2rDT7fJOiEbUDiMkKIAhqQnMWJmDtVbD0iIiIj4ut2Af41+Yz66iUt68cQCdmkW7XZKI1KOUhEgeOr8Hd5zSiVe/X8+r32/gq+XbSW/XlJuGd2B0WhIBfrYPnUbUjmBUWhLb9hSzYmu+26WIiIjIUezeW8pVLy5g8669vHRNP3q2jnW7JBFxSWJ0KHed1pm5E0fx57O7snV3MTe+msGp/5rF/xZuZF9ZudslVpuC2hGMSEsE0PRHERERH1a4r4xrX17Amux8nrsynf4pcW6XJCI+IDI0iGsHpzDr7hE8eWlvwoIDuee9nxny6Ayemelh995St0s8JgW1I0iKDqNHqximr1RQExER8UXFpeXc+GoGS7J28+/L+jC8U6LbJYmIjwkKDODc3q349PYhvH79ANKaR/P3qas4+eFvePDT5Wzxbp/gi7RG7ShGdk7kPzM87CwsoWlkiNvliIiIiFdpeQW3vfkDczN38M+LezGmewu3SxIRH2aMYUjHBIZ0TGDZlt1Mnr2Wl+eu55W56zmnV0smDG9PWvMmbpd5EI2oHcXItCQqLMxek+N2KSIiIuJVXmG56+0lTFuRzV/P7cYFJ7V2uyQR8SPdWsbw5KV9mHX3CK4c1I6py7Yx5olvueblBczNzMVa63aJgILaUfVqHUt8ZIimP4qIiPgIay1/+OBnPl6yhYlj07hyULLbJYmIn2rdNII/n92NuRNH8bvTOrF0824uf34+5z49h09/2kJZeYWr9SmoHUVAgGF4p0Rmrc6hvMI3krWIiEhjZa3lwc9WMGXhJm4bmcrNwzu4XZKINACxESHcNqoj390zir+d34P84jJue/NHRv1jFq9+v569Je50iqxWUDPGjDHGrDLGeIwxEw/zfIwx5hNjzBJjzDJjzLW1X6o7RqYlsauolMWb/HvDPBEREX/35DdrePG7dVxzcjJ3ndbJ7XJEpIEJCw7k8gFtmXbncCaN70t8VAh/+mgZJz/yDf/6ejV5hSX1Ws8xg5oxJhB4GhgLdAUuM8Z0PeSwW4Hl1tpewAjgH8aYBtF9Y1inRAIDjKY/ioiIuOiFb9fyxLQ1XNS3NX86qyvG+NfGtSLiPwIDDGO6N+f9X53MOzcPom+7pjz5zRpOfuQb7vtwKRt2FNZLHdUZUesPeKy1a621JcAU4NxDjrFAtHH+1YwC8oCyWq3UJTHhwfRt15TpK9VQRERExA1vzt/Ig5+t4MweLXhkXE8CAhTSRKTuGWPolxzHC1f34+vfDuOcXi2ZsnAjIx+fya1v/MBPWbvq9PrVCWqtgE1V7md5H6vqP0AXYAvwM/Aba+0vVt8ZYyYYYzKMMRk5Of4TfEZ2TmLF1j1s213sdikiIiKNykeLN/OHD39mZOdE/nVJbwIV0kTEBR2bRfP3C3vx3T2jmDCsA7NX5/DcrLV1es3qBLXD/Yt4aGeN04HFQEugN/AfY8wvNiKw1k621qZba9MTE/1nU8pRaUkAzFil6Y8iIiL15evl27nz7SX0T47j2fF9CQlSDzQRcVezJmFMHJvG3HtHcd9Zh64Gq13V+RcvC2hT5X5rnJGzqq4F3rcOD7AOSKudEt3XqVkUrWLDtU5NRESknny3Jpdb3/iB7q1iePGafoQFB7pdkohIpeiwYJrHhNXpNaoT1BYCHY0xKd4GIZcCHx9yzEZgNIAxphnQGajbscB6ZIxhROdE5nhy2VfmTntOERGRxmLRhjxufDWD9omR/PfafkSFBrldkohIvTtmULPWlgG3AV8CK4C3rbXLjDE3G2Nu9h72V+BkY8zPwDfAPdba3Loq2g2j0pIoKilnwbo8t0sRERFpsJZu3s01Ly+keUwYr17fn9iIBtFEWkSkxqr1EZW19nPg80Mem1Tl6y3AabVbmm85uUMCIUEBTF+ZzdCO/rO+TkRExF94svO56qUFNAkL5vUbBpAUXbfTikREfJlW5VZTeEggg9rHM0Pr1EREGh1jzBhjzCpjjMcYM/Ewz8cYYz4xxiwxxiwzxlxb5bn1xpifjTGLjTEZ9Vu5/9iUV8T4FxYQYAyv3zCAVrHhbpckIuIqBbUaGJWWxPodRazLrZ9N7kRExH3GmEDgaWAs0BW4zBhzaKuvW4Hl1tpewAjgH9513fuNtNb2ttam10fN/mb7nmKueGE+e0vLef2G/qQkRLpdkoiI6xTUamB/m351fxQRaVT6Ax5r7VprbQkwBTj3kGMsEG2MMUAUkAeU1W+Z/imvsIQrXpjPjoJ9/Pe6/qQ1/8XuPiIijZKCWg20iYsgNSlK0x9FRBqXVsCmKvezvI9V9R+gC872NT8Dv7HWVnifs8BXxphFxpgJR7qIMWaCMSbDGJORk5NTe9X7sD3FpVz10nw25RXx4jX96N0m1u2SRER8hoJaDY3snMj8dTso3KcPSkVEGglzmMfsIfdPBxYDLYHewH+MMfuHhgZba0/CmTp5qzFm2OEuYq2dbK1Nt9amJyY2/KZVRSVlXPfyQlZty2fSlX0Z2D7e7ZJERHyKgloNjUxLorTc8p2nQe0+ICIiR5YFtKlyvzXOyFlV1wLvW4cHWAekQWVnZKy12cAHOFMpG7V9ZeXc9Noifti4kycv7cPIzklulyQi4nMU1GqoX3IcUaFBmv4oItJ4LAQ6GmNSvA1CLgU+PuSYjcBoAGNMM6AzsNYYE2mMifY+Homzlc3SeqvcB5WWV3D7mz/y7ZpcHh3XkzN6tHC7JBERn1StfdTkgODAAIZ2TGDGqmystTjrxkVEpKGy1pYZY24DvgQCgZestcuMMTd7n58E/BV4xRjzM85UyXustbnGmPbAB973iiDgTWvtVFe+ER9QUWG5+50lfLV8O/ef042L0tsc+yQRkUZKQe04jExL4oul21i+dQ/dWsa4XY6IiNQxa+3nwOeHPDapytdbcEbLDj1vLdCrzgv0E996cvlw8RbuPLUTV5+c7HY5IiI+TVMfj8OIzs4ib01/FBERqb4VW/cAKKSJiFSDgtpxSIoOo2frGO2nJiIiUgOe7AISo0OJCQ92uxQREZ+noHacRnRO4sdNu8grLHG7FBEREb/gyS4gNTHK7TJERPyCgtpxGpWWhLUwe3Xj2JRURETkRFhrycwuIDVJQU1EpDoU1I5Tz1YxxEeGaPqjiIhINWTn7yN/X5mCmohINSmoHaeAAMPwzonMWp1DeYV1uxwRERGf5skuAFBQExGpJgW1EzAqLYnde0v5ceNOt0sRERHxaQpqIiI1o6B2AoZ2TCQwwGj6o4iIyDF4sguIDg0iKTrU7VJERPyCgtoJiAkPJr1dUwU1ERGRY/BkF9AhKQpjjNuliIj4BQW1EzQyLYmV2/LZunuv26WIiIj4LE+OOj6KiNSEgtoJGpWWBMCMlWrTLyIicji795aSk79PQU1EpAYU1E5Qx6QoWsWGa/qjiIjIEVQ2EtFm1yIi1ebfQW1fvtsVYIxhZFoiczy57Csrd7scERERn5Opjo8iIjXmv0Etczr8Iw22LXW7EkalJbG3tJz5a/PcLkVERMTneHIKCAkKoE1chNuliIj4Df8Nai37gAmEmQ+7XQmD2icQGhSg6Y8iIiKH4ckuoH1CJIEB6vgoIlJd/hvUwpvCoFth5aewZbG7pYQEcnKHeGasysZa62otIiIivmZ/a34REak+/w1qAANvhrBYnxhVG5mWxIYdRazLLXS7FBEREZ9RXFrOpp1FaiQiIlJD/h3UwmLg5Nth9VTIWuRqKSM7O236Nf1RRETkgLU5hVirRiIiIjXl30ENYMBNEB4HM//mahlt4iLomBTFjFUKaiIiIvt5ctTxUUTkePh/UAuNhiF3gGcabJzvaikj05JYsC6Pgn1lrtYhIiLiKzzZBQQYSEmIdLsUERG/4v9BDaDfDRCZ6Pqo2sjOSZSWW75bk+tqHSIiIr4iM7uANnERhAUHul2KiIhfaRhBLSQShvwW1s6E9XNcKyM9uSnRYUHM0Do1ERERwBlRUyMREZGaaxhBDSD9OohqBjMeApda5AcHBjCsY6La9IuIiABl5RWsyy3U+jQRkePQcIJacDgMvQs2zIF1s10rY0TnRLLz97Fsyx7XahAREfEFm3bupaS8QnuoiYgch4YT1ABOuhqatIIZf3NtVG2Et02/pj+KiEhj58lWx0cRkePVsIJacJgzqrZpHmROd6WExOhQ+ifH8dq8Der+KCIijZqCmojI8WtYQQ2gz5UQ08bVtWq/P7MLOQX7eHLaaleuLyIi4gs82QUkRYfSJCzY7VJERPxOwwtqQSEw7G7YvAjWfOVKCb3bxHJpvza8NGc9q7blu1KDiIiI2zw5BXRQx0cRkePS8IIaQO/LIbadq6Nqd5+eRnRYEPd9tFQdIEVEpNGx1rI2u0DTHkVEjlPDDGqBwTD8Hti6BFZ97koJcZEh3DMmjQXr8vho8RZXahAREXFLdv4+8veVKaiJiBynhhnUAHpeAnEdYMbDUFHhSgmXpLehV5tYHvp8BXuKS12pQURExA1qJCIicmKqFdSMMWOMMauMMR5jzMTDPH+3MWax97bUGFNujImr/XJrIDAIRkyE7T/Dyk9cKSEgwPDXc7uRW7CPJ75e40oNIiIiblBQExE5MccMasaYQOBpYCzQFbjMGNO16jHW2sestb2ttb2Be4FZ1tq8Oqi3ZrqPg4RO3lG1cldK6Nk6lsv7t+W/369nxVZtgi0iIo2DJ7uA6NAgkqJD3S5FRMQvVWdErT/gsdautdaWAFOAc49y/GXAW7VR3AkLCHRG1XJWwLIPXCvj7tM7ExMezJ/UWERExC9VY2ZJjDHmE2PMEmPMMmPMtdU9t6HyZBfQISkKY4zbpYiI+KXqBLVWwKYq97O8j/2CMSYCGAO8d4TnJxhjMowxGTk5OTWt9fh0PR8Su8DMR1wbVYuNCGHimDQWrt/J+z9sdqUGERE5PtWZWQLcCiy31vYCRgD/MMaEVPPcBsmTo46PIiInojpB7XAfhR1pWOhsYM6Rpj1aaydba9OttemJiYnVrfHEBATAyHthxxr4+d36ueZhXNi3NX3axvLwFyvYvVeNRURE/Eh1ZpZYINo4w0dRQB5QVs1zG5zde0vJyd+noCYicgKqE9SygDZV7rcGjtRv/lJ8ZdpjVWlnQ/MeMOsRKC9zpQSnsUh38gpL+NfXq12pQUREjkt1Zpb8B+iC8/74M/Aba21FNc9tcCobiWizaxGR41adoLYQ6GiMSTHGhOCEsY8PPcgYEwMMBz6q3RJrQUAAjPg95K2Fn6a4Vkb3VjGMH9iOV79fz9LNu12rQ0REaqQ6M0tOBxYDLYHewH+MMU2qea5zETeWB9SRTHV8FBE5YccMatbaMuA24EtgBfC2tXaZMeZmY8zNVQ49H/jKWltYN6WeoM5joUVvmPUolLs39fCuUzvTNCKEP320lIoKNRYREfED1ZlZci3wvnV4gHVAWjXPBVxaHlBHPDkFhAQF0CYuwu1SRET8VrX2UbPWfm6t7WSt7WCtfcj72CRr7aQqx7xirb20rgo9YcbAyD/Aro2w+A3XyoiJCObeM7rww8ZdvPtDlmt1iIhItVVnZslGYDSAMaYZ0BlYW81zGxxPdgHtEyIJDFDHRxGR41WtoNZgdDwVWqXD7MehbJ9rZVzQpxXp7ZryyBcr2V2kxiIiIr6smjNL/gqcbIz5GfgGuMdam3ukc+v/u6hf+1vzi4jI8WtcQc0YGPUH2L0JfnzNtTICAgwPnNudXUUlPP7VKtfqEBGR6jnWzBJr7RZr7WnW2h7W2u7W2tePdm5DVlxazqadRWokIiJyghpXUANoPxLaDoLZ/4DSYtfK6NqyCVcNSub1+Rv4OUuNRUREpGFYm1OItWokIiJyohpfUDMGRv4e8rfAoldcLeXO0zoRHxnKH9VYREREGghPjjo+iojUhsYX1ABShkHyUPjun1BS5FoZTcKC+cOZaSzZtIu3MzYd+wQREREf58kuIMBASkKk26WIiPi1xhnUwBlVK9gOGS+5WsZ5vVvRPzmOR6euZGdhiau1iIiInKjM7ALaxEUQFhzodikiIn6t8Qa1dic769W++xeUuLf1mzGGB87rxp7iMv7+pRqLiIiIf/NkF6iRiIhILWi8QQ2cUbWiXFjwvKtlpDVvwjUnJzNl4UYWb9rlai0iIiLHq6y8gnW5hVqfJiJSCxp3UGvTH1JPhTlPwr58V0u545SOJEaF8qePllKuxiIiIuKHNu3cS0l5hfZQExGpBY07qAGMvBf25sH8Sa6WER0WzB/O7MJPWbuZsnCjq7WIiIgcD0+2Oj6KiNQWBbVWfaHTWJj7byh2dz+zc3q1ZGD7OP4+dRV5aiwiIiJ+RkFNRKT2KKiBM6pWvBvmPetqGcYY/npudwr3lfHoFytdrUVERKSmPNkFJEWH0iQs2O1SRET8noIaQIte0OVs+P5p2LvT1VI6Novm+iEp/C9jEz9sdLcWERGRmvDkFGg0TUSkliio7TfiXti3B+b+x+1KuH10R5o3CeO+D9VYRERE/IO1lsxsBTURkdqioLZfs27Q7XynqUjhDldLiQoN4o9ndWHZlj28OX+Dq7WIiIhUx/Y9+yjYV6agJiJSSxTUqho+0dn8eu5TblfCmT1aMDg1nse+XEVuwT63yxERETmqykYi2uxaRKRWKKhVlZQGPS6EBZOhIMfVUowx3H9Od/aWlvOIGouIiIiP82Q7+5FqRE1EpHYoqB1q+EQoK4Y5T7hdCalJUdwwtD3vLsoiY32e2+WIiIgckSengOiwIBKjQ90uRUSkQVBQO1RCKvS8FBa+APnb3K6G20el0jImjPs+WkZZeYXb5YiIiByWx9tIxBjjdikiIg2CgtrhDL8bykvhu3+5XQkRIUHcd1ZXVmzdw+vz1FhERER8kye7UOvTRERqkYLa4cS1h96XQ8bLsHuz29UwpntzhnZM4B9frSY7v9jtckRERA6yu6iU3IJ9Wp8mIlKLFNSOZNjdYMvhu3+6XYm3sUg39pVV8MjnaiwiIiK+xZOjRiIiIrVNQe1ImraDPlfCov/Crk1uV0P7xCgmDGvP+z9uZv5ad/d5ExERqaqyNb+CmohIrVFQO5phvwNj4NvH3a4EgFtHptIqNpw/fbSMUjUWERERH+HJLiAkKIDWTSPcLkVEpMFQUDuamNbQ9xr48XXIW+d2NYSHBPKns7uyans+r36vxiIiIuIbPNkFtE+IJDBAHR9FRGqLgtqxDLkTTCDM9o1RtdO6NmNE50T+9fVqtu9RYxEREXGfJ6dA0x5FRGqZgtqxNGkB/a6HJW/Bjky3q6lsLFJSXsHfPl/hdjkiItLIFZeWk7Vzr4KaiEgtU1CrjsF3QGAIzPq725UA0C4+kpuHd+CjxVv4PlONRURExD2ZOQVYq0YiIiK1TUGtOqKbQf8b4ee3IWe129UAcMuIDrRuGs6fPlqqxiIiIuIadXwUEakbCmrVNfg3EBQOsx5xuxIAwoID+cvZ3ViTXcDLc9xvdCIiIo1TZnYBAQZSEiLdLkVEpEFRUKuuyAQYcBMsfR+2L3e7GgBO6dqM0WlJPDFtDVt373W7HBERaYQ8OQW0jYsgNCjQ7VJERBoUBbWaOPl2CInymVE1gL+c043yCstDn6mxiIiI1D9Ptjo+iojUBQW1moiIg4G/guUfwbaf3a4GgDZxEdwyIpVPf9rKHE+u2+WIiEgjUlZewbrcQjooqImI1DoFtZoadAuExsBM3xlVu2l4e9rGRfCnj5ZSUqbGIiIiUj825hVRWm5JTVRQExGpbQpqNRXeFE6+DVZ+Clt+dLsawNtY5JyuZOYU8uJ3aiwiIiL1Qx0fRUTqjoLa8RhwM4TFwoyH3a6k0qi0ZpzatRlPfbOGLbvUWEREROqeJ8cJapr6KCJS+xTUjkdYExj8a1jzJWRluF1NpT+d1RWL5cHPfKMrpYhIQ2GMGWOMWWWM8RhjJh7m+buNMYu9t6XGmHJjTJz3ufXGmJ+9z/nOm0Yt8GQX0KxJKE3Cgt0uRUSkwVFQO179J0BEPMz4m9uVVGoTF8FtI1P5/OdtzF6d43Y5IiINgjEmEHgaGAt0BS4zxnSteoy19jFrbW9rbW/gXmCWtTavyiEjvc+n11fd9SFTHR9FROqMgtrxCo12NsHO/AY2znO7mko3DmtPcnwEf/54GfvKyt0uR0SkIegPeKy1a621JcAU4NyjHH8Z8Fa9VOYiay2ZOYVqJCIiUkcU1E5EvxshMsmnRtVCgwK5/9zurMst5IVv1VhERKQWtAI2Vbmf5X3sF4wxEcAY4L0qD1vgK2PMImPMhCNdxBgzwRiTYYzJyMnx/VkR2/YUU7CvTCNqIiJ1pFpB7Vhz873HjPDOv19mjJlVu2X6qJAIGPJbWDcL1n/ndjWVhndKZGz35vx7+hqydha5XY6IiL8zh3nMHuHYs4E5h0x7HGytPQln6uStxphhhzvRWjvZWpturU1PTEw8sYrrwf6Oj2okIiJSN44Z1KozN98YEws8A5xjre0GXFT7pfqo9GshqrkzqmaP9L5d//54VlcMhr9+qsYiIiInKAtoU+V+a2DLEY69lEOmPVprt3j/zAY+wJlK6ffUml9EpG5VZ0StOnPzLwfet9ZuhMo3o8YhOByG3gUb5jgjaz6iVWw4t49O5ctl25mxqvH85xARqQMLgY7GmBRjTAhOGPv40IOMMTHAcOCjKo9FGmOi938NnAYsrZeq65gnu4AmYUEkRoW6XYqISINUnaBWnbn5nYCmxpiZ3jn4Vx3uhfxt/n21nXQVNGnlc6NqNwxpT/vESP7y8TKKS9VYRETkeFhry4DbgC+BFcDb1tplxpibjTE3Vzn0fOAra21hlceaAd8ZY5YAC4DPrLVT66v2uuTxdnw05nAzQ0VE5ERVJ6hVZ25+ENAXOBM4HbjPGNPpFyf52fz7agsOg2G/g03znS6QPiIkKIAHzunOhh1FTJ691u1yRET8lrX2c2ttJ2ttB2vtQ97HJllrJ1U55hVr7aWHnLfWWtvLe+u2/9yGIDNHrflFROpSdYJadebmZwFTrbWF1tpcYDbQq3ZK9BO9x0NMW5j+kE+Nqg3pmMCZPVvw9AwPm/LUWERERE7crqIScgtKFNREROpQdYJadebmfwQMNcYEeVsTD8CZHtJ4BIXA8Lthyw+w+ku3qznIH8/sQmCA4f5P1FhEREROnBqJiIjUvWMGterMzbfWrgCmAj/hzMF/wVrbIBZL10ivy6BpMszwrVG1FjHh/GZ0R6at2M43K7a7XY6IiPi5yqCWGO1yJSIiDVe19lGr5tz8x6y1Xa213a21T9RRvb4tMBiG3wPbfoKVn7pdzUGuHZxCalIUf/lEjUVEROTEeLILCA0KoFXTcLdLERFpsKoV1KQGelwM8R3h8/+D/G1uV1MpJCiA+8/pxqa8vbw8Z73b5YiIiB/z5BTQPjGKwAB1fBQRqSsKarUtMAguehmKd8H/roSyfW5XVGlwagKj05J4ZoaHHQW+U5eIiPiX/a35RUSk7iio1YXmPeC8ZyFrAXx2p0+tV7v3jC4UlZbzxLQ1bpciIiJ+aG9JOZt37SU1UUFNRKQuKajVlW7nwbD/gx9fh/nPuV1NpdSkKC7v35Y3F2zEk53vdjkiIuJnMnMKsFYdH0VE6pqCWl0acS90PhO+/D2snel2NZXuOKUjEcGBPPLFSrdLERERP5OZo9b8IiL1QUGtLgUEwAXPQUIneOcayFvndkUAxEeFcsvIVKatyGZuZq7b5YiIiB/xZBcQYCA5IcLtUkREGjQFtboWGg2XvemsU3vrMtjnG9MNrx2cTKvYcB76bAUVFb6zhk5ERHybJ7uAdvGRhAYFul2KiEiDpqBWH+Law0WvQO5q+OBmqKhwuyLCggP5vzGdWbZlD+//uNntckRExE9k5hTQQY1ERETqnIJafekwEk5/yNkIe9ajblcDwNk9W9KrdQyPf7mKvSXaBFtERI6urLyCdbmFWp8mIlIPFNTq04CbofcVMOsRWP6R29UQEGD441ld2banmOe/Xet2OSIi4uM25hVRWm4V1ERE6oGCWn0yBs76F7Tu50yB3LbU7YrolxzHmG7NmTQrk+w9xW6XIyIiPsyTrY6PIiL1RUGtvgWFwiWvQ1gMTLkMCne4XRETx6ZRWl7BP79e7XYpIiLiwzze1vwdEiNdrkREpOFTUHNDdHO49A3I3w7vXA3lpa6Wk5wQyZUDk3k7YxMrt+1xtRYREfFdnuwCmjUJJTos2O1SREQaPAU1t7TqC+c8Beu/dTbEdtmvR6cSHRbM3z7XJtgiInJ4mdkFmvYoIlJPFNTc1OtSGHQbLJgMi/7raimxESHcPiqV2atzmLU6x9VaRETE91hrycwpJFWt+UVE6oWCmttOfQA6jILP7oKN81wt5apBybSLj+Bvn62gXJtgi4hIFdv2FFOwr0wjaiIi9URBzW0BgXDhSxDbFv43HnZnuVZKSFAAE8eksWp7Pm9nbHKtDhER8T37Oz52UFATEakXCmq+ILwpXPYWlBbDlMuhpMi1UsZ0b056u6b846vVFOwrc60OERHxLWrNLyJSvxTUfEViZxj3Amz9CT6+Haw7Uw+NMfzhzC7kFuzjuVmZrtQgIiK+x5NdQJOwIBKjQt0uRUSkUVBQ8yWdx8Do+2DpuzDnCdfK6NO2KWf3asnz365l6+69rtUhIiK+w+Pt+GiMcbsUEZFGQUHN1wy5E7pdANPuh9VfuVbG/53emQoLj325yrUaRETEd2TmqDW/iEh9UlDzNcbAuU9D8x7w3vWQs9qVMtrERXDt4GTe/2EzSzfvdqUGERHxDbuKSsgtKFFQExGpRwpqvigkAi59EwJDYMplsHeXK2XcOjKVuMgQHvxsOdalNXMiIuI+NRIREal/Cmq+KrYNXPIa7NzgjKxVlNd7CU3CgrnjlI7MW5vHNyuy6/36IiLiGyqDWmK0y5WIiDQeCmq+rN3JcMZj4JkG0/7iSgmX9W9L+8RI/vbFCkrLK1ypQURE3OXJLiA0KIBWTcPdLkVEpNFQUPN16ddC+vUw9yn46e16v3xwYAC/H9uFtTmFvLVgY71fX0RE3OfJKaB9YhSBAer4KCJSXxTU/MHYR6HdEGd/tc0/1PvlR3dJYlD7eJ6YtoY9xaX1fn0REXHX/tb8IiJSfxTU/EFgMFz8X4hMgilXQP72er38/k2wdxaV8PQMT71eW0RE3LW3pJzNu/aSmqigJiJSnxTU/EVkAlz2JhTvgv+Nh7J99Xr57q1iOL9PK17+bj2b8orq9doiIuKezJwCrFXHRxGR+qag5k+a94DznoGsBfDZnVDPLfPvPr0zAQHwd22CLSKNjDFmjDFmlTHGY4yZeJjn7zbGLPbelhpjyo0xcdU519dl5qg1v4iIGxTU/E2382HY3fDj67Bgcr1eukVMODcObc8nS7bw48ad9XptERG3GGMCgaeBsUBX4DJjTNeqx1hrH7PW9rbW9gbuBWZZa/Oqc66v82QXEGAgOSHC7VJERBoVBTV/NOL30PkMmHovrJ1Zr5e+aXgHEqJCefCzFdoEW0Qai/6Ax1q71lpbAkwBzj3K8ZcBbx3nuT7Hk11Au/hIQoMC3S5FRKRRUVDzRwEBcP5zkNAR3rkG8tbV26WjQoO467ROLNqwky+Wbqu364qIuKgVsKnK/SzvY79gjIkAxgDvHce5E4wxGcaYjJycnBMuurZ4sgvooEYiIiL1TkHNX4U1gcvectapTbkc9uXX26UvTm9D52bRPPLFSkrKtAm2iDR4h9s87EhTCs4G5lhr82p6rrV2srU23VqbnpiYeBxl1r6y8grW7yjU+jQRERcoqPmzuPZw0SuQsxI+uBkq6ic0BQYYfn9mFzbmFfHq9+vr5ZoiIi7KAtpUud8a2HKEYy/lwLTHmp7rczbkFVFabhXURERcoKDm7zqMhNMegpWfwqxH6+2ywzslMqxTIv+e7mFXUUm9XVdExAULgY7GmBRjTAhOGPv40IOMMTHAcOCjmp7rqzzZ6vgoIuIWBbWGYOCvoPcVMOsRWP7RsY+vJX84owv5xaU89Y02wRaRhstaWwbcBnwJrADettYuM8bcbIy5ucqh5wNfWWsLj3Vu/VV/YvYHtQ6JkS5XIiLS+AS5XYDUAmPgzH9Czir44FcQ1wGad6/zy3ZuHs3F6W14bd56rhrUjuQEvZGLSMNkrf0c+PyQxyYdcv8V4JXqnOsvMrMLaN4kjOiwYLdLERFpdDSi1lAEh8GlbzhNRqZcBoU76uWyd57WieDAAB75YmW9XE9EROqPJ6dA0x5FRFyioNaQRDeHS96A/O3wztVQXlrnl0yKDuPm4R2YumwbC9blHfsEERHxC9ZaMrMV1ERE3FKtoGaMGWOMWWWM8RhjJh7m+RHGmN3GmMXe259qv1SpltZ94ZynYP238OXv6+WSNw5tT/MmYTz02XIqKrQJtohIQ7B1dzGFJeV0UFATEXHFMYOaMSYQeBoYC3QFLjPGdD3Mod9aa3t7bw/Ucp1SE70uhUG3wYLJsOi/dX658JBAfnd6Z5Zk7eaTn/ym67SIiBxFZcdHbXYtIuKK6oyo9Qc81tq11toSYApwbt2WJSfslPuhwyj47C7YOK/OL3dBn1Z0a9mEv09dRXFpeZ1fT0RE6pZa84uIuKs6Qa0VsKnK/SzvY4caZIxZYoz5whjT7XAvZIyZYIzJMMZk5OTkHEe5Um2BQXDhSxDbBv43HnZn1enlAgIMfzizC5t37eXlOevr9FoiIlL3PDkFxIQHkxAV4nYpIiKNUnWCmjnMY4cuRPoBaGet7QX8G/jwcC9krZ1srU231qYnJibWqFA5DuFN4bIpUFoMUy6HkqI6vdzJHRI4pUsSz8zwsKNgX51eS0RE6pbH20jEmMP9GiAiInWtOkEtC2hT5X5r4KCFSNbaPdbaAu/XnwPBxpiEWqtSjl9iZxj3PGz9CT6+HWzdNvuYOLYLRaXlPDFtTZ1eR0RE6lZmdoHWp4mIuKg6QW0h0NEYk2KMCQEuBT6ueoAxprnxfuRmjOnvfd362chLjq3zWBj1R1j6Lsx5ok4vlZoUxeX92/Lmgo14svPr9FoiIlI3dhaWsKOwROvTRERcdMygZq0tA24DvgRWAG9ba5cZY242xtzsPexCYKkxZgnwFHCptXU8dCM1M/Qu6HYBTPsLTL3XmQ5ZR+44pSMRwYE8/Lk2wRYR8UeeHDUSERFxW1B1DvJOZ/z8kMcmVfn6P8B/arc0qVXGwHnPQmQCzHsG1s6CcS9As8PttHBi4qNCuWVkKo9OXclcTy4np2oWrIiIP1HHRxER91Vrw2tpIILD4IzH4PJ3oDAbJo+Aec9CRUWtX+rawcm0ig3nwc9WUK5NsEVE/Ionu4Cw4ABaxYa7XYqISKOloNYYdToNfvU9dBgJUyfCG+Mgf1utXiIsOJD/G9OZ5Vv38P4Pdbs1gIiI1C5PdgHtE6IICFDHRxERtyioNVZRiU7r/jP/ARu+h2cGwYpPa/US5/RqSa82sTz+1SqKSspq9bVFRKTu7G/NLyIi7lFQa8yMgX43wE2zvRtjXwEf/xpKCmvp5Q33ndmF7Xv28cK362rlNUVEpG4VlZSxeddeBTUREZcpqAkkdoLrp8HgO+CHV2HSUNi8qFZeOj05jrHdmzNpVibZe+qu06SIiNSOtTnOh3UKaiIi7lJQE0dQCJx6P1z9CZTtgxdPg9mPQUX5Cb/0xLFplJZX8M+vV9dCoSIiUpfU8VFExDcoqMnBUobCr76DLufA9AfhlTNh54YTesl28ZFcNSiZtzM2sXLbnloqVERE6oInu4DAAENyfKTbpYiINGoKavJL4U3hwpfg/Odg21KYNAR+evuEXvL2UalEhwXz0GcraqlIERGpC57sAtrFRRASpF8RRETcpH+F5fCMgV6XOqNrSV3h/Rvh3eth767jernYiBBuH5XKt2tymbkqu3ZrFRGRWuPJKaCDpj2KiLhOQU2OrmkyXPMZjPwjLPvAGV1bP+e4XuqqQcm0i4/gb5+voKy89jfZFhGRE1NaXsH63EKtTxMR8QEKanJsgUEw/G64/isIDHbWrU37C5SV1OhlQoICmDgmjdXbC3g7Q5tgi4j4mg07iiirsKQmKqiJiLhNQU2qr3U63PQt9BkP3/0LXjwVctfU6CXGdG9Ov+Sm/PPrVRTs0ybYIiK+RB0fRUR8h4Ka1ExoFJz7H7jkddi1wdlzbeGLYG21TjfG8Iczu5JbUMKkmZl1XKyIiNREZo4T1LRGTUTEfQpqcny6nA2/+h7aDoTP7oS3LoPC3Gqd2rtNLOf0asnz365l6+69dVyoiIhUlye7gBYxYUSFBrldiohIo6egJsevSQsY/z6c/jBkfgPPDII1X1fr1P8b0xkLPPblqrqtUUREqs2TXaBpjyIiPkJBTU5MQAAMugVunAGRCfDGhfD53VB69JGy1k0juG5wCu//sJmlm3fXU7EiInIkFRWWzJwCOqiRiIiIT1BQk9rRvLsT1gb8ChZMhskjYOtPRz3llpEdiIsM4cHPlmOrucZNRETqxtY9xRSVlGtETUTERyioSe0JDoOxjzjTIffuhBdGw9x/Q8Xh90xrEhbMHad0ZN7aPKat0CbYIiJuUsdHERHfoqAmtS91tNNopONp8NUf4bXzYM+Wwx56Wf+2tE+M5OHPV1CqTbBFRFyjoCYi4lsU1KRuRMY7LfzPfgqyFjqNRpZ9+IvDggMD+P3YLqzNLWT8C/N5eoaHhevzKC4tr/+aRUQaMU92AbERwcRHhrhdioiIAOq/K3XHGOh7NSQPgfdugHeuhjVXwNhHITS68rDRXZK445SOfPbT1soukCGBAfRsHUO/lDj6JTelb9s4YiKC3fpOREQavMzsAlITozDGuF2KiIigoCb1Ib4DXP8VzHwEvvsnbJgDF7wAbfoBzibYd5zSiTtO6UReYQmLNuxk4fo8Fq7P4/nZa3l2psUY6NwsmvTkpvRLjqNfchwtY8Nd/sZERBoOT04Bp3Vt5nYZIiLipaAm9SMwGEbf56xfe/8meOl0GP5/MPR3EHjgr2FcZAindm3Gqd5fFvaWlLN40y4y1uexcMNOPvxxC6/P2whAq9hw+iU3Jd0b3DomRREQoE+CRURqKq+whLzCEq1PExHxIQpqUr/anQy/+s7Za23mw+D5Bi6YDHEphz08PCSQQR3iGdQhHoCy8gpWbstn4fo8MtbvZE7mDj5c7DQqiQkPJr1d08rpkt1bxRAaFFhv31p92VdWzrbdxWTn76N5kzBaxYYroIrUMWPMGOBJIBB4wVr7yGGOGQE8AQQDudba4d7H1wP5QDlQZq1Nr5eia2B/I5EOCmoiIj5DQU3qX1iME846ngaf3gmThsDYv0Pvy511bUcRFBhA91YxdG8Vw7WDU7DWsjGviIXrd5KxPo8F6/P4ZqXT6j80KIBebWIrR936tmtKkzDfXudWUWHZUVjCll17ndvu4gNf79rL5l3F5BbsO+icyJBAOjaLJq15NJ32/9k8moSoUJe+C5GGxRgTCDwNnApkAQuNMR9ba5dXOSYWeAYYY63daIxJOuRlRlprc+ur5pqq7Pioza5FRHyGgpq4p8eF0GYAfHATfHQLLHsfTnsIktKq/RLGGNrFR9IuPpIL+7YGYEfBvsrgtnDDTibNWkv5jEyMgbTmTehfZbpk85iwuvruDqtwXxlbdzuBa2uV8LVl11627naCWUnZwdsUhAcH0jI2jJax4aQ1b0LL2HBaxoaRGB3Kll3FrN6ez6pt+Xy1fDtTFm6qPC8+MoTO3vDWuXl05ddRofrfXqSG+gMea+1aAGPMFOBcYHmVYy4H3rfWbgSw1vrV5pCZOQWEBwfSSmt/RUR8hn5jE3fFtoGrP4H5k2Dmo/DsydD3GhhxL0QlHtdLxkeFMqZ7c8Z0bw5AUUkZizfuYuF6p0nJO4uy+O/3GwBoExdOv3ZxpCfH0T+lKR1OoONZWXkF2fn7KoNY1ZGwLbuK2bJ7L7uKSg86J8BAsyZOCOvROpbTu4V5g5gTxlrGhBMbEVytmqy15BaUsHp7Piu35bN6Wz6rtufzdsYmikoObHfQKja8ctStszfEtU+MbJDTREVqSStgU5X7WcCAQ47pBAQbY2YC0cCT1tpXvc9Z4CtjjAWes9ZOPtxFjDETgAkAbdu2rb3qq8GTXUD7xEhNoxYR8SEKauK+gEAYdCv0vBRmPQILX4Sf34Ghd8GAmyH4xEa9IkKCODk1gZNTEwAnUC3fuqdy1G32mhze/3EzAE0jgr2jbc6oW/eWMYQEBWCtZU9x2S+mIW7dfSCIbdtTTHmFPejaTcKCaBkbTqvYcE5qF1v5dcvYcFrEhNGsSRjBgbWznaExhsToUBKjQxns/V7BmU65eddeVnmD26pt+azens/sNTmUljv1BgYYUhIinZG3ZgemULaJiyBQv7iJHO5/AnvI/SCgLzAaCAe+N8bMs9auBgZba7d4p0N+bYxZaa2d/YsXdALcZID09PRDX79OebILSE9uWp+XFBGRY1BQE98RGQ9nPAb9boSv74Npf4aMF+GUv0C3C465fq26ggID6Nk6lp6tY7l+iLPObf2OImdLgHV5ZGzYydfLtwMQFhxAy9hwtu8uprDk4E24gwMNLWKcka8BKXEHj4R5g1i0D6yJCwgwtImLoE1cBKdUab1dUlbB+h2FToDzhrifs3bz2U9bK48JCw6gY1L0gQDX3AlwSdGh2mtJGpMsoE2V+62BLYc5JtdaWwgUGmNmA72A1dbaLeBMhzTGfIAzlfIXQc0tRSVlbN61l0sT2xz7YBERqTcKauJ7EjvB5f+DtTPhyz/Au9fBvElw+t8q916rTcY4o0kpCZFcnO78opKdX8yi9TtZuH4nW3fvZXinxINGwlrFhpMQFerX04RCggLo5B09O7vXgceLSspYs73g4NG31Tm8uyir8piY8ODKaZOVUyibRWtTcmmoFgIdjTEpwGbgUpw1aVV9BPzHGBMEhOBMjfyXMSYSCLDW5nu/Pg14oP5KP7a1OYUAas0vIuJjFNTEd7UfATfNhsVvwvS/wounQPdxMPrP0LRdnV46KTqMsT1aMLZHizq9ji+KCAmiV5tYerWJPejxvEJn/VvVNXAfLt5MfnFZ5THNm4TRqXk0A1LiuKx/W+IiQ+q5epHaZ60tM8bcBnyJ057/JWvtMmPMzd7nJ1lrVxhjpgI/ARU4LfyXGmPaAx94R6CDgDettVPd+U4Or7Ljo4KaiIhPMdbW6zT4Sunp6TYjI8OVa4sf2lcAc56Euf8GWwGDboEhd0JYE7cra9SstWzdXcyq7d7mJdvyWbEtnxVb9xAaFMC4vq25bnCKfgEUjDGLfHH/MF9Vn++Rj3+5imdnZbLigTGEBNXOmlkREameo70/akRN/ENoFIz6A/S9Gr75K3z3L/jhNeexPldBoP4qu8EYU7k2b2TnA9tGrdmez0tz1vHuoizenL+RUWlJ3DAkhUEd4rW2TcTHeLILaBcfoZAmIuJj9K+y+JeY1nDBc3DjDEjsDJ/+FiYNhjXT3K5MqujYLJqHL+jJ3Imj+O0pnfgpaxeXvzCfM576jvcWZf1irzgRcY8np0AbXYuI+CAFNfFPrU6Caz6DS16Hsn3wxjh47QLYvvzY50q9SYgK5TendOS7e0bx93E9Ka+o4K53ljDk0ek8PcPDzsISt0sUadRKyytYn1uo6ckiIj5IQU38lzHQ5Wy4dYHTEXJzhjO69skdUJDtdnVSRVhwIBf3a8OXdwzj1ev6k9aiCY99uYpBj3zDHz/8mbU5BW6XKNIobdhRRFmFVVATEfFBWtgj/i8oxNkwu9dlMOtRWPgC/PwuDL0TBt5ywhtmS+0xxjCsUyLDOiWyals+L363lrcXZvHG/I2MTkvi+iHtGdg+TuvYROqJOj6KiPgujahJwxERB2MfhVvmQ8ow+OZ++E8/J7S51N1Ujqxz82j+fmEv5kwcxe2jOvLDxl1c9vw8zvr3d3zwo9axidSHTO9odnutURMR8TkKatLwJKTCZW/C1Z9AeAy8dz28cApsnO92ZXIYidGh3HlqJ+ZOHMXDF/SguLSc3/5vCcP+PoNnZ2ayu6jU7RJFGixPdgEtYsKICtUEGxERX6OgJg1XyjCYMAvOfQZ2Z8FLp8E718DO9W5XJocRFhzIZf3b8vVvh/Pytf1ITYri0akrGfjwN/z5o6Wszy10u0SRBseTXaBpjyIiPqpaQc0YM8YYs8oY4zHGTDzKcf2MMeXGmAtrr0SRExAQCH2ugF//AMMnwuovnemQX/8Jine7XZ0cRkCAYWTnJF6/YQBf/GYoZ/ZswZsLNjLyHzOZ8GoGC9blYTWVVeSEVVRYMnMK6KBpjyIiPumYQc0YEwg8DYwFugKXGWO6HuG4R4Eva7tIkRMWEgkj74XbF0GPi2DOU/BUH1jwPJSXuV2dHEGXFk14/KJezLlnFLeNTGXh+jwufu57zn16Dh8t3kxpudaxiRyvrXuKKSop14iaiIiPqs6IWn/AY61da60tAaYA5x7muNuB9wD1RRff1aQlnPcMTJgJSV3h89/BsyfD6q/UcMSHJTUJ467TOjN34mgePK87BcVl/GbKYob9fQbPzcpk916tYxOpKXV8FBHxbdUJaq2ATVXuZ3kfq2SMaQWcD0w62gsZYyYYYzKMMRk5OTk1rVWk9rTs7TQbufRNqCiDNy+C186H7cvcrkyOIjwkkPED2zHtzuG8eHU6yfGRPPzFSgY9/A1/+XgZG3cUuV2iiN9QUBMR8W3VCWqH29Do0KGHJ4B7rLXlR3sha+1ka226tTY9MTGxmiWK1BFjIO1MuGUejHkEtvwIk4bAx7+G/O1uVydHERBgGN2lGW9NGMintw9hTLfmvD5vAyMen8HNry0iY73WsYkciye7gNiIYOIjQ9wuRUREDqM6/XizgDZV7rcGthxyTDowxbtJbQJwhjGmzFr7YW0UKVKngkJg4K+g5yUw+3FYMBmWvgdD7oBBt0FwuNsVylF0bxXDPy/pzf+NSePV79fzxvyNTF22jd5tYrlhaApjujUnKFANbkUOlZldQGpilDaYFxHxUdX57WUh0NEYk2KMCQEuBT6ueoC1NsVam2ytTQbeBW5RSBO/ExEHY/4Gt86H9iNg+oPw73RY8j+oOOpgsfiA5jFh/N+YNL6/dxR/Pbcbu4pKuO3NHxn+2Exe+HYte4q1jk2kKk+OWvOLiPiyYwY1a20ZcBtON8cVwNvW2mXGmJuNMTfXdYEi9S6+A1z6BlzzGUTGwwcT4NnBsPxjNRzxAxEhQVw5KJlv7hrB81el07ppOA9+toKTH57OXz5exlfLtpG1s0hTI6VRyyssIa+wREFNRMSHVWfqI9baz4HPD3nssI1DrLXXnHhZIj4geQjcOBOWfwgz/gZvXwktesHIP0LHU501buKzAgMMp3Ztxqldm/Fz1m5e/G4tr8/bwCtz1wMQEx5M1xZN6NayCV29tw6JUQRrmqQ0AvsbiXRQUBMR8VnVCmoijVZAAHS/ALqcAz+/AzMfdjpEtu4Po/4I7Ye7XaFUQ4/WMTxxaR/+dkEPVm7LZ/mWPSzbsoflW/fw2rwN7Ctz9mMLCQqgc7PoA+GtRRO6tGhCZKj+qZSGpbLjoza7FhHxWfrtQ6Q6AoOg92XQ40L48XWY/Ri8eg4kD3UCW9uBblco1RAREsRJbZtyUtumlY+VlVewLreQ5Vu94W3LHr5cto0pC51dSYyB5PjIyuC2P8QlRYe59W2InDBPdgHhwYG0ilWzJBERX6WgJlITgcGQfi30ugwWvQLf/gNeOh1ST4VRf4CWfdyuUGooKDCAjs2i6dgsmnN7O1tEWmvZtqeYZZudUbflW/bwU9YuPvtpa+V5CVGhlaGtmzfEJcdHEhCgKbHi+zw5BbRP1N9XERFfpqAmcjyCw2DgzXDSlbDgeZjzBEweAWlnwcjfQ7NublcoJ8AYQ4uYcFrEhHNK12aVj+/eW8oKb3DbP3Vyzuy1lFU4jUkiQgLp0uLgkbdOzaIJCw5061sROazM7ALSk5se+0AREXGNgprIiQiJdPZbS78O5j0L3/8HVn4G3cfBiHshIdXtCqUWxYQHM7B9PAPbx1c+tq+snDXbCypH3pZv2cMHP27mtXkbAKepSWpi1EEjb11bNiE2QpsMizsK95WxeddeLk1sc+yDRUTENQpqIrUhrAmMuAf63+iEtXmTYNn7zhTJ4f8HTZPdrlDqSGhQIN1bxdC9VUzlYxUVlk07iw4aeZubmcsHP26uPKZVbDhdqoy8JUSFUmEtFRWWcmuxFsorv7aUV3DQ8xXWuU6FtZRXeI+v/Nr5s8J6zznK+eXe5yu8x1ee7z3HWsu9Y7sQExHsxo9X6sDanEIAteYXEfFxCmoitSkiDkb/CQb8Cr77Fyx8AX76H5x0FQz9HcS0crtCqQcBAYZ28ZG0i49kbI8WlY/nFuxzRt0qp0/u5puV213bni/AOCN+Aca5BQYYzC8eg9+e2okYFNQaCk9OPqCgJiLi6xTUROpCVCKM+RucfJvTcGTRf+HHN6Df9TDktxCV5HaF4oKEqFCGdUpkWKfEyseKSspYtS2fPcVlTnAyBuMNTYEBznq5QG9oCgigMlA5QapKqAowlecHeB8LNAYT4DxWGcLMgeOlcfJkFxDo/TBBRER8l4KaSF1q0hLO/Aec/GuY/XeY/5zTLXLATc5jEXFuVyguiwgJok9bNXWQ+uPJLqBdfAQhQdrcXUTEl+lfaZH60LQdnPs03LoA0s6E756AJ3vBzEegeI/b1YlII+LJLtBG1yIifkBBTaQ+JaTCuBfgV3Oh/XCY+TA82dNZz1ZS6HZ1ItLAlZZXsGFHkdaniYj4AQU1ETc06wqXvA4TZkLrfjDtL84I27xnobTY7epEpIHasKOQsgqroCYi4gcU1ETc1LIPXPEOXPcVJHWBqRPhqT6Q8RKUlbhdnYg0MJ7sAkAdH0VE/IGCmogvaDsArv4ErvoYYtvAp7+F/6TD4jehvMzt6kSkgdgf1DpojZqIiM9TUBPxJe2Hw3VfwhXvQngsfPgreGYgLH0PKircrk5E/Jwnu4CWMWFEhqrps4iIr1NQE/E1xkDHU2HCLGcdW0AQvHsdTBoCKz/Dtd2RRcTveXIK6KBpjyIifkFBTcRXGQNdzoZfzYFxL0JZMUy5HJ4fCWumKbCJ1CNjzBhjzCpjjMcYM/EIx4wwxiw2xiwzxsyqybn1oaLCkpldqPVpIiJ+QkFNxNcFBEKPC5092M59Ggp3wBvj4KUxsPgtKMpzu0KRBs0YEwg8DYwFugKXGWO6HnJMLPAMcI61thtwUXXPrS9bdu9lb2m5gpqIiJ/QJHURfxEYBH3GQ4+L4cdXnU2zP7wZTCC0OxnSzoK0MyC2rduVijQ0/QGPtXYtgDFmCnAusLzKMZcD71trNwJYa7NrcG69qOz4qEYiIiJ+QSNqIv4mKAT63QB3/Aw3zoAhv4XCXJh6DzzRAyYNhZmPwLafNT1SpHa0AjZVuZ/lfayqTkBTY8xMY8wiY8xVNTgXAGPMBGNMhjEmIycnp5ZKP0Ct+UVE/ItG1ET8lTHQ6iTnNvo+2JHpNBtZ9bkT1GY+7IyupZ0FaWdCm4HOqFxjUlEB2cthw1zYMAdyV0On0yH9Oo08Sk2Ywzx26KcgQUBfYDQQDnxvjJlXzXOdB62dDEwGSE9Pr/VPWTJzCmgaEUx8VGhtv7SIiNSBRvZbm0gDFt8BBv/auRVkw+qpTnBb+CLMewbC46DTGCe0dRgFIRFuV1z7ykth6xInlG2YCxu/h+LdznNNWkNcCsx50rl1Ggv9b4CUERCgyQVyVFlAmyr3WwNbDnNMrrW2ECg0xswGelXz3HrhyS7QaJqIiB9RUBNpiKKS4KSrnNu+Asj8BlZ+7oy2LXkTgsKdsJZ2phPeIuPdrvj4lBTB5gzY8L0TzrIWQmmR81x8R+h6LrQb7Kzh2z+CtmsTLHoZFv0XVn0G8anOVNJelzl714n80kKgozEmBdgMXIqzJq2qj4D/GGOCgBBgAPAvYGU1zq0XnuwCxnRv7salRUTkOCioiTR0oVFOYOl6rjPitGGuM9K28jMnqJgAaDvICW2dz3BGnXxV8W7YtODAiNnmH6CiFDDQvLsTTNsOcoJZVNLhXyO2DYz+Ewy/B5Z9CAufh6kT4ZsHoOfF0O9G57VEvKy1ZcaY24AvgUDgJWvtMmPMzd7nJ1lrVxhjpgI/ARXAC9bapQCHO7e+v4cdBfvYWVRKBzUSERHxG8a61GwgPT3dZmRkuHJtEcFpNLLtpwOhbftS5/Fm3Q+Etha9nLVwbinIgY1zD4yYbV8KtsLZBLzlSU4ga3cytBlwYqNhWxY7ge3nd5396tqe7EyLTDvbad4iJ8wYs8ham+52Hf6itt8j56/dwSWT5/HKtf0Y0fkIH2KIiEi9O9r7o0bURBorY5wg1qIXjPw95K1zpkau/AxmPwazHoWYNk5gSzvTCUSBwXVb065N3rVlc50/c1c7jweFQ5t+zihYu5OhVXrtrrFr2dvZo+7Uv8KPr0PGi/DudRDVDPpeA32vhSYtau96IvXMk6OOjyIi/kZBTUQccSkw6FbnVrjjQDOSH/4LC56DsFhvM5IzoMNoZ0rlibAWdni80xi/d4LZ7o3Oc6Ex0HYg9L7CWWPWolf9jGxFxDnNWAbdBp5pzijbrL/Dt/9wumf2v9Gpx81RRpHj4MkuIDw4kJYx4W6XIiIi1aSgJiK/FBkPfa5wbiWFkDnDCW2rp8JPUyAwFDqM9DYjGQtRicd+zYpy2L7s4BGzQu9eUZFJ0G4QnHybM2KW1BUCAuv2ezyagADodJpzy1vrdM788XVY/qFTW7/roeelJx5WReqJJ7uADkmRBAToQwYREX+hoCYiRxcSCV3Ocm7lZbBpnndd26dOcMM4o1/7p0jGd3DOKyuBrYsPjJhtnAf7vK3yY9tC6inexh+DnXN8dZQqrj2c/hCM/AMsfc8ZZfvsLvj6L9D7cqdjZGInt6sUOarM7AL6p8S5XYaIiNSAgpqIVF9gECQPcW6n/81p7rE/tH19n3NL7AKRCZCVAWV7nfMSOkP3C5zRsraDnM6L/iYkAk66EvqMd763BZOdNv8LnoOU4dB/gjM1tLFtKi4+r3BfGVt2F2t9moiIn9FvFCJyfIyB5j2c24iJsHMDrPrCCW378iH92gPBLDLB7WprjzFOY5M2/Zyw+sN/IeNl+N8Vzqba6dfCSVdXbzqoSD3IVCMRkQaptLSUrKwsiouL3S5FqiEsLIzWrVsTHFz9xmwKaiJSO5q2g4E3O7fGIioRhv0OBt8Bq7+ABc/D9L86HTO7nuc0H2ndz3endUqj4MlWUBNpiLKysoiOjiY5ORmj9xmfZq1lx44dZGVlkZJS/f1qFdRERE5UYBB0Odu55ayGhS/A4jfh57eheU9nWmT3cbW7pYBINXmyCwgKMLSLj3S7FBGpRcXFxQppfsIYQ3x8PDk5OTU6L6CO6hERaZwSO8EZf4e7VsCZ/4DyUvj4NvhnF/jyD04XSZF65MkuoF18BMGBessXaWgU0vzH8fy30r/aIiJ1ITTa6Qh5y/dwzWfQfgTMexaeOgneuAhWfwUVFW5XKY2AJ6dA0x5FRPyQpj6KiNQlYw50ytyzFRa94nSLfPMiaJoM6dc7nSQj1Dpdal9JWQUbdhQxtntzt0sREZEa0oiaiEh9adICRt4LdyyFC1+C6JbOlgb/7AIf3uK0/bfW7SqlAdmwo5DyCqsRNRGpdbt27eKZZ56p8XlnnHEGu3btqv2CGiCNqImI1LegEKe5SPdxsG2ps4n2T+/A4jec7Q76Xgs9L3amT4qcgMqOj4n6uyTSkN3/yTKWb9lTq6/ZtWUT/nx2tyM+vz+o3XLLLQc9Xl5eTmBg4BHP+/zzz2utxrpwrPrrk0bURETc1Lw7nP0k3LUSzvwnWOCzO+EfafDJb2DrErcrFD+2P6h1SFLHRxGpXRMnTiQzM5PevXvTr18/Ro4cyeWXX06PHj0AOO+88+jbty/dunVj8uTJleclJyeTm5vL+vXr6dKlCzfeeCPdunXjtNNOY+/evUe83vPPP0+/fv3o1asX48aNo6ioCIDt27dz/vnn06tXL3r16sXcuXMBePXVV+nZsye9evXiyiuvBOCaa67h3XffrXzNqChntsHMmTOrXf/UqVM56aST6NWrF6NHj6aiooKOHTtWdnSsqKggNTWV3NzcE/4Za0RNRMQXhDWBftdD+nWweRFkvARL/uesaWt5kvN49wsgRL9wS/V5cgpoFRtORIje7kUasqONfNWVRx55hKVLl7J48WJmzpzJmWeeydKlSyv3CXvppZeIi4tj79699OvXj3HjxhEfH3/Qa6xZs4a33nqL559/nosvvpj33nuP8ePHH/Z6F1xwATfeeCMAf/zjH3nxxRe5/fbb+fWvf83w4cP54IMPKC8vp6CggGXLlvHQQw8xZ84cEhISyMvLO+b3s2DBgmPWX1FRwY033sjs2bNJSUkhLy+PgIAAxo8fzxtvvMEdd9zBtGnT6NWrFwkJCSfy4wWqOaJmjBljjFlljPEYYyYe5vlzjTE/GWMWG2MyjDFDTrgyEZHGyBhonQ7nPeO0+B/zKJQWOS3+/5EGn98N25e7XaX4CU92AR20Pk1E6kH//v0P2sz5qaeeolevXgwcOJBNmzaxZs2aX5yTkpJC7969Aejbty/r168/4usvXbqUoUOH0qNHD9544w2WLVsGwPTp0/nVr34FQGBgIDExMUyfPp0LL7ywMizFxR27YVd16p83bx7Dhg2rPG7/61533XW8+uqrgBPwrr322mNerzqO+RGbMSYQeBo4FcgCFhpjPrbWVv1N4RvgY2utNcb0BN4G0mqlQhGRxiq8KQy8GQbcBBu/h4yXnRG2BZOhzUBIvxa6ngfBYW5XKj6oosKSmVPAgJT4Yx8sInKCIiMPzPiYOXMm06ZN4/vvvyciIoIRI0ZQXFz8i3NCQ0Mrvw4MDDzq1MdrrrmGDz/8kF69evHKK68wc+bMIx5rrT3svmVBQUFUeLfGsdZSUlJSo/qP9Lpt2rShWbNmTJ8+nfnz5/PGG28csbaaqM6IWn/AY61da60tAaYA51Y9wFpbYG1lq7JInFUWIiJSG4yBdifDuOfhzpVw2oNQmAMf3AT/THM20s795SeV0rht3rWX4tIKdXwUkToRHR1Nfn7+YZ/bvXs3TZs2JSIigpUrVzJv3rwTvl5+fj4tWrSgtLT0oCA0evRonn32WcBpBLJnzx5Gjx7N22+/zY4dOwAqpz4mJyezaNEiAD766CNKS0trVP+gQYOYNWsW69atO+h1AW644QbGjx/PxRdfXGvNSKoT1FoBm6rcz/I+dhBjzPnGmJXAZ8B1h3shY8wE79TIjP0L7kREpAYi4+Hk2+G2DLjqY0gZDvMnwX/S4ZWzYOl7UFZy7NeRBs+T4+34qKAmInUgPj6ewYMH0717d+6+++6DnhszZgxlZWX07NmT++67j4EDB57w9f76178yYMAATj31VNLSDkzce/LJJ5kxYwY9evSgb9++LFu2jG7duvGHP/yB4cOH06tXL+68804AbrzxRmbNmkX//v2ZP3/+QaNo1ak/MTGRyZMnc8EFF9Dr/9u79+CoyjSP49+HBMxFiUBQ0YCwSgkiJkjAC8WUiASYYWNRoIDgLLigoCDrDDuELW+Uo6JLoWuhy7IOlKKusDhQsytIEHG8zBYkwahcZriPxiAgDJdwG4LP/tEtEzS3bhpOd/L7VHVxztt93n76TeDh6fOe92RnM2zYsNPH5OfnU1FREbNpjwDmddyzx8zuBPq7+9jw/j1AT3efVMPrfwI85u6319Zvbm6uFxcXRxe1iIj8zeHdUPp6aFrkgS8hvTXkjITuo6Flh7qOPi/MrMTdc4OOI1HEIke+8tF2fv3OJtY92o+W6c1iFJmIxItNmzbRuXPnoMOQsOLiYh5++GE++uijGl9T3c+stvxYnzNqZUDbKvtZQHlNL3b3D4GrzOzslzoREZG6XXQp9P4lPPQZjHwbsnrCH16EF3NgwWDY9D9wqvrpHdJwbdtbQcv0ZirSRETOsRkzZjBkyBCeeeaZmPZbn/V6i4COZtYB+BoYDtxd9QVmdjWwLbyYyA1AM2BfTCMVEZHaNWkCHW8PPQ5+DZ8ugHWvwcJRcOFlcMPPQ4+L29bdlyS8rXsquLq1pj2KSGJ58MEH+eSTT85omzx5ckynFMZaQUEBBQU/Whj/rNVZqLl7pZlNBFYAScA8d99gZuPDz88BhgA/N7OTwDFgmNc1p1JERM6djCvg1gLoPQW2FELJfPjwX+GjmdAxD7qPgY79oElsLniW+LN1TwUDrmsTdBgiIhF56aWXgg4hbtTrDpjuvgxY9oO2OVW2nwWejW1oIiJy1pKSodNPQ4+//Dl0hm3da7D5XchoCzf8A3QbBc31H/qGZF/FCf5y9KQWEhERSWD1uuG1iIg0AC2uhL6Pwi82wl2vQaurYPWv4fkuoemRW1dB+P4ykti27tGKjyIiia5eZ9RERKQBSWoK194ReuzbFlotsvSN0KIjLdqHVovMGQUXtg44UImWluYXEUl8OqMmItKYtboK8p6EX2yCIb+B5lfAe0/ArM7w32Ngx0egS44TztY9FaQ1S+LyjJSgQxGRBurAgQO8/PLLUR37wgsvcPTo0RhH1PCoUBMREUi+ALoOhTHL4MG10GMsbFsFrw6CbzcHHZ1EaOueCq5qfSFmFnQoItJANZRCrbKyMugQaqSpjyIicqbW18DAGXD747D996F9SSiP/30XDh/XvfNEGo3lBfDNF7Ht87KuoVxQg4KCArZt20ZOTg79+vXjkksuYdGiRZw4cYLBgwczffp0jhw5wl133UVZWRmnTp3i0UcfZffu3ZSXl9OnTx8yMzNZvXp1tf1PmDCBoqIijh07xtChQ5k+fToARUVFTJ48mSNHjnDBBRewatUq0tLSmDp1KitWrMDMGDduHJMmTaJ9+/YUFxeTmZlJcXExU6ZM4YMPPuCJJ56gvLycnTt3kpmZydNPP80999zDkSNHAJg9eza33HILAM899xwLFiygSZMmDBw4kHHjxnHnnXeybt06ALZs2cLw4cMpKSmJ5egDKtRERKQmTVPhmgFBRyFR0LVpInKuzZgxg/Xr11NaWkphYSGLFy9m7dq1uDv5+fl8+OGH7N27l8svv5x33nkHgIMHD5KRkcGsWbNYvXo1mZmZNfb/1FNP0bJlS06dOkXfvn35/PPP6dSpE8OGDWPhwoX06NGDQ4cOkZqayty5c9mxYweffvopycnJ7N+/v874S0pK+Pjjj0lNTeXo0aOsXLmSlJQUtmzZwogRIyguLmb58uUsXbqUNWvWkJaWxv79+2nZsiUZGRmUlpaSk5PD/PnzGT16dKyG9Qwq1EREREREElktZ77Oh8LCQgoLC+nWrRsAFRUVbNmyhd69ezNlyhSmTp3KoEGD6N27d737XLRoEXPnzqWyspJdu3axceNGzIw2bdrQo0cPAJo3bw7Ae++9x/jx40lODpU2LVu2rLP//Px8UlNTATh58iQTJ06ktLSUpKQkNm/efLrfMWPGkJaWdka/Y8eOZf78+cyaNYuFCxeydu3aen+uSKhQExERERGRqLk706ZN4/777//RcyUlJSxbtoxp06aRl5fHY489Vmd/O3bsYObMmRQVFdGiRQtGjx7N8ePHcfdqr72tqT05OZnvwredOX78+BnPpaenn95+/vnnufTSS/nss8/47rvvSElJqbXfIUOGMH36dG677Ta6d+9Oq1at6vxM0dBiIiIiIiIiEpGLLrqIw4cPA9C/f3/mzZtHRUXo1iBff/01e/bsoby8nLS0NEaNGsWUKVNOX9dV9djqHDp0iPT0dDIyMti9ezfLly8HoFOnTpSXl1NUVATA4cOHqaysJC8vjzlz5pxeGOT7qY/t27c/fe3Y22+/XeP7HTx4kDZt2tCkSRMWLFjAqVOnAMjLy2PevHmnFz75vt+UlBT69+/PhAkTGDNmTBSjVz8q1EREROpgZgPM7E9mttXMCqp5/lYzO2hmpeHHY1We22lmX4Tbi89v5CIi50arVq3o1asX1113HStXruTuu+/m5ptvpmvXrgwdOpTDhw/zxRdf0LNnT3Jycnjqqad45JFHALjvvvsYOHAgffr0qbbv7OxsunXrRpcuXbj33nvp1asXAM2aNWPhwoVMmjSJ7Oxs+vXrx/Hjxxk7dizt2rXj+uuvJzs7mzfffBOAxx9/nMmTJ9O7d2+SkpJq/CwPPPAAr776KjfddBObN28+fbZtwIAB5Ofnk5ubS05ODjNnzjx9zMiRIzEz8vLyYjKe1TEP6P44ubm5XlysfCUi0hiYWYm75wYdRzTMLAnYDPQDyoAiYIS7b6zymluBKe4+qJrjdwK57v5tfd9TOVJE6rJp0yY6d+4cdBiN1syZMzl48CBPPvlkvY+p7mdWW37UNWoiIiK16wlsdfftAGb2FnAHsLHWo0REpEEaPHgw27Zt4/333z+n76NCTUREpHZXAF9V2S8DbqzmdTeb2WdAOaGzaxvC7Q4UmpkD/+Huc6t7EzO7D7gPoF27drGKXUQkrt14442cOHHijLYFCxbQtWvXgCKq25IlS87L+6hQExERqd2Pl/wKFV9VrQOudPcKM/spsBToGH6ul7uXm9klwEoz+6O7f/ijDkMF3FwITX2MWfQiInFszZo1QYcQt7SYiIiISO3KgLZV9rMInTU7zd0PuXtFeHsZ0NTMMsP75eE/9wBLCE2lFBE5a0GtNSGRi+ZnpUJNRESkdkVARzPrYGbNgOHA76q+wMwus/DNdsysJ6H8us/M0s3sonB7OpAHrD+v0YtIg5SSksK+fftUrCUAd2ffvn2n789WX5r6KCIiUgt3rzSzicAKIAmY5+4bzGx8+Pk5wFBggplVAseA4e7uZnYpsCRcwyUDb7r7u4F8EBFpULKysigrK2Pv3r1BhyL1kJKSQlZWVkTHqFATERGpQ3g647IftM2psj0bmF3NcduB7HMeoIg0Ok2bNqVDhw5BhyHnkKY+ioiIiIiIxBkVaiIiIiIiInFGhZqIiIiIiEicsaBWijGzvcCfz7KbTODbGITTmGjMIqcxi5zGLHINfcyudPfWQQeRKJQjA6Mxi5zGLHIas8g09PGqMT8GVqjFgpkVu3tu0HEkEo1Z5DRmkdOYRU5jJrGm36nIacwipzGLnMYsMo15vDT1UUREREREJM6oUBMREREREYkziV6ozQ06gASkMYucxixyGrPIacwk1vQ7FTmNWeQ0ZpHTmEWm0Y5XQl+jJiIiIiIi0hAl+hk1ERERERGRBkeFmoiIiIiISJxJ2ELNzAaY2Z/MbKuZFQQdT7wzs7ZmttrMNpnZBjObHHRMicDMkszsUzP736BjSRRmdrGZLTazP4Z/324OOqZ4ZmYPh/9Orjez/zKzlKBjksSm/BgZ5cfoKUdGRvkxco09RyZkoWZmScBLwEDgWmCEmV0bbFRxrxL4pbt3Bm4CHtSY1ctkYFPQQSSYfwPedfdOQDYavxqZ2RXAQ0Cuu18HJAHDg41KEpnyY1SUH6OnHBkZ5ccIKEcmaKEG9AS2uvt2d/8r8BZwR8AxxTV33+Xu68Lbhwn943BFsFHFNzPLAn4GvBJ0LInCzJoDPwF+A+Duf3X3A4EGFf+SgVQzSwbSgPKA45HEpvwYIeXH6ChHRkb5MWqNOkcmaqF2BfBVlf0y9I9qvZlZe6AbsCbgUOLdC8CvgO8CjiOR/B2wF5gfng7zipmlBx1UvHL3r4GZwJfALuCguxcGG5UkOOXHs6D8GJEXUI6MhPJjhJQjE7dQs2radJ+BejCzC4G3gX9y90NBxxOvzGwQsMfdS4KOJcEkAzcA/+7u3YAjgK6RqYGZtSB0tqMDcDmQbmajgo1KEpzyY5SUH+tPOTIqyo8RUo5M3EKtDGhbZT+LRnYqNBpm1pRQEnrD3X8bdDxxrheQb2Y7CU0dus3MXg82pIRQBpS5+/ffRi8mlJikercDO9x9r7ufBH4L3BJwTJLYlB+joPwYMeXIyCk/Rq7R58hELdSKgI5m1sHMmhG6sPB3AccU18zMCM2L3uTus4KOJ965+zR3z3L39oR+v95390b1LU403P0b4Cszuybc1BfYGGBI8e5L4CYzSwv/He2LLi6Xs6P8GCHlx8gpR0ZO+TEqjT5HJgcdQDTcvdLMJgIrCK0AM8/dNwQcVrzrBdwDfGFmpeG2f3H3ZcGFJA3UJOCN8H8StwNjAo4nbrn7GjNbDKwjtPLcp8DcYKOSRKb8GBXlRzlflB8joBwJ5q6p6yIiIiIiIvEkUac+ioiIiIiINFgq1EREREREROKMCjUREREREZE4o0JNREREREQkzqhQExERERERiTMq1EQiYGanzKy0yqMghn23N7P1sepPRETkfFF+FIm9hLyPmkiAjrl7TtBBiIiIxBnlR5EY0xk1kRgws51m9qyZrQ0/rg63X2lmq8zs8/Cf7cLtl5rZEjP7LPy4JdxVkpn9p5ltMLNCM0sNv/4hM9sY7uetgD6miIhIRJQfRaKnQk0kMqk/mNoxrMpzh9y9JzAbeCHcNht4zd2vB94AXgy3vwj83t2zgRuADeH2jsBL7t4FOAAMCbcXAN3C/Yw/Nx9NREQkasqPIjFm7h50DCIJw8wq3P3Catp3Are5+3Yzawp84+6tzOxboI27nwy373L3TDPbC2S5+4kqfbQHVrp7x/D+VKCpu//azN4FKoClwFJ3rzjHH1VERKTelB9FYk9n1ERix2vYruk11TlRZfsUf7uO9GfAS0B3oMTMdH2piIgkCuVHkSioUBOJnWFV/vy/8PYfgOHh7ZHAx+HtVcAEADNLMrPmNXVqZk2Atu6+GvgVcDHwo28tRURE4pTyo0gU9K2DSGRSzay0yv677v79EsQXmNkaQl+AjAi3PQTMM7N/BvYCY8Ltk4G5ZvaPhL4ZnADsquE9k4DXzSwDMOB5dz8Qo88jIiISC8qPIjGma9REYiA8Bz/X3b8NOhYREZF4ofwoEj1NfRQREREREYkzOqMmIiIiIiISZ3RGTUREREREJM6oUBMREREREYkzKtRERERERETijAo1ERERERGROKNCTUREREREJM78P+wofrsk6w6jAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 1080x504 with 2 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "from helper_functions import plot_loss_curves\n",
        "\n",
        "plot_loss_curves(effnetb2_results)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZA9p7C5oQaQM"
      },
      "source": [
        "Woah!\n",
        "\n",
        "Those are some nice looking loss curves.\n",
        "\n",
        "It looks like our model is performing quite well and perhaps would benefit from a little longer training and potentially some [data augmentation](https://www.learnpytorch.io/04_pytorch_custom_datasets/#6-other-forms-of-transforms-data-augmentation) (to help prevent potential overfitting occurring from longer training)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "ZJ4at0JSQaQN"
      },
      "source": [
        "### 3.5 Saving EffNetB2 feature extractor\n",
        "\n",
        "Now we've got a well-performing trained model, let's save it to file so we can import and use it later.\n",
        "\n",
        "To save our model we can use the [`utils.save_model()`](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/going_modular/going_modular/utils.py) function we created in [05. PyTorch Going Modular section 5](https://www.learnpytorch.io/05_pytorch_going_modular/#5-creating-a-function-to-save-the-model-utilspy).\n",
        "\n",
        "We'll set the `target_dir` to `\"models\"` and the `model_name` to `\"09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth\"` (a little comprehensive but at least we know what's going on)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Oxw1I6TAQaQN",
        "outputId": "b9bb38cc-4a55-4ab1-b84c-c5c3268e3df4"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[INFO] Saving model to: models/09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth\n"
          ]
        }
      ],
      "source": [
        "from going_modular.going_modular import utils\n",
        "\n",
        "# Save the model\n",
        "utils.save_model(model=effnetb2,\n",
        "                 target_dir=\"models\",\n",
        "                 model_name=\"09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yXZARnvRQaQN"
      },
      "source": [
        "### 3.6 Checking the size of EffNetB2 feature extractor\n",
        "\n",
        "Since one of our criteria for deploying a model to power FoodVision Mini is **speed** (~30FPS or better), let's check the size of our model.\n",
        "\n",
        "Why check the size?\n",
        "\n",
        "Well, while not always the case, the size of a model can influence its inference speed.\n",
        "\n",
        "As in, if a model has more parameters, it generally performs more operations and each one of these operations requires some computing power.\n",
        "\n",
        "And because we'd like our model to work on devices with limited computing power (e.g. on a mobile device or in a web browser), generally, the smaller the size the better (as long as it still performs well in terms of accuracy).\n",
        "\n",
        "To check our model's size in bytes, we can use Python's [`pathlib.Path.stat(\"path_to_model\").st_size`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.stat) and then we can convert it (roughly) to megabytes by dividing it by `(1024*1024)`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4Q4B34eeQaQN",
        "outputId": "096a10e1-9214-4f33-d210-5bb89d2b0d29"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Pretrained EffNetB2 feature extractor model size: 29 MB\n"
          ]
        }
      ],
      "source": [
        "from pathlib import Path\n",
        "\n",
        "# Get the model size in bytes then convert to megabytes\n",
        "pretrained_effnetb2_model_size = Path(\"models/09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth\").stat().st_size // (1024*1024) # division converts bytes to megabytes (roughly)\n",
        "print(f\"Pretrained EffNetB2 feature extractor model size: {pretrained_effnetb2_model_size} MB\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DKG9p9GYQaQO"
      },
      "source": [
        "### 3.7 Collecting EffNetB2 feature extractor stats\n",
        "\n",
        "We've got a few statistics about our EffNetB2 feature extractor model such as test loss, test accuracy and model size, how about we collect them all in a dictionary so we can compare them to the upcoming ViT feature extractor.\n",
        "\n",
        "And we'll calculate an extra one for fun, total number of parameters.\n",
        "\n",
        "We can do so by counting the number of elements (or patterns/weights) in `effnetb2.parameters()`. We'll access the number of elements in each parameter using the [`torch.numel()`](https://pytorch.org/docs/stable/generated/torch.numel.html) (short for \"number of elements\") method."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "06KrbQBaQaQO",
        "outputId": "52990df6-2966-410f-9627-ea5fc381a86d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "7705221"
            ]
          },
          "execution_count": 17,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Count number of parameters in EffNetB2\n",
        "effnetb2_total_params = sum(torch.numel(param) for param in effnetb2.parameters())\n",
        "effnetb2_total_params"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yXPDLLPgQaQO"
      },
      "source": [
        "Excellent!\n",
        "\n",
        "Now let's put everything in a dictionary so we can make comparisons later on."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dGMqybtzQaQO",
        "outputId": "34ae8250-3774-476a-86e1-8b780004bdf1"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'test_loss': 0.28128674924373626,\n",
              " 'test_acc': 0.96875,\n",
              " 'number_of_parameters': 7705221,\n",
              " 'model_size (MB)': 29}"
            ]
          },
          "execution_count": 18,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Create a dictionary with EffNetB2 statistics\n",
        "effnetb2_stats = {\"test_loss\": effnetb2_results[\"test_loss\"][-1],\n",
        "                  \"test_acc\": effnetb2_results[\"test_acc\"][-1],\n",
        "                  \"number_of_parameters\": effnetb2_total_params,\n",
        "                  \"model_size (MB)\": pretrained_effnetb2_model_size}\n",
        "effnetb2_stats"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "13EbQ66TQaQO"
      },
      "source": [
        "Epic!\n",
        "\n",
        "Looks like our EffNetB2 model is performing at over 95% accuracy!\n",
        "\n",
        "Criteria number 1: perform at 95%+ accuracy, tick!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wK4l3y8FQaQO"
      },
      "source": [
        "## 4. Creating a ViT feature extractor\n",
        "\n",
        "Time to continue with our FoodVision Mini modelling experiments.\n",
        "\n",
        "This time we're going to create a ViT feature extractor.\n",
        "\n",
        "And we'll do it in much the same way as the EffNetB2 feature extractor except this time with [`torchvision.models.vit_b_16()`](https://pytorch.org/vision/stable/models/generated/torchvision.models.vit_b_16.html#torchvision.models.vit_b_16) instead of `torchvision.models.efficientnet_b2()`.\n",
        "\n",
        "We'll start by creating a function called `create_vit_model()` which will be very similar to `create_effnetb2_model()` except of course returning a ViT feature extractor model and transforms rather than EffNetB2.\n",
        "\n",
        "Another slight difference is that `torchvision.models.vit_b_16()`'s output layer is called `heads` rather than `classifier`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iEcmx8ziQaQO",
        "outputId": "cc7eed73-da8a-4e1f-e6ea-13590ad67e6d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Sequential(\n",
              "  (head): Linear(in_features=768, out_features=1000, bias=True)\n",
              ")"
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Check out ViT heads layer\n",
        "vit = torchvision.models.vit_b_16()\n",
        "vit.heads"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IyAJtcXRQaQP"
      },
      "source": [
        "Knowing this, we've got all the pieces of the puzzle we need."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zNFCusI3QaQP"
      },
      "outputs": [],
      "source": [
        "def create_vit_model(num_classes:int=3,\n",
        "                     seed:int=42):\n",
        "    \"\"\"Creates a ViT-B/16 feature extractor model and transforms.\n",
        "\n",
        "    Args:\n",
        "        num_classes (int, optional): number of target classes. Defaults to 3.\n",
        "        seed (int, optional): random seed value for output layer. Defaults to 42.\n",
        "\n",
        "    Returns:\n",
        "        model (torch.nn.Module): ViT-B/16 feature extractor model.\n",
        "        transforms (torchvision.transforms): ViT-B/16 image transforms.\n",
        "    \"\"\"\n",
        "    # Create ViT_B_16 pretrained weights, transforms and model\n",
        "    weights = torchvision.models.ViT_B_16_Weights.DEFAULT\n",
        "    transforms = weights.transforms()\n",
        "    model = torchvision.models.vit_b_16(weights=weights)\n",
        "\n",
        "    # Freeze all layers in model\n",
        "    for param in model.parameters():\n",
        "        param.requires_grad = False\n",
        "\n",
        "    # Change classifier head to suit our needs (this will be trainable)\n",
        "    torch.manual_seed(seed)\n",
        "    model.heads = nn.Sequential(nn.Linear(in_features=768, # keep this the same as original model\n",
        "                                          out_features=num_classes)) # update to reflect target number of classes\n",
        "\n",
        "    return model, transforms"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rGkVM7EjQaQP"
      },
      "source": [
        "ViT feature extraction model creation function ready!\n",
        "\n",
        "Let's test it out."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cUxknc10QaQQ"
      },
      "outputs": [],
      "source": [
        "# Create ViT model and transforms\n",
        "vit, vit_transforms = create_vit_model(num_classes=3,\n",
        "                                       seed=42)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OEH84dz3QaQQ"
      },
      "source": [
        "No errors, lovely to see!\n",
        "\n",
        "Now let's get a nice-looking summary of our ViT model using `torchinfo.summary()`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Fz2qV20GQaQQ"
      },
      "outputs": [],
      "source": [
        "from torchinfo import summary\n",
        "\n",
        "# # Print ViT feature extractor model summary (uncomment for full output)\n",
        "# summary(vit,\n",
        "#         input_size=(1, 3, 224, 224),\n",
        "#         col_names=[\"input_size\", \"output_size\", \"num_params\", \"trainable\"],\n",
        "#         col_width=20,\n",
        "#         row_settings=[\"var_names\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nFmUvd2VQaQQ"
      },
      "source": [
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-vit-feature-extractor-3-classes.png\" width=900 alt=\"vit feature extractor with 3 output classes\"/>\n",
        "\n",
        "Just like our EffNetB2 feature extractor model, our ViT model's base layers are frozen and the output layer is customized to our needs!\n",
        "\n",
        "Do you notice the big difference though?\n",
        "\n",
        "Our ViT model has *far* more parameters than our EffNetB2 model. Perhaps this will come into play when we compare our models across speed and performance later on."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4kBH6T8GQaQQ"
      },
      "source": [
        "### 4.1 Create DataLoaders for ViT\n",
        "\n",
        "We've got our ViT model ready, now let's create some `DataLoader`s for it.\n",
        "\n",
        "We'll do this in the same way we did for EffNetB2 except we'll use `vit_transforms` to transform our images into the same format the ViT model was trained on."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "H3nTjWhiQaQQ"
      },
      "outputs": [],
      "source": [
        "# Setup ViT DataLoaders\n",
        "from going_modular.going_modular import data_setup\n",
        "train_dataloader_vit, test_dataloader_vit, class_names = data_setup.create_dataloaders(train_dir=train_dir,\n",
        "                                                                                       test_dir=test_dir,\n",
        "                                                                                       transform=vit_transforms,\n",
        "                                                                                       batch_size=32)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I7reu4KcQaQQ"
      },
      "source": [
        "### 4.2 Training ViT feature extractor\n",
        "\n",
        "You know what time it is...\n",
        "\n",
        "...it's traininggggggg time (sung in the same tune as the song [Closing Time](https://youtu.be/xGytDsqkQY8)).\n",
        "\n",
        "Let's train our ViT feature extractor model for 10 epochs using our `engine.train()` function with `torch.optim.Adam()` and a learning rate of `1e-3` as our optimizer and `torch.nn.CrossEntropyLoss()` as our loss function.\n",
        "\n",
        "We'll use our `set_seeds()` function before training to try and make our results as reproducible as possible."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-LUQFFIGQaQR",
        "outputId": "fce3042c-dc09-482c-ce04-73bd36013a7f",
        "colab": {
          "referenced_widgets": [
            "ded441208c9540c28035eb5ea07b4b39"
          ]
        }
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ded441208c9540c28035eb5ea07b4b39",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/10 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch: 1 | train_loss: 0.7023 | train_acc: 0.7500 | test_loss: 0.2714 | test_acc: 0.9290\n",
            "Epoch: 2 | train_loss: 0.2531 | train_acc: 0.9104 | test_loss: 0.1669 | test_acc: 0.9602\n",
            "Epoch: 3 | train_loss: 0.1766 | train_acc: 0.9542 | test_loss: 0.1270 | test_acc: 0.9693\n",
            "Epoch: 4 | train_loss: 0.1277 | train_acc: 0.9625 | test_loss: 0.1072 | test_acc: 0.9722\n",
            "Epoch: 5 | train_loss: 0.1163 | train_acc: 0.9646 | test_loss: 0.0950 | test_acc: 0.9784\n",
            "Epoch: 6 | train_loss: 0.1270 | train_acc: 0.9375 | test_loss: 0.0830 | test_acc: 0.9722\n",
            "Epoch: 7 | train_loss: 0.0899 | train_acc: 0.9771 | test_loss: 0.0844 | test_acc: 0.9784\n",
            "Epoch: 8 | train_loss: 0.0928 | train_acc: 0.9812 | test_loss: 0.0759 | test_acc: 0.9722\n",
            "Epoch: 9 | train_loss: 0.0933 | train_acc: 0.9792 | test_loss: 0.0729 | test_acc: 0.9784\n",
            "Epoch: 10 | train_loss: 0.0662 | train_acc: 0.9833 | test_loss: 0.0642 | test_acc: 0.9847\n"
          ]
        }
      ],
      "source": [
        "from going_modular.going_modular import engine\n",
        "\n",
        "# Setup optimizer\n",
        "optimizer = torch.optim.Adam(params=vit.parameters(),\n",
        "                             lr=1e-3)\n",
        "# Setup loss function\n",
        "loss_fn = torch.nn.CrossEntropyLoss()\n",
        "\n",
        "# Train ViT model with seeds set for reproducibility\n",
        "set_seeds()\n",
        "vit_results = engine.train(model=vit,\n",
        "                           train_dataloader=train_dataloader_vit,\n",
        "                           test_dataloader=test_dataloader_vit,\n",
        "                           epochs=10,\n",
        "                           optimizer=optimizer,\n",
        "                           loss_fn=loss_fn,\n",
        "                           device=device)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rQYborEeQaQS"
      },
      "source": [
        "### 4.3 Inspecting ViT loss curves\n",
        "\n",
        "Alright, alright, alright, ViT model trained, let's get visual and see some loss curves.\n",
        "\n",
        "> **Note:** Don't forget you can see what an ideal set of loss curves should look like in [04. PyTorch Custom Datasets section 8](https://www.learnpytorch.io/04_pytorch_custom_datasets/#8-what-should-an-ideal-loss-curve-look-like)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "y441aB1PQaQS",
        "outputId": "dea227c2-682f-42ef-f9d1-33b8ec3af260"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2oAAAG5CAYAAAD/HsejAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAB9VElEQVR4nO3deXhU5d3/8fc92ZMJkB1IWJOwg6ABxF1RFLXivu8Lta1WfaoVW1trt8df61Nt61bqrnWr+w7uG4oERWU1CWvYEsKWyZ7M/fvjTBYwQBKSnJnJ53VdcyVzzpmZb0bMyWfu+3xvY61FREREREREgofH7QJERERERERkVwpqIiIiIiIiQUZBTUREREREJMgoqImIiIiIiAQZBTUREREREZEgo6AmIiIiIiISZBTUREREREREgoyCmsh+MMasNsYc63YdIiIiXc0Y86ExZpsxJsbtWkR6AgU1EREREdkrY8xg4HDAAqd04+tGdtdriQQbBTWRTmaMiTHG3G2M2RC43d346aMxJtUY87oxZrsxZqsx5hNjjCew72ZjzHpjTLkxZoUxZqq7P4mIiEiTi4EvgEeBSxo3GmMGGGNeNMaUGmPKjDH3tNh3lTFmWeC8ttQYc2BguzXG5LQ47lFjzB8D3x9ljCkOnBM3AY8YY5IC587SwIje68aYrBaPTzbGPBI4524zxrwc2L7YGPOjFsdFGWO2GGPGd9F7JNKpFNREOt+vgYOB8cABwCTg1sC+XwDFQBqQAfwKsMaY4cA1wERrbSJwPLC6W6sWERHZs4uB/wRuxxtjMowxEcDrwBpgMJAJPANgjDkL+F3gcb1wRuHK2vhafYFkYBAwE+fv1UcC9wcCVcA9LY5/AogHRgPpwF2B7Y8DF7Y47kRgo7V2URvrEHGVhpNFOt8FwLXW2hIAY8ztwL+A3wB1QD9gkLW2EPgkcEwDEAOMMsaUWmtXu1G4iIjI7owxh+GEpOestVuMMUXA+TgjbP2Bm6y19YHDPw18vRL4i7V2QeB+YTte0g/cZq2tCdyvAl5oUc+fgA8C3/cDpgMp1tptgUM+Cnx9EviNMaaXtXYncBFOqBMJCRpRE+l8/XE+XWy0JrAN4K84J6u5xpiVxphZAIHQdj3Op48lxphnjDH9ERERcd8lwFxr7ZbA/acC2wYAa1qEtJYGAEUdfL1Sa2114x1jTLwx5l/GmDXGmJ3Ax0CfwIjeAGBri5DWxFq7AfgMOMMY0wcn0P2ngzWJdDsFNZHOtwHnk8dGAwPbsNaWW2t/Ya0dCvwI+J/Ga9GstU9Zaxs/tbTA/+veskVERHZljIkDzgaONMZsClw3dgPO1P7NwMA9NPxYB2Tv4WkrcaYqNuq723672/1fAMOBydbaXsARjeUFXic5EMRa8xjO9MezgM+ttev3cJxI0FFQE9l/UcaY2MYb8DRwqzEmzRiTCvwWZ/oFxpiTjTE5xhgD7AQagAZjzHBjzDGBpiPVONM8Gtz5cURERJqcinM+GoVz7fV4YCTO1P1TgY3AHcaYhMB58NDA4x4EbjTGHGQcOcaYxg8xFwHnG2MijDEnAEfuo4ZEnPPidmNMMnBb4w5r7UbgLeC+QNORKGPMES0e+zJwIHAdzjVrIiFDQU1k/72JcwJpvMUC+cC3wHfAV8AfA8fmAu8CPuBz4D5r7Yc416fdAWwBNuFcDP2rbvsJREREWncJ8Ii1dq21dlPjDaeZx3k4s0NygLU4zbLOAbDW/hf4E840yXKcwJQceM7rAo/bjnNd98v7qOFuIA7nHPkF8PZu+y/CuQZ8OVCCcykBgToar28bArzY9h9bxH3G2t1Hl0VEREREwoMx5rfAMGvthfs8WCSIqOujiIiIiISlwFTJK3BG3URCiqY+ioiIiEjYMcZchdNs5C1r7cdu1yPSXpr6KCIiIiIiEmQ0oiYiIiIiIhJkXLtGLTU11Q4ePNitlxcRkW60cOHCLdbaNLfrCBU6R4qI9Ax7Oz+6FtQGDx5Mfn6+Wy8vIiLdyBizxu0aQonOkSIiPcPezo+a+igiIiIiIhJkFNRERERERESCjIKaiIiIiIhIkNGC1yLS49XV1VFcXEx1dbXbpYS82NhYsrKyiIqKcrsUERGRkKagJiI9XnFxMYmJiQwePBhjjNvlhCxrLWVlZRQXFzNkyBC3yxEREQlpmvooIj1edXU1KSkpCmn7yRhDSkqKRiZFREQ6gYKaiAgopHUSvY8iIiKdQ0FNREREREQkyCioiYiIiIiIBBkFNRERl23fvp377ruv3Y878cQT2b59e7sfd+mll/L888+3+3EiIiLSfRTURERctqeg1tDQsNfHvfnmm/Tp06eLqhIRERE3qT2/iEgLt7+2hKUbdnbqc47q34vbfjR6j/tnzZpFUVER48ePJyoqCq/XS79+/Vi0aBFLly7l1FNPZd26dVRXV3Pdddcxc+ZMAAYPHkx+fj4+n4/p06dz2GGHMW/ePDIzM3nllVeIi4vbZ23vvfceN954I/X19UycOJH777+fmJgYZs2axauvvkpkZCTTpk3jzjvv5L///S+33347ERER9O7dm48//rjT3iMRERHZlYKaiIjL7rjjDhYvXsyiRYv48MMPOemkk1i8eHHTWmQPP/wwycnJVFVVMXHiRM444wxSUlJ2eY6CggKefvpp/v3vf3P22WfzwgsvcOGFF+71daurq7n00kt57733GDZsGBdffDH3338/F198MS+99BLLly/HGNM0vfL3v/89c+bMITMzs0NTLkVERKTtFNRERFrY28hXd5k0adIuC0b/4x//4KWXXgJg3bp1FBQU/CCoDRkyhPHjxwNw0EEHsXr16n2+zooVKxgyZAjDhg0D4JJLLuHee+/lmmuuITY2liuvvJKTTjqJk08+GYBDDz2USy+9lLPPPpvTTz+9E35SERER2ZM2XaNmjDnBGLPCGFNojJnVyv6bjDGLArfFxpgGY0xy55fbrLqugSUbdlBVu/drOEREQk1CQkLT9x9++CHvvvsun3/+Od988w0TJkxodUHpmJiYpu8jIiKor6/f5+tYa1vdHhkZyZdffskZZ5zByy+/zAknnADAAw88wB//+EfWrVvH+PHjKSsra++PJiIiEh52boSyoi59iX0GNWNMBHAvMB0YBZxnjBnV8hhr7V+tteOtteOBW4CPrLVbu6DeJl+sLOOkf3zKd+t3dOXLiIh0ucTERMrLy1vdt2PHDpKSkoiPj2f58uV88cUXnfa6I0aMYPXq1RQWFgLwxBNPcOSRR+Lz+dixYwcnnngid999N4sWLQKgqKiIyZMn8/vf/57U1FTWrVvXabWIiIgELWudUPbVE/DyT+HvB8DfRsD7f+jSl23L1MdJQKG1diWAMeYZYAawdA/Hnwc83Tnl7VlOuheAwhIfk4Z06eCdiEiXSklJ4dBDD2XMmDHExcWRkZHRtO+EE07ggQceYNy4cQwfPpyDDz640143NjaWRx55hLPOOqupmcjVV1/N1q1bmTFjBtXV1VhrueuuuwC46aabKCgowFrL1KlTOeCAAzqtFhERkaDh90PJEljzOaz5DNZ+Dr7Nzr64ZPwDp7Bl5CVUZx3KwC4sw+xp6kvTAcacCZxgrb0ycP8iYLK19ppWjo0HioGc1kbUjDEzgZkAAwcOPGjNmjUdLtzvt4y+bQ7nTx7Ib04ete8HiIjswbJlyxg5cqTbZYSN1t5PY8xCa22eSyWFnLy8PJufn+92GSIiPUN9LWxc5ISyNZ/Dui+g2pm1V5vQn019DmRp9Gjm1Q3j020prNlWRYPfctK4ftx7/oH79dJ7Oz+2ZUTNtLJtT+nuR8Bne5r2aK2dDcwG5yTUhtfeI4/HMDQtgaJS3/48jYiIiIiI9CS1FbDuS1j7OXbNPCjOx9RXAVAaM4jvIg/h44hhvFMxlPXVaVAGURGGwSkJDOvr5cRx/clJ9zK6f68uLbMtQa0YGNDifhawYQ/Hnks3THtslJ3m5au127rr5UREQsrPfvYzPvvss122XXfddVx22WUuVSQiIsFoe2UtlbUNJCdEExsV4XY5na9yK/41n+P7/mPsmnkkbluKx9bTgIcVDOLz+qP40j+cfP9wqv3JZKd7yRno5fx0LzmB28DkeKIi2tSHsdO0JagtAHKNMUOA9Thh7PzdDzLG9AaOBPa+cE8nyk7z8tq3G6iqbSAuOgz/UYmI7Id7773X7RJERCRIWGvZuKOawhKfcyt1vhaV+CirqG06Li4qguSE6F1uSfHRpHidr7vv6xMXhcfT2gQ8d9TUN1C8uoidKz4iYt0XpG5dSP/aVXiAGBvJIpvDAv9JLI8eS3naBDL7ZpCT5uWCdC+3pXvp1zsWY4Lj59lnULPW1htjrgHmABHAw9baJcaYqwP7Hwgcehow11pb0WXV7iYn3Yu1sHKLj9H9e3fXy4qIiIiIBKW6Bj9ryiqdEFbqBLHCwNeKFsta9YmPIifNy7EjM8hJ95IYG8nWylq2VdRSVuF83VpRy8otPrb6and5bEseA30aA1x8NEkJUSQnxJC8+9f4aJK9zjGdMcBSXl3nBM7N5Wxdt5yYDV/Qd/vXjKpbTLYpcY6xcSyNGMG8PpdRkTGJuMETGdovhQvSvCQlRO93DV2tTQteW2vfBN7cbdsDu91/FHi0swpri+x0Z62hotIKBTURERER6TEqa+tZWVrRPEIWCGRryiqoa2huBdGvdyw56V7OyhvQNI0vJ91LSkJ0u0aOqusa2FbphLeWt6ZgV1lLma+WVVsqWLhmO9sqa2nwt96SIjbKQ0pCTHOoi/9huGscxYuLjmRNWYUTOEt8FJXsgM1Lya76jkmeZRzlWUGacRp/lHt6szHlQL7rfzlxOYeROWISk2NjmLx/b7Vr2hTUgtXglAQ8BopK1FBERERERMLP1oraXcJYUWDK4vrtVU3HRHgMg1LiyU7zctwoZypfTrqX7HQv3pjO+XM/NiqCfr3j6Nc7rk3H+/2W8up6yipqmkLctsrm0bqWo3ar9jJqF00dY81KJnlWcEzkCm7yfI/XVkAUVMVnUp81lYbcw4kYfBiJqbkkBsm0xc4Q0kEtNiqCAcnxFKrzo4iIiEjXaqiHTd86a0oFOuXhTYdBhzi3gYeAN83tKoNLfQ2s/6p5La6N30DyUBg4xXnPBkyGuD5Ya9nQ8vqxkuYpi1tbXD8WG+UhO81L3uAkzk1rHiEblJJAdGT3NrrYF4/H0Ds+it7xUW1+THVdA9u3b6N61ed41n5B/Mb59Nn2LZH+GgBs6nDMoLNg0KEwcApxfQbs4xlDW0gHNXAaimhETURC2fbt23nqqaf46U9/2u7H3n333cycOZP4+Pg9HjN48GDy8/NJTU3dnzJFpKepq4b1C51Qtnae0868NvA3V9JgGHIE+DbBwsdgfuCKmJSc5tA26BDoMxDCaIRjn2rKYd38wELJ85z3r8EJGaSNxJ99LDWbvydm3r14PrsbP4bVEYP5vH4Yn9UNZ4F/BKX0abp+bNqojKaRsZw0L5l94oKqcUenqChzQuzaz4ldM4++G78B2wDGA33HwaQrAv+mpmASetZ5LOSDWk66l08Lt9Dgt0SE2z9cEekRtm/fzn333dfhoHbhhRfuNaiJiLRJ9Q4njK2Z59w2fAUNgdGc9NFwwLnNI0G9+jc/rr7WGSlqHDVa+gp89bizr1dm82MGHQKpw8ETXCM/+6ViSyDIBoLZpm/B+sFEQL8DYNJVTSGj0BfN2f/6nK0VtcRSwwRPIcfEFTElcgVn2Y+4wMwBoCFpKJ5Bh2Aa37OkweEVdncUB4Js4N9L6XJne0QMZOXBYTcERhsnQUyiu7W6LOSDWnZaArX1ftZvq2Jgiv5QEZH99NYs2PRd5z5n37Ew/Y497p41axZFRUWMHz+e4447jvT0dJ577jlqamo47bTTuP3226moqODss8+muLiYhoYGfvOb37B582Y2bNjA0UcfTWpqKh988ME+S/nb3/7Gww8/DMCVV17J9ddf3+pzn3POOcyaNYtXX32VyMhIpk2bxp133tlpb4mIBAFfya4hY/NiJ2R4IqHfeJj8Y2eK2YDJEJ+85+eJjIYBE50b14PfDyVLA8/7Gaz+FBY/7xwblxQYbZvifO03DiLaPjXOddvX7hoytnzvbI+MhayJcPiNzs+WNQlivE0Pa/BbfvnoPBr8lv876wByM7xkp51CQuP1Yw11sPFbWPMZEWs/h+Wvw6InnX2J/XYNu2kjQyfsWgtlhc77teZzZ2R2+1pnX3QiDDwYxp3t/FvIPBAiY9ytN8iEfFDLSXf+Jygq9SmoiUhIuuOOO1i8eDGLFi1i7ty5PP/883z55ZdYaznllFP4+OOPKS0tpX///rzxxhsA7Nixg969e/O3v/2NDz74oE3TGhcuXMgjjzzC/PnzsdYyefJkjjzySFauXPmD5966dSsvvfQSy5cvxxjD9u3bu/ItEJGuZi1sX9P8x/Kaec4f0ACRcU7IOuKXgZAxEaITOv5aHg/0HePcJl3lvPa2VYGRusDrr3B+3xCV4Lx241TJrDyIaluzii5nLZSuCLxfztQ8dqxz9sX0hoGTYfz5Tu39x+81ZDw6bzVfrd3OXeccwGkTsn54QEQUZB3k3A79uRN2S5c3/7da8zksedE5NrZPILhNaX7tYAm7/gbnw87GkL72C6godfYlpDl1H/xT5791xhjwaB3kvQn5oDY01QlqhSU+jh6R7nI1IhLy9jLy1R3mzp3L3LlzmTBhAgA+n4+CggIOP/xwbrzxRm6++WZOPvlkDj/88HY/96effsppp51GQoLzB9jpp5/OJ598wgknnPCD566vryc2NpYrr7ySk046iZNPPrlTf04R6WJ+P2xZ0WIk43PYud7ZF9vb+YN5wkXOiFm/A5xRsa5ijNNAI3koTLjQ2Va+adfRvA//F7DgiYL+E5pHjwLNNrrF7s1S1n4OlWXOvoRA05RDrnXeu4zRbQ4Zq7dU8Nc5y5k6Ip1Tx2e2rRaPBzJGObeJV7YI2vOaa/v+LefYqHgn4DaF3YkQ3U2DF3XVzhTZxprWzofacmdfn4GQPbX5v2VKTnhN4ewGIR/UkhKiSUmIpkidH0UkDFhrueWWW/jxj3/8g30LFy7kzTff5JZbbmHatGn89re/bfdzt2bYsGGtPveXX37Je++9xzPPPMM999zD+++/36GfSUS6QePUubUt/pCv2ubs8/Zt0ZlxCqSPcn/qXGJfGHO6cwOn1rXzm0evPr8HPrsbMM7Iy6ApzU1KEjM6p4a6qkCzlM9bb5aSe3zz+5Y8tEMhw++33PzCt0RFePjTaWPbtW7ZLoxxakoa7IziAZRvbhEq58FH/w8n7Aamrjb9Nz/YmXLaGZqapQT+O+3WLIVxZzVPbe3dysihtEvIBzWA7HQvher8KCIhKjExkfJy5xPI448/nt/85jdccMEFeL1e1q9fT1RUFPX19SQnJ3PhhRfi9Xp59NFHd3lsW6Y+HnHEEVx66aXMmjULay0vvfQSTzzxBBs2bPjBc/t8PiorKznxxBM5+OCDycnJ6cq3QETaq7YS1ue3CBkLoK7C2Zc8FIaf1BxukoYE/0hGXBIMP8G5QYufLxA8v34Svpzt7Ese2qKz5JS2/3xNzVICo4y7NEsZtedmKfvhP/PXMH/VVv5yxjj69o7tlOdskpgBo091bgBV252frzGsf3E/zPsHYJyfr2XY7dWvba+xS7OUz5xpja01SxlwMCSkdO7PJ2ES1NK8vL14o9tliIh0SEpKCoceeihjxoxh+vTpnH/++UyZMgUAr9fLk08+SWFhITfddBMej4eoqCjuv/9+AGbOnMn06dPp16/fPpuJHHjggVx66aVMmjQJcJqJTJgwgTlz5vzgucvLy5kxYwbV1dVYa7nrrru69k0Qkb2r2t5iJGMebPga/HU4I06jYcIFzSEjsa/b1e6/6Hin/f+QI5z7DXWBzpKB0LDsdSe8wZ6bbXRWs5QOWre1kv99azmH56ZyVl43jC7F9YFh05wbtBgxDPybWfQ0LHjQ2Zc0uGktsl1GDPfWLCUzb4/NUqRrmD1NhelqeXl5Nj8/v1Oe66FPV/GH15fy1W+OIzmhC+dYi0hYWrZsGSNHjnS7jLDR2vtpjFlorc1zqaSQ05nnSAkRDXVQvhF2bnSuJdu5IXBb7zT92LyEpmlt/Q9sbiQxcHLnTWsLJY3NNhoDxZp5zvsHTrON+GTYutK5HxnnXMM16NDOaZbSBtZaLnroS75eu425/3MkmX2CoEFKQz1s+qZ5jbe1n0PVVmefN8O5PnBnsXM/ppczZXLgFOd920ezFOm4vZ0fw2REzfmfrbDEx6Qhnf+JiIiIiEiH1VVD+YZdw9fu3/tKgN0+PI+Kd6bg9RkII3/kjHxk5nVfo4hg1rLZRlNnydXNoa1yKxx4ifOe9Rvftc1SWvHsgnV8WriFP546JjhCGkBEJGQe5NwOuSbQcOb75rDbUAeDft7uZinSdcIkqDW36FdQE5GeavLkydTU1Oyy7YknnmDs2LEuVSTSA9SUtz4K1vL7xlGLlmJ6OyGsV3+nWUavzMD9TOf6oV79nZGhYL+2LFgYA8lDnFtjsw2XbNxRxZ/eWMbBQ5M5f9JAV2vZK48H0kc4t4lXuF2NtCIsglpmnzhiozxqKCIiHWat7Xg3riAxf/58t0vYY2dJkZBjLVRvbyV8tQxhG6Bm5w8fG5/iBK3emc4aYb36Q2L/XYNYTGK3/0jS9ay1/OrF76j3W/5yxgF4PKF9XhF3hUVQ83gMQ1O9atEvIh0SGxtLWVkZKSkpIR/W3GStpaysjNjYTu5sJtLVtq+FJS/B5qVOECvf6ISwusrdDjTOtTy9+jtrQg05okX4CgSxxH7Bs2CzdLsXv1rPBytKue1HoxiYoimqsn/CIqgB5KR7+XrdNrfLEJEQlJWVRXFxMaWlpW6XEvJiY2PJytLaORICfKVOOFv8vNNNEaBXVvNUxNzjm8NXYxBL7AsRUe7WLUGrZGc1t7+2hLxBSVwyZbDb5UgYCJuglp3m5bVvN1Bd10BslC5+FJG2i4qKYsiQIW6XISJdrXqH09Z98fOw8iOwDc76Usf8Bsac4VzfJNIB1lpufXkxNfV+/nLmOE15lE4RPkEtPQFrYWVpBaP693K7HBEREQkGdVXw/RwnnH0/FxpqnC6Kh10PY850ugaK7KfXv93I3KWbuWX6CIamaX0x6RxhE9Ry0p3/KQpLfQpqIiIiPVlDnTNitvh5ZwStthwS0iHvMiecZeWpm6J0mjJfDbe9uoQDBvThysOHul2OhJGwCWqDUxLwGChS50cREZGex+93rjVb/Lxz7VllmdMCf/QMJ5wNOULrQkmX+O2rS/BV1/PXM8cRoSmP0onCJqjFRkUwIDlenR9FRER6Cmth03fw3X9h8Yuwsxgi42D4CTD2LMg5FiJj3K5SwtjbizfyxrcbuXHaMIZlaMkF6VxhE9TAaSiitdRERETCXFkRfPe8M3q25XvwREL2MXDsbTB8utYo60QNfsu2ylq2VdRSVrHr1/Kaek6bkMnIfj3zkpNtFbXc+vISRvfvxY+PzHa7HAlDYRbUEviscAsNfquhZxERkXCyc4Mzarb4edjwNWBg0KFw8E9g5AxISHG7wqBnraWytoGtFbXOrbKWrb5atlXWNm9rua+ilh1VdexpHfsIj+HZBet4+qqDe2R/gD+8vpTtlbU8fvkkoiI8bpcjYSisglpOupeaej/rt1VpkUEREZFQV7kVlr4Ci1+A1Z8CFvqNh2l/hNGnQ+9Mtyt01d5Gu8oqWg9gNfX+Vp8r0mNISogmOT6a5IRoRvbtRVJCFMkJMSTHR5HsjWnal5wQTVJCFCU7azj7X59z4UPzeXbmweT2oKl/7y/fzItfr+fnU3N7ZEiV7hFWQS070A61qNSnoCYiIhKKanyw4i3nurOi98BfDym5cNQtzlpnqTluV9htauv9zCvawoLVWynztW+0KzEm0gleCdFk9IplZL9eTsCKjyYlIbppX+OtV2wkpp2dMAckx/PUVQdz9r8+54IH5/Pcj6cwODWhE37y4Lajqo5bXvyO4RmJXHN0z/n3KN0vbIPa0SPSXa5GRERE2qS+Bgrfda47W/EW1FdBr0w4+Kcw9kzoO67HtNOvrK3noxWlvL1kE+8vK6G8pr5ptCslELRG9uu12+hW874UbzR94qOIieyeDpdDUhN46srJnDP7Cy54cD7P/vhgspLC+8PyP7+xjC2+Wv59cR7RkZryKF0nrIJa4y8qNRQREREJcv4GWP2JE86WvQrVOyA+Bcaf74SzAQeDp2f8Ebyjso53l21mzpJNfPR9KTX1fpLio5g+ti8njOnLIdmpxEYF79ICuRmJPHHFJM6b/QXn/9sZWevbO9btsrrEx9+X8mz+On5yVDbjsvq4XY6EubAKauCMqqlFv4iISJBavxC+/S8seRF8myHaCyNOdsLZ0KMgIsrtCrtFSXk1c5c44ezzojLq/Za+vWI5b9JAjh/dl4mDk4gMoQYVo/v35vErJnPhg/O54MEvePbHU0j1htfSCL6aem558Tuy0xK4bmqu2+VIDxB+QS3dy9uLN7pdhoiIiOzuiwfg7ZshIhpypznhbNgJEBXndmXdYt3WSuYs2cTbizexcO02rIXBKfFcefhQThjTl3GZvfGEcNfq8QP68PClE7nk4S+58MH5PDPzYPrER7tdVqe5461lbNhRxfNXHxLUI5wSPsIvqKUlsK2yjq0VtSQnhM8vBxERkZBW8C7MuQWGnwSn3Q+xvd2uqMtZayko8fH2YiecLd24E4BR/Xpx/dRhnDCmL8MyvO1u4hHMJg1J5t8X53H5Ywu46KEv+c9Vk+kVG/qjpPOKtvDkF2u58rAhHDQoye1ypIcIu6CWk97cUCQ5IdnlakRERITSFfD8ZZAxGk6fDTFetyvqMtZavinewZwlm5izeBMrt1QAcNCgJH594kiOH9037DtTH5abygMXHsiPn1jIZY8s4PHLJ5EQE7p/clbW1jPrhe8YnBLPL6YNd7sc6UFC9/+aPWjs/FhY4mPiYAU1ERERV1VuhafOhshYOO+ZsAxp9Q1+Fqze5oSzJZvYuKOaSI9hSnYKlx82hGmjMkjvFZ7NNfbkmBEZ/OPcCVzz9Ndc+Vg+j1w2MWSnC/51zgrWbq3k2ZkHExcdmj+DhKawC2qZfeKIjfJQpM6PIiIi7qqvhWcvgp0b4dI3oHeW2xV1mpr6BuYVlvH24k28s2wzWytqiYn0cMSwNG6cNpypI9PD6vqsjpg+th//V+/nhucW8eMnFjL74oO6bdmAzpK/eiuPzlvNJVMGMXloitvlSA8TdkHN4zEMTfVSqM6PIiIi7rEW3rwR1nwKpz8IAya6XdF+q6ip58PAGmcfLC/BV1NPYkwkx4xM5/jRfTlyWFpIT/HrCqdOyKSmvoGbX/iOa5/6mnsvOJCoEOlmWV3XwC+f/5bMPnH88oQRbpcjPVBY/jbJTveyaN02t8sQERHpub64H756DA6/Ecad5XY1HbatojawxtlmPi4opbbeT3JCNCeP68fxY/pySHZKyI0SdbdzJg6kus7Pba8u4X+e+4a7zxlPRAh0t7zrne9ZuaWC/1w5WQFcXBGW/+py0ry8/u0GqusaQnY+tIiISMgqeAfm/tpZH+3oX7tdTbtt3lnN3CWbeHvJJr5YuZUGv6V/71gumNy4xllySASNYHLJIYOprmvgf99aTkykh7+cMS6olyJYtG47//5kJedNGsChOalulyM9VFgGtez0BKyFlaUVjOrfy+1yREREeo6S5fD85c0dHj2hMc1t885qXlm0nrcXb+KrtdsBGJqWwI+PcNY4G5vZO6za6Lvhx0dmU1XXwN3vFhAb5eEPM8YE5XtaU9/ATf/9hoxesdxy4ki3y5EeLDyDWlpzi34FNRERkW5SUQZPn9Pc4TE6we2K2mRndR0/+uenlJTXMCazFzdOc9Y4y0lPdLu0sHPd1Fyq6/w88FERsZER/PqkkUEX1u55v5CCEh+PXDYxLNaAk9AVlkFtSGoCxjgt+kVERKQb1NfCc4EOj5e9GVIdHu9+p4BSXw3/vXqKlvbpYsYYbj5hONV1DTz46SrioiOCam2yxet3cN+HRZxxYBZHD093uxzp4cIyqMVGRTAgKZ4idX4UERHpetbCG/8Daz6DMx6CrDy3K2qzFZvKeezz1Zw7caBCWjcxxnDbj0ZRU9/AP98vJDYqgp8dneN2WdQ1+Lnp+W9JTojmtyePcrsckfAMagA56V6KSivcLkNERCT8fXEffP0EHHETjD3T7WrazFrLb19ZTGJsJL88PnhGdXoCYwx/PHUs1XV+/jpnBTGRHq48fKirNd3/YRHLNu5k9kUH0TteUx7FfaFxhW8HZKclsLLUR4Pful2KiIiEOGPMCcaYFcaYQmPMrFb2JxljXjLGfGuM+dIYM6bFvtXGmO+MMYuMMfndW3k3+H4uzL0VRp4CR/3K7Wra5bVvNzJ/1VZunDacpISevTi1GyI8hr+eOY4Tx/blj28s48kv1rhWy/JNO/nn+wWcckB/po3u61odIi2F7YhadpqXmno/G7ZXMSA53u1yREQkRBljIoB7geOAYmCBMeZVa+3SFof9ClhkrT3NGDMicPzUFvuPttZu6baiu0vJskCHxzFw2gMh0+ERwFdTz5/eWMqYzF6cN2mg2+X0WJERHu4+ZwI1dQu59eXFxEZFcOZB3Xt9Y32Dn18+/y29YqP43Smju/W1RfYmdH6jtlNOutP5UQ1FRERkP00CCq21K621tcAzwIzdjhkFvAdgrV0ODDbGZHRvmd2sogyeOgei40Oqw2Ojf75fwOadNdx+yhitieay6EgP915wIIfnpvLL57/htW82dOvr//uTVXxbvIPfzxhDskZWJYiEbVBr2aJfRERkP2QC61rcLw5sa+kb4HQAY8wkYBDQOCxggbnGmIXGmJl7ehFjzExjTL4xJr+0tLTTiu8S9bXw7IXg2wznPg29d387glthiY+HPlnFWQdlcdCgJLfLEZxGcLMvyiNvcDLXP7uIuUs2dcvrFpb4uOvd75k+pi8njevXLa8p0lZtCmr7mpsfOOaowPz7JcaYjzq3zPZLSogmJSFaQU1ERPZXa8Mtu18AfQeQZIxZBFwLfA3UB/Ydaq09EJgO/MwYc0RrL2KtnW2tzbPW5qWlpXVO5V3BWnjjBlg7D2bcC1kHuV1Ru1hr+d2rS4iLjuDm6SPcLkdaiIuO4OFLJzI2szfXPPU1H64o6dLXa/Bbfvn8N8RHR/D7GWP2/QCRbrbPoNZibv50nKkd5xljRu12TB/gPuAUa+1o4KzOL7X9stO8mvooIiL7qxgY0OJ+FrDL3Cxr7U5r7WXW2vHAxUAasCqwb0PgawnwEs5UytD1+b3w9ZNwxC9DqsNjo7cXb+LTwi384rhhpHpj3C5HduONieSxyyeRm+Hlx08sZF5R113a+chnq/hq7XZ+96PRpCXq34IEn7aMqLVlbv75wIvW2rXQdDJyXXZ6glr0i4jI/loA5BpjhhhjooFzgVdbHmCM6RPYB3Al8LG1dqcxJsEYkxg4JgGYBizuxto71/dznA6Po2bAUbe4XU27VdU28IfXlzKibyIXHjzI7XJkD3rHRfHEFZMZlBLPlY/ls3DN1k5/jdVbKrhz7gqOHZnOjPH9O/35RTpDW4JaW+bmD8OZ8vFhYA7+xa09UXfPv89O87K1opatFbVd/loiIhKerLX1wDXAHGAZ8Jy1dokx5mpjzNWBw0YCS4wxy3FmoFwX2J4BfGqM+Qb4EnjDWvt29/4EnWTzUnj+Cuh3AJwaWh0eG937QSEbdlTz+xljiIwIvfp7kuSEaJ68cjIZvWK59OEFfFu8vdOe2++3/PKFb4mK8PDHU8dijJrJSHBqS3v+tszNjwQOwmlFHAd8boz5wlr7/S4PsnY2MBsgLy+vyxc4y05vbiiSnJDc1S8nIiJhylr7JvDmbtseaPH950BuK49bCRzQ5QV2tYot8PQ5TmfH8552Oj2GmFVbKpj98UpOm5DJpCH6myAUpCfG8p8rJ3P2vz7nooe+5JmZBzOyX6/9ft7/zF/Dl6u28pczx9G3d2wnVCrSNdrycdI+5+YHjnnbWlsRWCfmY4LgxJTT2PlR16mJiIh0TH1NoMNjCZz3FPQKvWli1lpuf20J0ZEeblEDkZDSv08cT191MHFREVz00Pz97j2wbmsl//vWco4YlsZZ3bxem0h7tSWo7XNuPvAKcLgxJtIYEw9Mxpke4qrMPnHERHrUUERERKQjrIXX/wfWfg6n3geZodXhsdG7y0r4cEUp1x+bS3ovjaCEmgHJ8Tx11WTAcMGDX7CmrGP9B6y13PLid3iM4X9P15RHCX77DGptmZtvrV0GvA18izMH/0FrresXS3s8hqFpXrXoFxER6Yh5/4RFT8KRs2DMGW5X0yHVdQ38/vUl5KZ7ueSQwW6XIx00NM3Lf66cTG29n/P/PZ/126va/RzPLljHp4VbuOXEEWT2ieuCKkU6V5uupLXWvmmtHWatzbbW/imw7YHd5uf/1Vo7ylo7xlp7dxfV22456V4KFdRERETaZ8Xb8M5vYdSpcOTNblfTYQ98VMS6rVXcfspootRAJKQN75vIE1dMZmd1HRf8+ws276xu82M37qjiT28sY8rQFM6bOLALqxTpPGH/Gys7LYHibVVU1zW4XYqIiEho2LwEXmjs8Hh/SHZ4BOd6pPs/LOKkcf04JCfV7XKkE4zJ7M1jl0+itLyGCx6cT5mvZp+Psdbyqxe/o95v+X9njMPj0ZRHCQ2h+Zu3HXLSvVjrdHsSERGRffCVwtPnQrQ3ZDs8Nvr960vxGMOtJ410uxTpRAcOTOKhSydSvK2SCx/6kh2VdXs9/sWv1vPBilJuPmE4A1NC99+z9DxhH9SyA50f1VBERERkH3bp8Ph0SHZ4bPTBihLeWbqZa6fm0K+3rkcKNwcPTWH2RXkUlfi4+JEvKa9uPayV7Kzm9teWMHFwEhdPGdy9RYrsp7APakNSEzAGNRQRERHZG2vh9Rtg3RfOdMfMA92uqMNq6hu4/dUlDE1N4IrDhrhdjnSRI4alcd8FB7Jk/Q4uf3QBlbX1u+y31vLrlxdTU+/XlEcJSWEf1GKjIhiQFK8RNRERkb2Z9w9Y9B846hYYc7rb1eyXBz9ZxeqySm47ZTQxkRFulyNd6NhRGfz93AksXLONqx7P36UnwWvfbuSdpZv5xbRhDA3MsBIJJWEf1MBpKFJUqmvUREREWrXiLXjnNhh9ekh3eARYv72Ke94v5PjRGRw5LM3tcqQbnDSuH3eedQDzisr4yZMLqa33U+ar4XevLmH8gD5ccdhQt0sU6ZBItwvoDjnpXuYVleH3Ww17i4iItLRpMbxwJfQf7yxqHeKLAP/pjaX4reXWk0a5XYp0o9MPzKK6zs+vXvqOnz/9NR4P+Krr+euZ44jQ334SonpEUMtO81JT72f99ioGJKvbj4iICNDc4TEmEc59GqJCu+nGpwVbePO7TfzPccN0vu+Bzp88MLDA+VIAbjp+OLkZiS5XJdJxPSOopQc6P5b69ItbREQEAh0eL4CKLXD5W9Crn9sV7Zfaej+3vbqYgcnxzDxCU916qssPG4LHwFdrt+vfgYS8HnGNWk7gAtIiNRQRERFxOjy+dh2smw+n3Q/9J7hd0X57dN4qikoruO1Ho4iNUgORnuzSQ4fwj/MmEBXRI/7MlTDWI/4FJyVEk5wQrRb9IiIiAJ/9Hb55Go76FYw+ze1q9tvmndX8/d0Cpo5IZ+rIDLfLERHpFD0iqIEzqlZUos6PIiLSwy1/E979HYw5A478pdvVdIo/v7mMOr/ltz9SAxERCR89JqhlpydQqBE1ERHpyZo6PE6AGfeGfIdHgC9WlvHKog1cfcRQBqUkuF2OiEin6TlBLc3L1opatlbUul2KiIhI9/OVOB0eY3vDeaHf4RGgvsHPba8sIbNPHD85KsftckREOlXPCWqBzo+6Tk1ERHqcump49kKnw+N5T0NiX7cr6hSPf76GFZvL+c3Jo4iLVgMREQkvPSaoqfOjiIj0SC07PJ7+L2dh6zBQWl7DXe98z+G5qRw/Wg1ERCT89JigltknjphIj0bURESkZ/nsbvj2GTj6Vhg1w+1qOs0dby2nur6B350yGhMG19qJiOyuxwQ1j8cwNM1LoUbURESkp1j2Orx7O4w5E4640e1qOs3CNVt54atirjhsKNmBGTMiIuGmxwQ1gOy0BIpK1aJfRER6gE3fwYszIfNAmHFPWHR4BGjwW37z8hL69orl2mPUQEREwlePCmo56V7Wbaukuq7B7VJERES6TvlmeOpciOsD5z4VFh0eGz01fw1LN+7k1yeNJCEm0u1yRES6TI8KatlpXqyFVVs0qiYiImHKWnjhCqjaGlYdHgHKfDX8dc4KDslO4eRx/dwuR0SkS/Woj6JyWrToH9mvl8vViIiIdAFjYOpvnVb8/Q5wu5pO9dc5K6isbeB2NRARkR6gRwW1IakJGIMaioiISHgbMMntCjrdonXbeTZ/HVccOoTcjES3yxER6XI9aupjbFQEWUlxaigiIiISQvx+y22vLCbVG8N1x+a6XY6ISLfoUUENnIWvNaImIiISOp7LX8c3xTv41YkjSIyNcrscEZFu0eOCWnaal5WlPvx+63YpIiIisg/bK2v5f28vZ+LgJE4dn+l2OSIi3abHBbWcdC819X7Wb69yuxQRERHZh/+b+z07quq4/ZQxaiAiIj1Kjwtq2YHOj4Wlmv4oIiISzBav38F/5q/h4imDGdVf3ZpFpGfpeUEtLdCiX9epiYiIBC2/3/LbVxaTFB/NDccNc7scEZFu1+OCWnJCNMkJ0RRpRE1ERCRovfj1er5au52bp4+gd5waiIhIz9PjghpAdloCRSVq0S8iIhKMdlTVccdbyxg/oA9nHpjldjkiIq7okUEtJ92rETUREZEgdfe731NWUcsfZozB41EDERHpmXpkUMtO81JWUcu2ilq3SxEREZEWlm/ayeOfr+G8SQMZm9Xb7XJERFzTY4MaoFE1ERGRIGKt5bevLCExNpKbpg13uxwREVf1yKCW09iiX50fRUREgsar32zgy1Vbuen44SQlRLtdjoiIq3pkUOvfJ46YSI9G1ERERIKEr6aeP72xjLGZvTl34kC3yxERcV2k2wW4IcJjGJrmpahUnR9FRESCwT/eK6CkvIZ/XXQQEWogIiLSM0fUwGnRr6mPIiIi7issKefhT1dxdl4WEwYmuV2OiEhQ6MFBzcu6bZVU1zW4XYqIiEiPZa3ltleXEB8dwc0njHC7HBGRoNFjg1pOuhdrYdUWTX8UERFxy5vfbeKzwjJ+MW04Kd4Yt8sREQkaPTaoqUW/iIiIuypr6/njG0sZ2a8XF0xWAxERkZZ6bFAbmpaAMVBUohE1ERERN9zzfiEbd1Tz+xmjiYzosX+SiIi0qsf+VoyNiiArKY5CjaiJiIh0u5WlPv79yUpOn5DJxMHJbpcjIhJ0emxQA2f6Y5E6P4qIiHQray23v7aUmMgIZk1XAxERkdb06KCWk+Zl5RYffr91uxQREZEe452lm/no+1KuPzaX9F6xbpcjIhKU2hTUjDEnGGNWGGMKjTGzWtl/lDFmhzFmUeD2284vtfNlp3uprvOzfnuV26WIiIj0CNV1Dfz+9aUMy/ByySGD3S5HRCRoRe7rAGNMBHAvcBxQDCwwxrxqrV2626GfWGtP7oIau0xOenPnxwHJ8S5XIyIiEv7u/7CI4m1VPHXVZKLUQEREZI/a8htyElBorV1pra0FngFmdG1Z3aOxRX+hrlMTERHpcuu3V3H/R0WcPK4fh2Snul2OiEhQa0tQywTWtbhfHNi2uynGmG+MMW8ZY0a39kTGmJnGmHxjTH5paWkHyu1cyQnRJMVHUVSqFv0iIiJdbV7hFmrr/Vx7TK7bpYiIBL22BDXTyrbdu298BQyy1h4A/BN4ubUnstbOttbmWWvz0tLS2lVoV8lJV+dHERGR7lBY6iMqwpCdluB2KSIiQa8tQa0YGNDifhawoeUB1tqd1lpf4Ps3gShjTEjMachO81KktdRERES6XOFmH0NSE7S4tYhIG7TlN+UCINcYM8QYEw2cC7za8gBjTF9jjAl8PynwvGWdXWxXyEn3UlZRy7aKWrdLERERCWuFpT5y0xPdLkNEJCTsM6hZa+uBa4A5wDLgOWvtEmPM1caYqwOHnQksNsZ8A/wDONdaGxKLkzU2FNGomoiISNeprmtg7dbKpo7LIiKyd/tszw9N0xnf3G3bAy2+vwe4p3NL6x4tg1re4GSXqxEREQlPRaU+rEVBTUSkjXr8JPHMpDhiIj1q0S8iItKFGs+zuRkKaiIibdHjg1qExzAkNUEt+kVERLpQYYkPj4Ehqer4KCLSFj0+qEGgRb+uURMREekyhSU+BqUkEBMZ4XYpIiIhQUEN5zq1dVsrqa5rcLsUERGRsFRQ4tP1aSIi7aCgBmSne/FbWF2m6Y8iIiKdra7Bz+otFQpqIiLtoKAG5AQ6P6qhiIiISOdbU1ZBvd+Sq6AmItJmCmrA0LQEjIGiEo2oiYjIDxljTjDGrDDGFBpjZrWyP8kY85Ix5ltjzJfGmDFtfWxPULDZ+SBUI2oiIm2noAbERkWQlRSnhiIiIvIDxpgI4F5gOjAKOM8YM2q3w34FLLLWjgMuBv7ejseGvcYZK41rl4qIyL4pqAVkp3k19VFERFozCSi01q601tYCzwAzdjtmFPAegLV2OTDYGJPRxseGvYISH5l94kiIiXS7FBGRkKGgFpCd5mXlFh9+v3W7FBERCS6ZwLoW94sD21r6BjgdwBgzCRgEZLXxsQQeN9MYk2+MyS8tLe2k0oODOj6KiLSfglpATrqX6jo/67dXuV2KiIgEF9PKtt0/1bsDSDLGLAKuBb4G6tv4WGejtbOttXnW2ry0tLT9KDe4NPgtK0t9aiQiItJOmoMQ0DhvvqjUx4DkeJerERGRIFIMDGhxPwvY0PIAa+1O4DIAY4wBVgVu8ft6bLgr3lZJTb1fI2oiIu2kEbWAxhNIUak6P4qIyC4WALnGmCHGmGjgXODVlgcYY/oE9gFcCXwcCG/7fGy4a7z+OzdDQU1EpD00ohaQnBBNUnyUGoqIiMgurLX1xphrgDlABPCwtXaJMebqwP4HgJHA48aYBmApcMXeHuvGz+GWgsB5NSct0eVKRERCi4JaC9lpXrXoFxGRH7DWvgm8udu2B1p8/zmQ29bH9iQFm32kJcbQOz7K7VJEREKKpj62kJPupUgjaiIiIp2mUI1EREQ6REGthew0L2UVtWyrqHW7FBERkZBnraVIrflFRDpEQa2F7PQEAFZu0aiaiIjI/tq0sxpfTb1G1EREOkBBrYXGC53VUERERGT/FWwONBJJVyMREZH2UlBrITMpjuhIj1r0i4iIdIKmjo8aURMRaTcFtRYiPIahqQkaURMREekEhSU++sRHkeqN3vfBIiKyCwW13WSnq0W/iIhIZygsKSc33Ysxxu1SRERCjoLabrLTvKzbWkl1XYPbpYiIiIQsay0F6vgoItJhCmq7yUn34rewukzXqYmIiHRUWUUt2yvr1EhERKSDFNR2k53mtOgvKlFQExER6ajmjo8aURMR6QgFtd0MTfVijFr0i4iI7I/CwPXeWkNNRKRjFNR2ExcdQWafODUUERER2Q+Fm8tJiI6gX+9Yt0sREQlJCmqtyE5T50cREZH9UVjqNBJRx0cRkY5RUGtFTqBFv99v3S5FREQkJBVs9qmRiIjIflBQa0V2mpfqOj8bdlS5XYqIiEjI2VFVR0l5jRqJiIjsBwW1VjSeWNRQREREpP0az59qJCIi0nEKaq1oatFfqhb9IiIi7VVYUg5AboaCmohIRymotSI5IZo+8VFqKCIiItIBhSU+oiM9ZCXFu12KiEjIUlBrhTGGnDSvpj6KiIh0QEGJj+w0LxEedXwUEekoBbU9yE7zslIjaiIiIu1WWOJTIxERkf2koLYHOeletvhq2V5Z63YpIiIiIaOytp7ibVVqJCIisp8U1PYgO72xoYhG1URERNqqqMRpxKWgJiKyfxTU9iA7zTnBNJ5wREREZN8KS52Oj5r6KCKyfxTU9iArKZ7oSA+FGlETERFps4LNPiI9hkEpCW6XIiIS0hTU9iDCYxiamkCROj+KiIi0WWGJj0EpzoedIiLScfotuhfZ6V6NqImIiLRDYYmP3PREt8sQEQl5Cmp7kZ3mZd3WSqrrGtwuRUREJOjV1DewuqyC3AxdnyYisr8U1PYiOy0Bv4U1ZZVulyIiIhL0Vm+pxG/VSEREpDMoqO1F44mmUNepiYiI7FNBiTo+ioh0FgW1vRia6sUYraUmIiLSFoUlPoxpXuJGREQ6rk1BzRhzgjFmhTGm0Bgzay/HTTTGNBhjzuy8Et0TFx1BZp84jaiJiIi0QUGJjwFJ8cRGRbhdiohIyNtnUDPGRAD3AtOBUcB5xphRezju/wFzOrtIN2WneTWiJiIi0gaFm33katqjiEinaMuI2iSg0Fq70lpbCzwDzGjluGuBF4CSTqzPddlpXlaWVuD3W7dLERERCVr1DX5WbanQ9WkiIp2kLUEtE1jX4n5xYFsTY0wmcBrwwN6eyBgz0xiTb4zJLy0tbW+trshJ91JV18CGHVVulyIiIhK01m6tpLbBr6AmItJJ2hLUTCvbdh9euhu42Vq71wXHrLWzrbV51tq8tLS0Npboruy0BACKSitcrkRERCR4NV7PraAmItI52hLUioEBLe5nARt2OyYPeMYYsxo4E7jPGHNqZxToNrXoFxER2bcCBTURkU4V2YZjFgC5xpghwHrgXOD8lgdYa4c0fm+MeRR43Vr7cueV6Z7khGj6xEepoYiIiMheFJb46Nc7lsTYKLdLEREJC/sMatbaemPMNTjdHCOAh621S4wxVwf27/W6tFBnjHE6P2pETUREZI8KS3waTRMR6URtGVHDWvsm8OZu21oNaNbaS/e/rOCSk+blveWb3S5DREQkKPn9lsISH+dOGrDvg0VEpE3atOB1T5ednsAWXy3bK2vdLkVERCTobNhRRVVdg0bUREQ6kYJaGzSeeHSdmoiIyA81NhLJTU90uRIRkfChoNYG2WmBoFaiFv0iIiK7K9zcGNQ0oiYi0lkU1NogKyme6EiPRtRERERaUVjiIyUhmqSEaLdLEREJGwpqbRDhMQxNTdBaaiIiIq0oKCnX9WkiIp1MQa2NstO8GlETERHZjbVWrflFRLqAglobZad7Wbu1kuq6BrdLERERCRql5TXsrK7X9WkiIp1MQa2NstMS8FtYU1bpdikiIiJBo6njY4Y6PoqIdCYFtTZq6vyo6Y8iIiJNGq/f1tRHEZHOpaDWRo1BTQ1FREREmhWUlJMYG0l6YozbpYiIhBUFtTaKi44gs0+cRtRERERaaGwkYoxxuxQRkbCioNYOOelejaiJiIi0UFjiUyMREZEuoKDWDtlpXlaWVuD3W7dLERERcd22ilq2+GrJTVcjERGRzqag1g7Z6QlU1TWwcWe126WIiIi4rrBUjURERLqKglo75KihiIiISJOCzQpqIiJdRUGtHbIDJ6IiBTUREREKS3zERTnNtkREpHMpqLVDSkI0feKjmqZ6iIiI9GQFJeVkpyfg8ajjo4hIZ1NQawdjDNlpXo2oiYiI0NjxUY1ERES6goJaO2WnJVBUWuF2GSIiIq4qr65j445qXZ8mItJFFNTaKSfdyxZfDTsq69wuRURExDWNH1oqqImIdA0FtXbKbuz8qOvURESkB2vsgKygJiLSNRTU2ilHnR9FREQoKCknKsIwKDne7VJERMKSglo7ZSXFEx3hoUgjaiIi0oMVbvYxNNVLZIT+lBAR6Qr67dpOER7DkNQEBTUREenRCkt9mvYoItKFFNQ6ICfd2zQ3X0REpKeprmtg7dZKBTURkS6koNYB2WkJrN1aSU19g9uliIiIdLuVpRVYq0YiIiJdSUGtA7LTvfgtrN5S6XYpIiLSDYwxJxhjVhhjCo0xs1rZ39sY85ox5htjzBJjzGUt9q02xnxnjFlkjMnv3sq7RkFJOQC5GQpqIiJdRUGtAxpb9Os6NRGR8GeMiQDuBaYDo4DzjDGjdjvsZ8BSa+0BwFHA/xljolvsP9paO95am9cdNXe1whIfHgNDUhPcLkVEJGwpqHXA0DTnxKQW/SIiPcIkoNBau9JaWws8A8zY7RgLJBpjDOAFtgL13Vtm9yks8TEoJYGYyAi3SxERCVsKah0QHx1JZp84LXotItIzZALrWtwvDmxr6R5gJLAB+A64zlrrD+yzwFxjzEJjzMw9vYgxZqYxJt8Yk19aWtp51XeBghJ1fBQR6WoKah2Une7V1EcRkZ7BtLLN7nb/eGAR0B8YD9xjjOkV2HeotfZAnKmTPzPGHNHai1hrZ1tr86y1eWlpaZ1SeFeoa/CzekuFgpqISBdTUOugnDQvRSUV+P27n6tFRCTMFAMDWtzPwhk5a+ky4EXrKARWASMArLUbAl9LgJdwplKGrDVlFdT7LbkKaiIiXUpBrYOy0xOoqmtg485qt0sREZGutQDINcYMCTQIORd4dbdj1gJTAYwxGcBwYKUxJsEYkxjYngBMAxZ3W+VdoGCzM5skNz3R5UpERMJbpNsFhKqmzo8lPjL7xLlcjYiIdBVrbb0x5hpgDhABPGytXWKMuTqw/wHgD8CjxpjvcKZK3myt3WKMGQq85PQYIRJ4ylr7tis/SCcpDDTSyk5Xx0cRka6koNZBjXPzC0t8HDEseK8lEBGR/WetfRN4c7dtD7T4fgPOaNnuj1sJHNDlBXajgsAHlPHR+hNCRKQraepjB6UkRNM7LkoNRUREpEcpVMdHEZFuoaDWQcYYctK9TVNAREREwl2D31JU6lMjERGRbqCgth+y0xIoKq1wuwwREZFuUbytkpp6P7kZCmoiIl1NQW0/ZKd52eKrYUdlnduliIiIdLnGWSSa+igi0vUU1PZDU0MRXacmIiI9QEFjUEtTa34Rka6moLYfmlr0K6iJiEgPUFjiIy0xht7xUW6XIiIS9hTU9sOA5HiiIzwUqaGIiIj0AAUlaiQiItJdFNT2Q4THMCQ1QSNqIiIS9qy1FCmoiYh0GwW1/ZSdrs6PIiIS/jbtrMZXU69GIiIi3URBbT/lpHlZU1ZBTX2D26WIiIh0mYLNjR0f1UhERKQ7KKjtp+x0L34La8oq3S5FRESky6g1v4hI92pTUDPGnGCMWWGMKTTGzGpl/wxjzLfGmEXGmHxjzGGdX2pwauz8WKiGIiIiEsYKSnz0iY8i1RvtdikiIj1C5L4OMMZEAPcCxwHFwAJjzKvW2qUtDnsPeNVaa40x44DngBFdUXCwGZqWAKDOjyIiEtYKS8rJTfdijHG7FBGRHqEtI2qTgEJr7UprbS3wDDCj5QHWWp+11gbuJgCWHiI+OpLMPnHq/CgiImHLWktBiU/THkVEulFbglomsK7F/eLAtl0YY04zxiwH3gAub+2JjDEzA1Mj80tLSztSb1DKTvdSqKAmIiJhqqyilu2VdWokIiLSjdoS1Fqb4/CDETNr7UvW2hHAqcAfWnsia+1sa22etTYvLS2tXYUGs+y0BIpKKvD7e8xAooiI9CBqJCIi0v3aEtSKgQEt7mcBG/Z0sLX2YyDbGJO6n7WFjJx0L1V1DWzcWe12KSIiIp2uIBDUtNi1iEj3aUtQWwDkGmOGGGOigXOBV1seYIzJMYGri40xBwLRQFlnFxusGjs/qqGIiIiEo8LN5SRER9Cvd6zbpYiI9Bj7DGrW2nrgGmAOsAx4zlq7xBhztTHm6sBhZwCLjTGLcDpEntOiuUjYawpquk5NRETCUGGp00hEHR9FRLrPPtvzA1hr3wTe3G3bAy2+/3/A/+vc0kJHqjea3nFRWktNRETCUsFmH4fnhs+15SIioaBNC17L3hljnIYiGlETEZEws6OqjpLyGnIzdH2aiEh3UlDrJDnpXgpLKtwuQ0REpFM1dXxMU1ATEelOCmqdJDvNyxZfDTsq69wuRUREpNMUlpQDaERNRKSbKah1kqaGIls0/VFERMJHYYmP6EgPWUnxbpciItKjKKh1ksZFQNVQREREwklBiY/sNC8RHnV8FBHpTgpqnSQrKY7oCI8aioiISFgpLPFpoWsRERcoqHWSyAgPQ1ITtOi1iIiEjcraeoq3VTXNGhERke6joNaJstMTKCpV50cREQkPRYFuxhpRExHpfqEb1Hash5d+AtU73a6kSXaal7VbK6mpb3C7FBERkf1WWOp0fNSImohI9wvdoFa+Eb59Bub8yu1KmuSke2nwW9aUVbpdioiIyH4r2Owj0mMYlJLgdikiIj1O6Aa1rDw49Hr4+gn4fo7b1QAtWvTrOjUREQkDhSU+BqcmEB0Zun8uiIiEqtD+zXvULEgfDa/+HCq3ul0NQ9OcTxzVol9ERMJBYYmPnDRNexQRcUNoB7XIGDjtfqjcAm/90u1qiI+OJLNPnFr0i4hIyKupb2B1WQW5GQpqIiJuCO2gBtDvADjyZvjuv7D0FberYWiaOj+KiEjoW72lEr9VIxEREbeEflADOOwG6DceXr8BfKWulpKT7qWo1Iffb12tQ0REZH8UlKjjo4iIm8IjqEVEwWkPQI0PXr8erHshKTvNS2VtA5t2VrtWg4iIyP4qLPFhTHOjLBER6V7hEdQA0kfCMbfC8tfh2+dcK6Pxk0c1FBERkVBWUOJjQFI8sVERbpciItIjhU9QA5jyMxhwMLx1E+zc4EoJTS361VBERERCWOFmH7ma9igi4prwCmqeCDj1Pmiog1evdWUKZKo3ml6xkSzbuLPbX1tERKQz1Df4WbWlQteniYi4KLyCGkBKNhx7OxS+C1891u0vb4zhmBHp/HdhMW98u7HbX19ERGR/rd1aSW2DX0FNRMRF4RfUACZeCUOOgDm/hm1ruv3l//f0cRw0MInrn/2aj793twuliIhIezVeZ52bkehyJSIiPVd4BjWPB2bcCxh45Wfg93fry8dFR/DQpRPJTvPy4ycW8tXabd36+iIiIvujIBDUstMSXK5ERKTnCs+gBtBnIJzwZ1j9CSz4d7e/fO+4KB6/YhLpvWK47JEFrNhU3u01iIiIdERRiY9+vWNJjI1yuxQRkR4rfIMawISLIHcavHMbbCns9pdPT4zlySsmExPp4aKH5rNua2W31yAiItJeBSU+XZ8mIuKy8A5qxsCP/gGRMfDyT8Df0O0lDEiO54krJlNT7+fCh+ZTUq6FsEVEJHj5/ZZCBTUREdeFd1AD6NUPTrwTir+Eef90pYThfRN55LKJlJbXcPFDX7Kjqs6VOkRERPZlw44qquoayE1XIxERETeFf1ADGHsmjDwFPvgTbF7qSgkHDkziXxcdRFGpjyseXUBVbfeP7omIiOxLYyMRjaiJiLirZwQ1Y+DkuyCmF7z0Y2dBbBccnpvG3edMYOHabfzkPwupre/ebpQiIiL7UtTYml9BTUTEVT0jqAEkpDphbdO38Mn/uVbGSeP68efTxvLhilJu/O83+P3WtVpERER2V7DZR0pCNEkJ0W6XIiLSo/WcoAYw6hQYdw58/FfY8LVrZZw3aSC/PGE4r36zgdteXYK1CmsiIhIcCkrKNe1RRCQI9KygBjD9/0FCGrz0E6ivca2MnxyZzcwjhvLEF2u4653vXatDRESkkbVOx8fcDAU1ERG39bygFpcEp/wTSpfBB392rQxjDLdMH8HZeVn84/1CHv50lWu1iIiIAJSW17Czup6cNAU1ERG39bygBpB7HBx4Ccz7B6yd71oZxhj+fNpYjh+dwe9fX8oLC4tdq0VERKSwsZFIhlrzi4i4rWcGNYDj/wS9spyFsGsrXSsjMsLD38+dwKE5KfzyhW95Z+lm12oREZGeTa35RUSCR88NajGJcOq9sLUI3rvd1VJioyL410V5jOnfi5899RWfF5W5Wo+IiPRMBSXlJMZGkp4Y43YpIiI9Xs8NagBDjoDJV8P8B2DVx66W4o2J5NHLJjEwOZ6rHs/nu+IdrtYjIiI9T2GJj9x0L8YYt0sREenxenZQA5h6GyRnw8s/g+qdrpaSlBDNE1dMondcFJc88iVFpT5X6xERkZ6lsMSnaY8iIkFCQS06Hk69H3YWw9xb3a6Gfr3jeOKKSRjgogfns2F7ldsliYhID7CtopYtvlpy09VIREQkGCioAQycDIdcC189BgXvuF0NQ9O8PHb5JMqr67nooflsrah1uyQREQlzhaVqJCIiEkwU1Bod9StIGwmvXgtV29yuhjGZvXnwkjyKt1Vx6SNf4qupd7skEREJYwWbFdRERIKJglqjqFg47X6oKIW3bna7GgAmD03h3vMPZMmGnVz1WD7VdQ1ulyQiImGqsMRHXFQEmX3i3C5FRERQUNtV/wlw+I3w7bOw7DW3qwHg2FEZ3HnWOD5fWca1T39NfYPf7ZJERCQMFZSUk52egMejjo8iIsFAQW13R9wIfcfBa9dDxRa3qwHgtAlZ3PajUbyzdDOzXvwOv9+6XZKIiISZohKfGomIiAQRBbXdRUTBaf+Cmp3w+vVggyMUXXboEK6bmsvzC4v585vLsEFSl4iIhL7y6jo27KjW9WkiIkFEQa01GaPg6F850x+/e97tappcf2wul0wZxIOfruK+D4vcLkdERMJEUWkFoEYiIiLBpE1BzRhzgjFmhTGm0Bgzq5X9Fxhjvg3c5hljDuj8UrvZIT+HrInw5o2wc6Pb1QBgjOG2H43m1PH9+eucFTz5xRq3SxIR6RHacB7sbYx5zRjzjTFmiTHmsrY+NhgUljgdH3MV1EREgsY+g5oxJgK4F5gOjALOM8aM2u2wVcCR1tpxwB+A2Z1daLfzRMCpD0B9Dbz286CZAunxGP561gEcMyKd37yymNe+2eB2SSIiYa2N58GfAUuttQcARwH/Z4yJbuNjXVdQUk50hIeByfFulyIiIgFtGVGbBBRaa1daa2uBZ4AZLQ+w1s6z1jYuPvYFkNW5ZbokNQeO/R0UzIWvn3C7miZRER7uPf9A8gYl8T/PLeKj70vdLklEJJzt8zwIWCDRGGMAL7AVqG/jY11XVOJjSGoCkRG6IkJEJFi05TdyJrCuxf3iwLY9uQJ4q7UdxpiZxph8Y0x+aWmIhItJM2Hw4fD2r2D7WreraRIXHcGDl0wkJz2Rq59YyMI1W90uSUQkXLXlPHgPMBLYAHwHXGet9bfxsYC758iCEp+uTxMRCTJtCWqtLajS6jxAY8zROEGt1RWjrbWzrbV51tq8tLS0tlfpJo8HZtwLWHjlZ+APnnXMesdF8fjlk8joFcNljyxg2cadbpckIhKO2nIePB5YBPQHxgP3GGN6tfGxzkaXzpHVdQ2s3VqpoCYiEmTaEtSKgQEt7mfhfGK4C2PMOOBBYIa1tqxzygsSSYPg+D/Bqo9hwYNuV7OLtMQYnrhiMnHREVz88JesKatwuyQRkXDTlvPgZcCL1lGIc+32iDY+1lUrSyuwFnIzFNRERIJJW4LaAiDXGDPEGBMNnAu82vIAY8xA4EXgImvt951fZhA48BLIORbevQ3Kgqs1/oDkeJ64YjJ1DX4ueuhLSnZWu12SiEg42ed5EFgLTAUwxmQAw4GVbXysqwpKygG15hcRCTb7DGrW2nrgGmAOsAx4zlq7xBhztTHm6sBhvwVSgPuMMYuMMfldVrFbjIFT/uksiP3yT8Df4HZFuxiWkcgjl05ki6+Gix76kh2VdW6XJCISFtp4HvwDcIgx5jvgPeBma+2WPT22+3+KPSsq8eExMCQ1we1SRESkBWNdajufl5dn8/NDMM998yy8NBOO+z0cep3b1fzApwVbuPzRBYzJ7MWTV04mPjrS7ZJERDDGLLTW5rldR6joznPkT55cyPJN5Xxw41Hd8noiItJsb+dH9eFtr3Fnw4iT4f0/Qskyt6v5gcNyU/n7ueNZtG47Vz/5FbX1wdP8REREgo86PoqIBCcFtfYyBk6+C2IS4aWroSH4phhOH9uPP582lo+/L+V/nltEgz84FusWEZHgUtfgZ/WWCnIV1EREgo6CWkd40+Gkv8HGRfDJ39yuplXnThrIrOkjeP3bjfz2lcW4NcVVRESC15qyCur9ViNqIiJBSEGto0afCmPOhI//Ahu/cbuaVl19ZDY/PnIo/5m/lv+bG57NOEVEpOMKS3wA5KYnulyJiIjsTkFtf5z4V4hPcaZA1te4XU2rZp0wgnMnDuCeDwp58JOVbpcjIiJBpGCzE9Sy09XxUUQk2Cio7Y/4ZKdlf8lS+PAOt6tplTGGP502lulj+vLHN5bxXP46t0sSEZEgUVDiI7NPnDoEi4gEIQW1/TXseJhwIXx2N6xb4HY1rYrwGO4+dzyH5aRy8wvf8r9vLqOmPrjWgRMRke5XWOIjN0PXp4mIBCMFtc5w/P9Cr0x4+WqorXS7mlbFREYw++KDOHfiQP718Upm3PMZSzfsdLssERFxSYPfUlTqIydNQU1EJBgpqHWG2F4w4x4oK4T3/+B2NXsUHx3J/54+locvzWOLr5YZ937KfR8Wqn2/iEgPtH5bFTX1fo2oiYgEKQW1zjL0KJh4FXxxH6z6xO1q9uqYERnMveEIjhuVwV/eXsHZ//qc1Vsq3C5LRES6UUFJOYBa84uIBCkFtc503O2QNARe+SnUlLtdzV4lJ0Rz7/kH8vdzx1OwuZzpf/+EJ79Yo/XWRER6iIJAa/6cNLXmFxEJRgpqnSk6AU57ALavg2cugC0Fble0V8YYZozPZM4NR3DQoCRufXkxlz6ygM07q90uTUREulhhiY/0xBh6x0e5XYqIiLRCQa2zDTwYTr4L1n8F9x0Mb90MlVvdrmqv+vWO4/HLJ/H7GaOZv6qMaXd9zGvfbHC7LBER6UIFJT5NexQRCWIKal0h7zL4+ddw4MXw5Wz4x3iYdw/U17pd2R55PIaLpwzmjZ8fzuDUBK59+mt+/vTXbK8M3ppFRKRjrLUUlfjIVVATEQlaCmpdxZvmjKz9ZB5kTYS5v4Z7J8HSVyGIrwPLTvPywtVT+MVxw3jzu40cf/fHfPR9qdtliYhIJ9q0sxpfTb1G1EREgpiCWldLHwkXvgAXvACRsfDcRfDoSbDha7cr26PICA/XTs3l5Z8dSq/YKC55+Etuffk7Kmvr3S5NREQ6QcHmQCORdDUSEREJVgpq3SX3WLj6U2eUrXQFzD4KXroadqx3u7I9GpPZm9euPYwrDxvCf+av5cS/f8LCNdvcLktERPZTYaDjo9ZQExEJXgpq3SkiEvIud65fO+wGWPwi/PMg+ODPUONzu7pWxUZFcOvJo3jqyoOpa7Cc9cA8/jpnObX1frdLExGRDioo8dEnPoqUhGi3SxERkT1QUHNDbC849ndwzQIYcSJ89P+cwPb1k+BvcLu6Vk3JTuHt6w/njAOzuPeDIk699zNWbAruteJERKR1jY1EjDFulyIiInugoOampEFw5sNwxTvQZwC88jOYfSSs/MjtylqVGBvFX886gNkXHcTmndX86J+fMvvjIhr8wdscRUREdmWt5fuScjUSEREJcgpqwWDAJCesnfkwVO2Ax0+Bp88L2gWzp43uy5wbjuCo4Wn8+c3lnPfvL1i3tdLtskREpA3KKmrZXlmnRiIiIkFOQS1YGANjznCmQx77O1j1SVAvmJ3qjeFfFx3EnWcdwNINOznh7o95dsFabBAvPSAiIi0aiWhETUQkqCmoBZuoWKfRSAgsmG2M4cyDsnj7+sMZl9WHm1/4jisfy6ekvNrt0kREZA8KShpb8yuoiYgEMwW1YBVCC2ZnJcXznysn85uTR/FJ4RaOv+tj3l680e2yRESkFUUlPhKiI+jXO9btUkREZC8U1IJdiCyY7fEYrjhsCG9cexhZSfFc/eRX/M+zi9hRVed2aSIi0kJBoJGIOj6KiAQ3BbVQESILZudmJPLiTw/h51NzeeWbDUy/+2M+K9zidlkiIhJQsNmnRiIiIiFAQS2UhMiC2VERHv7nuGG88JNDiI2K4IIH5/O7V5dQVRuca8SJiPQUO6rqKCmvITdD16eJiAQ7BbVQFCILZo8f0Ic3fn44lx4ymEfnreakf37CN+u2u12WiEiP1djxMSdNQU1EJNgpqIWyEFgwOy46gt+dMponr5hMVW0Dp98/j7ve+Z66Br/bpYmI9DhFja35NaImIhL0FNTCQQgsmH1YbipvX38EpxzQn7+/V8Dp982jsKTc7bJERHqUgpJyoiM9ZCXFu12KiIjsg4JauNjTgtlv/jJoFszuHRfFXeeM574LDqR4WyUn/eNTHv50FX5/cC03ICISrgpKfGSneYnwqOOjiEiwU1ALN7svmL3g30G3YPaJY/sx54YjODQnld+/vpQLHpzP+u1VbpclIhL2Ckt85GqhaxGRkKCgFq72tGD24hegzv1QlJ4Yy0OX5HHH6WP5tng7J9z1Mbe8+B0Pf7qKj78vZeOOKmyQLewdLOoa/BSV+vikoBRfTb3b5YhIiKisrad4WxU5CmoiIiEh0u0CpIs1Lphd8C7MvRWev9xZOHvQIZBzLGRPhbThztTJbmaM4dxJAzkkO5Xfv76EtxZv5OnK5gWyvTGRZKd7yUnzkpvR/DUrKT7sp+1Ya9niq2XVlgpWlvpY2fi1tIK1WyupD0wXTU6I5qdHZXPhwYOIjYpwuWoRCWYrSysANKImIhIiFNR6itxjYehRsPJDKHoPCt+DOb9y9vXKhJypTmgbehTE9enW0gamxPPgJROx1lJWUUthiY+CEh9FJT4KSsr5tLCUF74qbjo+JtLD0DQvOelectObvw5KSSA6MrQGiavrGlhdVsHK0uYgVrSlglWlPnZWN4+WRUd6GJKSwPC+iUwf25ehqV76xEfxyGer+eMby3jo01X8fGouZx6URVREaL0HItI9CgINnNTxUUQkNCio9SQRkU5gyz3Wub99XSC0vQtLXoavHgfjcaZKZk91Rtz6jwdP94zUGGNI9caQ6o3h4KEpu+zbUVVHUamPws1OeCss8fH12m289s2G5h/PYxicEh8IbonkBEJcdpqXuGj3RpustWzcUc3K0gpWbfFRVFrRNEK2fnsVLWd49usdy9C0BE4Z35+hqV6GpiWQnealf5+4VkcRp47MYF7hFv4yZwW3vPgdsz9eyQ3HDePksf3whPmoo4i0T8FmH5Eew6CUBLdLERGRNjBuXQeUl5dn8/PzXXltaUVDHRTnN4+2bfgasBCXBEOPDkyTPAZ69XO70l1U1tazsrQiMApX3jQat6askobA9EBjICspLjB1MpGcNC85GU6I6xUb1Wm1+GrqWVVawcrGMBYYIVu1pYKquuaFyBOiIxiSltAUxIameRmamsCQ1AQSYjr22Ym1lneXlXDnnBWs2FzOiL6J3HT8cI4ZkY5xYVqryO6MMQuttXlu1xEquuIcOfPxfFZuqeDd/zmyU59XREQ6bm/nR42oiSMiCgZNcW7H3AoVZbDyA2e0rfA9WPKic1zGGCew5UyFgVMgMsbVsuOjIxmT2Zsxmb132V5b72d1WSDAbfZRWOqjYHM5nxWVUVvfvNh2Rq+YphG47BZTKVMSolsNOA1+S/G2SmeKYuDascZwtnlnTdNxHgNZSfEMTUvg4KEpDElLIDvVCWUZvWI6PTwZYzhuVAZTR6Tz2rcb+Ns733PFY/kcOLAPNx0/ginZKft+EhEJa4UlPoZlJLpdhoiItJFG1GTfrIXNi5tD29ovwF8HUfEw+HBntC1nKiQPdaUpSXs0+C3rtlY2jbwVlvgoDIzEVdQ2j3olxUcFpk4m0isuktVbnOvI1pRVUtvQHPR6x0U5o2JN0xSdMDYwOd7V5h51DX7+m1/MP94rYNPOag7PTeXGacM5YEAf12qSnk0jau3T2efImvoGRv12Dj89KptfTBveac8rIiL7RyNqsn+Mgb5jndthN0CND1Z/4oS2wnehYI5zXJ9BzaFtyBEQE3yf3EZ4DINTExicmsCxozKatjdeR7Z7gHtr8UYqauoZmBzP0DQvx4xMJzvVG5i6mEDyHkbe3BYV4eH8yQM5/cBMnvxiDfd+UMiMez/jhNF9+cW0YeTqU3WRHmX1Fmc6uFrzi4iEDgU1ab8YLwyf7twAyoqg6H0nuH3zDOQ/BJ5IGHAw5BzjhLeMseAJ3m6Exhj694mjf584jhiW1rTdWou1hGxjjtioCK48fCjnTBzAw5+u5t+frGTO0k2cNiGTG44dxoDkeLdLFJFu0NjxUUFNRCR0KKjJ/kvJdm6TroL6Wlj3RWC07T147/fOLSEtcG3bsU5zEm/avp83CBhjgn02Z5skxkZx3bG5XDxlEPd/VMRj81bz2jcbOHfiQK49Jof0XrFulygiXaiwxIcxkJ2moCYiEioU1KRzRUY70x6HHAHH3Q7lmwOjbe9CwTvw7bPOcf3GN6/dNmCS08xEulxSQjS/OnEklx86hH++X8DTX67lvwvXcckhg7n6iGySEqLdLlFEukBBiY8BSe5eOysiIu2joCZdKzEDxp/n3PwNsHERFAaC26d3wyf/B9GJMPRIZ/229FGQPhJ6ZwV9Y5JQ1rd3LH86bSwzjxjK3e8WMPvjlTz1xVquOmIolx82BG8HlwkQkeBUVOIjV9MeRURCiv4ak+7jiYDMg5zbkTdB1XZY9bET2lZ+AMtfbz42phekjXBCW8Zo52v6KEhIda38cDQoJYG7zhnP1Udm839zV/C3d77nsXmr+enROVwweaA+fRcJA/UNflaWVnDk8NCYci4iIo42BTVjzAnA34EI4EFr7R277R8BPAIcCPzaWntnZxcqYSiuD4w6xbkBVG2DkuVQshRKljm3Za/CV481PyYhrTm0NX5NGwGxvVz5EcLF8L6JzL44j6/XbuP/5n7PH15fyoOfrOS6qbmceVAWkRHB2whGRPZu7VZnWZEcXZ8mElbq6uooLi6murra7VKkDWJjY8nKyiIqqu2X++wzqBljIoB7geOAYmCBMeZVa+3SFodtBX4OnNquikVaiktqXnS7kbXg29wivAW+fvUE1FU0H9d7YCC4tQhxqcMgSk0y2mPCwCSevHIy8wq38Jc5K5j14nf86+OV3HDcME4e2y9ku1+K9GSFJT4ALcshEmaKi4tJTExk8ODBQblUkDSz1lJWVkZxcTFDhgxp8+PaMqI2CSi01q4EMMY8A8wAmoKatbYEKDHGnNS+skX2wRhI7Ovcso9p3u73w/Y1u4a3kmVO4xJ/XeCxHkjOhoxRu47AJQ2BCM363ZtDclJ5KTuFd5eVcOecFfz86a+574NCbjp+OMeMSNcJQSSEFASCWnZagsuViEhnqq6uVkgLEcYYUlJSKC0tbdfj2vLXaiawrsX9YmByu14lwBgzE5gJMHDgwI48hYjD44HkIc5txInN2xvqnHXdSpY2B7hN38HSVwHrHBMRA2nDAuFtlBqY7IExhuNGZTB1RDqvfbuBv73zPVc8ls+BA/tw0/EjmJKd4naJItIGRSU++vWOJTFW3XVFwo1CWujoyH+rtgS11p7VtvuVAGvtbGA2QF5eXoeeQ2SvIqIgfYRz4/Tm7bWVsGVF8wjc5qWw6pPm5QLA6T65+/TJ9FEhs+ZbV/F4DDPGZ3Li2H78N7+Yf7xXwHn//oLDc1O56fjhjMvq43aJIrIXBSU+LXQtIhKC2hLUioEBLe5nARu6phyRLhIdD/0nOLeW2tLAJD61OcCljQiEuBHONXU9SFSEh/MnD+T0AzN58os13PtBIafc8xknjO7LL6YN0/UvbWStxVrwW4s/8NVaiI3y6JNR6XR+v6WwxMe5kwbs+2AREQkqbQlqC4BcY8wQYD1wLnB+l1Yl0l3a3MBkOSx6Cmp9zcd5+/4wwKUND/sOlLFREVx5+FDOmTiAhz5dxYOfrGLO0k2cNiGTG44dxoDk+E5/zfoGP1V1Dc6ttsXXxu/rGqisbaA6sL3p+8D2qroGqgNfG/wtg5ITlmyL0NR8f9cg1XS8v53H7/b8dg9zCbKS4jjroAGcmZdFZp+4Tn8PpWfasKOKqroGctP1QYqIdK7t27fz1FNP8dOf/rRdjzvxxBN56qmn6NOnT9cUFkb2GdSstfXGmGuAOTjt+R+21i4xxlwd2P+AMaYvkA/0AvzGmOuBUdbanV1XukgX2VMDE2thx7rmEbjS5U6Qy38E6quaj+uVFQhwI5qXD0gbDtHhdSF/YmwU1x87jIunDOaBj4p4bN5qXvtmA+dOHMipEzKpC4Sr6toWYalFcGoZtHYPVs2Bq57qOj+1Df521xcT6SE+OoK4qAhioyOIj44gJjKCCI/B44FI4yHCYzDG4DHgCXzd9b7BtNjn3G+x39PO45ueP7DN4xxvLXxeVMbd733P3e99z2E5qZyVN4BpozK0lp3sl8ZGIpr6KBLebn9tCUs3dO6f3aP69+K2H43e4/7t27dz3333/SCoNTQ0EBGx53PXm2++2Wk1doV91d+d2tT6zlr7JvDmbtseaPH9JpwpkSLhyxjoM9C5DZvWvN3fEOhA2TLALYdVH0FDbeODIWkQpO0W4MJgCYHkhGh+deJILj90CP98v4Cnv1zLE1+s2etjoiM8xEZ5iI+OJC46gtioiKZQlRQf3fR9XHTgFtXifivb4wPPERcIZLGRESG3lMDPjs5h3dZKXviqmP/mF/Pzp7+mV2wkp07I5Oy8AYzu30tTI6Xdihpb8yuoiUgnmzVrFkVFRYwfP56oqCi8Xi/9+vVj0aJFLF26lFNPPZV169ZRXV3Nddddx8yZMwEYPHgw+fn5+Hw+pk+fzmGHHca8efPIzMzklVdeIS6u9Vkl//73v5k9eza1tbXk5OTwxBNPEB8fz+bNm7n66qtZuXIlAPfffz+HHHIIjz/+OHfeeSfGGMaNG8cTTzzBpZdeysknn8yZZ54JgNfrxefz8eGHH3L77be3qf63336bX/3qVzQ0NJCamso777zD8OHDmTdvHmlpafj9foYNG8YXX3xBamrqfr3Hxu5pHk4Xy8vLs/n5+a68tki3aKiHbauar30rXeYEuLIC8Nc7xxgPJA8NTJ1snEY5ElJyIDLa3fo7aN3WSr7fXL5LqIqPiiQ22tMUrLSA9t75/ZbPV5bxXP463lq8idp6PyP79eLsvCxOHZ9JUkLo/dswxiy01ua5XUeo6Kxz5M3Pf8t7yzeTf+txnVCViASTZcuWMXLkSNdef/Xq1Zx88sksXryYDz/8kJNOOonFixc3rRO2detWkpOTqaqqYuLEiXz00UekpKTsEtRycnLIz89n/PjxnH322ZxyyilceOGFrb5eWVkZKSlOx+lbb72VjIwMrr32Ws455xymTJnC9ddfT0NDAz6fj+LiYk4//XQ+++wzUlNTm2rZW1BrS/1+v58DDzyQjz/+mCFDhjQdc/vtt9O7d2+uv/565s6dy7/+9S9eeOGFH/wMrf0329v5UYtJiXSViEhIzXVuo05p3l5fC1uLmq99Kw0EuRVvgg1M8fNEOmGtZfOStJFOqHNrDThrndFDf52zDIK/PvB11/sD/HUMSIuHXplOExdpN4/HcGhOKofmpPL7yjpe/XYD/81fx+2vLeV/31zOcaMyOCsvi8Nz04gIsZFD6V4FJeVkp2k0TUS63qRJk3ZZzPkf//gHL730EgDr1q2joKCgKWg1GjJkCOPHjwfgoIMOYvXq1Xt8/sWLF3Prrbeyfft2fD4fxx9/PADvv/8+jz/+OAARERH07t2bxx9/nDPPPLNpRCs5OblT6i8tLeWII45oOq7xeS+//HJmzJjB9ddfz8MPP8xll122z9drCwU1ke4WGd08etZSXbUz2tY0ArccNi6Cpa/QvAZctDNdsvG6t4joQFCq30OAqnemX+5p3z637/Yc7RWfCn0GQO/Aren7LGcKaVyS1q7bh97xUVx08CAuOngQyzbu5L/5xbz0dTFvfLeRfr1jOePALM7Ky2JQSnhdAyn7z1qn4+Mp4/u7XYqI9AAJCc3noQ8//JB3332Xzz//nPj4eI466iiqq6t/8JiYmJim7yMiIqiqqvrBMY0uvfRSXn75ZQ444AAeffRRPvzwwz0ea61t9XKByMhI/H5/0zG1tbVN+9pS/56ed8CAAWRkZPD+++8zf/58/vOf/+yxtvZQUBMJFlGx0Hesc2uptgK2fN8c4EqWwbr5sPj5XY8zEc46cp4oZ9TNE+Xcb9oW5YzUtbwfGdP6Y3Y/bpf7bTiuthJ2rIXt62BHsRM6C97ZtekKQFRCILxltQhyAwNBbgAk9gNPcFzQGwxG9uvFb380ilnTR/Dess08l7+O+z4s5J4PCpk8JJmz8wYwfWxf4qP1q12gtLyGndX15GhETUS6QGJiIuXl5a3u27FjB0lJScTHx7N8+XK++OKL/X698vJy+vXrR11dHf/5z3/IzMwEYOrUqdx///1NUx8rKiqYOnUqp512GjfccAMpKSlNUxQHDx7MwoULOfvss3nllVeoq2v9Q+g91T9lyhR+9rOfsWrVql2mPgJceeWVXHjhhVx00UWd1oxEZ3ORYBed0PoacHVVzlTJxpAU7CNT1kJlmdM5c/s65+uOYti+1vl+/VdQtXXXx3gioVf/3UbkGkNdINBF9bxW9tGRHqaP7cf0sf3YtKM60IBkHb/47zfc9uoSfnRAP87KG8CEAX3UgKQHK2xsJKI1DkWkC6SkpHDooYcyZswY4uLiyMjIaNp3wgkn8MADDzBu3DiGDx/OwQcfvN+v94c//IHJkyczaNAgxo4d2xQS//73vzNz5kweeughIiIiuP/++5kyZQq//vWvOfLII4mIiGDChAk8+uijXHXVVcyYMYNJkyYxderUXUbRWtpT/WlpacyePZvTTz8dv99Peno677zzDgCnnHIKl112WadNewQ1ExGRYFJbEQhvjUFut1C3c33zdXyN9jS9svFrD5leaa1lweptPJe/jje+3UhVXQM56V7OzsvitAlZpCXG7PtJupCaibRPZ5wjH5u3mtteXcKXv5pKeq/Q7i4rIj/kdjMR2VV+fj433HADn3zyyR6PUTMREQld0QnOtXdpw1vf31AP5Ruap1S2a3rlAGd0LibReZ2o+BZf453jouKav2/5NTI26MOeMYZJQ5KZNCSZ350ymje+3cBz+cX8+c3l/OXtFRw9Ip2z8wZw9PA0dd3sIQpKykmMjXQ9pIuIhLs77riD+++/v9OuTWukoCYioSMisnktu9a0Nr2y5ejcxkXOqF1dZTtf2ARCXVzrAS863rnf2rZdjm1tW0Knd/L0xkRyzsSBnDNxIIUlPv67cB0vLFzPO0s3k5YYw+kTMjkrb4AWQQ5zhSU+ctO9mv4qIiHlZz/7GZ999tku26677rpOnVLY2WbNmsWsWbM6/XkV1EQkfBgDCanObfdr+lry+52Rt9pKqKsIfK1sDnF1lT/ctvuxjduqtrU4NnBM4zp5bRUR3TyFs8/A5mvwGpur9BnQ4WvxctK93DJ9JDdOG85HK0p5Ln8dD326in99vJIDB/bh7LwBnDSuH4mxUR16fglehSU+jhmR7nYZIiLtcu+997pdQtBQUBORnsfjcUa1ohOAtM5//oa6PQS8xq9VPwyIvhJn1G/dl7DkpR+GvYS0VgLcwOZpnbG99lpSVISHY0dlcOyoDErLa3j56/U8m7+OWS9+x+2vLeXEsf04Z+IAJg5O0ghMGNhWUcsWXy256WokIiISqhTUREQ6W0QUxPVxbh3hb4Dyjc60ze1rm6/F274WNi+GFW9BQ82uj4nt03qAa5wq2qKpSlpiDFcdMZQrDx/ConXbeS6/mNe+2cALXxUzOCWes/IGcMaBWfTtrQYUjYwxJwB/ByKAB621d+y2/ybggsDdSGAkkGat3WqMWQ2UAw1AfXc0VSksdTo+5mRoequISKhSUBMRCTaeiMAyBFkwaMoP9/v9UFEauAZvbfMSB9vXwdaVsOojqPXt+phob3M3zMD0StNnABP6DGLCcQP47UkjeWvJJp7LX8df56zg/+au4IhhaZydN4CpI9OJiey569kZYyKAe4HjgGJggTHmVWvt0sZjrLV/Bf4aOP5HwA3W2pbrTRxtrd3SXTUXbA4ENa2hJiISshTURERCjccDiRnOLauVwRlrnWvnmgLc2hbNVdY40yurt+/ykLiIGE7vM4DTew+g/MB+fOPrxbvrY3jk+96Mvu5SBvVN6Z6fLThNAgqttSsBjDHPADOApXs4/jzg6W6qrVWFJT7ioiLI7NPz1hkUke6xfft2nnrqKX7605+2+7F33303M2fOJD4+vgsqCx8KaiIi4cYYiE92bv3Ht35M9c7dumOuafo+cfNiDqso5TCAGCD2bKBHB7VMYF2L+8XA5NYONMbEAycA17TYbIG5xhgL/MtaO7urCm1UUFJOdnoCHo+uNxSRrrF9+3buu+++Dge1Cy+8MCiCWn19PZGRwRmJgrMqERHpWrG9IHY0ZIxufX9dVWDx8TWQ2K97aws+raUdu4djfwR8ttu0x0OttRuMMenAO8aY5dbaj3/wIsbMBGYCDBy4hyUo2ui2H42mvLpuv55DRELIW7Ng03ed+5x9x8L0O/a4e9asWRQVFTF+/HiOO+440tPTee6556ipqeG0007j9ttvp6KigrPPPpvi4mIaGhr4zW9+w+bNm9mwYQNHH300qampfPDBB60+/09+8hMWLFhAVVUVZ555JrfffjsACxYs4LrrrqOiooKYmBjee+894uPjufnmm5kzZw7GGK666iquvfZaBg8eTH5+PqmpqeTn53PjjTfy4Ycf8rvf/Y4NGzawevVqUlNT+fOf/8xFF11ERUUFAPfccw+HHHIIAH/5y1944okn8Hg8TJ8+nauuuoqzzjqLr776CoCCggLOPfdcFi5c2JnvPqCgJiIirYmKg9Rc5ybFwIAW97OADXs49lx2m/Zord0Q+FpijHkJZyrlD4JaYKRtNkBeXt6egmCbaI08Eelqd9xxB4sXL2bRokXMnTuX559/ni+//BJrLaeccgoff/wxpaWl9O/fnzfeeAOAHTt20Lt3b/72t7/xwQcfkJqausfn/9Of/kRycjINDQ1MnTqVb7/9lhEjRnDOOefw7LPPMnHiRHbu3ElcXByzZ89m1apVfP3110RGRrJ169Y9Pm+jhQsX8umnnxIXF0dlZSXvvPMOsbGxFBQUcN5555Gfn89bb73Fyy+/zPz584mPj2fr1q0kJyfTu3dvFi1axPjx43nkkUe49NJLO+tt3YWCmoiIyN4tAHKNMUOA9Thh7PzdDzLG9AaOBC5ssS0B8FhrywPfTwN+3y1Vi0jPsZeRr+4wd+5c5s6dy4QJzhqmPp+PgoICDj/8cG688UZuvvlmTj75ZA4//PA2P+dzzz3H7Nmzqa+vZ+PGjSxduhRjDP369WPixIkA9OrlLE3z7rvvcvXVVzdNYUxOTt7n859yyinExTnX8dbV1XHNNdewaNEiIiIi+P7775ue97LLLmuaotn4vFdeeSWPPPIIf/vb33j22Wf58ssv2/xztYeCmoiIyF5Ya+uNMdcAc3Da8z9srV1ijLk6sP+BwKGnAXOttRUtHp4BvBRYmy4SeMpa+3b3VS8i0vWstdxyyy38+Mc//sG+hQsX8uabb3LLLbcwbdo0fvvb3+7z+VatWsWdd97JggULSEpK4tJLL6W6uhprbatrfe5pe2RkJH6/H4Dq6upd9iUkJDR9f9ddd5GRkcE333yD3+8nNjZ2r897xhlncPvtt3PMMcdw0EEHkZLSNddxe7rkWUVERMKItfZNa+0wa222tfZPgW0PtAhpWGsftdaeu9vjVlprDwjcRjc+VkQk1CUmJlJeXg7A8ccfz8MPP4zP5ywNsn79ekpKStiwYQPx8fFceOGF3HjjjU3XdbV8bGt27txJQkICvXv3ZvPmzbz11lsAjBgxgg0bNrBgwQIAysvLqa+vZ9q0aTzwwAPU19cDNE19HDx4cNO1Yy+88MIeX2/Hjh3069cPj8fDE088QUNDAwDTpk3j4YcfprKycpfnjY2N5fjjj+cnP/kJl112WQfevbZRUBMRERERkXZJSUnh0EMPZcyYMbzzzjucf/75TJkyhbFjx3LmmWdSXl7Od999x6RJkxg/fjx/+tOfuPXWWwGYOXMm06dP5+ijj271uQ844AAmTJjA6NGjufzyyzn00EMBiI6O5tlnn+Xaa6/lgAMO4LjjjqO6uporr7ySgQMHMm7cOA444ACeeuopAG677Tauu+46Dj/8cCIi9rwe6E9/+lMee+wxDj74YL7//vum0bYTTjiBU045hby8PMaPH8+dd97Z9JgLLrgAYwzTpk3rlPezNcba/bpeucPy8vJsfn6+K68tIiLdyxiz0FrbyqJv0hqdI0VkX5YtW8bIkSPdLqPHuvPOO9mxYwd/+MMf2vyY1v6b7e38qGvURERERERE2ui0006jqKiI999/v0tfR0FNRERERERcMXnyZGpqanbZ9sQTTzB27FiXKtq3l156qVteR0FNRERERERcMX/+fLdLCFpqJiIiIiIiEoLc6jUh7deR/1YKaiIiIiIiISY2NpaysjKFtRBgraWsrKxpfba20tRHEREREZEQk5WVRXFxMaWlpW6XIm0QGxtLVlZWux6joCYiIiIiEmKioqIYMmSI22VIF9LURxERERERkSCjoCYiIiIiIhJkFNRERERERESCjHGrU4wxphRYs59Pkwps6YRyehK9Z+2n96z99J61X7i/Z4OstWluFxEqdI50jd6z9tN71n56z9on3N+vPZ4fXQtqncEYk2+tzXO7jlCi96z99J61n96z9tN7Jp1N/6baT+9Z++k9az+9Z+3Tk98vTX0UEREREREJMgpqIiIiIiIiQSbUg9pstwsIQXrP2k/vWfvpPWs/vWfS2fRvqv30nrWf3rP203vWPj32/Qrpa9RERERERETCUaiPqImIiIiIiIQdBTUREREREZEgE7JBzRhzgjFmhTGm0Bgzy+16gp0xZoAx5gNjzDJjzBJjzHVu1xQKjDERxpivjTGvu11LqDDG9DHGPG+MWR749zbF7ZqCmTHmhsD/k4uNMU8bY2LdrklCm86P7aPzY8fpHNk+Oj+2X08/R4ZkUDPGRAD3AtOBUcB5xphR7lYV9OqBX1hrRwIHAz/Te9Ym1wHL3C4ixPwdeNtaOwI4AL1/e2SMyQR+DuRZa8cAEcC57lYloUznxw7R+bHjdI5sH50f20HnyBANasAkoNBau9JaWws8A8xwuaagZq3daK39KvB9Oc4vh0x3qwpuxpgs4CTgQbdrCRXGmF7AEcBDANbaWmvtdleLCn6RQJwxJhKIBza4XI+ENp0f20nnx47RObJ9dH7ssB59jgzVoJYJrGtxvxj9Um0zY8xgYAIw3+VSgt3dwC8Bv8t1hJKhQCnwSGA6zIPGmAS3iwpW1tr1wJ3AWmAjsMNaO9fdqiTE6fy4H3R+bJe70TmyPXR+bCedI0M3qJlWtmmdgTYwxniBF4DrrbU73a4nWBljTgZKrLUL3a4lxEQCBwL3W2snABWArpHZA2NMEs5oxxCgP5BgjLnQ3aokxOn82EE6P7adzpEdovNjO+kcGbpBrRgY0OJ+Fj1sKLQjjDFROCeh/1hrX3S7niB3KHCKMWY1ztShY4wxT7pbUkgoBoqttY2fRj+Pc2KS1h0LrLLWllpr64AXgUNcrklCm86PHaDzY7vpHNl+Oj+2X48/R4ZqUFsA5BpjhhhjonEuLHzV5ZqCmjHG4MyLXmat/Zvb9QQ7a+0t1tosa+1gnH9f71tre9SnOB1hrd0ErDPGDA9smgosdbGkYLcWONgYEx/4f3Qqurhc9o/Oj+2k82P76RzZfjo/dkiPP0dGul1AR1hr640x1wBzcDrAPGytXeJyWcHuUOAi4DtjzKLAtl9Za990ryQJU9cC/wn8kbgSuMzleoKWtXa+MeZ54CucznNfA7PdrUpCmc6PHaLzo3QXnR/bQedIMNZq6rqIiIiIiEgwCdWpjyIiIiIiImFLQU1ERERERCTIKKiJiIiIiIgEGQU1ERERERGRIKOgJiIiIiIiEmQU1ETawRjTYIxZ1OI2qxOfe7AxZnFnPZ+IiEh30flRpPOF5DpqIi6qstaOd7sIERGRIKPzo0gn04iaSCcwxqw2xvw/Y8yXgVtOYPsgY8x7xphvA18HBrZnGGNeMsZ8E7gdEniqCGPMv40xS4wxc40xcYHjf26MWRp4nmdc+jFFRETaRedHkY5TUBNpn7jdpnac02LfTmvtJOAe4O7AtnuAx62144D/AP8IbP8H8JG19gDgQGBJYHsucK+1djSwHTgjsH0WMCHwPFd3zY8mIiLSYTo/inQyY611uwaRkGGM8Vlrva1sXw0cY61daYyJAjZZa1OMMVuAftbausD2jdbaVGNMKZBlra1p8RyDgXestbmB+zcDUdbaPxpj3gZ8wMvAy9ZaXxf/qCIiIm2m86NI59OImkjnsXv4fk/HtKamxfcNNF9HehJwL3AQsNAYo+tLRUQkVOj8KNIBCmoineecFl8/D3w/Dzg38P0FwKeB798DfgJgjIkwxvTa05MaYzzAAGvtB8AvgT7ADz61FBERCVI6P4p0gD51EGmfOGPMohb337bWNrYgjjHGzMf5AOS8wLafAw8bY24CSoHLAtuvA2YbY67A+WTwJ8DGPbxmBPCkMaY3YIC7rLXbO+nnERER6Qw6P4p0Ml2jJtIJAnPw86y1W9yuRUREJFjo/CjScZr6KCIiIiIiEmQ0oiYiIiIiIhJkNKImIiIiIiISZBTUREREREREgoyCmoiIiIiISJBRUBMREREREQkyCmoiIiIiIiJB5v8DYJiRmB0ZFP8AAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 1080x504 with 2 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "from helper_functions import plot_loss_curves\n",
        "\n",
        "plot_loss_curves(vit_results)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A5Fb91zlQaQT"
      },
      "source": [
        "Ohh yeah!\n",
        "\n",
        "Those are some nice looking loss curves. Just like our EffNetB2 feature extractor model, it looks our ViT model might benefit from a little longer training time and perhaps some [data augmentation](https://www.learnpytorch.io/04_pytorch_custom_datasets/#6-other-forms-of-transforms-data-augmentation) (to help prevent overfitting)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3tk0F9ckQaQT"
      },
      "source": [
        "### 4.4 Saving ViT feature extractor\n",
        "\n",
        "Our ViT model is performing outstanding!\n",
        "\n",
        "So let's save it to file so we can import it and use it later if we wish.\n",
        "\n",
        "We can do so using the `utils.save_model()` function we created in [05. PyTorch Going Modular section 5](https://www.learnpytorch.io/05_pytorch_going_modular/#5-creating-a-function-to-save-the-model-utilspy)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "L7lAWeniQaQT",
        "outputId": "b9bb7f07-4951-4aed-f1ff-91f6fa1fe93d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[INFO] Saving model to: models/09_pretrained_vit_feature_extractor_pizza_steak_sushi_20_percent.pth\n"
          ]
        }
      ],
      "source": [
        "# Save the model\n",
        "from going_modular.going_modular import utils\n",
        "\n",
        "utils.save_model(model=vit,\n",
        "                 target_dir=\"models\",\n",
        "                 model_name=\"09_pretrained_vit_feature_extractor_pizza_steak_sushi_20_percent.pth\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YLpwhS_SQaQT"
      },
      "source": [
        "### 4.5 Checking the size of ViT feature extractor\n",
        "\n",
        "And since we want to compare our EffNetB2 model to our ViT model across a number of characteristics, let's find out its size.\n",
        "\n",
        "To check our model's size in bytes, we can use Python's `pathlib.Path.stat(\"path_to_model\").st_size` and then we can convert it (roughly) to megabytes by dividing it by `(1024*1024)`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8mjRxgFyQaQU",
        "outputId": "cac7a6e7-602e-4757-fa10-850dc779f1ce"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Pretrained ViT feature extractor model size: 327 MB\n"
          ]
        }
      ],
      "source": [
        "from pathlib import Path\n",
        "\n",
        "# Get the model size in bytes then convert to megabytes\n",
        "pretrained_vit_model_size = Path(\"models/09_pretrained_vit_feature_extractor_pizza_steak_sushi_20_percent.pth\").stat().st_size // (1024*1024) # division converts bytes to megabytes (roughly)\n",
        "print(f\"Pretrained ViT feature extractor model size: {pretrained_vit_model_size} MB\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-ww0EdGgQaQU"
      },
      "source": [
        "Hmm, how does the ViT feature extractor model size compare to our EffNetB2 model size?\n",
        "\n",
        "We'll find this out shortly when we compare all of our model's characteristics."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jX2EC6aRQaQU"
      },
      "source": [
        "### 4.6 Collecting ViT feature extractor stats\n",
        "\n",
        "Let's put together all of our ViT feature extractor model statistics.\n",
        "\n",
        "We saw it in the summary output above but we'll calculate its total number of parameters."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MOlyrImGQaQV",
        "outputId": "a6b1cfdf-78f3-4f5e-ae0e-86978b884f04"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "85800963"
            ]
          },
          "execution_count": 28,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Count number of parameters in ViT\n",
        "vit_total_params = sum(torch.numel(param) for param in vit.parameters())\n",
        "vit_total_params"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C9k7Wj76QaQV"
      },
      "source": [
        "Woah, that looks like a fair bit more than our EffNetB2!\n",
        "\n",
        "> **Note:** A larger number of parameters (or weights/patterns) generally means a model has a higher *capacity* to learn, whether it actually uses this extra capacity is another story. In light of this, our EffNetB2 model has 7,705,221 parameters where as our ViT model has 85,800,963 (11.1x more) so we could assume that our ViT model has more of a capacity to learn, if given more data (more opportunities to learn). However, this larger capacity to learn ofen comes with an  increased model filesize and a longer time to perform inference.\n",
        "\n",
        "Now let's create a dictionary with some important characteristics of our ViT model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "x0vXDi8fQaQV",
        "outputId": "0fe0631d-b2e0-4fc6-a658-94eef237e71c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'test_loss': 0.06418210905976593,\n",
              " 'test_acc': 0.984659090909091,\n",
              " 'number_of_parameters': 85800963,\n",
              " 'model_size (MB)': 327}"
            ]
          },
          "execution_count": 29,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Create ViT statistics dictionary\n",
        "vit_stats = {\"test_loss\": vit_results[\"test_loss\"][-1],\n",
        "             \"test_acc\": vit_results[\"test_acc\"][-1],\n",
        "             \"number_of_parameters\": vit_total_params,\n",
        "             \"model_size (MB)\": pretrained_vit_model_size}\n",
        "\n",
        "vit_stats"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KEb0632sQaQV"
      },
      "source": [
        "Nice! Looks like our ViT model achieves over 95% accuracy too."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VIX_LUuNQaQX"
      },
      "source": [
        "## 5. Making predictions with our trained models and timing them\n",
        "\n",
        "We've got a couple of trained models, both performing pretty well.\n",
        "\n",
        "Now how about we test them out doing what we'd like them to do?\n",
        "\n",
        "As in, let's see how they go making predictions (performing inference).\n",
        "\n",
        "We know both of our models are performing at over 95% accuracy on the test dataset, but how fast are they?\n",
        "\n",
        "Ideally, if we're deploying our FoodVision Mini model to a mobile device so people can take photos of their food and identify it, we'd like the predictions to happen at real-time (~30 frames per second).\n",
        "\n",
        "That's why our second criteria is: a fast model.\n",
        "\n",
        "To find out how long each of our models take to performance inference, let's create a function called `pred_and_store()` to iterate over each of the test dataset images one by one and perform a prediction.\n",
        "\n",
        "We'll time each of the predictions as well as store the results in a common prediction format: a list of dictionaries (where each element in the list is a single prediction and each sinlge prediction is a dictionary).\n",
        "\n",
        "> **Note:** We time the predictions one by one rather than by batch because when our model is deployed, it will likely only be making a prediction on one image at a time. As in, someone takes a photo and our model predicts on that single image.\n",
        "\n",
        "Since we'd like to make predictions across all the images in the test set, let's first get a list of all of the test image paths so we can iterate over them.\n",
        "\n",
        "To do so, we'll use Python's [`pathlib.Path(\"target_dir\").glob(\"*/*.jpg\"))`](https://docs.python.org/3/library/pathlib.html#basic-use) to find all of the filepaths in a target directory with the extension `.jpg` (all of our test images)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ryNK3ZMWQaQX",
        "outputId": "c824a72f-8b52-4226-8e15-59a969599073"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[INFO] Finding all filepaths ending with '.jpg' in directory: data/pizza_steak_sushi_20_percent/test\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "[PosixPath('data/pizza_steak_sushi_20_percent/test/steak/831681.jpg'),\n",
              " PosixPath('data/pizza_steak_sushi_20_percent/test/steak/3100563.jpg'),\n",
              " PosixPath('data/pizza_steak_sushi_20_percent/test/steak/2752603.jpg'),\n",
              " PosixPath('data/pizza_steak_sushi_20_percent/test/steak/39461.jpg'),\n",
              " PosixPath('data/pizza_steak_sushi_20_percent/test/steak/730464.jpg')]"
            ]
          },
          "execution_count": 30,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from pathlib import Path\n",
        "\n",
        "# Get all test data paths\n",
        "print(f\"[INFO] Finding all filepaths ending with '.jpg' in directory: {test_dir}\")\n",
        "test_data_paths = list(Path(test_dir).glob(\"*/*.jpg\"))\n",
        "test_data_paths[:5]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VxkL8KHtQaQX"
      },
      "source": [
        "### 5.1 Creating a function to make predictions across the test dataset\n",
        "\n",
        "Now we've got a list of our test image paths, let's get to work on our `pred_and_store()` function:\n",
        "\n",
        "1. Create a function that takes a list of paths, a trained PyTorch model, a series of transforms (to prepare images), a list of target class names and a target device.\n",
        "2. Create an empty list to store prediction dictionaries (we want the function to return a list of dictionaries, one for each prediction).\n",
        "3. Loop through the target input paths (steps 4-14 will happen inside the loop).\n",
        "4. Create an empty dictionary for each iteration in the loop to store prediction values per sample.\n",
        "5. Get the sample path and ground truth class name (we can do this by infering the class from the path).\n",
        "6. Start the prediction timer using Python's [`timeit.default_timer()`](https://docs.python.org/3/library/timeit.html#timeit.default_timer).\n",
        "7. Open the image using [`PIL.Image.open(path)`](https://pillow.readthedocs.io/en/stable/reference/Image.html#functions).\n",
        "8. Transform the image so it's capable of being used with the target model as well as add a batch dimension and send the image to the target device.\n",
        "9. Prepare the model for inference by sending it to the target device and turning on `eval()` mode.\n",
        "10. Turn on [`torch.inference_mode()`](https://pytorch.org/docs/stable/generated/torch.inference_mode.html) and pass the target transformed image to the model and calculate the prediction probability using `torch.softmax()` and the target label using `torch.argmax()`.\n",
        "11. Add the prediction probability and prediction class to the prediction dictionary created in step 4. Also make sure the prediction probability is on the CPU so it can be used with non-GPU libraries such as NumPy and pandas for later inspection.\n",
        "12. End the prediction timer started in step 6 and add the time to the prediction dictionary created in step 4.\n",
        "13. See if the predicted class matches the ground truth class from step 5 and add the result to the prediction dictionary created in step 4.\n",
        "14. Append the updated prediction dictionary to the empty list of predictions created in step 2.\n",
        "15. Return the list of prediction dictionaries.\n",
        "\n",
        "A bunch of steps, but nothing we can't handle!\n",
        "\n",
        "Let's do it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YVDF5yqzQaQY"
      },
      "outputs": [],
      "source": [
        "import pathlib\n",
        "import torch\n",
        "\n",
        "from PIL import Image\n",
        "from timeit import default_timer as timer\n",
        "from tqdm.auto import tqdm\n",
        "from typing import List, Dict\n",
        "\n",
        "# 1. Create a function to return a list of dictionaries with sample, truth label, prediction, prediction probability and prediction time\n",
        "def pred_and_store(paths: List[pathlib.Path],\n",
        "                   model: torch.nn.Module,\n",
        "                   transform: torchvision.transforms,\n",
        "                   class_names: List[str],\n",
        "                   device: str = \"cuda\" if torch.cuda.is_available() else \"cpu\") -> List[Dict]:\n",
        "\n",
        "    # 2. Create an empty list to store prediction dictionaires\n",
        "    pred_list = []\n",
        "\n",
        "    # 3. Loop through target paths\n",
        "    for path in tqdm(paths):\n",
        "\n",
        "        # 4. Create empty dictionary to store prediction information for each sample\n",
        "        pred_dict = {}\n",
        "\n",
        "        # 5. Get the sample path and ground truth class name\n",
        "        pred_dict[\"image_path\"] = path\n",
        "        class_name = path.parent.stem\n",
        "        pred_dict[\"class_name\"] = class_name\n",
        "\n",
        "        # 6. Start the prediction timer\n",
        "        start_time = timer()\n",
        "\n",
        "        # 7. Open image path\n",
        "        img = Image.open(path)\n",
        "\n",
        "        # 8. Transform the image, add batch dimension and put image on target device\n",
        "        transformed_image = transform(img).unsqueeze(0).to(device)\n",
        "\n",
        "        # 9. Prepare model for inference by sending it to target device and turning on eval() mode\n",
        "        model.to(device)\n",
        "        model.eval()\n",
        "\n",
        "        # 10. Get prediction probability, predicition label and prediction class\n",
        "        with torch.inference_mode():\n",
        "            pred_logit = model(transformed_image) # perform inference on target sample\n",
        "            pred_prob = torch.softmax(pred_logit, dim=1) # turn logits into prediction probabilities\n",
        "            pred_label = torch.argmax(pred_prob, dim=1) # turn prediction probabilities into prediction label\n",
        "            pred_class = class_names[pred_label.cpu()] # hardcode prediction class to be on CPU\n",
        "\n",
        "            # 11. Make sure things in the dictionary are on CPU (required for inspecting predictions later on)\n",
        "            pred_dict[\"pred_prob\"] = round(pred_prob.unsqueeze(0).max().cpu().item(), 4)\n",
        "            pred_dict[\"pred_class\"] = pred_class\n",
        "\n",
        "            # 12. End the timer and calculate time per pred\n",
        "            end_time = timer()\n",
        "            pred_dict[\"time_for_pred\"] = round(end_time-start_time, 4)\n",
        "\n",
        "        # 13. Does the pred match the true label?\n",
        "        pred_dict[\"correct\"] = class_name == pred_class\n",
        "\n",
        "        # 14. Add the dictionary to the list of preds\n",
        "        pred_list.append(pred_dict)\n",
        "\n",
        "    # 15. Return list of prediction dictionaries\n",
        "    return pred_list"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a5xrKALnQaQY"
      },
      "source": [
        "Ho, ho!\n",
        "\n",
        "What a good looking function!\n",
        "\n",
        "And you know what, since our `pred_and_store()` is a pretty good utility function for making and storing predictions, it could be stored to [`going_modular.going_modular.predictions.py`](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/going_modular/going_modular/predictions.py) for later use. That might be an extension you'd like to try, check out [05. PyTorch Going Modular](https://www.learnpytorch.io/05_pytorch_going_modular/) for ideas."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BohuUBrcQaQY"
      },
      "source": [
        "### 5.2 Making and timing predictions with EffNetB2\n",
        "\n",
        "Time to test out our `pred_and_store()` function!\n",
        "\n",
        "Let's start by using it to make predictions across the test dataset with our EffNetB2 model, paying attention to two details:\n",
        "\n",
        "1. **Device** - We'll hard code the `device` parameter to use `\"cpu\"` because when we deploy our model, we won't always have access to a `\"cuda\"` (GPU) device.\n",
        "    * Making the predictions on CPU will be a good indicator of speed of inference too because generally predictions on CPU devices are slower than GPU devices.\n",
        "2. **Transforms** - We'll also be sure to set the `transform` parameter to `effnetb2_transforms` to make sure the images are opened and transformed in the same way our `effnetb2` model has been trained on."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QJsK_LVrQaQY",
        "outputId": "9f345cdd-045b-4aa0-81ca-1e89ba3e26ef",
        "colab": {
          "referenced_widgets": [
            "9b516a8ba5ce4603a25ae0b6d5f8573a"
          ]
        }
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "9b516a8ba5ce4603a25ae0b6d5f8573a",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/150 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Make predictions across test dataset with EffNetB2\n",
        "effnetb2_test_pred_dicts = pred_and_store(paths=test_data_paths,\n",
        "                                          model=effnetb2,\n",
        "                                          transform=effnetb2_transforms,\n",
        "                                          class_names=class_names,\n",
        "                                          device=\"cpu\") # make predictions on CPU"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-dgxFqGrQaQZ"
      },
      "source": [
        "Nice! Look at those predictions fly!\n",
        "\n",
        "Let's inspect the first couple and see what they look like."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FAfxrd8gQaQZ",
        "outputId": "c481bd88-23d1-4243-fda1-adef45ccd1be"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'image_path': PosixPath('data/pizza_steak_sushi_20_percent/test/steak/831681.jpg'),\n",
              "  'class_name': 'steak',\n",
              "  'pred_prob': 0.9293,\n",
              "  'pred_class': 'steak',\n",
              "  'time_for_pred': 0.0494,\n",
              "  'correct': True},\n",
              " {'image_path': PosixPath('data/pizza_steak_sushi_20_percent/test/steak/3100563.jpg'),\n",
              "  'class_name': 'steak',\n",
              "  'pred_prob': 0.9534,\n",
              "  'pred_class': 'steak',\n",
              "  'time_for_pred': 0.0264,\n",
              "  'correct': True}]"
            ]
          },
          "execution_count": 33,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Inspect the first 2 prediction dictionaries\n",
        "effnetb2_test_pred_dicts[:2]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "i8re7AT1QaQZ"
      },
      "source": [
        "Woohoo!\n",
        "\n",
        "It looks like our `pred_and_store()` function worked nicely.\n",
        "\n",
        "Thanks to our list of dictionaries data structure, we've got plenty of useful information we can further inspect.\n",
        "\n",
        "To do so, let's turn our list of dictionaries into a pandas DataFrame."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "O3vWhxRNQaQZ",
        "outputId": "9091fefe-4dde-404b-fddf-aa8b70926052"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>image_path</th>\n",
              "      <th>class_name</th>\n",
              "      <th>pred_prob</th>\n",
              "      <th>pred_class</th>\n",
              "      <th>time_for_pred</th>\n",
              "      <th>correct</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>data/pizza_steak_sushi_20_percent/test/steak/8...</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.9293</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.0494</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>data/pizza_steak_sushi_20_percent/test/steak/3...</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.9534</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.0264</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>data/pizza_steak_sushi_20_percent/test/steak/2...</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.7532</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.0256</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>data/pizza_steak_sushi_20_percent/test/steak/3...</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.5935</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.0263</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>data/pizza_steak_sushi_20_percent/test/steak/7...</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.8959</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.0269</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                          image_path class_name  pred_prob  \\\n",
              "0  data/pizza_steak_sushi_20_percent/test/steak/8...      steak     0.9293   \n",
              "1  data/pizza_steak_sushi_20_percent/test/steak/3...      steak     0.9534   \n",
              "2  data/pizza_steak_sushi_20_percent/test/steak/2...      steak     0.7532   \n",
              "3  data/pizza_steak_sushi_20_percent/test/steak/3...      steak     0.5935   \n",
              "4  data/pizza_steak_sushi_20_percent/test/steak/7...      steak     0.8959   \n",
              "\n",
              "  pred_class  time_for_pred  correct  \n",
              "0      steak         0.0494     True  \n",
              "1      steak         0.0264     True  \n",
              "2      steak         0.0256     True  \n",
              "3      steak         0.0263     True  \n",
              "4      steak         0.0269     True  "
            ]
          },
          "execution_count": 34,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Turn the test_pred_dicts into a DataFrame\n",
        "import pandas as pd\n",
        "effnetb2_test_pred_df = pd.DataFrame(effnetb2_test_pred_dicts)\n",
        "effnetb2_test_pred_df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7YI4XAqjQaQZ"
      },
      "source": [
        "Beautiful!\n",
        "\n",
        "Look how easily those prediction dictionaries turn into a structured format we can perform analysis on.\n",
        "\n",
        "Such as finding how many predictions our EffNetB2 model got wrong..."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8SlNb2yNQaQZ",
        "outputId": "082ba875-21ca-4528-aed0-309cf4b5a4ff"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "True     145\n",
              "False      5\n",
              "Name: correct, dtype: int64"
            ]
          },
          "execution_count": 35,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Check number of correct predictions\n",
        "effnetb2_test_pred_df.correct.value_counts()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JCXiZvGmQaQa"
      },
      "source": [
        "Five wrong predictions out of 150 total, not bad!\n",
        "\n",
        "And how about the average prediction time?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "THl6lsHCQaQa",
        "outputId": "f921b5af-9606-4f1e-f828-c3ecd5a6743f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "EffNetB2 average time per prediction: 0.0269 seconds\n"
          ]
        }
      ],
      "source": [
        "# Find the average time per prediction\n",
        "effnetb2_average_time_per_pred = round(effnetb2_test_pred_df.time_for_pred.mean(), 4)\n",
        "print(f\"EffNetB2 average time per prediction: {effnetb2_average_time_per_pred} seconds\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "drNp7EZUQaQa"
      },
      "source": [
        "Hmm, how does that average prediction time live up to our criteria of our model performing at real-time (~30FPS or 0.03 seconds per prediction)?\n",
        "\n",
        "> **Note:** Prediction times will be different across different hardware types (e.g. a local Intel i9 vs Google Colab CPU). The better and faster the hardware, generally, the faster the prediction. For example, on my local deep learning PC with an Intel i9 chip, my average prediction time with EffNetB2 is around 0.031 seconds (just under real-time). However, on Google Colab (I'm not sure what CPU hardware Colab uses but it looks like it might be an [Intel(R) Xeon(R)](https://stackoverflow.com/questions/47805170/whats-the-hardware-spec-for-google-colaboratory)), my average prediction time with EffNetB2 is about 0.1396 seconds (3-4x slower).\n",
        "\n",
        "Let's add our EffNetB2 average time per prediction to our `effnetb2_stats` dictionary."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-nUJsna6QaQa",
        "outputId": "71b105cb-549a-43b9-9395-3d29b76c3ff1"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'test_loss': 0.28128674924373626,\n",
              " 'test_acc': 0.96875,\n",
              " 'number_of_parameters': 7705221,\n",
              " 'model_size (MB)': 29,\n",
              " 'time_per_pred_cpu': 0.0269}"
            ]
          },
          "execution_count": 37,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Add EffNetB2 average prediction time to stats dictionary\n",
        "effnetb2_stats[\"time_per_pred_cpu\"] = effnetb2_average_time_per_pred\n",
        "effnetb2_stats"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cNmsdQxiQaQa"
      },
      "source": [
        "### 5.3 Making and timing predictions with ViT\n",
        "\n",
        "We've made predictions with our EffNetB2 model, now let's do the same for our ViT model.\n",
        "\n",
        "To do so, we can use the `pred_and_store()` function we created above except this time we'll pass in our `vit` model as well as the `vit_transforms`.\n",
        "\n",
        "And we'll keep the predictions on the CPU via `device=\"cpu\"` (a natural extension here would be to test the prediction times on CPU and on GPU)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9ody3FKLQaQb",
        "outputId": "4b2bf34e-9257-42c4-96c0-12e4c25d75b6",
        "colab": {
          "referenced_widgets": [
            "2c6e2d8224d84f1c9158c60fe3f7fd9f"
          ]
        }
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "2c6e2d8224d84f1c9158c60fe3f7fd9f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/150 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Make list of prediction dictionaries with ViT feature extractor model on test images\n",
        "vit_test_pred_dicts = pred_and_store(paths=test_data_paths,\n",
        "                                     model=vit,\n",
        "                                     transform=vit_transforms,\n",
        "                                     class_names=class_names,\n",
        "                                     device=\"cpu\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RAna6hklQaQb"
      },
      "source": [
        "Predictions made!\n",
        "\n",
        "Now let's check out the first couple."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HQOq-cF3QaQb",
        "outputId": "fbacbddf-641f-4179-9bb0-486ab60f7b16"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[{'image_path': PosixPath('data/pizza_steak_sushi_20_percent/test/steak/831681.jpg'),\n",
              "  'class_name': 'steak',\n",
              "  'pred_prob': 0.9933,\n",
              "  'pred_class': 'steak',\n",
              "  'time_for_pred': 0.1313,\n",
              "  'correct': True},\n",
              " {'image_path': PosixPath('data/pizza_steak_sushi_20_percent/test/steak/3100563.jpg'),\n",
              "  'class_name': 'steak',\n",
              "  'pred_prob': 0.9893,\n",
              "  'pred_class': 'steak',\n",
              "  'time_for_pred': 0.0638,\n",
              "  'correct': True}]"
            ]
          },
          "execution_count": 39,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Check the first couple of ViT predictions on the test dataset\n",
        "vit_test_pred_dicts[:2]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "udpy-g9OQaQb"
      },
      "source": [
        "Wonderful!\n",
        "\n",
        "And just like before, since our ViT model's predictions are in the form of a list of dictionaries, we can easily turn them into a pandas DataFrame for further inspection."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qtnlNYg8QaQb",
        "outputId": "70393d84-f4ac-4745-d261-7f5e73ad2e9c"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>image_path</th>\n",
              "      <th>class_name</th>\n",
              "      <th>pred_prob</th>\n",
              "      <th>pred_class</th>\n",
              "      <th>time_for_pred</th>\n",
              "      <th>correct</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>data/pizza_steak_sushi_20_percent/test/steak/8...</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.9933</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.1313</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>data/pizza_steak_sushi_20_percent/test/steak/3...</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.9893</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.0638</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>data/pizza_steak_sushi_20_percent/test/steak/2...</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.9971</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.0627</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>data/pizza_steak_sushi_20_percent/test/steak/3...</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.7685</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.0632</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>data/pizza_steak_sushi_20_percent/test/steak/7...</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.9499</td>\n",
              "      <td>steak</td>\n",
              "      <td>0.0641</td>\n",
              "      <td>True</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                          image_path class_name  pred_prob  \\\n",
              "0  data/pizza_steak_sushi_20_percent/test/steak/8...      steak     0.9933   \n",
              "1  data/pizza_steak_sushi_20_percent/test/steak/3...      steak     0.9893   \n",
              "2  data/pizza_steak_sushi_20_percent/test/steak/2...      steak     0.9971   \n",
              "3  data/pizza_steak_sushi_20_percent/test/steak/3...      steak     0.7685   \n",
              "4  data/pizza_steak_sushi_20_percent/test/steak/7...      steak     0.9499   \n",
              "\n",
              "  pred_class  time_for_pred  correct  \n",
              "0      steak         0.1313     True  \n",
              "1      steak         0.0638     True  \n",
              "2      steak         0.0627     True  \n",
              "3      steak         0.0632     True  \n",
              "4      steak         0.0641     True  "
            ]
          },
          "execution_count": 40,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Turn vit_test_pred_dicts into a DataFrame\n",
        "import pandas as pd\n",
        "vit_test_pred_df = pd.DataFrame(vit_test_pred_dicts)\n",
        "vit_test_pred_df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NiXr8pVkQaQb"
      },
      "source": [
        "How many predictions did our ViT model get correct?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gmQzMVLvQaQb",
        "outputId": "3aec293c-9bc7-4327-8333-f0c17db6d430"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "True     148\n",
              "False      2\n",
              "Name: correct, dtype: int64"
            ]
          },
          "execution_count": 41,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Count the number of correct predictions\n",
        "vit_test_pred_df.correct.value_counts()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XnEBjt7GQaQb"
      },
      "source": [
        "Woah!\n",
        "\n",
        "Our ViT model did a little better than our EffNetB2 model in terms of correct predictions, only two samples wrong across the whole test dataset.\n",
        "\n",
        "As an extension you might want to visualize the ViT model's wrong predictions and see if there's any reason why it might've got them wrong.\n",
        "\n",
        "How about we calculate how long the ViT model took per prediction?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZHVW-rU5QaQc",
        "outputId": "71ab8bf8-9a93-4116-dd3d-5869b3ae9f17"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "ViT average time per prediction: 0.0641 seconds\n"
          ]
        }
      ],
      "source": [
        "# Calculate average time per prediction for ViT model\n",
        "vit_average_time_per_pred = round(vit_test_pred_df.time_for_pred.mean(), 4)\n",
        "print(f\"ViT average time per prediction: {vit_average_time_per_pred} seconds\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b2Mm2vGuQaQc"
      },
      "source": [
        "Well, that looks a little slower than our EffNetB2 model's average time per prediction but how does it look in terms of our second criteria: speed?\n",
        "\n",
        "For now, let's add the value to our `vit_stats` dictionary so we can compare it to our EffNetB2 model's stats.\n",
        "\n",
        "> **Note:** The average time per prediction values will be highly dependent on the hardware you make them on. For example, for the ViT model, my average time per prediction (on the CPU) was 0.0693-0.0777 seconds on my local deep learning PC with an Intel i9 CPU. Where as on Google Colab, my average time per prediction with the ViT model was 0.6766-0.7113 seconds."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2XZOKi3WQaQc",
        "outputId": "f9e2d643-fe90-4ce7-fd04-a2c2e28c79f0"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'test_loss': 0.06418210905976593,\n",
              " 'test_acc': 0.984659090909091,\n",
              " 'number_of_parameters': 85800963,\n",
              " 'model_size (MB)': 327,\n",
              " 'time_per_pred_cpu': 0.0641}"
            ]
          },
          "execution_count": 43,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Add average prediction time for ViT model on CPU\n",
        "vit_stats[\"time_per_pred_cpu\"] = vit_average_time_per_pred\n",
        "vit_stats"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R45OZP1EQaQc"
      },
      "source": [
        "## 6. Comparing model results, prediction times and size\n",
        "\n",
        "Our two best model contenders have been trained and evaluated.\n",
        "\n",
        "Now let's put them head to head and compare across their different statistics.\n",
        "\n",
        "To do so, let's turn our `effnetb2_stats` and `vit_stats` dictionaries into a pandas DataFrame.\n",
        "\n",
        "We'll add a column to view the model names as well as the convert the test accuracy to a whole percentage rather than decimal."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DhYJOpaiQaQc",
        "outputId": "ca36c6d7-af56-43c6-b4e7-51e5722def56"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>test_loss</th>\n",
              "      <th>test_acc</th>\n",
              "      <th>number_of_parameters</th>\n",
              "      <th>model_size (MB)</th>\n",
              "      <th>time_per_pred_cpu</th>\n",
              "      <th>model</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.281287</td>\n",
              "      <td>96.88</td>\n",
              "      <td>7705221</td>\n",
              "      <td>29</td>\n",
              "      <td>0.0269</td>\n",
              "      <td>EffNetB2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.064182</td>\n",
              "      <td>98.47</td>\n",
              "      <td>85800963</td>\n",
              "      <td>327</td>\n",
              "      <td>0.0641</td>\n",
              "      <td>ViT</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   test_loss  test_acc  number_of_parameters  model_size (MB)  \\\n",
              "0   0.281287     96.88               7705221               29   \n",
              "1   0.064182     98.47              85800963              327   \n",
              "\n",
              "   time_per_pred_cpu     model  \n",
              "0             0.0269  EffNetB2  \n",
              "1             0.0641       ViT  "
            ]
          },
          "execution_count": 44,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Turn stat dictionaries into DataFrame\n",
        "df = pd.DataFrame([effnetb2_stats, vit_stats])\n",
        "\n",
        "# Add column for model names\n",
        "df[\"model\"] = [\"EffNetB2\", \"ViT\"]\n",
        "\n",
        "# Convert accuracy to percentages\n",
        "df[\"test_acc\"] = round(df[\"test_acc\"] * 100, 2)\n",
        "\n",
        "df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EL9YXSAYQaQc"
      },
      "source": [
        "Wonderful!\n",
        "\n",
        "It seems our models are quite close in terms of overall test accuracy but how do they look across the other fields?\n",
        "\n",
        "One way to find out would be to divide the ViT model statistics by the EffNetB2 model statistics to find out the different ratios between the models.\n",
        "\n",
        "Let's create another DataFrame to do so."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uTvMkuVTQaQc",
        "outputId": "a5f2d29e-b723-4b21-f854-86812827a71e"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>test_loss</th>\n",
              "      <th>test_acc</th>\n",
              "      <th>number_of_parameters</th>\n",
              "      <th>model_size (MB)</th>\n",
              "      <th>time_per_pred_cpu</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>ViT to EffNetB2 ratios</th>\n",
              "      <td>0.228173</td>\n",
              "      <td>1.016412</td>\n",
              "      <td>11.135432</td>\n",
              "      <td>11.275862</td>\n",
              "      <td>2.3829</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                        test_loss  test_acc  number_of_parameters  \\\n",
              "ViT to EffNetB2 ratios   0.228173  1.016412             11.135432   \n",
              "\n",
              "                        model_size (MB)  time_per_pred_cpu  \n",
              "ViT to EffNetB2 ratios        11.275862             2.3829  "
            ]
          },
          "execution_count": 45,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Compare ViT to EffNetB2 across different characteristics\n",
        "pd.DataFrame(data=(df.set_index(\"model\").loc[\"ViT\"] / df.set_index(\"model\").loc[\"EffNetB2\"]), # divide ViT statistics by EffNetB2 statistics\n",
        "             columns=[\"ViT to EffNetB2 ratios\"]).T"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GElZkePGQaQc"
      },
      "source": [
        "It seems our ViT model outperforms the EffNetB2 model across the performance metrics (test loss, where lower is better and test accuracy, where higher is better) but at the expense of having:\n",
        "* 11x+ the number of parameters.\n",
        "* 11x+ the model size.\n",
        "* 2.5x+ the prediction time per image.\n",
        "\n",
        "Are these tradeoffs worth it?\n",
        "\n",
        "Perhaps if we had unlimited compute power but for our use case of deploying the FoodVision Mini model to a smaller device (e.g. a mobile phone), we'd likely start out with the EffNetB2 model for faster predictions at a slightly reduced performance but dramatically smaller size."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oG_-UKWSQaQc"
      },
      "source": [
        "### 6.1 Visualizing the speed vs. performance tradeoff\n",
        "\n",
        "We've seen that our ViT model outperforms our EffNetB2 model in terms of performance metrics such as test loss and test accuracy.\n",
        "\n",
        "However, our EffNetB2 model performs predictions faster and has a much smaller model size.\n",
        "\n",
        "> **Note:** Performance or inference time is also often referred to as \"latency\".\n",
        "\n",
        "How about we make this fact visual?\n",
        "\n",
        "We can do so by creating a plot with matplotlib:\n",
        "1. Create a scatter plot from the comparison DataFrame to compare EffNetB2 and ViT `time_per_pred_cpu` and `test_acc` values.\n",
        "2. Add titles and labels respective of the data and customize the fontsize for aesthetics.\n",
        "3. Annotate the samples on the scatter plot from step 1 with their appropriate labels (the model names).\n",
        "4. Create a legend based on the model sizes (`model_size (MB)`)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "r7yuqfNvQaQd",
        "outputId": "6c5c33c3-f233-404c-d2a8-536d606d00e4"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuEAAAH7CAYAAABi5UHyAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABa4klEQVR4nO3deZgcVfmw4efNRkKGQDAQ9oR9X0LizpKwIwgBFEF2URQ+F1RwAcSIiAv4wwUVUBACkYAgICAGgYRdJcEQkQASSCAsYY+ZbGQ53x9VEzqdnpmemZ6azOS5r6uvma46derU6eqZt0+/dSpSSkiSJEkqTreOboAkSZK0qjEIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhUo1FxPCISBFxYkfUERETImJ6a/e9MsiP/ao2bD8qr2Nw7VrV5P5Wj4hfRMQLEbGks/e/mtcV3me1EBG7RMQ9EfF2/p4b1dFtkjoLg3B1WiWBamOPD3V0GwEi4o95e3ZpokxExPMR8U5E9Cmwee2iJAhOEfGJRsocVlJmVMFNLG3HVXkbBrShmm8CXwKuB04ETq9B07qciNgsIi6PiKciYl4euD0ZEVdHxIiObl9n1sjfw/qImBQRX4mI7u2wzx7ATcCWwHeA44A/1Xo/UlfVo6MbINXAdcBfKix/tuiGNOIK4BPAScBXGikzAhgMXJZSmh8R9wN9gEWt2N9+QLRiu/awgOy4b6yw7jP5+t4V1vUBlrRhv+cDPwIWtqGOltgX+HdK6cyC9tfpRMQw4D6yc3o08B+y13kr4OPAHGB8hzWw62j4exjABmQfCn8GbA+cUuN9bZY/vp5SuqTGdUtdnkG4uoLHUkrXdnQjmnAX8CJwTEScmVJ6t0KZk/KfVwCklJaSBagt1kj9HeVm4MiI2CCl9HLDwohYDzgAuAH4dPlGKaVWHXvJ9ouBxW2po4XWA16odaURsUZKaU6t6+0g3wVWB4aklCaXroiIL5L1odpuub+HEfEbYCrw2Yj4TkppVlt3UHJeNrxmb7W1zrL6A+ibUqqvZb3SysZ0FK0SImKPiPhbRMyOiPkR8VhEnFyDsodGxL8iYkFEvBgR5wE9S8vkAfVVwPuAQyrU0Q84HHgipfRovmyFnPA8ZeX0iJgSEXMi4n8R8XREXBERPUvKVcxVrfa4GraPiA0i4ro8ZWBuRIyLiK0q9UMTrgWWkn1NXep4IOXrVxAVcsIblkXEhyPivrxNb0TE7yKirqxsm3LCS7bfOiIuiIiZEbEwIh6PiI+VlDsxIhKwKbBnpfSaiBgWETfnbV2Yv2Zn51/ll+6zod83i4gbI+It4H8l69ePiN9Elnf+bkS8nKd2rNuatpdtc0REjI8sHWpe3sZfRESvkjIREadGlt4wLz8Hx0f1aSRbAm+WB+CQvUdKP6Tl+2t4vfeJiL/n+3w1In4eEX0rHMOaEfHjiHg2P97X8/N3swplV4uIsyLiP/l7952IuC0ihlQo2z8ifpu/fnPz12loNQccEWvl9VdM0YiIH0ZJqlpErB0RF0fEtHy7N/P+bvU3LCml/wGPkI2ML+uLiPhURDyYv47zIuIfUSF1rOR12DsvXw/cFhETyL7ZAPh9ybk/ON+ub3580/LX49WIGB0Rg8rqX/a3LiL+X0Q8STYAcUZEDG54P0XEkRExObK/Xc9GxEn59ps0vF/yY7k2ItYo28c2EfHr/PVuON5JEfG5Cse7sr5/1AU5Eq6uYPVYMZ93YcMIYkR8nGxE9lXgp2Rfex8F/C4iNkspnd2wUQvLHkaWDzkdOI9s5PUk4OAKbfw9cA6VUzOOIhshvKKZ4zwn389twKVk6RqbkgX2q9FE6kpLjivXF7gf+DtwVr6frwC3RsQOKaVqU0VeA+4gO+4flyw/CbgdeL3Kehrskm/3e+APwHDgZLJAv9ZftQNcTdavFwG9yHK9b4mIrVJK08n66DjgYuAN4Af5dlMA8n/aN5OlRv2UbMTww2Sv4y7AJ8v2V0cW2DwEnA2sm9ezCVkg1YvsPJkGbAGcCoyIiGEppdktbDt53T8ge42fzI/jFWBz4AjgXKDhm5VrgKPJzt/fk51zxwB/i4jDU0p/bqYvpwFb52WrzRvelSyV67dkKSwjgC8DO0TEvvkHXCJiTeBhYBPgSrJUl/WB04B/5P0zIy/bE/gr8JH8mC4B1gQ+BzwUEXuklCaWlB0HvD8v+3ey1+1u4M3mGp9Seici/gwcGhFrp5SWjRhHRDey/ptS8sHkj8AewGXA42R/F7YhO88vrLLPlhMRQXauQHaOEhHnk51ffyXL5V4KHAb8MSK+mFL6VVk1w8jOh9+SnVeQ/d17iOzcuRx4IF/+emQfMMcBHyU7X35K9iHsVGC//PWYWbaP08kGKn5L9nfqxZJ1BwNfAH5N9h46GbgyIt4FLgDuzdvxft5Lc/tsyfbDyfr1duB5sr9vnwQuj4gBKaUfVui6le39o64opeTDR6d8kP1hTY08xuZlugMzgHeADUq27UX2D2QJsGUry75A9k9tQEnZNfM6EnBiWXvvIQvUNyhb/ghZ7vKACsd2Ysmyx4Anq+iXCcD0kudVH1fJ9gn4Rlm9Z+bL96+iDaPyssPI8n0T8JF83Ufy5wfn6xMwqmz7BFxVYdlS4ENly+8g+2dZV2H/g6to61V52QEVtr8diJLl78+X/7CsjunAhLJlvcmCifuBHmXrvprXM7xCv59foY23kn2g2ahs+bD8nBrVmrYDH8iX3Qv0Lqs7GrYnC9AScEpZmR7ARLLAJsrbXVb2w2QBSQKeIQuWTwW2baR8w3t5ZNnyn+fLjypbNh/YuazsILJvE64qWdbQ9/uXle1H9p6eULLslLzs98rKnp4vn97UMedlD8rLnla2fO98+dfy52vmz3/dXJ2N7Gd4vv25wABgHWAnsqA2AY/k5XbNn19QoY5b8v5ao8LrsE8T+yz/W/e5fPlPGumLayrU8Rawbln5wfm6ucCgkuXrkAXaSxv6r2Tdn/LzrPTvQd8Kbe9G9p6bDfRc2d8/Prrmw3QUdQWXk10YV/o4P183lHx0LJV83Z2yvOkLyf4QH9rKshsDv08pvVFSdjbZKHUlV5AFxMtSMyJiG+BDwJ9L62nEbGDDiNitmXLlWnJcDZYCvyhbdm/+c8sW7v8vZMFoQ977SWSjRXe2sB7IAom/V2hXD7J/2LX285Sy/5YAKUsXmkN1fbAvMJBs1GutiBjQ8OC9C4n3q7DdRaVP8lHeg4E/AwvK6plONspeqZ5q2n5M/vPbqSwPP+Xyp8fm295Stv+1yL6ZGUwzfZJSeoTsXLyaLOA8iWxk88mIeCAqpI0AT6eUbilb9qP852GwbKT3GLIPOy+VtW8u2eh1af8cCzwFTCor2wv4G7BbvDdD0UiyD6k/LWvDbyhJFWrGOGAWWQpWqePzusfkz+eTfRj/YLRtas3vkX3D9BrZaPpnyM6dkfn6Y8gCwqtLjz/vgz8Da5B9YCr1eErp7ha04TCyvyHLjTCnlO4AJpN9M1Aef4xOKb3WSH23pPybjLye14Gn832Uj9o/QJYSOLik/NyG3yOid0S8D1ib7HqdfmTfNpRbqd4/6ppMR1FX8N8m/kFsmv/8T4V1T+Q/N2tF2YafT1Uo+2QjbfkT2Wh0aWrGZ/KfVzayTamzyEaqHoiIl8lGce4AbkxNX4zZkuNq8HL5PxXe+/r9fVW0dZmU0pKIuAb4QkScBXwK+E2+vCVVATxXYVmr2tWG/b1V5b62zX829doOLHv+ekrpnbJlW5N9UDo5f1RSqZ3VtH1LsoDs8SbaCNmxrEEWTDZmINkId6NSSv8mm62DyHKD9yRLG9idLNVpaNm5PLVCHa9ExDu8d86uQ3ZM+9F4etPSsmPp00RZyEaSX8z38UrK8qpL27AwIp4D+jdRR0PZxRHxB+CreSrDM5HltB8O/DXlF0qmlN6NiNPJRvWfjyw3+l6yAPSe5vZT4nKytJaGEeRnUkkaDNnxB5X/djUoPy+bfF0r2JTsb8jbFdb9hyylZwDZB4Vq9lHpXH6b7LUpnwGpYZ/LzvPIrhkZBRxJNnhSrtLruNK9f9T1GISrq2tJlNeasqmJdctJKS3I/xmfFhEfAf5BNio+k2xEpkkppUciYnNgf7Lc2BFkM4ucExG7lf2jbbY9zWgq57s19V1Jls4yhuyfUTUfOiqpdbtau79q9tVQ5kyy0b9KXi57Pq+Jeq7lvXzccvMrLKum7UHlc7jSNq9TYSabEk80sW4F+cjm6PwD2gNk+cMfAB4sLdZEe8p/v5vlrztoTAD/Br7WRJnXS8pW04bmXE2WBnM82bUdh5Pl/48uLZRSujQibiVL29iTLB/+ixFxfUrpqCr31dSgREO7E3AgjZ8j5R/YK52XTWnNe7GpfTTWzmr/HvyB7Nuky8m+MXmLLI3rY2SvS6WsgJX6/aOuwSBcXd20/Of2FdZtl/98rg1lt61QttKyBleQXSx2EtnXoesBP0hVXuiYsim7bsofRMRpZF/HnkzjF2615LjaRUrpqYh4hCxF4+GU0tPtub+VxH/zn3Nb+FV+uWfJ/tH3amM9lTxNNlXkTsA/myj3X7L5vP+eajxtXEopRcQ/yILwDctWb1dePiLWJ0tnaThnXyf7hqlflf3zX7LR83tTfmFnE6aRXUjYr3Q0PCJWIxvtrTTSu4KU0uMR8ThwbER8hywYf4cs/aO87CvA78gumu5OfkFfRPw0T4loq/+SveYvpJRW+KahRqYBB0TEWhW+2dmOLJWnufS7moiItcgC8GtSSl8oW7dPG6vv8PePOjdzwtXVPUZ2sdVJkc1NDSyb9aDhQsNbW1F2EtkI9kmlM7NENt3gcn/oS6WUHiMbFf0U8MW8zt9XcyAVZoBpaDNkAX1jWnJc7elbZPmq3y5gXyuDcWRft38rIlZ4fSKiT5RNpVZJSulNshzyw6PCXWAjs04r2/iH/OcFeWC5Qt35r6PJ/l9UmkWCiChPX6hUZt8om5YxX96H93K2y1O5to6IkWXLvpn/vAWWTQE6BvhANH531tJpHEeTffitOBJediy3kl3H8fWyYqeS5RK3xNVkF4p+GtgLuL405SsiVo+I1Us3yD+cT8mfNvUeb4lr8p8XRIW7aJb1VWvdQna+fKus7gOBIWTXwDT3AahWGgY4lhudzz/MfXbF4i1S2PtHXZMj4erS8rzjL5JNE/doRFxOdoHMp8guiLwgpfTfVpb9KtnNZv4ZEb8l+3rzM2Q5yps00awrgF+SpZVMSClNa6JsqakR8XeyNJaXyaZgO4VsJoCxteiD9pRSup/sq+BVQkppbkQcTxaQPB0RV5KNaq9FdiHY4WQXsE2oorpTydI07o+I0cC/yP6pb0Z2Ue1ospzXlrbxnxHxY7LAdlJEXE92Ee2mZKkQHwDeSSndGBG/J0uN2JVs5og3gI3ILuLbghWvKyh3MfC+yKbs+zdZ+sHGZEHpVmQX5v27bJt/A9fm76//kqVgfYJsGsfrS8qdTTaSfkNE3EB2Mea7ZEHvx8g+NJ+Yl/052TcyF0bEXmR51/8je8/uTTbrRsPczb8ne4+dGxGbks1kNIRsertptOx/6BjgJ2QXo3ZjxdSirYD7IuJmstSEt8m+VTuVbPaMB6iBlNKjEfFdsg/EkyPij7z392QoWX/1aqKKalwFnAB8M7/I9H6yc+Q0srzos9pYf9VSSnMi4i6ybyHmA4+SnRefJ+vXVl9LUvD7R12QQbi6vJTSbRGxN1ku5plk/2CmAp9LKf2uDWVvzEfeziULgF4j++dzP03neI8hSx3pTctyo39K9g/yy2Rfx79GFmz8MKXU5IVBLTku1U5KaVxEvJ9sRPBYsjSIt8kCuP/jvVHO5up5MbIbxHyTLOg+lixYfJFsdoUb2tDGb+WpEl8EvkEWIL5INvo+r6TcZyJiPFlQ+m2yc+hVsm9aqvl242t523cjm0N5LbIZf6aQ5XJfVWGbx/LtfkD2DdP/yOb1Pqt0JDWlNDsiPko2Yn1kvp/FZN9WPUiW3tFQdlFEHEQWEB5HFoxCFoj+k5LgOL9Ycl+y9+vIvN2PkgXxF9GCGXlSSq9FxF/JUiP+m88WU+pFsr8HI/J9rQa8RDbF4I9TSi3Ny26qLedFxCSyvyWnk82b/RpZ8P+VGtS/KCL2J/t78ymyD5zvkF0wek5K6cUmNm8Px5LNqvNxsg8H/yX74LaIKr+JbEyB7x91QQ1zWEqStNKI7E6kV6eUTuzotkhSezAnXJIkSSqYQbgkSZJUMINwSZIkqWDmhEuSJEkFcyRckiRJKtgqNUXhgAED0uDBgzu6GZ3a3Llz6du3b0c3o9OzH2vDfmw7+7A27MfasB9rw36sjbb046RJk95IKTV5I7VVKggfPHgwEydO7OhmdGoTJkxg+PDhHd2MTs9+rA37se3sw9qwH2vDfqwN+7E22tKPETGjuTKmo0iSJEkFMwiXJEmSmrD99tszYcKEmtZpEC5JkqRV2v7778+55567wvJbb72V9dZbj8cff5zhw4fzhS98gbq6Ourq6ujVqxc9e/Zc9vzAAw9s0T4NwiVJkrRKO/HEE7nmmmson7r7mmuu4ZhjjqFHj+wyyksvvZT6+nrq6+s566yz+NSnPrXs+Z133tmifRqES5IkaZU2cuRI3nrrLR544IFly+bMmcPtt9/O8ccfz+DBg7n77rtruk+DcEmSJK3S+vTpw5FHHsno0aOXLRs/fjzbbLMNO++8c7vs0yBckiRJq7wTTjiBP/7xj8yfPx+Au+66ixNOOKHd9mcQLkmSpFXebrvtxjrrrMOtt97Kc889x9NPP82nP/3pdtvfKnWzHkmSJHVh81+B/14KM2+Fd9/OlvXqDxsfAVucAn0GNrn58ccfz+jRo3n66acZNmwYAwc2Xb4tDMIlSZLUub39ODx+Nrx6NxCwdMF76+a9AHOehv9cAOvvBzv/ANbaoWI1xx9/POeffz5Tpkzhs5/9bLs22XQUSZIkdV4zb4O7PgIv/wWWLlw+AG+wZEG2/KXbYNwH4eXK0wkOHjyYj3zkI8ydO5ePfOQj7dpsR8IlSZLUOb1yFzz0KVgyv8oNEiyZBw8cAcPvgIEjVijRcGfM0jtkTp8+fYVyo0aNanFzSzkSLkmSpM5n/qvwwOEtCMBLLJkP9x0CC96ofbuqZBAuSZKkzue/l8HSJa3fPi2BaVfUrj0tZBAuSZKkzmXpYnjmF5Xzv6u1ZD489X+QltauXS1gEC5JkqTO5eU7YemittezZH6WV94BDMIlSZLUubwzBRbPbXs9SxbA7CfaXk8rGIRLkiSpc1n4JlCDNJK0CBa+3fZ6WsEgXJIkSZ1Lz7oaVdQNevatUV0t3rMkSZLUifTZALqv3vZ6uveB3uu1vZ5WMAiXJElS57LxEbWZ1SQtgY0Pa3s9rWAQLkmSpM6l9zqwwcdoUygb3bIAvFf/mjWrJQzCJUmS1Plse0aWTtJa3XrDNl+rXXtauvsO27MkSZLUWgM+BIOOal1uePfVYbMT4X3Dat6sahmES5IkqfOJgA9cBuvt27JAvPvqsMFBMOyX7de2KhiES5IkqXPq1h32+BNsdVqWXtJUMN59dejeG7Y+HXa7PssJ70A9OnTvkiRJUltENxhyIWz3bZh2JTz1f7BoNnTLw9yli6HXWrDNGbD5iR12IWY5g3BJkiR1fqutDdudAdt+DebOgHfzO2H26g99B2fpKysRg3BJkiR1HdEN6jYFNu3oljTJnHBJkiSpYAbhkiRJUsEKDcIjYtuIuDciZkfEsxFxWMm6IyNiakTMiYgnI2JkFfVtGRELIuLadm24JEmSVEOFBeER0QO4FbgdWBs4Bbg2IraKiA2Ba4GvAf2AM4E/RMS6zVT7K+DR9mu1JEmSVHtFjoRvA2wAXJxSWpJSuhd4CDgO2Ah4J6V0Z8rcAcwFNm+ssog4CngHuKfdWy5JkiTVUJFBeKV5YQLYAZgITI2IQyKie56KshCYUrGiiH7AecDX26mtkiRJUruJlFIxO4roCTwNXApcDIwgS00Zn1LaPyJOBn4O9AbeBT6Zj4hXquvnwMsppR9HxChgi5TSsY2UPYUs9YWBAwcOHTt2bG0PbBVTX19PXV1dRzej07Mfa8N+bDv7sDbsx9qwH2vDfqyNtvTjiBEjJqWUhjVVprAgHCAidgJ+yXuj36+TjXhfB1wP7A88BgwF/gwcmFKaXFbHLsAYYEhK6d3mgvBSw4YNSxMnTqzV4aySJkyYwPDhwzu6GZ2e/Vgb9mPb2Ye1YT/Whv1YG/ZjbbSlHyOi2SC80Jv1pJSmAHs2PI+Ih4GrgV2A+1NKDRHyoxHxD2AfYHJZNcOBwcALkd35qA7oHhHbpZR2bcfmS5IkSTVR9BSFO0VE74hYPSLOANYHriKb4WT3fJSbiBgC7E7lnPDLyS7Y3CV/XArcQTaKLkmSJK30ir5t/XHAZ4GewAPAvimlhcB9eVrJjRExkCxN5YKU0l0AEXEWsHtK6cCU0jxgXkOFEVEPLEgpvV7soUiSJEmtU3Q6yplkc4BXWncJcEkj6y5oos5RNWmcJEmSVBBvWy9JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpWaBAeEdtGxL0RMTsino2Iw0rWHRkRUyNiTkQ8GREjG6ljtYi4IiJm5GX/FREHFnYQkiRJUhsVFoRHRA/gVuB2YG3gFODaiNgqIjYErgW+BvQDzgT+EBHrVqiqB/AisCewJvAd4IaIGNzuByFJkiTVQJEj4dsAGwAXp5SWpJTuBR4CjgM2At5JKd2ZMncAc4HNyytJKc1NKY1KKU1PKS1NKd0OPA8MLe5QJEmSpNYrMgiPRpbtAEwEpkbEIRHRPU9FWQhMabbSiIHAVsB/athWSZIkqd1ESqmYHUX0BJ4GLgUuBkaQpaaMTyntHxEnAz8HegPvAp/MR8Sbq/NOYFpK6fONlDmFLPWFgQMHDh07dmyNjmjVVF9fT11dXUc3o9OzH2vDfmw7+7A27MfasB9rw36sjbb044gRIyallIY1VaawIBwgInYCfsl7o9+vk414XwdcD+wPPEaWWvJn4MCU0uRG6uoG/IEsh/zQlNKi5vY/bNiwNHHixLYfyCpswoQJDB8+vKOb0enZj7VhP7adfVgb9mNt2I+1YT/WRlv6MSKaDcILnR0lpTQlpbRnSul9KaX9gc2AfwK7APenlCbmed6PAv8A9qlUT0QEcAUwEDiimgBckiRJWlkUPUXhThHROyJWj4gzgPWBq4BHgd0jYpe83BBgdxrPCf8NsC3w8ZTS/HZvuCRJklRDRd+s5zjgFeA1YG9g35TSwpTSfcAo4MaImAPcBFyQUroLICLOiog7898HAZ8nGz1/NSLq88cxBR+LJEmS1Co9itxZSulMsjnAK627BLikkXUXlPw+g8ozrUiSJEmdgretlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBSs0CI+IbSPi3oiYHRHPRsRhJeuOjIipETEnIp6MiJFN1LN2RNwcEXMjYkZEfLqQA5AkSZJqoLAgPCJ6ALcCtwNrA6cA10bEVhGxIXAt8DWgH3Am8IeIWLeR6n4FvAsMBI4BfhMR27fzIUiSJEk1UeRI+DbABsDFKaUlKaV7gYeA44CNgHdSSnemzB3AXGDz8koioi9wBPCdlFJ9SulB4M95PZIkSdJKr8ggPBpZtgMwEZgaEYdERPc8FWUhMKXCNlsBS1JKz5QsexxwJFySJEmdQqSUitlRRE/gaeBS4GJgBFlqyviU0v4RcTLwc6A3WarJJ/MR8fJ6dgf+mFJar2TZ54BjUkrDK5Q/hSz1hYEDBw4dO3ZsrQ9tlVJfX09dXV1HN6PTsx9rw35sO/uwNuzH2rAfa8N+rI229OOIESMmpZSGNVWmR6tqboWU0qJ8hPuXwDfJRr9vABZGxD7AT4DhwGPAUODPEXFgSmlyWVX1ZHnjpfoBcxrZ7+XA5QDDhg1Lw4cPr8HRrLomTJiAfdh29mNt2I9tZx/Whv1YG/ZjbdiPtdHe/Vjo7CgppSkppT1TSu9LKe0PbAb8E9gFuD+lNDGltDSl9CjwD2CfCtU8A/SIiC1Llu0M/Kedmy9JkiTVRNFTFO4UEb0jYvWIOANYH7gKeBTYPSJ2ycsNAXanQk54Smku8CfgvIjoGxEfBQ4FrinmKCRJkqS2KfpmPccBrwCvAXsD+6aUFqaU7gNGATdGxBzgJuCClNJdABFxVkTcWVLPaUCfvJ7rgFNTSo6ES5IkqVMoLCccIKV0Jtkc4JXWXQJc0si6C8qevwWMrHX7JEmSpCJ423pJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKliP5gpExFrA4cCewGCgD/A68BhwZ0rp4XZsnyRJktTlNDoSHhEbRMTvgFeAs4FewETgLmAGWVD+t4h4MiI+VURjJUmSpK6gqZHwycDVwLCU0n8qFYiIPsBI4GsRsXFK6aKat1CSJEnqYpoKwrdPKb3e1MYppfnAdcB1EbFOTVsmSZIkdVGNpqM0F4C3tbwkSZK0qmrR7CgRURcRP46IRyPisYj4RUSs3V6NkyRJkrqiZmdHKXMpkIDvkl2oeSpwDXBQjdslSZIkdVlNBuER8dmU0u9KFn0Y2DKltDRf/yTwaDu2T5IkSepymktHOTAiJkTElvnzvwNXRMSBEXEI8DPgofZsoCRJktTVNDkSnlI6IiJGAndGxJXAl4BvA+eTBfAPAqPauY2SJElSl9LshZkppVuAXYGNgPHATSmloSmlISmlL6WU3mznNkqSJEldSlWzo6SU/pdSOo3sQszL81lR+rZv0yRJkqSuqckgPCI2iYgbIuLfETEGeB4YCrwBTI6IjxfRSEmSJKkraW4kfDSwFDgTeA24LKW0KKV0HvBx4IyI+GM7t1GSJEnqUpqbJ3wYsHNKaVpEjCMbCQcgpfQUsGdEnNKeDZQkSZK6muaC8EnAeRFxNbAP8O/yAimly9ujYZIkSVJX1Vw6yvHAasDFwIbA59u9RZIkSVIX19w84TOATxTUFkmSJGmV0OhIeESs0ZKKWlpekiRJWlU1lY7y34g4JyI2aqxARHTLb2H/N+D/1b55kiRJUtfTVDrK7sAPgOciYgowEXgFWAD0B7YDPgTMBy4Aftu+TZUkSZK6hkaD8JTSf4EjI2Jj4EiyoPyDQB+ym/X8C7gc+EtKaWkBbZUkSZK6hOamKCSl9CLw0/whSZIkqY2am6JQkiRJUo0ZhEuSJEkFKzQIj4htI+LeiJgdEc9GxGH58mMior7kMS8iUkQMbaSewRHxl4h4OyJejYhLIqLZ1BpJkiRpZVBYEJ4HybcCtwNrA6cA10bEVimlMSmluoYHcBrwHPBYI9X9GngNWB/YBdgz30aSJEla6RU5Er4NsAFwcUppSUrpXuAh4LgKZU8ARqeUUiN1bQrckFJakFJ6FfgrsH17NFqSJEmqtWg8zi0pFDEZ+B0wJqX0dqt2FLEj8AiwRkNwnd/kpz6ldFhJuUFko+BbpJSeb6SuLwAfAb5ANmf5OOA7KaWbK5Q9hWzUnYEDBw4dO3Zsa5qvXH19PXV1dR3djE7PfqwN+7Ht7MPasB9rw36sDfuxNtrSjyNGjJiUUhrWVJlqg/AfkI1YrwPcAvwupXRPSxoTET2Bp4FLgYuBEWSpKeNTSvuXlPsOsHdKaXgTdW0LXAvsDHQHrgZOamLkHIBhw4aliRMntqTZKjNhwgSGDx/e0c3o9OzH2rAf284+rA37sTbsx9qwH2ujLf0YEc0G4VWlo6SUzgYGAYeTBb13RMT0iDg3Ijapso5FwEjgIOBV4OvADcDMsqLHkwXVFUVEN7KR7z8BfYEBZKPhP66mHZIkSVJHqzonPGXuTCkdSZbbfRlwFtlt7cdFxAFV1DElpbRnSul9+ej3ZsA/G9ZHxEfzum9sopq1gY2BS1JKC1NKbwK/Bz5W7bFIkiRJHanFF2ZGxIeAHwHfAl4GvgdMA26MiJ81s+1OEdE7IlaPiDPIZje5qqTICcBNKaU5jdWRUnoDeB44NSJ6RMRa+XaPt/RYJEmSpI5QVRAeEetGxBkR8R9gArAW8ImU0mYppe+nlE4DDgE+20xVxwGvkE0vuDewb0ppYb6P3sCRVEhFiYizIuLOkkWHAwcArwPPAouBr1ZzLJIkSVJHq/YGNzPJgt0rgKvz0ehyE4FHm6okpXQmcGYj6xaQBfeV1l1Q9nwyMLyZNkuSJEkrpWqD8L1TSg80VSCl9D+yGU8kSZIkNaHanPC3ImKn8oV5jvd2NW6TJEmS1KVVG4RfDuxQYfl2+TpJkiRJVao2CN+JkqkESzwK7Fi75kiSJEldX7VB+BJgzQrL+wNRu+ZIkiRJXV+1Qfh9wNkR0b1hQUT0AM4G7m+PhkmSJEldVbWzo3wDeBB4NiIezJftBtQBe7RHwyRJkqSuqqqR8JTS02R54X8gu238+4AxwM4ppant1zxJkiSp66l2JJyU0itk6SeSJEmS2qDqIBwgIjYANgF6lS5PKZkXLkmSJFWpqiA8D77/QJb/nchmREklRbpX2k6SJEnSiqqdHeVnZNMUbgfMA3YHPglMBQ5ol5ZJkiRJXVS16Sh7AgellJ6KiAS8nlJ6KCIWAt8H/tZuLZQkSZK6mGpHwvsAb+S/vwWsm//+JNmsKZIkSZKqVG0Q/hSwTf77ZOALETEI+H/AS+3QLkmSJKnLqjYd5efAevnv5wF/BY4GFgIntEO7JEmSpC6rqiA8pTSm5PfHImIw2cj4CymlNxrdUJIkSdIKmk1HiYieEfFqRGzfsCylNC+l9JgBuCRJktRyzQbhKaVFwCKWnxdckiRJUitVe2HmL4FvR0SL7rApSZIkaUXVBtW7k80V/lJEPAHMLV2ZUjqk1g2TJEmSuqpqg/A3gJvasyGSJEnSqqLa2VFOau+GSJIkSauKanPCJUmSJNVIVSPhEfFvmpgdJaXkreslSZKkKlWbE35j2fOewC7AR4Ff1bJBkiRJUldXbU749yotj4gzgUE1bZEkSZLUxbU1J/xPwDG1aIgkSZK0qmhrEL4HMK8WDZEkSZJWFdVemPnn8kXA+sAQoGKqiiRJkqTKqr0w882y50uB/wBnpZTuqm2TJEmSpK7Nm/VIkiRJBasqJzwito+IFeYCj4idImK72jdLkiRJ6rqqvTDzcmCHCsu3y9dJkiRJqlK1QfhOwD8rLH8U2LF2zZEkSZK6vmqD8CXAmhWW9yebKUWSJElSlaoNwu8Dzo6I7g0LIqIHcDZwf3s0TJIkSeqqqp2i8BvAg8CzEfFgvmw3oI7shj2SJEmSqlTVSHhK6WmyvPA/AGsD7wPGADunlKa2X/MkSZKkrqfq29anlF5JKZ2dUjoopfSxlNI5KaWXW7KziNg2Iu6NiNkR8WxEHJYvPyYi6kse8yIiRcTQJuo6KiKmRsTciJgWEbu3pC2SJElSR6l2nvAvRsSxFZYfGxGnVVlHD+BW4Hay0fRTgGsjYquU0piUUl3DAzgNeA54rJG69gV+DJwErEGWEvNcNe2QJEmSOlq1I+GnAy9WWD4d+GqVdWwDbABcnFJaklK6F3gIOK5C2ROA0Sml1Ehd3wPOSyn9PaW0NKX0UkrppSrbIUmSJHWoaoPwjYAZFZbPzNdVo9JUhkHZTYAiYhDZyPboipVkM7QMA9bJU1pmRsQlEdGnynZIkiRJHSoaH2wuKRQxHTg9pXRL2fLDgZ+nlDauoo6ewNPApcDFwAiy1JTxKaX9S8p9B9g7pTS8kXo2AF4CJgEfBxaRpblMSCmdXaH8KWSpLwwcOHDo2LFjm2uqmlBfX09dXV1HN6PTsx9rw35sO/uwNuzH2rAfa8N+rI229OOIESMmpZSGNVWm2ikK/wD8IiLmAhMa6gd+RjZLSrNSSosiYiTwS+CbwETgBmBhWdHjgQuaqGp+/vOXKaVXACLi/4BzyOYtL9/v5cDlAMOGDUvDhw+vprlqxIQJE7AP285+rA37se3sw9qwH2vDfqwN+7E22rsfqw3CvwtsCowju3smZKksfwS+U+3OUkpTgD0bnkfEw8DVJc8/SpY3fmMTdbwdETOB5ofwJUmSpJVQVUF4SmkRcHREnAvsQpbL/VhK6dmW7CwidgKeIQvgTwPWB64qKXICcFNKaU4zVf0e+FJE/JUsHeV0stQWSZIkaaVX7Ug4ACml/wL/bcP+jgM+C/QEHgD2TSktBIiI3sCRwBHlG0XEWcDuKaUD80XfBwaQBfQLyNJaftCGdkmSJEmFqToIj4itgE8AmwC9StellD5TTR0ppTOBMxtZtwBYq5F1F5Q9X0Q2kl7VHOWSJEnSyqSqIDwiDgJuAv4FDAUeBTYHViMb0ZYkSZJUpWrnCT8P+F5K6cNks5kcBwwG7ua92VIkSZIkVaHaIHxr4Pr890XA6nn6yHlkF0VKkiRJqlK1QfgcoHf++yvAFvnvPYD+tW6UJEmS1JVVe2HmP4DdgCeBO4CfRsTOwGHAI+3UNkmSJKlLqjYI/xrQcN/OUcAaZFMJPpOvkyRJklSlam/W81zJ7/OAU9utRZIkSVIXV21OuCRJkqQaMQiXJEmSCmYQLkmSJBXMIFySJEkqWFVBeEQcHxGrVVjeKyKOr32zJEmSpK6r2pHw3wNrVli+Rr5OkiRJUpWqDcIDSBWWbwLMrl1zJEmSpK6vyXnCI+LfZMF3Au6LiMUlq7sDg4C/tF/zJEmSpK6nuZv13Jj/3IHsdvX1JeveBaYDN9W+WZIkSVLX1WQQnlL6HkBETAfGppQWFtEoSZIkqSurNif8L0C/hicRsWNEnB8RR7dPsyRJkqSuq9og/Abg4wARMQC4HzgMuDQivt5ObZMkSZK6pGqD8J2Av+e/fwJ4NqW0PXA88Pn2aJgkSZLUVVUbhPfhvYsy9wH+nP/+GLBxrRslSZIkdWXVBuH/BQ6PiI2B/YC78uUDgXfaoV2SJElSl1VtEP494MdkUxL+PaX0j3z5/sC/2qFdkiRJUpfV3DzhAKSU/hQRmwAbAI+XrLob5wmXJEmSWqSqIBwgpTQLmBURAyPi9ZTS0pIRcUmSJElVqiodJSJ6RsRPImIO8BIwOF/+44g4rR3bJ0mSJHU51eaEf5dsnvBjgdK7Zv4TOLHGbZIkSZK6tGrTUY4GPpNSui8ilpYsfwLYqvbNkiRJkrquakfCNwBmVFjegxbklUuSJEmqPgj/D7BHheVHApNq1xxJkiSp62tyFDsirgS+QjZP+LX5zXq6A5+MiG2ATwMHtXsrJUmSpC6kuZHwE4A+KaXbyEa99wOWkl2ouSXw8ZTS3e3bREmSJKlraS6fOxp+SSmNA8a1b3MkSZKkrq+anPDU7q2QJEmSViHVzGzyakQ0WSCl1L02zZEkSZK6vmqC8FOAd9q5HZIkSdIqo5og/LaU0mvt3hJJkiRpFdFcTrj54JIkSVKNNReEN50MLkmSJKnFmkxHSSlVe0dNSZIkSVUqNMiOiG0j4t6ImB0Rz0bEYfnyYyKivuQxLyJSRAxtpr4tI2JBRFxbzBFIkiRJbVdYEB4RPYBbgduBtclmXbk2IrZKKY1JKdU1PIDTgOeAx5qp9lfAo+3ZbkmSJKnWihwJ3wbYALg4pbQkpXQv8BBwXIWyJwCjU0qNXhgaEUeRTZ14Tzu0VZIkSWo3RQbhlS7yDGCH5RZEDAL2AEY3WlFEP+A84Ou1bKAkSZJUhGhisLm2O4roCTwNXApcDIwgS00Zn1Lav6Tcd4C9U0rDm6jr58DLKaUfR8QoYIuU0rGNlD2FLPWFgQMHDh07dmxtDmgVVV9fT11dXUc3o9OzH2vDfmw7+7A27MfasB9rw36sjbb044gRIyallIY1Vaaam/XUREppUUSMBH4JfBOYCNwALCwrejxwQWP1RMQuwD7AkCr3ezlwOcCwYcPS8OHDW9hylZowYQL2YdvZj7VhP7adfVgb9mNt2I+1YT/WRnv3Y2FBOEBKaQqwZ8PziHgYuLrk+UfJ8sZvbKKa4cBg4IWIAKgDukfEdimlXWvfakmSJKm2Cg3CI2In4BmyXPTTgPWBq0qKnADclFKa00Q1lwOlOSVnkAXlp9ayrZIkSVJ7KfpmPMcBrwCvAXsD+6aUFgJERG/gSEpGxhtExFkRcSdASmleSunVhgdQDyxIKb1e1EFIkiRJbVF0OsqZwJmNrFsArNXIukZzxFNKo2rRNkmSJKko3pZekiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVzCBckiRJKphBuCRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgpmEC5JkiQVrNAgPCK2jYh7I2J2RDwbEYfly4+JiPqSx7yISBExtEIdq0XEFRExIyLmRMS/IuLAIo9DkiRJaovCgvCI6AHcCtwOrA2cAlwbEVullMaklOoaHsBpwHPAYxWq6gG8COwJrAl8B7ghIgYXcBiSJElSmxU5Er4NsAFwcUppSUrpXuAh4LgKZU8ARqeUUvmKlNLclNKolNL0lNLSlNLtwPPACqPmkiRJ0sooKsS57bOjiB2BR4A1GoLriPgbUJ9SOqyk3CCyUfAtUkrPV1HvQGAGsEtK6akK608hG3Vn4MCBQ8eOHVuLw1ll1dfXU1dX19HN6PTsx9qwH9vOPqwN+7E27MfasB9roy39OGLEiEkppWFNlSkyCO8JPA1cClwMjCBLTRmfUtq/pNx3gL1TSsOrrPNOYFpK6fPNlR82bFiaOHFi6w5AAEyYMIHhw4d3dDM6PfuxNuzHtrMPa8N+rA37sTbsx9poSz9GRLNBeGHpKCmlRcBI4CDgVeDrwA3AzLKixwNXN1dfRHQDrgHeBb5Yy7ZKkiRJ7alHkTtLKU0hu6ASgIh4mJKAOyI+SpY3fmNT9UREAFcAA4GP5QG+JEmS1CkUGoRHxE7AM2Qj8KcB6wNXlRQ5AbgppTSnmap+A2wL7JNSmt8OTZUkSZLaTdE36zkOeAV4Ddgb2DeltBAgInoDR1IhFSUizoqIO/PfBwGfB3YBXi2ZW/yYYg5BkiRJapui01HOBM5sZN0CYK1G1l1Q8vsMINqjfZIkSVIRvG29JEmSVDCDcEmSJKlgBuGSJElSwQzCJUmSpIIZhEuSJEkFMwiXJEmSCmYQLkmSJBXMIFySJEkqmEG4JEmSVDCDcEmSJKlgBuGSJElSwQzCJUmSpIIZhEuSJEkFMwiXJEmSCmYQLkmSJBXMIFySJEkqmEG4JEmSVDCDcEmSJKlgBuGSJElSwQzCJUmSpIIZhEuSJEkFMwiXJEmSCmYQLkmSJBXMIFySJEkqmEG4JEmSVDCDcEmSJKlgBuGSJElSwQzCJUmSpIIZhEuSJEkF69HRDZAkSVL7WLp0KTNnzmTu3Lkd3ZROZ80112Tq1KmNru/bty8bbbQR3bq1bkzbIFySJKmLeuONN4gItt5661YHi6uqOXPmsMYaa1Rct3TpUl566SXeeOMN1l133VbV76shSZLURb3zzjsMHDjQALzGunXrxsCBA5k9e3br66hheyRJkrQSWbJkCT179uzoZnRJPXv2ZPHixa3e3iBckiSpC4uIjm5Cl9TWfjUIlyRJkgpmEC5JkrSKiQiOO+64Zc8XL17MOuusw8EHH9yiegYPHswbb7zR5jINzj33XO6+++4WtaGSW265hfPOOw+AUaNGERE8++yzy9ZffPHFRAQTJ05c1sYdd9yRXXbZhR133JFbb70VgHfffZc99tijTWknjTEIlyRJWsX07duXJ554gvnz5wPwt7/9jQ033LCDWwXnnXce++yzT5vr+clPfsJpp5227PmOO+7I2LFjlz2/8cYb2W677ZbbZvz48UyePJkbb7yRL3/5ywD06tWLvffem+uvv77NbSpnEC5JkrQKOvDAA7njjjsAuO666zj66KOXrXvrrbcYOXIkO+20Ex/60IeYMmUKAG+++Sb77bcfQ4YM4fOf/zwppWXbXHvttXzgAx9gl1124fOf/zxLlixpdN9LlizhxBNPZIcddmDHHXfk4osvBuDEE0/kxhtvZOLEieyyyy7LRqYb8q+nTZvGAQccwNChQ9l999156qmnVqj7mWeeYbXVVmPAgAHLlo0cOXLZ6PZzzz3HmmuuyTrrrFOxbf/73//o37//ctuOGTOm6c5sBYNwSZKkVdBRRx3F2LFjWbBgAVOmTOGDH/zgsnXf/e53GTJkCFOmTOGCCy7g+OOPB+B73/seu+22G//617845JBDeOGFFwCYOnUq119/PQ899BCTJ0+me/fuTQaukydP5qWXXuKJJ57g3//+NyeddNJy64cNG8bkyZOZPHkyBxxwAGeccQYAp5xyCr/85S+ZNGkSF1100XKj3Q0eeughdt111+WW9evXj4033pgnnniC6667jk996lMrbDdixAh22GEH9txzT84///xly3fYYQceffTR5rqzxQq9WU9EbAv8ChgKvA6cmVK6OSKOAS4rKdoN6AMMSylNqlDP2sAVwH7AG8C3U0p/aO/2S5IkdRU77bQT06dP57rrruNjH/vYcusefPBBbrrpJgD22msv3nzzTWbPns3999/Pn/70JwAOOuigZSPG99xzD5MmTeL9738/APPnz2/yJjabbbYZzz33HF/60pc46KCD2G+//SqWu+GGG3jssce46667qK+v5+GHH+aTn/zksvULFy5cYZtXXnml4ih3w4eOcePGcc899/D73/9+ufXjx49nwIABTJs2jb333ptHHnmENdZYg+7du9OrV68mb97TGoUF4RHRA7gVuBTYF9gTuC0ihqSUxgBjSsqeCHwHeKyR6n4FvAsMBHYB7oiIx1NK/2m3A5AkSepiDjnkEM444wwmTJjAm2++uWx5aZpJg4aUkEpT86WUOOGEE/jhD39Y1X779+/P448/zrhx4/jVr37FDTfcwJVXXrlcmf/85z9897vf5f7776d79+4sXbqUtdZai8mTJzdZd58+fSreROfjH/84Z555JsOGDaNfv36Nbr/55pszcOBAnnrqKdZff30gC/Z79+5d1bFVq8h0lG2ADYCLU0pLUkr3Ag8Bx1UoewIwOlU4AyKiL3AE8J2UUn1K6UHgz43UI0mSpEZ85jOf4dxzz2XHHXdcbvkee+yxLJ1kwoQJDBgwgH79+i23/M477+Ttt98GYO+99+bGG2/ktddeA7Kc8hkzZjS63zfeeIOlS5dyxBFH8P3vf5/HHlt+3HX27NkcddRRjB49etmodr9+/dh000354x//CGSB/+OPP75C3dtuu+1yM6E06NOnDz/+8Y85++yzm+yT1157jeeff55NNtkEyPLg11lnnZrf9KjIdJRKM5oHsMNyCyIGAXsAn2mknq2AJSmlZ0qWPU42si5JkqQqbbTRRnzlK19ZYfmoUaM46aST2GmnnVh99dW5+uqrgSxX/Oijj2bXXXdlzz33XBaobrfddpx//vnst99+LF26lJ49e/KrX/2KQYMGVdzvSy+9xEknncTSpUsBVhhBv+WWW5gxYwaf+9znli2bPHkyY8aM4dRTT+X8889n0aJFHHXUUey8887LbbvHHnvw9a9/nZTSCqP2Rx11VKN9MWLECLp3786iRYv40Y9+tCydZvz48Suk69RCVPq6oT1ERE/gabJ0lIuBEcDtwPiU0v4l5b4D7J1SGt5IPbsDf0wprVey7HPAMZW2iYhTgFMABg4cOLR0ehq1XH19PXV1dR3djE7PfqwN+7Ht7MPasB9rw36sjdJ+XHPNNdliiy06uEXF+8Y3vsGBBx7IiBEjWl3HkiVL6N69O8cccwyjRo1iyy23XKHMs88+WzH1ZcSIEZNSSsOaqr+wkfCU0qKIGAn8EvgmMBG4ASjPqD8euKCJquqB8kSefsCcRvZ7OXA5wLBhw9Lw4cNb2nSVmDBhAvZh29mPtWE/tp19WBv2Y23Yj7VR2o9Tp06t6cWEncWoUaP4xz/+0aZjnzNnDqutthqf+MQnVphtpUHv3r0ZMmRIq+ovdIrClNKUlNKeKaX35aPfmwH/bFgfER8lyxu/sYlqngF6RETpx5GdAS/KlCRJEgMHDuSQQw5pcz29evVaNj1jrRUahEfEThHROyJWj4gzgPWBq0qKnADclFKqOKoNkFKaC/wJOC8i+uaB+6HANe3YdEmSJKlmir5Zz3HAK8BrwN7AvimlhQAR0Rs4Eri6fKOIOCsi7ixZdBrZPOKvAdcBpzo9oSRJkjqLQm/Wk1I6EzizkXULgLUaWXdB2fO3gJE1bp4kSZJUCG9bL0mSJBXMIFySJEkqmEG4JEmSVmoLFy7k5JNPZtCgQayxxhoMGTKEO+9873LBqVOnstdeey2bF/3mm2/uwNZWxyBckiRJK7XFixez8cYbc9999zF79my+//3vc+SRRzJ9+nQWL17MoYceysEHH8xbb73F5ZdfzrHHHsszzzzTfMUdyCBckiRJrTZ37lzuvvtuLrvsMu6++27mzp1b83307duXUaNGMXjwYLp168bBBx/MpptuyqRJk3jqqad4+eWX+epXv0r37t3Za6+9+OhHP8o116zcs1cXOjuKJEmSuo65c+fygx/8gJdffpm+ffvyyCOPcO+993L22WfTt2/fdtvvrFmzeOaZZ9h+++1ZtGjRCutTSjzxxBPttv9acCRckiRJrfLII4/w8ssvM3jwYNZZZx0GDx7MSy+9xN///vd22+eiRYs45phjOOGEE9hmm23YZpttWHfddbnwwgtZtGgRd911F/fddx/z5s1rtzbUgkF4J3LOOecwYMAA1ltvPQBuvvlmNt54Y+rq6vjXv/7Vwa2TJEmrmmnTpq0w4l1XV8e0adPaZX9Lly7luOOOo1evXlxyySUA9OzZk1tuuYU77riD9dZbj5/+9KcceeSRbLTRRu3ShloxCF/JDB48mD59+lBXV7fs8cUvfpEXX3yRn/70pzz55JO8+uqrAJxxxhlccskl1NfXM2TIEAYPHszAgQOXy8X63e9+x/Dhw6va94knnsg555zTaHv69+/Pt771LV588cVl6y+88EJ22GEH1lhjDTbddFMuvPDCtneCJEnqFDbffPMVcsDr6+vZfPPNa76vlBInn3wys2bN4qabbqJnz57L1u20007cd999vPnmm4wbN47nnnuOD3zgAzVvQy0ZhK+EbrvtNurr65c9LrnkEmbMmMH73vc+1l133WXlZsyYwfbbb7/ctosXL+bnP/95u7TnlVdeoX///nzpS19ati6lxOjRo3n77bf561//yiWXXMLYsWNrun9JkrRy+vCHP8wGG2zA888/z+uvv87zzz/PhhtuyIc+9KGa7+vUU09l6tSp3HbbbfTp02e5dVOmTGHBggXMmzePiy66iFdeeYUTTzyx5m2oJYPwTuDuu+9m33335eWXX6auro6jjz6auro6lixZws4777zcp80zzzyTiy66iHfeeadiXU899RT77rsva6+9NltvvTU33HADAJdffjljxozhJz/5CXV1dXz84x9fYdvevXuz55578uSTTy5b9o1vfINdd92VHj16sPXWW3PooYfy0EMP1bYDJEnSSqlv376cffbZHH/88ey8884cf/zx7XJR5owZM7jsssuYPHky66233rJsgTFjxgBwzTXXsP7667Puuutyzz338Le//Y3VVlutpm2oNWdH6QT22Wcf7rzzTo499lhmzpy5bHlE8Pjjj7PFFlssWzZs2DCGDx/ORRddxPnnn79cPXPnzmXfffflvPPO484772TKlCnst99+bL/99pxyyik8/PDDbLTRRits12DevHmMHz++0U+3KSUeeOABPv/5z9fgqCVJUmfQt29f9t57b/bee+9228egQYNIKTW6/sILL+x0KbEG4SuhkSNH0qPHey/NhRdeyJZbbln19ueddx4f/ehH+cpXvrLc8ttvv53Bgwdz0kknAbDrrrtyxBFHcOONN66Q1lKpPfX19ay11lqMHz++YrlRo0axdOnSZfVLkiSpMtNRVkK33HIL77zzzrLH5z73uRZtv8MOO3DwwQfzox/9aLnlM2bM4B//+AdrrbXWsseYMWOWXejZXHsWLlzIl7/8Zfbcc88VtrnkkksYPXo0d9xxx0r/9Y8kSVJHMwjvor73ve/x29/+lpdeemnZso033pg999xzuQC/vr6e3/zmN0CW3tKU7t27s8cee9C9e3cefPDBZcuvvPJKfvSjH3HPPfes9NMBSZIkrQwMwruoLbbYgk996lP84he/WLbs4IMP5plnnuGaa65h0aJFLFq0iEcffZSpU6cCMHDgQJ577rlG60wp8eCDD/L222+z7bbbAjBmzBjOOuss/va3v7HZZpu170FJkiR1EQbhK6GPf/zjy80Tfthhh7WqnnPPPXe5uTvXWGMN7rrrLsaOHcsGG2zAeuutxze/+U0WLlwIwMknn8yTTz7JWmutxciRI1doT79+/bjiiiu4+uqrl+WQn3POObz55pu8//3vX9beL3zhC60/eEmS1OmklFi8eHGTF09qeV6YuZKZPn16o+tKZ0YBVjjRy7fdeOONWbBgwXLLtt56a+64446K9W+55ZZMnjy5yTonTJiw3M1/nn/++UbbK0mSuq558+YxceJExo0bx6xZs1i8eDE9evRg4MCBHHDAAQwdOpTVV1+9o5u50jIIlyRJUtXmz5/PzTffzPjx41m8eDH9+/dngw02oFu3bixdupT6+nquvPJKRo8ezV577cXIkSNXuLmODMIlSZJUpdmzZ3PxxRczffp0Ntxww+VuHQ/QrVs3+vXrR79+/Vi0aBHjxo3jmWee4fTTT2fNNdfsoFavnMwJb2dvvQXPPgtLlnR0SyRJklpv/vz5XHzxxcycOZPBgwevEICX69mzJ4MHD+bFF1/kZz/7GfPnzy+opZ2DQXg7efddOPZY2GAD2GWX7Oe993Z0qyRJklrn5ptvZvr06S2ejnijjTbi+eef55ZbbmnT/o899ljWX399+vXrx1ZbbcXvfvc7ABYuXMjJJ5/MoEGDWGONNRgyZAh33nnnctuWTnhRV1dH9+7d+dKXvtSm9rSVQXg7Ofdc+NOfYOFCmDsXXnsNDjkE3nijo1smSZLUMnPnzmX8+PFsuOGGrdp+ww035N5772XevHmtbsO3v/1tpk+fzv/+9z/+/Oc/c8455zBp0iQWL17MxhtvzH333cfs2bP5/ve/z5FHHrnc5BL19fXLHrNmzaJPnz588pOfbHVbasEgvJ1ceSVU+tbl5puLb4skSVJbNAS7zaWgNKZnz54sXryYSZMmtboN22+//bK7ckcEEcG0adPo27cvo0aNYvDgwXTr1o2DDz6YTTfdtNF93Xjjjay77rrsvvvurW5LLRiEt5NKOeApmRsuSZI6n3HjxtG/f/821dG/f3/++te/tqmO0047jdVXX51tttmG9ddfn4997GMrlJk1axbPPPPMsnualLv66qs5/vjjm71TeHszCG8nxxwD+Ye1ZVKCknvgSJIkrfRSSrz66qvU1dW1qZ66ujpmzZrVphv6/PrXv2bOnDk88MADHH744ctGxhssWrSIY445hhNOOIFtttlmhe1feOEF7rvvPk444YRWt6FWDMLbyY9+BHvtBb17Q79+2eO662C99Tq6ZZIkSdVbsmQJS5YsoVu3toWN3bp1Y/HixSxpY1pA9+7d2W233Zg5cya/+c1vli1funQpxx13HL169eKSSy6puO3o0aPZbbfd2HTTTdvUhlpwnvB2svrq8Je/wAsvZBdl7rjjiiPjkiRJK7vu3bvTvXt3li5d2qZAfOnSpfTo0YPu3bvXpF2LFy9m2rRpQDZaf/LJJzNr1iz+8pe/NJq7Pnr0aL71rW/VZP9t5Uh4O9tkExg2zABckiR1ThHBeuutR319fZvqqa+vZ+DAga3KxX7ttdcYO3Ys9fX1LFmyhHHjxnHdddex1157AXDqqacydepUbrvttkbvzvnwww/z0ksvdfisKA0MwiVJktSk/fffn7fffrtNdbz99tsccMABrdo2IvjNb37DRhttRP/+/TnjjDP42c9+xqGHHsqMGTO47LLLmDx5Muutt96yucDHjBmzXB1XX301hx9+OGussUabjqNWTEeRJElSk4YOHco111zDokWLWjVN4aJFi+jRowdDhw5t1f7XWWcd7rvvvorrBg0aVNXFnpdddlmr9t1eHAmXJElSk/r27cuIESN46aWXWrX9Sy+9xF577cXqq69e45Z1XgbhkiRJatZhhx3GpptuysyZM1u03cyZM9l0000Z6TzNyzEIlyRJUrP69OnD6aefzkYbbcT06dNZtGhRk+UXLVrE9OnT2XjjjTn99NMbvWByVWVOuCRJkqqy5ppr8s1vfpNbbrmFe++9l8WLF9O/f3/q6uro1q0bS5cupb6+nrfffpuePXuy//77M3LkSAPwCgzCJUmSurCUUk1v0d6nTx+OPvpoDj30UCZNmsRf//pXXn75ZRYvXkyPHj0YOHAghx12GEOHDu3SOeBtufMnGIRLkiR1Wd27d2fRokX06tWr5nWvvvrq7L777uy+++6klFiyZAndu3evacC/MmuY8aW1zAmXJEnqotZaay1mzZrF0qVL23U/EUGPHj1WmQB86dKlzJo1izXXXLPVdTgSLkmS1EUNGDCAmTNn8vTTT3d0UzqdBQsW0Lt370bX9+3blwEDBrS6foNwSZKkLqpbt25ssskmHd2MTmnChAkMGTKk3eo3HUWSJEkqmEG4JEmSVDCDcEmSJKlgBuGSJElSwaKtE413JhHxOjCjo9vRyQ0A3ujoRnQB9mNt2I9tZx/Whv1YG/ZjbdiPtdGWfhyUUlqnqQKrVBCutouIiSmlYR3djs7OfqwN+7Ht7MPasB9rw36sDfuxNtq7H01HkSRJkgpmEC5JkiQVzCBcLXV5Rzegi7Afa8N+bDv7sDbsx9qwH2vDfqyNdu1Hc8IlSZKkgjkSLkmSJBXMIFySJEkqmEH4Kigi1o6ImyNibkTMiIhPN1H2qxHxakTMjogrI2K1knXXRsQrEfG/iHgmIj5btu3eEfFURMyLiPERMag9j6tIRfRhRAyOiBQR9SWP77T3sRWpVv1YUmbLiFgQEdeWLe+y5yIU04+ejyuUbep9PSHvv4Z+erpsW8/H98q2qh89H1co2+T7OiKOioipeV3TImL3knVd9nwsog/bdC6mlHysYg/gOuB6oA7YDZgNbF+h3P7ALGB7oD8wAfhRyfrtgdXy37cBXgWG5s8H5PV+EugNXAj8vaOPvZP14WAgAT06+nhX9n4sKXcX8ABwbcmyLn0uFtiPno9V9mP+/LON7MPzsTb96PlYfT/uS3ajwg+RDb5uCGy4KpyPBfVhq8/FDu8gH8U+gL7Au8BWJcuuaeQf8R+AC0qe7w282ki9WwOvAEfmz08BHi7b73xgm47ug07Uh136n0yt+xE4CrgBGMXywWOXPRcL7kfPxyr7kaaDR8/H2vSj52P1/fgwcHIj++my52OBfdjqc9F0lFXPVsCSlNIzJcseJ/v0V277fF1puYER8b6GBRHx64iYBzxFFkD+pdK2KaW5wLRG9tPZFNWHDWZExMyI+H1EDKjJEawcataPEdEPOA/4enPbdrFzEYrrxwaej1W8r4EfRsQbEfFQRAxvbFvPx1b3YwPPxyb6MSK6A8OAdSLi2byvLomIPpW27WLnY1F92KDF56JB+KqnjuzrmFKzgTWqKNvw+7KyKaXT8ue7A38CFrZiP51NUX34BvB+YBAwNC8zpo1tX5nUsh+/D1yRUnqxjfvpjIrqR8/HxsuW9+M3gc3IvrK+HLgtIjZvxX46o6L60fOx8bKl/TgQ6Al8gux/zC7AEOCcVuynsymqD1t9LhqEr3rqgX5ly/oBc6oo2/D7cmVTSktSSg8CGwGntmI/nU0hfZhSqk8pTUwpLU4pzQK+COyXj1Z2BTXpx4jYBdgHuLgG++mMCulHz8cmyy73vk4p/SOlNCeltDCldDXwEPCxVuynMyqkHz0fmyxb2o/z899/mVJ6JaX0BvB/rBrnYyF92JZz0SB81fMM0CMitixZtjPwnwpl/5OvKy03K6X0ZiN19wAaRimW2zYi+ubrKu2nsymqD8s13FkrWtDWlVmt+nE4WU7eCxHxKnAGcEREPFZp2y52LkJx/VjO83H5ck29rxPv9ZPn43va0o+V1tHE+s6mJv2YUnobmMl7/dPktl3sfCyqD8tVfy52dOK8j+IfwFiyK4b7Ah+l8auFDyCbrWM7squF7yW/oAFYl+wCrjqgO9mVxXOBQ/P16+T1HkF2xfWP6VpXXBfRhx8ku1izG/A+siu8x3f0sa+E/bg6sF7J4yLgRmCdVeFcLLAfPR+r68e18vdyb7IP1cfk7+utPR9r2o+ej1X0Y77+POBRsv85/clmPvr+qnA+FtSHrT4XO7yDfBT/ANYGbsn/oL0AfDpfvgnZVzKblJT9Gtm0Pf8Dfs970+mtA9wHvJOv+zfwubL97EN2seF8sqvcB3f0sXemPgSOBp7P9/EKMBpYr6OPfWXrxwp1jqJkVo+ufi4W1Y+ejy16Xz9K9jX2O8DfgX09H2vbj56P1b+vyfKZf53346vAL4Deq8L5WEQftuVcjLwCSZIkSQUxJ1ySJEkqmEG4JEmSVDCDcEmSJKlgBuGSJElSwQzCJUmSpIIZhEuSJEkFMwiXtNKJiE9ERCp5fmJE1LexzuERkSJiQNtb2Kr9T4+IMzpi30WqxWu1MoiIURFxZUe3o60i4vaIuKrKsl+MiD+3c5Mk5QzCJVUlIq7Kg9gUEYsi4rmIuCi/zXF7ux7YrNrCjQS8DwPrA43dErsm8uDtiQqr3k92s4eurkWv1cooItYlu3HH+R3dloL9FhgWEbt3dEOkVUGPjm6ApE7lbuA4sruH7Q78jux2wKeWF4yIHsCSVIM7gqWU5pPdza0tdbxLdqezDpFSer2j9t0eIqJX3qfLqcVrtRL4LPDPlNJzHd2QIqWUFkbEH4Avk92WW1I7ciRcUkssTCm9mlJ6MaX0B2AMMBLeGwHO0xGmAQuBvhGxZkRcHhGvRcSciLgvIoaVVhoRx0fEjIiYFxG3AwPL1q+Q4hARB0XEPyJifkS8GRG3RUTviJgADAIubBi5z8uvkI4SEYdHxL8jYmFEvBgRZ0dElKyfHhHnRMRlEfG/iJgZEWc21jkRcSLwXWD7km8NTiyp64ySsikiTo2IW/PjfiYiRkTERhExLiLmRsTkiNi1bB8fyftwXkS8FBG/iYh+TbSp4bgPzutbEBGTImJoS+qNiAn5sosi4nXgocb6oPS1KjkvTsj7oD4ifh8RvSLitLzf34yI/4uIbiXbHRsRj+bnzGsR8ceI2LBsXwdFxNP5Md0fEUflxzq4tf2V+zSwXFpGROwREX/P2z87P/d2aEH/RUR8PSL+m59vMyPihyXrd4yIu/Pz+a3Ivnlas2T9VZGllnwlr//tvB9XLymzel6uPiJmRcRZFV6fwyNiSsl+7ouI0vfbn4FDSuuV1D4MwiW1xXyyUfEGm5IFMJ8EdiYLxO8ANgQOBoYA9wP3RsT6ABHxQeAq4HJgF+A24LymdhoRBwC3An8DhgIjgPvI/qYdDszM61g/f1SqYyjwR+BPwI7At4BvA18sK/pV4N/ArsCPgZ9ExIcbadr1wE+Bp0v2fX0Th3IOMJasryYC1wFXkKWtDAFeJuubhjbvCNxFFijtnB/rLkA1ucsXAd8EhgHPAXc0BFotqPdYIMi+BTm+in02GAwcSnYOHEF2ftxKlqKzH9nI85eAw0q26UX2gWbnfLsBZP1D3uZNyF67O/IyvwB+UrrT1vRXRKwNbEf2ejQs65G398G8ng8CPweWtGA/FwDfAX4IbJ/3wYv59qsDfwXqgQ/k/fCRCu3cHdgB2Af4VF7uKyXrLwL2JevjvcnOoT1KjmM9svPtamDbfN01ZfuYSPYteWPnuKRaSSn58OHDR7MPsmDw9pLnHwDeAK7Pn48CFgEDS8rsRRZY9CmrazLwjfz3PwB/K1v/u+zP07LnJwL1Jc8fAsY20dbpwBlly4YDCRiQPx8D3FtWZhQws6ye68rK/Bc4p4l9jwKeaK5NeVt+WPJ8h3zZ15po82jgirJ6d8nLrNtIexrqOKZkWR3wDvDZausFJgBTqjhPyl+rUWQf1tYsWXYj8DrQq2TZBOCSJurdJm/PRvnzHwJTgSgpc1ZeZnAb+qth/aYly9bOl+3ZyDZN7ifv7wXAFxrZ/nPAbGCNCq/bFiXvvxeBHiVlfgvcXfKaLmzkdb4qf75rXuegZl7Dt4CTm3utffjw0baHI+GSWuKA/KvuBcAjZKPaXypZPzOlNKvk+VBgdeD1fLv6PFVhB2DzvMy2eV2lyp+XGwLc09qDKNlveUrFg8CGZekKU8rKvEwWWNVCad0N/fbvCssa9jcUOLasLxuOYXOatqxPU0r1+X62a2G9k5o7oEa8kFKaXfJ8FvBMWj6nfBYl/RoRu0aWqjMjIubw3sj0JvnPbYBHU0ql1xz8o2y/remvPvnPBQ0LUkpvkQXB4yLijoj4WkRs3IL9bAesRuPn7LZkH3DmlCx7GFjKe68RwJMppcUlz0vPxc3Jvj2o9Do3eJzsuo4nIuKmyNKh1qnQnvkl/SCpnXhhpqSWuB84hWzE++WU0qKy9XPLnncjC64qzbbwv/xnVFhXhCAbFaykdHn5MSZql8pXWndqYlm3kp+/Ay6uUNdLbWhHtfWWv77VqtSHlZZ1B4hsxp1xvHch8Gtk6SgPkAWa0PTr16A1/fVG/rM/8MqyxqV0UkT8DDgAOAT4QUSMTCmNq2I/OzXTzlqci82+j1JKSyJiP+BDZGlAJwM/jIg9U0qPlxRdm+ybCkntyCBcUkvMSyk924Lyj5FdZLk0NT7TxJNkQUGp8ufl/kWW8/rbRta/Sx7QNeFJYLeyZbuRjebPqVC+WtXsu7UeA7Zv4WvQ4ENkueANQe4OZGkUba23PWxDFnSflVJ6HrILCsvKTCXLMy/1gbLnrTmuaWQfELcjO0eWyQPVx4EfR8SdwAlkHxaa3E9EPEmWKrI3WTpTuSeBz0TEGiXn3kfIAuypVbb7WbIgvdLrPK3kGBLZaPkjEXEe8B+y/PLH8202B3rnxySpHZmOIqk93U32tfytEXFgRGwaER+OiO/Fe3MR/wLYJyK+HRFbRsTnWP4CvUp+AHwyIs6PiO0iYvuI+GrJjA7Tgd0jYsNo/OY8PwX2jGz2jq0i4hjg65Rd3NcK04FBeTrFgIhYrY31lfox8IGIuDQihkTEFpHNenJZFdueExH7RsT2ZBf8vUuWj9/WetvDC2RB6xcjYrOIOAj4flmZS4HNI5utZes8SP98vq5h9LjFx5VSWkp23i77gJaftz+KbAaUQRExgmx0uyFIb3I/eWD9c7JR55MiYvOI+EBENEztOYbsW4bRkc2SsgdwGfCnaj9A5KknV5B9QCh9nZd9IIyID0U228/78wtbDwE2ZvkPG7sDz6WUKn1YkFRDBuGS2k0+6vYx4F6yUeungRuArcnyWUkp/Z3sa/FTyXKkDye7mK+pev9CFqgfSDYqfh/ZDClL8yLnkgUX02jka/WU0mNkM1QcATwB/Ch/XNKKQy11E/AXsvzf14Gj21jfMimlKWQzWgwmO+bHyS5QnNXEZg2+RfbB4zFgS+DglNLcGtRbcymbU/0EsukvnySbJeVrZWVmkL12h5C196vA9/LVC/IyrT2uy4FPRURDADsP2IpsNp1nyGYXGUMWfFe7n2/n5b9DNrp9E7BRvv08YH+gH/BPsplYHgE+00w7y50BjAduzn8+QZZC1mA28FHgdrIR+Z8C308pXVtS5mga/4ZJUg3F8te0SJK6kogYThaQrZNSeqPp0p1bRHyFbGrK/vmIdlvqegT4dUqpfAq/Liuyec/vAbYqu5BWUjswJ1yS1ClFxP8DHiX7xuFDZKPMV7U1AM99nmwWnlXJBsDxBuBSMQzCJUmd1RZkc4O/j+wGTZfSzI2eqpWnmJRPT9mlpZTu6ug2SKsS01EkSZKkgnlhpiRJklQwg3BJkiSpYAbhkiRJUsEMwiVJkqSCGYRLkiRJBTMIlyRJkgr2/wEfYcDa9GKEAwAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 864x576 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# 1. Create a plot from model comparison DataFrame\n",
        "fig, ax = plt.subplots(figsize=(12, 8))\n",
        "scatter = ax.scatter(data=df,\n",
        "                     x=\"time_per_pred_cpu\",\n",
        "                     y=\"test_acc\",\n",
        "                     c=[\"blue\", \"orange\"], # what colours to use?\n",
        "                     s=\"model_size (MB)\") # size the dots by the model sizes\n",
        "\n",
        "# 2. Add titles, labels and customize fontsize for aesthetics\n",
        "ax.set_title(\"FoodVision Mini Inference Speed vs Performance\", fontsize=18)\n",
        "ax.set_xlabel(\"Prediction time per image (seconds)\", fontsize=14)\n",
        "ax.set_ylabel(\"Test accuracy (%)\", fontsize=14)\n",
        "ax.tick_params(axis='both', labelsize=12)\n",
        "ax.grid(True)\n",
        "\n",
        "# 3. Annotate with model names\n",
        "for index, row in df.iterrows():\n",
        "    ax.annotate(text=row[\"model\"], # note: depending on your version of Matplotlib, you may need to use \"s=...\" or \"text=...\", see: https://github.com/faustomorales/keras-ocr/issues/183#issuecomment-977733270\n",
        "                xy=(row[\"time_per_pred_cpu\"]+0.0006, row[\"test_acc\"]+0.03),\n",
        "                size=12)\n",
        "\n",
        "# 4. Create a legend based on model sizes\n",
        "handles, labels = scatter.legend_elements(prop=\"sizes\", alpha=0.5)\n",
        "model_size_legend = ax.legend(handles,\n",
        "                              labels,\n",
        "                              loc=\"lower right\",\n",
        "                              title=\"Model size (MB)\",\n",
        "                              fontsize=12)\n",
        "\n",
        "# Save the figure\n",
        "!mdkir images/\n",
        "plt.savefig(\"images/09-foodvision-mini-inference-speed-vs-performance.jpg\")\n",
        "\n",
        "# Show the figure\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LE-cX6wiQaQd"
      },
      "source": [
        "Woah!\n",
        "\n",
        "The plot really visualizes the **speed vs. performance tradeoff**, in other words, when you have a larger, better performing deep model (like our ViT model), it *generally* takes longer to perform inference (higher latency).\n",
        "\n",
        "There are exceptions to the rule and new research is being published all the time to help make larger models perform faster.\n",
        "\n",
        "And it can be tempting to just deploy the *best* performing model but it's also good to take into consideration where the model is going to be performing.\n",
        "\n",
        "In our case, the differences between our model's performance levels (on the test loss and test accuracy) aren't too extreme.\n",
        "\n",
        "But since we'd like to put an emphasis on speed to begin with, we're going to stick with deploying EffNetB2 since it's faster and has a much smaller footprint.\n",
        "\n",
        "> **Note:** Prediction times will be different across different hardware types (e.g. Intel i9 vs Google Colab CPU vs GPU) so it's important to think about and test where your model is going to end up. Asking questions like \"where is the model going to be run?\" or \"what is the ideal scenario for running the model?\" and then running experiments to try and provide answers on your way to deployment is very helpful."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x6IvwNf8QaQd"
      },
      "source": [
        "## 7. Bringing FoodVision Mini to life by creating a Gradio demo\n",
        "\n",
        "We've decided we'd like to deploy the EffNetB2 model (to begin with, this could always be changed later).\n",
        "\n",
        "So how can we do that?\n",
        "\n",
        "There are several ways to deploy a machine learning model each with specific use cases (as discussed above).\n",
        "\n",
        "We're going to be focused on perhaps the quickest and certainly one of the most fun ways to get a model deployed to the internet.\n",
        "\n",
        "And that's by using [Gradio](https://gradio.app/).\n",
        "\n",
        "What's Gradio?\n",
        "\n",
        "The homepage describes it beautifully:\n",
        "\n",
        "> Gradio is the fastest way to demo your machine learning model with a friendly web interface so that anyone can use it, anywhere!\n",
        "\n",
        "Why create a demo of your models?\n",
        "\n",
        "Because metrics on the test set look nice but you never really know how you're model performs until you use it in the wild.\n",
        "\n",
        "So let's get deploying!\n",
        "\n",
        "We'll start by importing Gradio with the common alias `gr` and if it's not present, we'll install it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nHduiPOAQaQd",
        "outputId": "6c2b074f-2b15-4c75-e2ba-424c41b500cf"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Gradio version: 3.1.4\n"
          ]
        }
      ],
      "source": [
        "# Import/install Gradio\n",
        "try:\n",
        "    import gradio as gr\n",
        "except:\n",
        "    !pip -q install gradio\n",
        "    import gradio as gr\n",
        "\n",
        "print(f\"Gradio version: {gr.__version__}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "91jhir2IQaQd"
      },
      "source": [
        "Gradio ready!\n",
        "\n",
        "Let's turn FoodVision Mini into a demo application."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z7MXLuWJQaQd"
      },
      "source": [
        "### 7.1 Gradio overview\n",
        "\n",
        "The overall premise of Gradio is very similar to what we've been repeating throughout the course.\n",
        "\n",
        "What are our **inputs** and **outputs**?\n",
        "\n",
        "And how should we get there?\n",
        "\n",
        "Well that's what our machine learning model does.\n",
        "\n",
        "```\n",
        "inputs -> ML model -> outputs\n",
        "```\n",
        "\n",
        "In our case, for FoodVision Mini, our inputs are images of food, our ML model is EffNetB2 and our outputs are classes of food (pizza, steak or sushi).\n",
        "\n",
        "```\n",
        "images of food -> EffNetB2 -> outputs\n",
        "```\n",
        "\n",
        "Though the concepts of inputs and outputs can be bridged to almost any other kind of ML problem.\n",
        "\n",
        "Your inputs and outputs might be any combination of the following:\n",
        "* Images\n",
        "* Text\n",
        "* Video\n",
        "* Tabular data\n",
        "* Audio\n",
        "* Numbers\n",
        "* & more\n",
        "\n",
        "And the ML model you build will depend on your inputs and outputs.\n",
        "\n",
        "Gradio emulates this paradigm by creating an interface ([`gradio.Interface()`](https://gradio.app/docs/#interface-header)) from inputs to outputs.\n",
        "\n",
        "```\n",
        "gradio.Interface(fn, inputs, outputs)\n",
        "```\n",
        "\n",
        "Where, `fn` is a Python function to map the `inputs` to the `outputs`.\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-gradio-workflow.png\" alt=\"gradio workflow of inputs flowing into some kind of model or function and then producing outputs\" width=900/>\n",
        "\n",
        "*Gradio provides a very helpful `Interface` class to easily create an inputs -> model/function -> outputs workflow where the inputs and outputs could be almost anything you want. For example, you might input Tweets (text) to see if they're about machine learning or not or [input a text prompt to generate images](https://huggingface.co/blog/stable_diffusion).*\n",
        "\n",
        "> **Note:** Gradio has a vast number of possible `inputs` and `outputs` options known as \"Components\" from images to text to numbers to audio to videos and more. You can see all of these in the [Gradio Components documentation](https://gradio.app/docs/#components)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jDFaX48AQaQe"
      },
      "source": [
        "### 7.2 Creating a function to map our inputs and outputs\n",
        "\n",
        "To create our FoodVision Mini demo with Gradio, we'll need a function to map our inputs to our outputs.\n",
        "\n",
        "We created a function earlier called `pred_and_store()` to make predictions with a given model across a list of target files and store them in a list of dictionaries.\n",
        "\n",
        "How about we create a similar function but this time focusing on a making a prediction on a single image with our EffNetB2 model?\n",
        "\n",
        "More specifically, we want a function that takes an image as input, preprocesses (transforms) it, makes a prediction with EffNetB2 and then returns the prediction (pred or pred label for short) as well as the prediction probability (pred prob).\n",
        "\n",
        "And while we're here, let's return the time it took to do so too:\n",
        "\n",
        "```\n",
        "input: image -> transform -> predict with EffNetB2 -> output: pred, pred prob, time taken\n",
        "```\n",
        "\n",
        "This will be our `fn` parameter for our Gradio interface.\n",
        "\n",
        "First, let's make sure our EffNetB2 model is on the CPU (since we're sticking with CPU-only predictions, however you could change this if you have access to a GPU)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [],
        "id": "8z1aaUy4QaQe",
        "outputId": "c18ba06e-4320-43a8-d2bb-8df09e57be5c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "device(type='cpu')"
            ]
          },
          "execution_count": 48,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Put EffNetB2 on CPU\n",
        "effnetb2.to(\"cpu\")\n",
        "\n",
        "# Check the device\n",
        "next(iter(effnetb2.parameters())).device"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u4wjK5SXQaQe"
      },
      "source": [
        "And now let's create a function called `predict()` to replicate the workflow above."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0Akjyz1oQaQe"
      },
      "outputs": [],
      "source": [
        "from typing import Tuple, Dict\n",
        "\n",
        "def predict(img) -> Tuple[Dict, float]:\n",
        "    \"\"\"Transforms and performs a prediction on img and returns prediction and time taken.\n",
        "    \"\"\"\n",
        "    # Start the timer\n",
        "    start_time = timer()\n",
        "\n",
        "    # Transform the target image and add a batch dimension\n",
        "    img = effnetb2_transforms(img).unsqueeze(0)\n",
        "\n",
        "    # Put model into evaluation mode and turn on inference mode\n",
        "    effnetb2.eval()\n",
        "    with torch.inference_mode():\n",
        "        # Pass the transformed image through the model and turn the prediction logits into prediction probabilities\n",
        "        pred_probs = torch.softmax(effnetb2(img), dim=1)\n",
        "\n",
        "    # Create a prediction label and prediction probability dictionary for each prediction class (this is the required format for Gradio's output parameter)\n",
        "    pred_labels_and_probs = {class_names[i]: float(pred_probs[0][i]) for i in range(len(class_names))}\n",
        "\n",
        "    # Calculate the prediction time\n",
        "    pred_time = round(timer() - start_time, 5)\n",
        "\n",
        "    # Return the prediction dictionary and prediction time\n",
        "    return pred_labels_and_probs, pred_time"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0xIl2lkyQaQe"
      },
      "source": [
        "Beautiful!\n",
        "\n",
        "Now let's see our function in action by performing a prediction on a random image from the test dataset.\n",
        "\n",
        "We'll start by getting a list of all the image paths from the test directory and then randomly selecting one.\n",
        "\n",
        "Then we'll open the randomly selected image with [`PIL.Image.open()`](https://pillow.readthedocs.io/en/stable/reference/Image.html#functions).\n",
        "\n",
        "Finally, we'll pass the image to our `predict()` function."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JRpqVFQzQaQh",
        "outputId": "05b75a05-e8b3-4078-e71f-f1acef8e354b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[INFO] Predicting on image at path: data/pizza_steak_sushi_20_percent/test/pizza/3770514.jpg\n",
            "\n",
            "Prediction label and probability dictionary: \n",
            "{'pizza': 0.9785208702087402, 'steak': 0.01169557310640812, 'sushi': 0.009783552028238773}\n",
            "Prediction time: 0.027 seconds\n"
          ]
        }
      ],
      "source": [
        "import random\n",
        "from PIL import Image\n",
        "\n",
        "# Get a list of all test image filepaths\n",
        "test_data_paths = list(Path(test_dir).glob(\"*/*.jpg\"))\n",
        "\n",
        "# Randomly select a test image path\n",
        "random_image_path = random.sample(test_data_paths, k=1)[0]\n",
        "\n",
        "# Open the target image\n",
        "image = Image.open(random_image_path)\n",
        "print(f\"[INFO] Predicting on image at path: {random_image_path}\\n\")\n",
        "\n",
        "# Predict on the target image and print out the outputs\n",
        "pred_dict, pred_time = predict(img=image)\n",
        "print(f\"Prediction label and probability dictionary: \\n{pred_dict}\")\n",
        "print(f\"Prediction time: {pred_time} seconds\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KWhL6vbKQaQh"
      },
      "source": [
        "Nice!\n",
        "\n",
        "Running the cell above a few times we can see different prediction probabilities for each label from our EffNetB2 model as well as the time it took per prediction."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "93WfvhkQQaQi"
      },
      "source": [
        "### 7.3 Creating a list of example images\n",
        "\n",
        "Our `predict()` function enables us to go from inputs -> transform -> ML model -> outputs.\n",
        "\n",
        "Which is exactly what we need for our Graido demo.\n",
        "\n",
        "But before we create the demo, let's create one more thing: a list of examples.\n",
        "\n",
        "Gradio's [`Interface`](https://gradio.app/docs/#interface) class takes a list of `examples` of as an optional parameter (`gradio.Interface(examples=List[Any])`).\n",
        "\n",
        "And the format for the `examples` parameter is a list of lists.\n",
        "\n",
        "So let's create a list of lists containing random filepaths to our test images.\n",
        "\n",
        "Three examples should be enough."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RMP7kWrBQaQi",
        "outputId": "1c3d5127-e596-45a0-eee5-43795488a91c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[['data/pizza_steak_sushi_20_percent/test/sushi/804460.jpg'],\n",
              " ['data/pizza_steak_sushi_20_percent/test/steak/746921.jpg'],\n",
              " ['data/pizza_steak_sushi_20_percent/test/steak/2117351.jpg']]"
            ]
          },
          "execution_count": 51,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Create a list of example inputs to our Gradio demo\n",
        "example_list = [[str(filepath)] for filepath in random.sample(test_data_paths, k=3)]\n",
        "example_list"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "86VtSB-sQaQi"
      },
      "source": [
        "Perfect!\n",
        "\n",
        "Our Gradio demo will showcase these as example inputs to our demo so people can try it out and see what it does without uploading any of their own data."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rTIYafWhQaQi"
      },
      "source": [
        "### 7.4 Building a Gradio interface\n",
        "\n",
        "Time to put everything together and bring our FoodVision Mini demo to life!\n",
        "\n",
        "Let's create a Gradio interface to replicate the workflow:\n",
        "\n",
        "```\n",
        "input: image -> transform -> predict with EffNetB2 -> output: pred, pred prob, time taken\n",
        "```\n",
        "\n",
        "We can do with the [`gradio.Interface()`](https://gradio.app/docs/#interface) class with the following parameters:\n",
        "* `fn` - a Python function to map `inputs` to `outputs`, in our case, we'll use our `predict()` function.\n",
        "* `inputs` - the input to our interface, such as an image using [`gradio.Image()`](https://gradio.app/docs/#image) or `\"image\"`.\n",
        "* `outputs` - the output of our interface once the `inputs` have gone through the `fn`, such as a label using [`gradio.Label()`](https://gradio.app/docs/#label) (for our model's predicted labels) or number using [`gradio.Number()`](https://gradio.app/docs/#number) (for our model's prediction time).\n",
        "    * **Note:** Gradio comes with many in-built `inputs` and `outputs` options known as [\"Components\"](https://gradio.app/docs/#components).\n",
        "* `examples` - a list of examples to showcase for the demo.\n",
        "* `title` - a string title of the demo.\n",
        "* `description` - a string description of the demo.\n",
        "* `article` - a reference note at the bottom of the demo.\n",
        "\n",
        "Once we've created our demo instance of `gr.Interface()`, we can bring it to life using [`gradio.Interface().launch()`](https://gradio.app/docs/#launch-header) or `demo.launch()` command.\n",
        "\n",
        "Easy!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fi_KMfpyQaQj",
        "outputId": "a503956e-a95c-45fc-a9da-9e61f582a361"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Running on local URL:  http://127.0.0.1:7860/\n",
            "Running on public URL: https://27541.gradio.app\n",
            "\n",
            "This share link expires in 72 hours. For free permanent hosting, check out Spaces: https://huggingface.co/spaces\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "<div><iframe src=\"https://27541.gradio.app\" width=\"900\" height=\"500\" allow=\"autoplay; camera; microphone;\" frameborder=\"0\" allowfullscreen></iframe></div>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "(<gradio.routes.App at 0x7f122dd0f0d0>,\n",
              " 'http://127.0.0.1:7860/',\n",
              " 'https://27541.gradio.app')"
            ]
          },
          "execution_count": 52,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import gradio as gr\n",
        "\n",
        "# Create title, description and article strings\n",
        "title = \"FoodVision Mini 🍕🥩🍣\"\n",
        "description = \"An EfficientNetB2 feature extractor computer vision model to classify images of food as pizza, steak or sushi.\"\n",
        "article = \"Created at [09. PyTorch Model Deployment](https://www.learnpytorch.io/09_pytorch_model_deployment/).\"\n",
        "\n",
        "# Create the Gradio demo\n",
        "demo = gr.Interface(fn=predict, # mapping function from input to output\n",
        "                    inputs=gr.Image(type=\"pil\"), # what are the inputs?\n",
        "                    outputs=[gr.Label(num_top_classes=3, label=\"Predictions\"), # what are the outputs?\n",
        "                             gr.Number(label=\"Prediction time (s)\")], # our fn has two outputs, therefore we have two outputs\n",
        "                    examples=example_list,\n",
        "                    title=title,\n",
        "                    description=description,\n",
        "                    article=article)\n",
        "\n",
        "# Launch the demo!\n",
        "demo.launch(debug=False, # print errors locally?\n",
        "            share=True) # generate a publically shareable URL?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z74k0ptfQaQj"
      },
      "source": [
        "<img src=\"https://github.com/mrdbourke/pytorch-deep-learning/raw/main/images/09-gradio-running-in-google-colab-and-in-browser.gif\" alt=\"Gradio demo running in Google Colab and on the web\" width=750/>\n",
        "\n",
        "*FoodVision Mini Gradio demo running in Google Colab and in the browser (the link when running from Google Colab only lasts for 72 hours). You can see the [permanent live demo on Hugging Face Spaces](https://huggingface.co/spaces/mrdbourke/foodvision_mini).*\n",
        "\n",
        "Woohoo!!! What an epic demo!!!\n",
        "\n",
        "FoodVision Mini has officially come to life in an interface someone could use and try out.\n",
        "\n",
        "If you set the parameter `share=True` in the `launch()` method, Gradio also provides you with a shareable link such as `https://123XYZ.gradio.app` (this link is an example only and likely expired) which is valid for 72-hours.\n",
        "\n",
        "The link provides a proxy back to the Gradio interface you launched.\n",
        "\n",
        "For more permanent hosting, you can upload your Gradio app to [Hugging Face Spaces](https://huggingface.co/spaces) or anywhere that runs Python code."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d3qtvhcxQaQj"
      },
      "source": [
        "## 8. Turning our FoodVision Mini Gradio Demo into a deployable app\n",
        "\n",
        "We've seen our FoodVision Mini model come to life through a Gradio demo.\n",
        "\n",
        "But what if we wanted to share it with our friends?\n",
        "\n",
        "Well, we could use the provided Gradio link, however, the shared link only lasts for 72-hours.\n",
        "\n",
        "To make our FoodVision Mini demo more permanent, we can package it into an app and upload it to [Hugging Face Spaces](https://huggingface.co/spaces/launch)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GjCNKgNEQaQj"
      },
      "source": [
        "### 8.1 What is Hugging Face Spaces?\n",
        "\n",
        "Hugging Face Spaces is a resource that allows you to host and share machine learning apps.\n",
        "\n",
        "Building a demo is one of the best ways to showcase and test what you've done.\n",
        "\n",
        "And Spaces allows you to do just that.\n",
        "\n",
        "You can think of Hugging Face as the GitHub of machine learning.\n",
        "\n",
        "If having a good GitHub portfolio showcases your coding abilities, having a good Hugging Face portfolio can showcase your machine learning abilities.\n",
        "\n",
        "> **Note:** There are many other places we could upload and host our Gradio app such as, Google Cloud, AWS (Amazon Web Services) or other cloud vendors, however, we're going to use Hugging Face Spaces due to the ease of use and wide adoption by the machine learning community."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WhJP0DuuQaQk"
      },
      "source": [
        "### 8.2 Deployed Gradio app structure\n",
        "\n",
        "To upload our demo Gradio app, we'll want to put everything relating to it into a single directory.\n",
        "\n",
        "For example, our demo might live at the path `demos/foodvision_mini/` with the file structure:\n",
        "\n",
        "```\n",
        "demos/\n",
        "└── foodvision_mini/\n",
        "    ├── 09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth\n",
        "    ├── app.py\n",
        "    ├── examples/\n",
        "    │   ├── example_1.jpg\n",
        "    │   ├── example_2.jpg\n",
        "    │   └── example_3.jpg\n",
        "    ├── model.py\n",
        "    └── requirements.txt\n",
        "```\n",
        "\n",
        "Where:\n",
        "* `09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth` is our trained PyTorch model file.\n",
        "* `app.py` contains our Gradio app (similar to the code that launched the app).\n",
        "    * **Note:** `app.py` is the default filename used for Hugging Face Spaces, if you deploy your app there, Spaces will by default look for a file called `app.py` to run. This is changable in settings.\n",
        "* `examples/` contains example images to use with our Gradio app.\n",
        "* `model.py` contains the model defintion as well as any transforms assosciated with the model.\n",
        "* `requirements.txt` contains the dependencies to run our app such as `torch`, `torchvision` and `gradio`.\n",
        "\n",
        "Why this way?\n",
        "\n",
        "Because it's one of the simplest layouts we could begin with.\n",
        "\n",
        "Our focus is: *experiment, experiment, experiment!*\n",
        "\n",
        "The quicker we can run smaller experiments, the better our bigger ones will be.\n",
        "\n",
        "We're going to work towards recreating the structure above but you can see a live demo app running on Hugging Face Spaces as well as the file structure:\n",
        "* [Live Gradio demo of FoodVision Mini 🍕🥩🍣](https://huggingface.co/spaces/mrdbourke/foodvision_mini).\n",
        "* [FoodVision Mini file structure on Hugging Face Spaces](https://huggingface.co/spaces/mrdbourke/foodvision_mini/tree/main)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RC9zPTsNQaQk"
      },
      "source": [
        "### 8.3 Creating a `demos` folder to store our FoodVision Mini app files\n",
        "\n",
        "To begin, let's first create a `demos/` directory to store all of our FoodVision Mini app files.\n",
        "\n",
        "We can do with Python's [`pathlib.Path(\"path_to_dir\")`](https://docs.python.org/3/library/pathlib.html#basic-use) to establish the directory path and [`pathlib.Path(\"path_to_dir\").mkdir()`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.mkdir) to create it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "weyZLoAJQaQl"
      },
      "outputs": [],
      "source": [
        "import shutil\n",
        "from pathlib import Path\n",
        "\n",
        "# Create FoodVision mini demo path\n",
        "foodvision_mini_demo_path = Path(\"demos/foodvision_mini/\")\n",
        "\n",
        "# Remove files that might already exist there and create new directory\n",
        "if foodvision_mini_demo_path.exists():\n",
        "    shutil.rmtree(foodvision_mini_demo_path)\n",
        "    foodvision_mini_demo_path.mkdir(parents=True, # make the parent folders?\n",
        "                                    exist_ok=True) # create it even if it already exists?\n",
        "else:\n",
        "    # If the file doesn't exist, create it anyway\n",
        "    foodvision_mini_demo_path.mkdir(parents=True,\n",
        "                                    exist_ok=True)\n",
        "\n",
        "# Check what's in the folder\n",
        "!ls demos/foodvision_mini/"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bdx4yv2mQaQl"
      },
      "source": [
        "### 8.4 Creating a folder of example images to use with our FoodVision Mini demo\n",
        "\n",
        "Now we've got a directory to store our FoodVision Mini demo files, let's add some examples to it.\n",
        "\n",
        "Three example images from the test dataset should be enough.\n",
        "\n",
        "To do so we'll:\n",
        "1. Create an `examples/` directory within the `demos/foodvision_mini` directory.\n",
        "2. Choose three random images from the test dataset and collect their filepaths in a list.\n",
        "3. Copy the three random images from the test dataset to the `demos/foodvision_mini/examples/` directory."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EsXTjaFRQaQl",
        "outputId": "a0d807ed-45ef-4b53-cfc1-fc500dea3bff"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[INFO] Copying data/pizza_steak_sushi_20_percent/test/sushi/592799.jpg to demos/foodvision_mini/examples/592799.jpg\n",
            "[INFO] Copying data/pizza_steak_sushi_20_percent/test/steak/3622237.jpg to demos/foodvision_mini/examples/3622237.jpg\n",
            "[INFO] Copying data/pizza_steak_sushi_20_percent/test/pizza/2582289.jpg to demos/foodvision_mini/examples/2582289.jpg\n"
          ]
        }
      ],
      "source": [
        "import shutil\n",
        "from pathlib import Path\n",
        "\n",
        "# 1. Create an examples directory\n",
        "foodvision_mini_examples_path = foodvision_mini_demo_path / \"examples\"\n",
        "foodvision_mini_examples_path.mkdir(parents=True, exist_ok=True)\n",
        "\n",
        "# 2. Collect three random test dataset image paths\n",
        "foodvision_mini_examples = [Path('data/pizza_steak_sushi_20_percent/test/sushi/592799.jpg'),\n",
        "                            Path('data/pizza_steak_sushi_20_percent/test/steak/3622237.jpg'),\n",
        "                            Path('data/pizza_steak_sushi_20_percent/test/pizza/2582289.jpg')]\n",
        "\n",
        "# 3. Copy the three random images to the examples directory\n",
        "for example in foodvision_mini_examples:\n",
        "    destination = foodvision_mini_examples_path / example.name\n",
        "    print(f\"[INFO] Copying {example} to {destination}\")\n",
        "    shutil.copy2(src=example, dst=destination)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A44vvs_wQaQl"
      },
      "source": [
        "Now to verify our examples are present, let's list the contents of our `demos/foodvision_mini/examples/` directory with [`os.listdir()`](https://docs.python.org/3/library/os.html#os.listdir) and then format the filepaths into a list of lists (so it's compatible with Gradio's [`gradio.Interface()`](https://gradio.app/docs/#interface) `example` parameter)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_WtmXEa0QaQm",
        "outputId": "643af04e-617f-4ac4-adba-09e6a06866b4"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[['examples/3622237.jpg'], ['examples/592799.jpg'], ['examples/2582289.jpg']]"
            ]
          },
          "execution_count": 55,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import os\n",
        "\n",
        "# Get example filepaths in a list of lists\n",
        "example_list = [[\"examples/\" + example] for example in os.listdir(foodvision_mini_examples_path)]\n",
        "example_list"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kAI5EE7DQaQm"
      },
      "source": [
        "### 8.5 Moving our trained EffNetB2 model to our FoodVision Mini demo directory\n",
        "\n",
        "We previously saved our FoodVision Mini EffNetB2 feature extractor model under `models/09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth`.\n",
        "\n",
        "And rather double up on saved model files, let's move our model to our `demos/foodvision_mini` directory.\n",
        "\n",
        "We can do so using Python's [`shutil.move()`](https://docs.python.org/3/library/shutil.html#shutil.move) method and passing in `src` (the source path of the target file) and `dst` (the destination path of the target file to be moved to) parameters."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-P1e9-YsQaQm",
        "outputId": "0892ce94-cf93-41cc-bc6f-43bd442e6d3d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[INFO] Attempting to move models/09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth to demos/foodvision_mini/09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth\n",
            "[INFO] Model move complete.\n"
          ]
        }
      ],
      "source": [
        "import shutil\n",
        "\n",
        "# Create a source path for our target model\n",
        "effnetb2_foodvision_mini_model_path = \"models/09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth\"\n",
        "\n",
        "# Create a destination path for our target model\n",
        "effnetb2_foodvision_mini_model_destination = foodvision_mini_demo_path / effnetb2_foodvision_mini_model_path.split(\"/\")[1]\n",
        "\n",
        "# Try to move the file\n",
        "try:\n",
        "    print(f\"[INFO] Attempting to move {effnetb2_foodvision_mini_model_path} to {effnetb2_foodvision_mini_model_destination}\")\n",
        "\n",
        "    # Move the model\n",
        "    shutil.move(src=effnetb2_foodvision_mini_model_path,\n",
        "                dst=effnetb2_foodvision_mini_model_destination)\n",
        "\n",
        "    print(f\"[INFO] Model move complete.\")\n",
        "\n",
        "# If the model has already been moved, check if it exists\n",
        "except:\n",
        "    print(f\"[INFO] No model found at {effnetb2_foodvision_mini_model_path}, perhaps its already been moved?\")\n",
        "    print(f\"[INFO] Model exists at {effnetb2_foodvision_mini_model_destination}: {effnetb2_foodvision_mini_model_destination.exists()}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z2QzGbqdQaQm"
      },
      "source": [
        "### 8.6 Turning our EffNetB2 model into a Python script (`model.py`)\n",
        "\n",
        "Our current model's `state_dict` is saved to `demos/foodvision_mini/09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth`.\n",
        "\n",
        "To load it in we can use `model.load_state_dict()` along with `torch.load()`.\n",
        "\n",
        "> **Note:** For a refresh on saving and loading a model (or a model's `state_dict` in PyTorch, see [01. PyTorch Workflow Fundamentals section 5: Saving and loading a PyTorch model](https://www.learnpytorch.io/01_pytorch_workflow/#5-saving-and-loading-a-pytorch-model) or see the PyTorch recipe for [What is a `state_dict` in PyTorch?](https://pytorch.org/tutorials/recipes/recipes/what_is_state_dict.html)\n",
        "\n",
        "But before we can do this, we first need a way to instantiate a `model`.\n",
        "\n",
        "To do this in a modular fashion we'll create a script called `model.py` which contains our `create_effnetb2_model()` function we created in [section 3.1: *Creating a function to make an EffNetB2 feature extractor*](https://www.learnpytorch.io/09_pytorch_model_deployment/#31-creating-a-function-to-make-an-effnetb2-feature-extractor).\n",
        "\n",
        "That way we can import the function in *another* script (see `app.py` below) and then use it to create our EffNetB2 `model` instance as well as get its appropriate transforms.\n",
        "\n",
        "Just like in [05. PyTorch Going Modular](https://www.learnpytorch.io/05_pytorch_going_modular/), we'll use the `%%writefile path/to/file` magic command to turn a cell of code into a file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qfBSLAVoQaQm",
        "outputId": "f1c82c2c-a00e-4c17-ded0-3b3a82977f1b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Writing demos/foodvision_mini/model.py\n"
          ]
        }
      ],
      "source": [
        "%%writefile demos/foodvision_mini/model.py\n",
        "import torch\n",
        "import torchvision\n",
        "\n",
        "from torch import nn\n",
        "\n",
        "\n",
        "def create_effnetb2_model(num_classes:int=3,\n",
        "                          seed:int=42):\n",
        "    \"\"\"Creates an EfficientNetB2 feature extractor model and transforms.\n",
        "\n",
        "    Args:\n",
        "        num_classes (int, optional): number of classes in the classifier head.\n",
        "            Defaults to 3.\n",
        "        seed (int, optional): random seed value. Defaults to 42.\n",
        "\n",
        "    Returns:\n",
        "        model (torch.nn.Module): EffNetB2 feature extractor model.\n",
        "        transforms (torchvision.transforms): EffNetB2 image transforms.\n",
        "    \"\"\"\n",
        "    # Create EffNetB2 pretrained weights, transforms and model\n",
        "    weights = torchvision.models.EfficientNet_B2_Weights.DEFAULT\n",
        "    transforms = weights.transforms()\n",
        "    model = torchvision.models.efficientnet_b2(weights=weights)\n",
        "\n",
        "    # Freeze all layers in base model\n",
        "    for param in model.parameters():\n",
        "        param.requires_grad = False\n",
        "\n",
        "    # Change classifier head with random seed for reproducibility\n",
        "    torch.manual_seed(seed)\n",
        "    model.classifier = nn.Sequential(\n",
        "        nn.Dropout(p=0.3, inplace=True),\n",
        "        nn.Linear(in_features=1408, out_features=num_classes),\n",
        "    )\n",
        "\n",
        "    return model, transforms"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "svPuApwYQaQm"
      },
      "source": [
        "### 8.7 Turning our FoodVision Mini Gradio app into a Python script (`app.py`)\n",
        "\n",
        "We've now got a `model.py` script as well as a path to a saved model `state_dict` that we can load in.\n",
        "\n",
        "Time to construct `app.py`.\n",
        "\n",
        "We call it `app.py` because by default when you create a HuggingFace Space, it looks for a file called `app.py` to run and host (though you can change this in settings).\n",
        "\n",
        "Our `app.py` script will put together all of the pieces of the puzzle to create our Gradio demo and will have four main parts:\n",
        "\n",
        "1. **Imports and class names setup** - Here we'll import the various dependencies for our demo including the `create_effnetb2_model()` function from `model.py` as well as setup the different class names for our FoodVision Mini app.\n",
        "2. **Model and transforms preparation** - Here we'll create an EffNetB2 model instance along with the transforms to go with it and then we'll load in the saved model weights/`state_dict`. When we load the model we'll also set `map_location=torch.device(\"cpu\")` in [`torch.load()`](https://pytorch.org/docs/stable/generated/torch.load.html) so our model gets loaded onto the CPU regardless of the device it trained on (we do this because we won't necessarily have a GPU when we deploy and we'll get an error if our model is trained on GPU but we try to deploy it to CPU without explicitly saying so).\n",
        "3. **Predict function** - Gradio's `gradio.Interface()` takes a `fn` parameter to map inputs to outputs, our `predict()` function will be the same as the one we defined above in [section 7.2: *Creating a function to map our inputs and outputs*](https://www.learnpytorch.io/09_pytorch_model_deployment/#72-creating-a-function-to-map-our-inputs-and-outputs), it will take in an image and then use the loaded transforms to preprocess it before using the loaded model to make a prediction on it.\n",
        "    * **Note:** We'll have to create the example list on the fly via the `examples` parameter. We can do so by creating a list of the files inside the `examples/` directory with: `[[\"examples/\" + example] for example in os.listdir(\"examples\")]`.\n",
        "4. **Gradio app** - This is where the main logic of our demo will live, we'll create a `gradio.Interface()` instance called `demo` to put together our inputs, `predict()` function and outputs. And we'll finish the script by calling `demo.launch()` to launch our FoodVision Mini demo!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GVOnDzs7QaQn",
        "outputId": "a67cc41b-851d-47e8-c97c-bc801cfaf13a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Writing demos/foodvision_mini/app.py\n"
          ]
        }
      ],
      "source": [
        "%%writefile demos/foodvision_mini/app.py\n",
        "### 1. Imports and class names setup ###\n",
        "import gradio as gr\n",
        "import os\n",
        "import torch\n",
        "\n",
        "from model import create_effnetb2_model\n",
        "from timeit import default_timer as timer\n",
        "from typing import Tuple, Dict\n",
        "\n",
        "# Setup class names\n",
        "class_names = [\"pizza\", \"steak\", \"sushi\"]\n",
        "\n",
        "### 2. Model and transforms preparation ###\n",
        "\n",
        "# Create EffNetB2 model\n",
        "effnetb2, effnetb2_transforms = create_effnetb2_model(\n",
        "    num_classes=3, # len(class_names) would also work\n",
        ")\n",
        "\n",
        "# Load saved weights\n",
        "effnetb2.load_state_dict(\n",
        "    torch.load(\n",
        "        f=\"09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth\",\n",
        "        map_location=torch.device(\"cpu\"),  # load to CPU\n",
        "    )\n",
        ")\n",
        "\n",
        "### 3. Predict function ###\n",
        "\n",
        "# Create predict function\n",
        "def predict(img) -> Tuple[Dict, float]:\n",
        "    \"\"\"Transforms and performs a prediction on img and returns prediction and time taken.\n",
        "    \"\"\"\n",
        "    # Start the timer\n",
        "    start_time = timer()\n",
        "\n",
        "    # Transform the target image and add a batch dimension\n",
        "    img = effnetb2_transforms(img).unsqueeze(0)\n",
        "\n",
        "    # Put model into evaluation mode and turn on inference mode\n",
        "    effnetb2.eval()\n",
        "    with torch.inference_mode():\n",
        "        # Pass the transformed image through the model and turn the prediction logits into prediction probabilities\n",
        "        pred_probs = torch.softmax(effnetb2(img), dim=1)\n",
        "\n",
        "    # Create a prediction label and prediction probability dictionary for each prediction class (this is the required format for Gradio's output parameter)\n",
        "    pred_labels_and_probs = {class_names[i]: float(pred_probs[0][i]) for i in range(len(class_names))}\n",
        "\n",
        "    # Calculate the prediction time\n",
        "    pred_time = round(timer() - start_time, 5)\n",
        "\n",
        "    # Return the prediction dictionary and prediction time\n",
        "    return pred_labels_and_probs, pred_time\n",
        "\n",
        "### 4. Gradio app ###\n",
        "\n",
        "# Create title, description and article strings\n",
        "title = \"FoodVision Mini 🍕🥩🍣\"\n",
        "description = \"An EfficientNetB2 feature extractor computer vision model to classify images of food as pizza, steak or sushi.\"\n",
        "article = \"Created at [09. PyTorch Model Deployment](https://www.learnpytorch.io/09_pytorch_model_deployment/).\"\n",
        "\n",
        "# Create examples list from \"examples/\" directory\n",
        "example_list = [[\"examples/\" + example] for example in os.listdir(\"examples\")]\n",
        "\n",
        "# Create the Gradio demo\n",
        "demo = gr.Interface(fn=predict, # mapping function from input to output\n",
        "                    inputs=gr.Image(type=\"pil\"), # what are the inputs?\n",
        "                    outputs=[gr.Label(num_top_classes=3, label=\"Predictions\"), # what are the outputs?\n",
        "                             gr.Number(label=\"Prediction time (s)\")], # our fn has two outputs, therefore we have two outputs\n",
        "                    # Create examples list from \"examples/\" directory\n",
        "                    examples=example_list,\n",
        "                    title=title,\n",
        "                    description=description,\n",
        "                    article=article)\n",
        "\n",
        "# Launch the demo!\n",
        "demo.launch()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ubnqogf7QaQn"
      },
      "source": [
        "### 8.8 Creating a requirements file for FoodVision Mini (`requirements.txt`)\n",
        "\n",
        "The last file we need to create for our FoodVision Mini app is a [`requirements.txt` file](https://learnpython.com/blog/python-requirements-file/).\n",
        "\n",
        "This will be a text file containing all of the required dependencies for our demo.\n",
        "\n",
        "When we deploy our demo app to Hugging Face Spaces, it will search through this file and install the dependencies we define so our app can run.\n",
        "\n",
        "The good news is, there's only three!\n",
        "\n",
        "1. `torch==1.12.0`\n",
        "2. `torchvision==0.13.0`\n",
        "3. `gradio==3.1.4`\n",
        "\n",
        "The \"`==1.12.0`\" states the version number to install.\n",
        "\n",
        "Defining the version number is not 100% required but we will for now so if any breaking updates occur in future releases, our app still runs (PS if you find any errors, feel free to post on the course [GitHub Issues](https://github.com/mrdbourke/pytorch-deep-learning/issues))."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZNytCBfOQaQn",
        "outputId": "cc537684-0a51-4a45-ef19-cb3e6bc8cfaa"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Writing demos/foodvision_mini/requirements.txt\n"
          ]
        }
      ],
      "source": [
        "%%writefile demos/foodvision_mini/requirements.txt\n",
        "torch==1.12.0\n",
        "torchvision==0.13.0\n",
        "gradio==3.1.4"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7i2QPXoRQaQn"
      },
      "source": [
        "Nice!\n",
        "\n",
        "We've officially got all the files we need to deploy our FoodVision Mini demo!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "X8_1QPOKQaQo"
      },
      "source": [
        "## 9. Deploying our FoodVision Mini app to HuggingFace Spaces\n",
        "\n",
        "We've got a file containing our FoodVision Mini demo, now how do we get it to run on Hugging Face Spaces?\n",
        "\n",
        "There are two main options for uploading to a Hugging Face Space (also called a [Hugging Face Repository](https://huggingface.co/docs/hub/repositories-getting-started#getting-started-with-repositories), similar to a git repository):\n",
        "1. [Uploading via the Hugging Face Web interface (easiest)](https://huggingface.co/docs/hub/repositories-getting-started#adding-files-to-a-repository-web-ui).\n",
        "2. [Uploading via the command line or terminal](https://huggingface.co/docs/hub/repositories-getting-started#terminal).\n",
        "    * **Bonus:** You can also use the [`huggingface_hub` library](https://huggingface.co/docs/huggingface_hub/index) to interact with Hugging Face, this would be a good extension to the above two options.\n",
        "\n",
        "Feel free to read the documentation on both options but we're going to go with option two.\n",
        "\n",
        "> **Note:** To host anything on Hugging Face, you will to [sign up for a free Hugging Face account](https://huggingface.co/join)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7RleoaG5QaQo"
      },
      "source": [
        "### 9.1 Downloading our FoodVision Mini app files\n",
        "\n",
        "Let's check out the demo files we've got inside `demos/foodvision_mini`.\n",
        "\n",
        "To do so we can use the `!ls` command followed by the target filepath.\n",
        "\n",
        "`ls` stands for \"list\" and the `!` means we want to execute the command at the shell level."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AeOMb8ZAQaQo",
        "outputId": "fc5f2e36-f2c9-41f7-8507-0b77b22f881f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth\n",
            "app.py\n",
            "examples\n",
            "model.py\n",
            "requirements.txt\n"
          ]
        }
      ],
      "source": [
        "!ls demos/foodvision_mini"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vdH3l3LCQaQo"
      },
      "source": [
        "These are all files that we've created!\n",
        "\n",
        "To begin uploading our files to Hugging Face, let's now download them from Google Colab (or wherever you're running this notebook).\n",
        "\n",
        "To do so, we'll first compress the files into a single zip folder via the command:\n",
        "\n",
        "```\n",
        "zip -r ../foodvision_mini.zip * -x \"*.pyc\" \"*.ipynb\" \"*__pycache__*\" \"*ipynb_checkpoints*\"\n",
        "```\n",
        "\n",
        "Where:\n",
        "* `zip` stands for \"zip\" as in \"please zip together the files in the following directory\".\n",
        "* `-r` stands for \"recursive\" as in, \"go through all of the files in the target directory\".\n",
        "* `../foodvision_mini.zip` is the target directory we'd like our files to be zipped to.\n",
        "* `*` stands for \"all the files in the current directory\".\n",
        "* `-x` stands for \"exclude these files\".\n",
        "\n",
        "We can download our zip file from Google Colab using [`google.colab.files.download(\"demos/foodvision_mini.zip\")`](https://colab.research.google.com/notebooks/io.ipynb) (we'll put this inside a `try` and `except` block just in case we're not running the code inside Google Colab, and if so we'll print a message saying to manually download the files).\n",
        "\n",
        "Let's try it out!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "tags": [],
        "id": "zrtAFm-yQaQo",
        "outputId": "267cef2a-d0c6-4862-f4aa-92ebd92ca3f1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "updating: 09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth (deflated 8%)\n",
            "updating: app.py (deflated 57%)\n",
            "updating: examples/ (stored 0%)\n",
            "updating: examples/3622237.jpg (deflated 0%)\n",
            "updating: examples/592799.jpg (deflated 1%)\n",
            "updating: examples/2582289.jpg (deflated 17%)\n",
            "updating: model.py (deflated 56%)\n",
            "updating: requirements.txt (deflated 4%)\n",
            "Not running in Google Colab, can't use google.colab.files.download(), please manually download.\n"
          ]
        }
      ],
      "source": [
        "# Change into and then zip the foodvision_mini folder but exclude certain files\n",
        "!cd demos/foodvision_mini && zip -r ../foodvision_mini.zip * -x \"*.pyc\" \"*.ipynb\" \"*__pycache__*\" \"*ipynb_checkpoints*\"\n",
        "\n",
        "# Download the zipped FoodVision Mini app (if running in Google Colab)\n",
        "try:\n",
        "    from google.colab import files\n",
        "    files.download(\"demos/foodvision_mini.zip\")\n",
        "except:\n",
        "    print(\"Not running in Google Colab, can't use google.colab.files.download(), please manually download.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rfDCzeU9QaQo"
      },
      "source": [
        "Woohoo!\n",
        "\n",
        "Looks like our `zip` command was successful.\n",
        "\n",
        "If you're running this notebook in Google Colab, you should see a file start to download in your browser.\n",
        "\n",
        "Otherwise, you can see the `foodvision_mini.zip` folder (and more) on the [course GitHub under the `demos/` directory](https://github.com/mrdbourke/pytorch-deep-learning/tree/main/demos)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "K63qe5s9QaQo"
      },
      "source": [
        "### 9.2 Running our FoodVision Mini demo locally\n",
        "\n",
        "If you download the `foodvision_mini.zip` file, you can test it locally by:\n",
        "1. Unzipping the file.\n",
        "2. Opening terminal or a command line prompt.\n",
        "3. Changing into the `foodvision_mini` directory (`cd foodvision_mini`).\n",
        "4. Creating an environment (`python3 -m venv env`).\n",
        "5. Activating the environment (`source env/bin/activate`).\n",
        "5. Installing the requirements (`pip install -r requirements.txt`, the \"`-r`\" is for recursive).\n",
        "    * **Note:** This step may take 5-10 minutes depending on your internet connection. And if you're facing errors, you may need to upgrade `pip` first: `pip install --upgrade pip`.\n",
        "6. Run the app (`python3 app.py`).\n",
        "\n",
        "This should result in a Gradio demo just like the one we built above running locally on your machine at a URL such as `http://127.0.0.1:7860/`.\n",
        "\n",
        "> **Note:** If you run the app locally and you notice a `flagged/` directory appear, it contains samples that have been \"flagged\".\n",
        ">\n",
        "> For example, if someone tries the demo and the model produces an incorrect result, the sample can be \"flagged\" and reviewed for later.\n",
        ">\n",
        "> For more on flagging in Gradio, see the [flagging documentation](https://gradio.app/docs/#flagging)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "tags": [],
        "id": "3cjxWoiiQaQo"
      },
      "source": [
        "### 9.3 Uploading to Hugging Face\n",
        "\n",
        "We've verfied our FoodVision Mini app works locally, however, the fun of creating a machine learning demo is to show it to other people and allow them to use it.\n",
        "\n",
        "To do so, we're going to upload our FoodVision Mini demo to Hugging Face.\n",
        "\n",
        "> **Note:** The following series of steps uses a Git (a file tracking system) workflow. For more on how Git works, I'd recommend going through the [Git and GitHub for Beginners tutorial](https://youtu.be/RGOj5yH7evk) on freeCodeCamp.\n",
        "\n",
        "1. [Sign up](https://huggingface.co/join) for a Hugging Face account.\n",
        "2. Start a new Hugging Face Space by going to your profile and then [clicking \"New Space\"](https://huggingface.co/new-space).\n",
        "    * **Note:** A Space in Hugging Face is also known as a \"code repository\" (a place to store your code/files) or \"repo\" for short.\n",
        "3. Give the Space a name, for example, mine is called `mrdbourke/foodvision_mini`, you can see it here: https://huggingface.co/spaces/mrdbourke/foodvision_mini\n",
        "4. Select a license (I used [MIT](https://opensource.org/licenses/MIT)).\n",
        "5. Select Gradio as the Space SDK (software development kit).\n",
        "   * **Note:** You can use other options such as Streamlit but since our app is built with Gradio, we'll stick with that.\n",
        "6. Choose whether your Space is it's public or private (I selected public since I'd like my Space to be available to others).\n",
        "7. Click \"Create Space\".\n",
        "8. Clone the repo locally by running something like: `git clone https://huggingface.co/spaces/[YOUR_USERNAME]/[YOUR_SPACE_NAME]` in terminal or command prompt.\n",
        "    * **Note:** You can also add files via uploading them under the \"Files and versions\" tab.\n",
        "9. Copy/move the contents of the downloaded `foodvision_mini` folder to the cloned repo folder.\n",
        "10. To upload and track larger files (e.g. files over 10MB or in our case, our PyTorch model file) you'll need to [install Git LFS](https://git-lfs.github.com/) (which stands for \"git large file storage\").\n",
        "11. After you've installed Git LFS, you can activate it by running `git lfs install`.\n",
        "12. In the `foodvision_mini` directory, track the files over 10MB with Git LFS with `git lfs track \"*.file_extension\"`.\n",
        "    * Track EffNetB2 PyTorch model file with `git lfs track \"09_pretrained_effnetb2_feature_extractor_pizza_steak_sushi_20_percent.pth\"`.\n",
        "13. Track `.gitattributes` (automatically created when cloning from HuggingFace, this file will help ensure our larger files are tracked with Git LFS). You can see an example `.gitattributes` file on the [FoodVision Mini Hugging Face Space](https://huggingface.co/spaces/mrdbourke/foodvision_mini/blob/main/.gitattributes).\n",
        "    * `git add .gitattributes`\n",
        "14. Add the rest of the `foodvision_mini` app files and commit them with:\n",
        "    * `git add *`\n",
        "    * `git commit -m \"first commit\"`\n",
        "15. Push (upload) the files to Hugging Face:\n",
        "    * `git push`\n",
        "16. Wait 3-5 minutes for the build to happen (future builds are faster) and your app to become live!\n",
        "\n",
        "If everything worked, you should see a live running example of our FoodVision Mini Gradio demo like the one here: https://huggingface.co/spaces/mrdbourke/foodvision_mini\n",
        "\n",
        "And we can even embed our FoodVision Mini Gradio demo into our notebook as an [iframe](https://gradio.app/sharing_your_app/#embedding-with-iframes) with [`IPython.display.IFrame`](https://ipython.readthedocs.io/en/stable/api/generated/IPython.display.html#IPython.display.IFrame) and a link to our space in the format `https://hf.space/embed/[YOUR_USERNAME]/[YOUR_SPACE_NAME]/+`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YonNrC4TQaQp",
        "outputId": "e815e0bf-f75c-479a-c10a-1b8453b83c27"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "        <iframe\n",
              "            width=\"900\"\n",
              "            height=\"750\"\n",
              "            src=\"https://hf.space/embed/mrdbourke/foodvision_mini/+\"\n",
              "            frameborder=\"0\"\n",
              "            allowfullscreen\n",
              "            \n",
              "        ></iframe>\n",
              "        "
            ],
            "text/plain": [
              "<IPython.lib.display.IFrame at 0x7f122dd77700>"
            ]
          },
          "execution_count": 62,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# IPython is a library to help make Python interactive\n",
        "from IPython.display import IFrame\n",
        "\n",
        "# Embed FoodVision Mini Gradio demo\n",
        "IFrame(src=\"https://hf.space/embed/mrdbourke/foodvision_mini/+\", width=900, height=750)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MDVguTYMQaQp"
      },
      "source": [
        "## 10. Creating FoodVision Big\n",
        "\n",
        "We've spent the past few sections and chapters working on bringing FoodVision Mini to life.\n",
        "\n",
        "And now we've seen it working in a live demo, how about we step things up a notch?\n",
        "\n",
        "How?\n",
        "\n",
        "FoodVision Big!\n",
        "\n",
        "Since FoodVision Mini is trained on pizza, steak and sushi images from the [Food101 dataset](https://pytorch.org/vision/main/generated/torchvision.datasets.Food101.html) (101 classes of food x 1000 images each), how about we make FoodVision Big by training a model on all 101 classes!\n",
        "\n",
        "We'll go from three classes to 101!\n",
        "\n",
        "From pizza, steak, sushi to pizza, steak, sushi, hot dog, apple pie, carrot cake, chocolate cake, french fires, garlic bread, ramen, nachos, tacos and more!\n",
        "\n",
        "How?\n",
        "\n",
        "Well, we've got all the steps in place, all we have to do is alter our EffNetB2 model slightly as well as prepare a different dataset.\n",
        "\n",
        "To finish Milestone Project 3, let's recreate a Gradio demo similar to FoodVision Mini (three classes) but for FoodVision Big (101 classes).\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-model-deployment-foodvision-mini-to-foodvision-big.png\" alt=\"foodvision mini model on three classes: pizza, steak, sushi and foodvision big on all of the 101 classes in the food101 dataset\" width=900/>\n",
        "\n",
        "*FoodVision Mini works with three food classes: pizza, steak and sushi. And FoodVision Big steps it up a notch to work across 101 food classes: all of the [classes in the Food101 dataset](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/extras/food101_class_names.txt).*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2D19gy87QaQp"
      },
      "source": [
        "### 10.1 Creating a model and transforms for FoodVision Big\n",
        "\n",
        "When creating FoodVision Mini we saw that the EffNetB2 model was a good tradeoff between speed and performance (it performed well with a fast speed).\n",
        "\n",
        "So we'll continue using the same model for FoodVision Big.\n",
        "\n",
        "We can create an EffNetB2 feature extractor for Food101 by using our `create_effnetb2_model()` function we created above, in [section 3.1](https://www.learnpytorch.io/09_pytorch_model_deployment/#31-creating-a-function-to-make-an-effnetb2-feature-extractor), and passing it the parameter `num_classes=101` (since Food101 has 101 classes)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oyF-b6-pQaQp"
      },
      "outputs": [],
      "source": [
        "# Create EffNetB2 model capable of fitting to 101 classes for Food101\n",
        "effnetb2_food101, effnetb2_transforms = create_effnetb2_model(num_classes=101)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tz0GbXQYQaQp"
      },
      "source": [
        "Beautiful!\n",
        "\n",
        "Let's now get a summary of our model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8AAFQBA6QaQq"
      },
      "outputs": [],
      "source": [
        "from torchinfo import summary\n",
        "\n",
        "# # Get a summary of EffNetB2 feature extractor for Food101 with 101 output classes (uncomment for full output)\n",
        "# summary(effnetb2_food101,\n",
        "#         input_size=(1, 3, 224, 224),\n",
        "#         col_names=[\"input_size\", \"output_size\", \"num_params\", \"trainable\"],\n",
        "#         col_width=20,\n",
        "#         row_settings=[\"var_names\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5vZWaH3MQaQq"
      },
      "source": [
        "<img src=\"https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/09-effnetb2-feature-extractor-101-classes.png\" width=900 alt=\"effnetb2 feature extractor with 100 output classes model summary\"/>\n",
        "                                                                                                                                                      \n",
        "Nice!\n",
        "\n",
        "See how just like our EffNetB2 model for FoodVision Mini the base layers are frozen (these are pretrained on ImageNet) and the outer layers (the `classifier` layers) are trainble with an ouput shape of `[batch_size, 101]` (`101` for 101 classes in Food101).\n",
        "\n",
        "Now since we're going to be dealing with a fair bit more data than usual, how about we add a little data augmentation to our transforms (`effnetb2_transforms`) to augment the training data.\n",
        "\n",
        "> **Note:** Data augmentation is a technique used to alter the appearance of an input training sample (e.g. rotating an image or slightly skewing it) to artificially increase the diversity of a training dataset to hopefully prevent overfitting. You can see more on data augmentation in [04. PyTorch Custom Datasets section 6](https://www.learnpytorch.io/04_pytorch_custom_datasets/#6-other-forms-of-transforms-data-augmentation).\n",
        "\n",
        "Let's compose a `torchvision.transforms` pipeline to use [`torchvision.transforms.TrivialAugmentWide()`](https://pytorch.org/vision/main/generated/torchvision.transforms.TrivialAugmentWide.html) (the same data augmentation used by the PyTorch team in their [computer vision recipes](https://pytorch.org/blog/how-to-train-state-of-the-art-models-using-torchvision-latest-primitives/#break-down-of-key-accuracy-improvements)) as well as the `effnetb2_transforms` to transform our training images."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_3AzZsypQaQq"
      },
      "outputs": [],
      "source": [
        "# Create Food101 training data transforms (only perform data augmentation on the training images)\n",
        "food101_train_transforms = torchvision.transforms.Compose([\n",
        "    torchvision.transforms.TrivialAugmentWide(),\n",
        "    effnetb2_transforms,\n",
        "])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GF_MfSzyQaQq"
      },
      "source": [
        "Epic!\n",
        "\n",
        "Now let's compare `food101_train_transforms` (for the training data) and `effnetb2_transforms` (for the testing/inference data)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_jnfhdr7QaQq",
        "outputId": "18793b5c-2521-4336-d7d5-b8df911af27d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Training transforms:\n",
            "Compose(\n",
            "    TrivialAugmentWide(num_magnitude_bins=31, interpolation=InterpolationMode.NEAREST, fill=None)\n",
            "    ImageClassification(\n",
            "    crop_size=[288]\n",
            "    resize_size=[288]\n",
            "    mean=[0.485, 0.456, 0.406]\n",
            "    std=[0.229, 0.224, 0.225]\n",
            "    interpolation=InterpolationMode.BICUBIC\n",
            ")\n",
            ")\n",
            "\n",
            "Testing transforms:\n",
            "ImageClassification(\n",
            "    crop_size=[288]\n",
            "    resize_size=[288]\n",
            "    mean=[0.485, 0.456, 0.406]\n",
            "    std=[0.229, 0.224, 0.225]\n",
            "    interpolation=InterpolationMode.BICUBIC\n",
            ")\n"
          ]
        }
      ],
      "source": [
        "print(f\"Training transforms:\\n{food101_train_transforms}\\n\")\n",
        "print(f\"Testing transforms:\\n{effnetb2_transforms}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TmgpPsCJQaQq"
      },
      "source": [
        "### 10.2 Getting data for FoodVision Big\n",
        "\n",
        "For FoodVision Mini, we made our own [custom data splits](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/extras/04_custom_data_creation.ipynb) of the entire Food101 dataset.\n",
        "\n",
        "To get the whole Food101 dataset, we can use [`torchvision.datasets.Food101()`](https://pytorch.org/vision/main/generated/torchvision.datasets.Food101.html).\n",
        "\n",
        "We'll first setup a path to directory `data/` to store the images.\n",
        "\n",
        "Then we'll download and transform the training and testing dataset splits using `food101_train_transforms` and `effnetb2_transforms` to transform each dataset respectively.\n",
        "\n",
        "> **Note:** If you're using Google Colab, the cell below will take ~3-5 minutes to fully run and download the Food101 images from PyTorch.\n",
        ">\n",
        "> This is because there is over 100,000 images being downloaded (101 classes x 1000 images per class). If you restart your Google Colab runtime and come back to this cell, the images will have to redownload. Alternatively, if you're running this notebook locally, the images will be cached and stored in the directory specified by the `root` parameter of `torchvision.datasets.Food101()`.  "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mO5bocLcQaQr"
      },
      "outputs": [],
      "source": [
        "from torchvision import datasets\n",
        "\n",
        "# Setup data directory\n",
        "from pathlib import Path\n",
        "data_dir = Path(\"data\")\n",
        "\n",
        "# Get training data (~750 images x 101 food classes)\n",
        "train_data = datasets.Food101(root=data_dir, # path to download data to\n",
        "                              split=\"train\", # dataset split to get\n",
        "                              transform=food101_train_transforms, # perform data augmentation on training data\n",
        "                              download=True) # want to download?\n",
        "\n",
        "# Get testing data (~250 images x 101 food classes)\n",
        "test_data = datasets.Food101(root=data_dir,\n",
        "                             split=\"test\",\n",
        "                             transform=effnetb2_transforms, # perform normal EffNetB2 transforms on test data\n",
        "                             download=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1K90qBZCQaQr"
      },
      "source": [
        "Data downloaded!\n",
        "\n",
        "Now we can get a list of all the class names using `train_data.classes`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "P3BB6B5RQaQr",
        "outputId": "ff042581-a033-40be-b00b-e74eda39f366"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['apple_pie',\n",
              " 'baby_back_ribs',\n",
              " 'baklava',\n",
              " 'beef_carpaccio',\n",
              " 'beef_tartare',\n",
              " 'beet_salad',\n",
              " 'beignets',\n",
              " 'bibimbap',\n",
              " 'bread_pudding',\n",
              " 'breakfast_burrito']"
            ]
          },
          "execution_count": 68,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Get Food101 class names\n",
        "food101_class_names = train_data.classes\n",
        "\n",
        "# View the first 10\n",
        "food101_class_names[:10]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cQpGMmVKQaQs"
      },
      "source": [
        "Ho ho! Those are some delicious sounding foods (although I've never heard of \"beignets\"... update: after a quick Google search, beignets also look delicious).\n",
        "                                                \n",
        "You can see a full list of the Food101 class names on the course GitHub under [`extras/food101_class_names.txt`](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/extras/food101_class_names.txt)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4F6GZSqtQaQs"
      },
      "source": [
        "### 10.3 Creating a subset of the Food101 dataset for faster experimenting\n",
        "\n",
        "This is optional.\n",
        "\n",
        "We don't *need* to create another subset of the Food101 dataset, we could train and evaluate a model across the whole 101,000 images.\n",
        "\n",
        "But to keep training fast, let's create a 20% split of the training and test datasets.\n",
        "\n",
        "Our goal will be to see if we can beat the original [Food101 paper's](https://data.vision.ee.ethz.ch/cvl/datasets_extra/food-101/) best results with only 20% of the data.\n",
        "\n",
        "To breakdown the datasets we've used/will use:\n",
        "\n",
        "| **Notebook(s)** | **Project name** | **Dataset** | **Number of classes** | **Training images** | **Testing images** |\n",
        "| ----- | ----- | ----- | ----- | ----- | ----- |\n",
        "| 04, 05, 06, 07, 08 | FoodVision Mini (10% data) | Food101 custom split | 3 (pizza, steak, sushi) | 225 | 75 |\n",
        "| 07, 08, 09 | FoodVision Mini (20% data) | Food101 custom split | 3 (pizza, steak, sushi) | 450 | 150 |\n",
        "| **09 (this one)** | FoodVision Big (20% data) | Food101 custom split | 101 (all Food101 classes) | 15150 | 5050 |\n",
        "| Extension | FoodVision Big | Food101 all data | 101 | 75750 | 25250 |\n",
        "\n",
        "Can you see the trend?\n",
        "\n",
        "Just like our model size slowly increased overtime, so has the size of the dataset we've been using for experiments.\n",
        "\n",
        "> **Note:** To truly beat the original Food101 paper's results with 20% of the data, we'd have to train a model on 20% of the training data and then evaluate our model on the *whole* test set rather than the split we created. I'll leave this as an extension exercise for you to try. I'd also encourage you to try training a model on the entire Food101 training dataset.\n",
        "\n",
        "To make our FoodVision Big (20% data) split, let's create a function called `split_dataset()` to split a given dataset into certain proportions.\n",
        "\n",
        "We can use [`torch.utils.data.random_split()`](https://pytorch.org/docs/stable/data.html#torch.utils.data.random_split) to create splits of given sizes using the `lengths` parameter.\n",
        "\n",
        "The `lengths` parameter accepts a list of desired split lengths where the total of the list must equal the overall length of the dataset.\n",
        "\n",
        "For example, with a dataset of size 100, you could pass in `lengths=[20, 80]` to receive a 20% and 80% split.\n",
        "\n",
        "We'll want our function to return two splits, one with the target length (e.g. 20% of the training data) and the other with the remaining length (e.g. the remaining 80% of the training data).\n",
        "\n",
        "Finally, we'll set `generator` parameter to a `torch.manual_seed()` value for reproducibility."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9rnqt2w4QaQs"
      },
      "outputs": [],
      "source": [
        "def split_dataset(dataset:torchvision.datasets, split_size:float=0.2, seed:int=42):\n",
        "    \"\"\"Randomly splits a given dataset into two proportions based on split_size and seed.\n",
        "\n",
        "    Args:\n",
        "        dataset (torchvision.datasets): A PyTorch Dataset, typically one from torchvision.datasets.\n",
        "        split_size (float, optional): How much of the dataset should be split?\n",
        "            E.g. split_size=0.2 means there will be a 20% split and an 80% split. Defaults to 0.2.\n",
        "        seed (int, optional): Seed for random generator. Defaults to 42.\n",
        "\n",
        "    Returns:\n",
        "        tuple: (random_split_1, random_split_2) where random_split_1 is of size split_size*len(dataset) and\n",
        "            random_split_2 is of size (1-split_size)*len(dataset).\n",
        "    \"\"\"\n",
        "    # Create split lengths based on original dataset length\n",
        "    length_1 = int(len(dataset) * split_size) # desired length\n",
        "    length_2 = len(dataset) - length_1 # remaining length\n",
        "\n",
        "    # Print out info\n",
        "    print(f\"[INFO] Splitting dataset of length {len(dataset)} into splits of size: {length_1} ({int(split_size*100)}%), {length_2} ({int((1-split_size)*100)}%)\")\n",
        "\n",
        "    # Create splits with given random seed\n",
        "    random_split_1, random_split_2 = torch.utils.data.random_split(dataset,\n",
        "                                                                   lengths=[length_1, length_2],\n",
        "                                                                   generator=torch.manual_seed(seed)) # set the random seed for reproducible splits\n",
        "    return random_split_1, random_split_2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WF-kAJbqQaQs"
      },
      "source": [
        "Dataset split function created!\n",
        "\n",
        "Now let's test it out by creating a 20% training and testing dataset split of Food101."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ATTj8WjWQaQs",
        "outputId": "4169bcc5-e085-429a-f23e-20a00646d028"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[INFO] Splitting dataset of length 75750 into splits of size: 15150 (20%), 60600 (80%)\n",
            "[INFO] Splitting dataset of length 25250 into splits of size: 5050 (20%), 20200 (80%)\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "(15150, 5050)"
            ]
          },
          "execution_count": 70,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Create training 20% split of Food101\n",
        "train_data_food101_20_percent, _ = split_dataset(dataset=train_data,\n",
        "                                                 split_size=0.2)\n",
        "\n",
        "# Create testing 20% split of Food101\n",
        "test_data_food101_20_percent, _ = split_dataset(dataset=test_data,\n",
        "                                                split_size=0.2)\n",
        "\n",
        "len(train_data_food101_20_percent), len(test_data_food101_20_percent)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PBfLGESCQaQs"
      },
      "source": [
        "Excellent!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aJsRnBs1QaQs"
      },
      "source": [
        "### 10.4 Turning our Food101 datasets into `DataLoader`s\n",
        "\n",
        "Now let's turn our Food101 20% dataset splits into `DataLoader`'s using `torch.utils.data.DataLoader()`.\n",
        "\n",
        "We'll set `shuffle=True` for the training data only and the batch size to `32` for both datasets.\n",
        "\n",
        "And we'll set `num_workers` to `4` if the CPU count is available or `2` if it's not (though the value of `num_workers` is very experimental and will depend on the hardware you're using, there's an [active discussion thread about this on the PyTorch forums](https://discuss.pytorch.org/t/guidelines-for-assigning-num-workers-to-dataloader/813))."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pDWhQB2ZQaQs"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import torch\n",
        "\n",
        "BATCH_SIZE = 32\n",
        "NUM_WORKERS = 2 if os.cpu_count() <= 4 else 4 # this value is very experimental and will depend on the hardware you have available, Google Colab generally provides 2x CPUs\n",
        "\n",
        "# Create Food101 20 percent training DataLoader\n",
        "train_dataloader_food101_20_percent = torch.utils.data.DataLoader(train_data_food101_20_percent,\n",
        "                                                                  batch_size=BATCH_SIZE,\n",
        "                                                                  shuffle=True,\n",
        "                                                                  num_workers=NUM_WORKERS)\n",
        "# Create Food101 20 percent testing DataLoader\n",
        "test_dataloader_food101_20_percent = torch.utils.data.DataLoader(test_data_food101_20_percent,\n",
        "                                                                 batch_size=BATCH_SIZE,\n",
        "                                                                 shuffle=False,\n",
        "                                                                 num_workers=NUM_WORKERS)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9foCcnQ5QaQt"
      },
      "source": [
        "### 10.5 Training FoodVision Big model\n",
        "\n",
        "FoodVision Big model and `DataLoader`s ready!\n",
        "\n",
        "Time for training.\n",
        "\n",
        "We'll create an optimizer using `torch.optim.Adam()` and a learning rate of `1e-3`.\n",
        "\n",
        "And because we've got so many classes, we'll also setup a loss function using `torch.nn.CrossEntropyLoss()` with `label_smoothing=0.1`, inline with [`torchvision`'s state-of-the-art training recipe](https://pytorch.org/blog/how-to-train-state-of-the-art-models-using-torchvision-latest-primitives/#label-smoothing).\n",
        "\n",
        "What's [**label smoothing**](https://paperswithcode.com/method/label-smoothing)?\n",
        "\n",
        "Label smoothing is a regularization technique (regularization is another word to describe the process of [preventing overfitting](https://www.learnpytorch.io/04_pytorch_custom_datasets/#81-how-to-deal-with-overfitting)) that reduces the value a model gives to anyone label and spreads it across the other labels.\n",
        "\n",
        "In essence, rather than a model getting *too confident* on a single label, label smoothing gives a non-zero value to other labels to help aid in generalization.\n",
        "\n",
        "For example, if a model *without* label smoothing had the following outputs for 5 classes:\n",
        "\n",
        "```\n",
        "[0, 0, 0.99, 0.01, 0]\n",
        "```\n",
        "\n",
        "A model *with* label smoothing may have the following outputs:\n",
        "\n",
        "```\n",
        "[0.01, 0.01, 0.96, 0.01, 0.01]\n",
        "```\n",
        "\n",
        "The model is still confident on its prediction of class 3 but giving small values to the other labels forces the model to at least consider other options.\n",
        "\n",
        "Finally, to keep things quick, we'll train our model for five epochs using the `engine.train()` function we created in [05. PyTorch Going Modular section 4](https://www.learnpytorch.io/05_pytorch_going_modular/#4-creating-train_step-and-test_step-functions-and-train-to-combine-them) with the goal of beating the original Food101 paper's result of 56.4% accuracy on the test set.\n",
        "\n",
        "Let's train our biggest model yet!\n",
        "\n",
        "> **Note:** Running the cell below will take ~15-20 minutes to run on Google Colab. This is because it's training the biggest model with the largest amount of data we've used so far (15,150 training images, 5050 testing images). And it's a reason we decided to split 20% of the full Food101 dataset off before (so training didn't take over an hour)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mGNd7RddQaQt",
        "outputId": "9fc3b7c0-924c-4a67-acab-40c1164d345e",
        "colab": {
          "referenced_widgets": [
            "41ba5e6cee154970aa960f83784b421f"
          ]
        }
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "41ba5e6cee154970aa960f83784b421f",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/5 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch: 1 | train_loss: 3.6317 | train_acc: 0.2869 | test_loss: 2.7670 | test_acc: 0.4937\n",
            "Epoch: 2 | train_loss: 2.8615 | train_acc: 0.4388 | test_loss: 2.4653 | test_acc: 0.5387\n",
            "Epoch: 3 | train_loss: 2.6585 | train_acc: 0.4844 | test_loss: 2.3547 | test_acc: 0.5649\n",
            "Epoch: 4 | train_loss: 2.5494 | train_acc: 0.5116 | test_loss: 2.3038 | test_acc: 0.5755\n",
            "Epoch: 5 | train_loss: 2.5006 | train_acc: 0.5239 | test_loss: 2.2805 | test_acc: 0.5810\n"
          ]
        }
      ],
      "source": [
        "from going_modular.going_modular import engine\n",
        "\n",
        "# Setup optimizer\n",
        "optimizer = torch.optim.Adam(params=effnetb2_food101.parameters(),\n",
        "                             lr=1e-3)\n",
        "\n",
        "# Setup loss function\n",
        "loss_fn = torch.nn.CrossEntropyLoss(label_smoothing=0.1) # throw in a little label smoothing because so many classes\n",
        "\n",
        "# Want to beat original Food101 paper with 20% of data, need 56.4%+ acc on test dataset\n",
        "set_seeds()\n",
        "effnetb2_food101_results = engine.train(model=effnetb2_food101,\n",
        "                                        train_dataloader=train_dataloader_food101_20_percent,\n",
        "                                        test_dataloader=test_dataloader_food101_20_percent,\n",
        "                                        optimizer=optimizer,\n",
        "                                        loss_fn=loss_fn,\n",
        "                                        epochs=5,\n",
        "                                        device=device)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nEYWx-QtQaQt"
      },
      "source": [
        "Woohoo!!!!\n",
        "\n",
        "Looks like we beat the original Food101 paper's results of 56.4% accuracy with only 20% of the training data (though we only evaluated on 20% of the testing data too, to fully replicate the results, we could evaluate on 100% of the testing data).\n",
        "\n",
        "That's the power of transfer learning!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vW0k45DsQaQt"
      },
      "source": [
        "### 10.6 Inspecting loss curves of FoodVision Big model\n",
        "\n",
        "Let's make our FoodVision Big loss curves visual.\n",
        "\n",
        "We can do so with the `plot_loss_curves()` function from `helper_functions.py`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_Dizf6JvQaQt",
        "outputId": "822bb63c-2224-425f-8b76-e69e8e4dedf4"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2oAAAG5CAYAAAD/HsejAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAB+3ElEQVR4nOzdd3xUVeL+8c9J74F0SAgJvfcOUkQRsCCi2BWxrHV1V/3p162uu67uqqu7WJZV7F1ELBRROiJVOkhLgNBbICRA2vn9cQcIMUCAJHcmed6v17xIZu5MntyEzDxzzj3XWGsRERERERER7+HndgARERERERE5mYqaiIiIiIiIl1FRExERERER8TIqaiIiIiIiIl5GRU1ERERERMTLqKiJiIiIiIh4GRU1ERERERERL6OiJnIejDGZxpiL3M4hIiJS2Ywx040x+40xwW5nEakJVNRERERE5LSMMWnABYAFrqjCrxtQVV9LxNuoqIlUMGNMsDHmRWPMNs/lxWPvPhpj4owxXxtjso0x+4wxs4wxfp7bHjPGbDXG5BhjfjbG9Hf3OxERETnuFuBH4C3g1mNXGmPqGWM+N8bsNsbsNcaMKnHbncaY1Z7ntVXGmA6e660xplGJ7d4yxvzV83FfY0yW5zlxB/CmMaa257lzt2dE72tjTEqJ+8cYY970POfuN8Z84bl+hTHm8hLbBRpj9hhj2lXSPhKpUCpqIhXvd0A3oB3QFugC/N5z28NAFhAPJAJPANYY0xS4H+hsrY0ELgEyqzS1iIjIqd0CvO+5XGKMSTTG+ANfA5uANCAZ+AjAGHMN8GfP/aJwRuH2lvNrJQExQH3gLpzXq296Pk8FDgOjSmz/LhAGtAQSgH95rn8HuKnEdoOB7dbaJeXMIeIqDSeLVLwbgQestbsAjDFPAv8F/gAUAHWA+tba9cAszzZFQDDQwhiz21qb6UZwERGR0owxvXBK0ifW2j3GmA3ADTgjbHWBR621hZ7NZ3v+vQP4h7V2gefz9WfxJYuBP1lrj3o+PwyMLZHnb8A0z8d1gEFArLV2v2eTGZ5/3wP+YIyJstYeBG7GKXUiPkEjaiIVry7Ou4vHbPJcB/BPnCerb40xG40xjwN4SttDOO8+7jLGfGSMqYuIiIj7bgW+tdbu8Xz+gee6esCmEiWtpHrAhnP8eruttUeOfWKMCTPG/NcYs8kYcxCYCdTyjOjVA/aVKGnHWWu3AXOAYcaYWjiF7v1zzCRS5VTURCreNpx3Ho9J9VyHtTbHWvuwtbYBcDnw22PHollrP7DWHnvX0gLPVm1sERGRkxljQoHhQB9jzA7PcWO/wZnavxNIPcWCH1uAhqd42DycqYrHJJW63Zb6/GGgKdDVWhsF9D4Wz/N1YjxFrCxv40x/vAaYa63deortRLyOiprI+Qs0xoQcuwAfAr83xsQbY+KAP+JMv8AYc5kxppExxgAHgSKgyBjT1BhzoWfRkSM40zyK3Pl2REREjrsS5/moBc6x1+2A5jhT968EtgPPGGPCPc+DPT33ex14xBjT0TgaGWOOvYm5BLjBGONvjBkI9DlDhkic58VsY0wM8KdjN1hrtwMTgVc8i44EGmN6l7jvF0AH4EGcY9ZEfIaKmsj5m4DzBHLsEgIsBJYBy4HFwF892zYGvgMOAXOBV6y103GOT3sG2APswDkY+okq+w5ERETKdivwprV2s7V2x7ELzmIe1+PMDmkEbMZZLOtaAGvtp8DfcKZJ5uAUphjPYz7ouV82znHdX5whw4tAKM5z5I/ApFK334xzDPgaYBfOoQR4chw7vi0d+Lz837aI+4y1pUeXRURERESqB2PMH4Em1tqbzrixiBfRqo8iIiIiUi15pkrejjPqJuJTNPVRRERERKodY8ydOIuNTLTWznQ7j8jZ0tRHERERERERL6MRNRERERERES/j2jFqcXFxNi0tza0vLyIiVWjRokV7rLXxbufwFXqOFBGpGU73/OhaUUtLS2PhwoVufXkREalCxphNbmfwJXqOFBGpGU73/KipjyIiIiIiIl5GRU1ERERERMTLqKiJiIiIiIh4GZ3wWkRqvIKCArKysjhy5IjbUXxeSEgIKSkpBAYGuh2l2tHvqW/R/wUROV8qaiJS42VlZREZGUlaWhrGGLfj+CxrLXv37iUrK4v09HS341Q7+j31Hfq/ICIVQVMfRaTGO3LkCLGxsXrxe56MMcTGxmrEp5Lo99R36P+CiFQEFTUREdCL3wqi/Vi5tH99h35WInK+VNRERERERES8jIqaiIiIiIiIl1FRExFxWXZ2Nq+88spZ32/w4MFkZ2ef9f1GjBjBZ599dtb3k5qtqn9PRURqOhU1ERGXneoFcFFR0WnvN2HCBGrVqlVJqUROVl1/T8+UX0TELVqeX0SkhCe/WsmqbQcr9DFb1I3iT5e3POXtjz/+OBs2bKBdu3YEBgYSERFBnTp1WLJkCatWreLKK69ky5YtHDlyhAcffJC77roLgLS0NBYuXMihQ4cYNGgQvXr14ocffiA5OZnx48cTGhp6xmzff/89jzzyCIWFhXTu3JlXX32V4OBgHn/8cb788ksCAgIYMGAAzz33HJ9++ilPPvkk/v7+REdHM3PmzArbR3J2asLv6f/+9z9Gjx5Nfn4+jRo14t133yUsLIydO3dy9913s3HjRgBeffVVevTowTvvvMNzzz2HMYY2bdrw7rvvMmLECC677DKuvvpqACIiIjh06BDTp0/nySefLFf+SZMm8cQTT1BUVERcXBxTpkyhadOm/PDDD8THx1NcXEyTJk348ccfiYuLq8gfiYjUcCpqIiIue+aZZ1ixYgVLlixh+vTpXHrppaxYseL4+ZfGjBlDTEwMhw8fpnPnzgwbNozY2NiTHmPdunV8+OGH/O9//2P48OGMHTuWm2666bRf98iRI4wYMYLvv/+eJk2acMstt/Dqq69yyy23MG7cONasWYMx5vi0tb/85S9MnjyZ5ORkTWWrgar69/Sqq67izjvvBOD3v/89b7zxBg888AC//vWv6dOnD+PGjaOoqIhDhw6xcuVK/va3vzFnzhzi4uLYt2/fGb+f+fPnnzF/cXExd955JzNnziQ9PZ19+/bh5+fHTTfdxPvvv89DDz3Ed999R9u2bVXSRKTCqaiJiJRwuhGFqtKlS5eTTpL773//m3HjxgGwZcsW1q1b94sXwOnp6bRr1w6Ajh07kpmZecav8/PPP5Oenk6TJk0AuPXWW3n55Ze5//77CQkJ4Y477uDSSy/lsssuA6Bnz56MGDGC4cOHc9VVV1XAdyrnqib8nq5YsYLf//73ZGdnc+jQIS655BIApk6dyjvvvANwfHT3nXfe4eqrrz5elmJiYiok/+7du+ndu/fx7Y497siRIxkyZAgPPfQQY8aM4bbbbjvj1xMROVs+e4yatZZFm/ZRXGzdjiIiUqHCw8OPfzx9+nS+++475s6dy9KlS2nfvn2ZJ9ENDg4+/rG/vz+FhYVn/DrWlv33MyAggPnz5zNs2DC++OILBg4cCMBrr73GX//6V7Zs2UK7du3Yu3fv2X5rUo1U9u/piBEjGDVqFMuXL+dPf/rTaU8eba0t87xlAQEBFBcXH98mPz//rPKf6nHr1atHYmIiU6dOZd68eQwaNOiU2USkmikuhrx9sGc97N9UqV/KZ0fUvl21k1+9u4i3R3ahT5N4t+OIiJyzyMhIcnJyyrztwIED1K5dm7CwMNasWcOPP/5YYV+3WbNmZGZmsn79+uPHAPXp04dDhw6Rl5fH4MGD6datG40aNQJgw4YNdO3ala5du/LVV1+xZcuWX4yYSPVV1b+nOTk51KlTh4KCAt5//32Sk5MB6N+/P6+++ioPPfQQRUVF5Obm0r9/f4YOHcpvfvMbYmNj2bdvHzExMaSlpbFo0SKGDx/O+PHjKSgoOKv83bt357777iMjI+P41Mdjo2p33HEHN910EzfffDP+/v7n/f2KiAuKCpzSdXif82/e3lIf7y9x+17n4yPZYJ03gGg1DK4eU2nxfLao9WuaQHxkMGNmZ6ioiYhPi42NpWfPnrRq1YrQ0FASExOP3zZw4EBee+012rRpQ9OmTenWrVuFfd2QkBDefPNNrrnmmuOLidx9993s27ePIUOGHB9R+Ne//gXAo48+yrp167DW0r9/f9q2bVthWcT7VfXv6VNPPUXXrl2pX78+rVu3Pl4SX3rpJe666y7eeOMN/P39efXVV+nevTu/+93v6NOnD/7+/rRv35633nqLO++8kyFDhtClSxf69+9/0ihaSafKHx8fz+jRo7nqqqsoLi4mISGBKVOmAHDFFVdw2223adqjiLfIzztF4SpRxEoWrsP74ehpFmUKCIGwWAiNgbDakNTa83HMievjGlXqt2RONfWlsnXq1MkuXLjwvB7jP9+v4/kpa/nut71plBBZQclEpKZZvXo1zZs3dztGtVHW/jTGLLLWdnIpks8p6zlSv6feZeHChfzmN79h1qxZp9xGPzORc2CtU6Dy9kLe/rLL1/HCtf/Ex4Wnnh5NcBSE1nYKVlhMqcJV++Tydez2oLAq+XZP9/zosyNqADd0TeU/09YzZk4mTw9t7XYcERERqQGeeeYZXn31Vd5//323o4h4t+KiUtMHyzG98PB+KD7F8avGD0JqnShc0SlQp00Z5atE4QqtDQFBVfptVxSfLmqxEcEMbZfM54uzeHRAU2qH++YPQUSkMtx3333MmTPnpOsefPBBTdUSr+KLv6ePP/44jz/+uNsxRKpW4dEypg+eYXrhkQPAKWbv+QWePMIV1+T0hSssxilpfj67FuJZ8+miBnBbrzQ+XriFDxds5t6+lTtPVETEl7z88stuRxA5I/2eilQxayE/95fHa5U52rXvxPTC/EOnfszAcE+h8kwvrJVaRuGqfXL5CoqAMlZVlRPOWNSMMSHATCDYs/1n1to/lbFdX+BFIBDYY63tU5FBT6VZUhS9GsXxzg+buPOCBgT615yWLSIiIiJynLWQsx32boB9G2HfBmcJ+ZNGwPZCUf6pHyMk2lOmYiEiEeKbe0a+ap96tCswpOq+xxqkPCNqR4ELrbWHjDGBwGxjzERr7fG1d40xtYBXgIHW2s3GmITKiVu2kb3SGPnWQiYs386QdslV+aVFRERERKqOtXBop6eMbShRyjyXgrwT2/oHOaNb4fFQOw2SO5x6emFYrDO10N/nJ9xVG2f8SVhnWchjY52BnkvpyaY3AJ9bazd77rOrIkOeSd8mCaTHhTNmdgZXtK1b5skpRURERER8grWQu7tUGTtWyDJOnoboF+iUsJgGkN7b+Te2IcQ0dBbb8NN5/nxVuSqzMcYfWAQ0Al621s4rtUkTINAYMx2IBF6y1r5TxuPcBdwFkJqaeh6xT+bnZ7itZxp/HL+SxZuz6Vi/doU9toiIiEB2djYffPAB995771nf98UXX+Suu+4iLKxqlrsW8QnWQu6eE1MUTyplGZBf4gTzfgFQq75TwOr38pSxBp4yVk+jYNVUuX6q1toioJ1niuM4Y0wra+2KUo/TEegPhAJzjTE/WmvXlnqc0cBocM4RUwH5jxvWIYXnJv/MmDkZKmoi4lMq+wVwWloaCxcuJC4u7nxiSg2XnZ3NK6+8cs6/pzfddJNXFLXCwkICAvSiVqqItc6xYcdGw0pPVyx5wmXj70xTjG0Iqd09o2INnEutVPAPdO/7EFec1V8qa222Z9RsIFCyqGXhLCCSC+QaY2YCbYG1v3yUyhEeHMD1XVJ5fXYGW7MPk1wrtKq+tIjIeakuL4Clenv88cfZsGED7dq14+KLLyYhIYFPPvmEo0ePMnToUJ588klyc3MZPnw4WVlZFBUV8Yc//IGdO3eybds2+vXrR1xcHNOmTSvz8e+55x4WLFjA4cOHufrqq3nyyScBWLBgAQ8++CC5ubkEBwfz/fffExYWxmOPPcbkyZMxxnDnnXfywAMPnPSmxMKFC3nkkUeYPn06f/7zn9m2bRuZmZnExcXx9NNPc/PNN5ObmwvAqFGj6NGjBwD/+Mc/ePfdd/Hz82PQoEHceeedXHPNNSxevBiAdevWcd1117Fo0aIq2OviM/L2lShipUbIjhw4sZ3xc0bAYhtCSucTUxRjG6qMyS+UZ9XHeKDAU9JCgYuAZ0ttNh4YZYwJAIKArsC/KjrsmdzSI43XZ2fwzg+Z/N/g5lX95UWkOpj4OOxYXrGPmdQaBj1zypsr+wVwSS+88AJjxowB4I477uChhx4q87GvvfZaHn/8cb788ksCAgIYMGAAzz33XIXtEjlPLvyePvPMM6xYsYIlS5bw7bff8tlnnzF//nystVxxxRXMnDmT3bt3U7duXb755hsADhw4QHR0NC+88ALTpk077aju3/72N2JiYigqKqJ///4sW7aMZs2ace211/Lxxx/TuXNnDh48SGhoKKNHjyYjI4OffvqJgIAA9u3bd8Zvb9GiRcyePZvQ0FDy8vKYMmUKISEhrFu3juuvv56FCxcyceJEvvjiC+bNm0dYWBj79u0jJiaG6OholixZQrt27XjzzTcZMWLEWe9eqQYOZ3sK2MZSi3hscJa3P85ArXpOAWt1dakyVt9nT74sVa88I2p1gLc9x6n5AZ9Ya782xtwNYK19zVq72hgzCVgGFAOvl5oaWSWSa4UysGUSH87fzK/7NyY8WFMbRMT7VfYL4GMWLVrEm2++ybx587DW0rVrV/r06cPGjRt/8dj79u1j3LhxrFmzBmMM2dnZlbkLxMd8++23fPvtt7Rv3x6AQ4cOsW7dOi644AIeeeQRHnvsMS677DIuuOCCcj/mJ598wujRoyksLGT79u2sWrUKYwx16tShc+fOAERFRQHw3Xffcffddx+fwhgTE3PGx7/iiisIDXVm2xQUFHD//fezZMkS/P39Wbt27fHHve22246PUB973DvuuIM333yTF154gY8//pj58+eX+/sSH3Pk4MklrOQiHnl7S2xonIU6YhpAy6GeKYqeMlY7DQKC3foOpBopz6qPy4D2ZVz/WqnP/wn8s+KinZuRvdL5Zvl2Pl+cxc3d09yOIyK+5jQjClWhMl4AHzN79myGDh1KeHg4AFdddRWzZs1i4MCBv3jswsJCQkJCuOOOO7j00ku57LLLKvT7lPPk8u+ptZb/+7//41e/+tUvblu0aBETJkzg//7v/xgwYAB//OMfz/h4GRkZPPfccyxYsIDatWszYsQIjhw5grW2zJWcT3V9QEAAxcXFABw5cuSk24793gP861//IjExkaVLl1JcXExISMhpH3fYsGE8+eSTXHjhhXTs2JHY2Ngzfk/ixY7mlJqiWGKELG/PydtGJTslrPnlJ4pYTAOona5zh0mlq3ZDTh1Sa9G2Xi3GzMnkxq718fPTUv0i4jsq+gVw6ccuS5MmTcp87Pnz5/P999/z0UcfMWrUKKZOnXpO35NUD5GRkeTkOKvQXXLJJfzhD3/gxhtvJCIigq1btxIYGEhhYSExMTHcdNNNRERE8NZbb51031ON/B48eJDw8HCio6PZuXMnEydOpG/fvjRr1oxt27axYMECOnfuTE5ODqGhoQwYMIDXXnuNvn37Hp/6GBMTQ1paGosWLWLQoEGMHTv2lN/LgQMHSElJwc/Pj7fffpuioiIABgwYwF/+8hduuOGGk6Y+hoSEcMkll3DPPffwxhtvVOyOlcpx9NCJIrZv48llLLfUWaQi6zrlq9ngEmXMMzIWpON/xT3VrqgZYxjZM40HP1rC9LW7uLBZotuRREROqzJfAJfUu3dvRowYweOPP461lnHjxvHuu++ybdu2Xzz2oUOHyMvLY/DgwXTr1o1GjRpV5i4QHxAbG0vPnj1p1aoVgwYN4oYbbqB79+4ARERE8N5777F+/XoeffRR/Pz8CAwM5NVXXwXgrrvuYtCgQdSpU6fMYynbtm1L+/btadmyJQ0aNKBnz54ABAUF8fHHH/PAAw9w+PBhQkND+e6777jjjjtYu3Ytbdq0ITAwkDvvvJP777+fP/3pT9x+++08/fTTdO3a9ZTfy7333suwYcP49NNP6dev3/HRtoEDB7JkyRI6depEUFAQgwcP5umnnwbgxhtv5PPPP2fAgAEVul/lPOTnlVq4Y+OJ6YqHdpy8bUSSU8CaDDi5jMWkQ1B42Y8v4jJzqndYK1unTp3swoULK+WxC4qKueDZaTRKiOC9O079h1pEBGD16tU0b+7uAkQ33HADy5YtY9CgQaSkpPD6668Dp38B3KlTJ/7zn//w8ssvn/IFMJy8PH9Zi4lMnjz5F4+dnJzMkCFDjk8/e+SRR7j11lvL9b2UtT+NMYustZ3OYxfVKGU9R3rD72lN9txzz3HgwAGeeuqpct9HP7MKUHDYOafYSSd9znA+ztl28rbhCSUW7mhw4rixmAYQHOFOfpEzON3zY7UsagCvTF/PPyb9zOSHetM0KbLSvo6I+D69mKpYKmrnT0XNuwwdOpQNGzYwderUszofoX5mZ+HgNtj20y/PM3Zw68nbhcWVKmMlzjUWEuVOdpHzcLrnx2o39fGY6zun8u/v1zFmdgbPXt3G7TgiIiICdO3alaNHj5503bvvvkvr1q1dSnRm48aNcztC9XQ4G1aNh+WfQuZswDN4EBbrFK+0C04s3nHs35BoNxOLVKlqW9RqhwdxVYcUPluUxf8b2JTYCC2TKiLVmy++AJaaZ968eW5HEDcVHIF138LyT2DtZCjKh9hG0O8JaNjfGSULre12ShGvUG2LGsDInml8MG8zH8zbzAP9G7sdR0S82KmW5fYl3vAC2K3p9DVFdfg9rSn0f6GE4mLYNAeWfQyrvoSjB5zjyTrfAa2vgbrtQb/XIr9QrYtao4RI+jSJ550fN3FXnwYEB/i7HUlEvFBISAh79+4lNjZWL4LPg7WWvXv3Hj8nlVQs/Z76Dv1f8NixwilnK8Y6x5oFRTjnI2t9DaT3Af9q/TJU5LxV+/8hI3ulc+uY+XyzbDtXdUhxO46IeKGUlBSysrLYvXu321F8XkhICCkp+ltbGfR76ltq7P+F7C3OMWfLP4Vdq8AvABpdBBf/BZoO1nnJRM5CtS9qvRvH0SghgjdmZzC0fbLehRSRXwgMDCQ9Pd3tGCKnpd9T8Vp5+04sCrJpjnNdva5w6fPQYiiEx7qbT8RHVfui5pwAO50nxi1nQeZ+uqTHuB1JRERExLcVHIG1k2DZJ87iIMUFENcELvy9M7WxdprbCUV8XrUvagBD2yfzj8lreGP2RhU1ERERkXNRXOQso7/sE1j9JRw9CBFJ0PVXTjmr01aLgohUoBpR1EKD/LmhSyqvztjA5r15pMZqfrSIiIjIGVkLO5afWBQkZzsERUKLKzyLgvQGPy3WJlIZakRRA7ilexqjZ27k7bmZ/OGyFm7HEREREfFe+zedWBRk9xrwC4TGF0Prp6HpIAgMdTuhSLVXY4paUnQIl7apw8cLtvDQRY2JDAl0O5KIiIiI98jbByvHOeVs81znutTucOkL0HIohOnwEZGqVGOKGsBtPdMZv2Qbny7MYmQvrZwlIiIiNVzBYfh5olPO1k1xFgWJbwYX/sGzKEh9txOK1Fg1qqi1q1eLjvVr89YPmdzaIw1/Px3wKiIiIjVMcRFkzPQsCvIV5OdAZB1nUZA210JSay0KIuIFalRRAxjZM537PljM96t3MqBlkttxRERERCqftbB9qVPOVoyFQzsgOApaDoHWwyGtlxYFEfEyNa6oXdIykeRaoYyZk6GiJiIiItXb/kxY9iks/wT2rHUWBWlyiTOtscklWhRExIvVuKIW4O/HrT3q8/SENazcdoCWdaPdjiQiIl7OGDMQeAnwB1631j5T6va+wHggw3PV59bav3huywRygCKg0FrbqWpSS42VuxdWfu4cd7ZlnnNd/Z7Q7V5oMUSLgoj4iBpX1ACu7ZTKi9+tY8zsTJ4f3tbtOCIi4sWMMf7Ay8DFQBawwBjzpbV2ValNZ1lrLzvFw/Sz1u6pzJxSw+Xnwc8TnHK2/jsoLoT45tD/T9D6aqiV6nZCETlLNbKoRYcFcnXHFD6av4XHBjUlITLE7UgiIuK9ugDrrbUbAYwxHwFDgNJFTaRqFRVCxgynnK3+CvIPQWRdZ+SszXBIbKVFQUR8WI0sauAs1f/O3E28/+NmfnNxE7fjiIiI90oGtpT4PAvoWsZ23Y0xS4FtwCPW2pWe6y3wrTHGAv+11o4u64sYY+4C7gJITdXoh5yCtbDtJ8/JqD+D3F0QHO2c56zNtc4URz8/t1OKSAWosUUtPS6c/s0SeO/HTdzTtyEhgVrpSEREylTWkIQt9flioL619pAxZjDwBdDYc1tPa+02Y0wCMMUYs8ZaO/MXD+gUuNEAnTp1Kv34UtPt23hiUZC968E/CBoPcMpZ4wEQqNlBItVNjS1qACN7pXPj6/P4cuk2hneq53YcERHxTllAySeJFJxRs+OstQdLfDzBGPOKMSbOWrvHWrvNc/0uY8w4nKmUvyhqIr+QuwdWfO6Us6wFznVpF0CPX0OLKyC0trv5RKRS1eii1qNhLM2SIhkzO4NrOqZgNI9bRER+aQHQ2BiTDmwFrgNuKLmBMSYJ2GmttcaYLoAfsNcYEw74WWtzPB8PAP5StfHFp+TnwpoJTjlb/z3YIudYs4uedBYFiU5xO6GIVJEaXdSMMYzsmc7/G7uMuRv30qNhnNuRRETEy1hrC40x9wOTcZbnH2OtXWmMudtz+2vA1cA9xphC4DBwnae0JQLjPG8EBgAfWGsnufKNiPcqKoSN051ytvprKMiFqBTo8YBnUZCWbicUERfU6KIGcEW7ujwzaQ1jZmeoqImISJmstROACaWue63Ex6OAUWXcbyOg88DIL1kLWxc75WzFWMjdDSHRzqhZm+GQ2kOLgojUcDW+qIUE+nNT11T+M209GXtySY8LdzuSiIiIVFd7NzgrNi77BPZtAP9gaHKJZ1GQiyEg2O2EIuIlanxRA7ipe31enbGBt3/I5M9XaHqBiIiIVKBDu04sCrJ1EWAgrRf0+g00vxxCa7mdUES8kIoakBAZwuVt6/LJwi385uImRIcGuh1JREREfNnRQ7DmG6ecbZjmLAqS1BoufgpaDYPoZLcTioiXU1HzGNkznc8Xb+WTBVu4s3cDt+OIiIiIrykqcErZ8k+cklaQB9Gp0PNB57izhOZuJxQRH6Ki5tEqOZou6TG89UMmt/VMI8BfB/CKiIjIGVgLWQs9i4J8Dnl7IKSWc8xZm+FQr5sWBRGRc6KiVsLtvdL51buLmLJqJ4Na13E7joiIiHirPeudcrbsE9ifAQEh0GSgU84aXQwBQW4nFBEfp6JWwkXNE6kXE8obszNU1ERERORk+Xmw+B1Y9hFs+wkwkN4bej/qLAoSEuV2QhGpRlTUSvD3M4zokc5TX69i6ZZs2tar5XYkERER8QY/T4KJj0L2ZkhqAwP+5iwKEqU3dkWkcmjSdCnDO6UQERzAm3My3I4iIiIibsveAh/dCB9eCwGhcOvXcPcs6HG/SpqIVCoVtVIiQwIZ3qkeXy/bzs6DR9yOIyIiIm4oKoA5L8HLXWD999D/T3D3bEi/wO1kIlJDqKiVYUSPNIqs5Z25mW5HERERkaq26Qd47QKY8kdo0BfumwcX/FYLhIhIlVJRK0NqbBgXN0/kg3mbOZxf5HYcERERqQq5e2DcPfDmIMg/BNd9CNd/CLXru51MRGogFbVTGNkrnf15BXyxZKvbUURERKQyFRfDwjfhPx2dJfd7/cYZRWs22O1kIlKDadXHU+iaHkPLulGMmZ3BdZ3rYYxxO5KIiIhUtO1L4evfwtaFUL8XXPo8JDRzO5WIyJlH1IwxIcaY+caYpcaYlcaYJ0+zbWdjTJEx5uqKjVn1jDGM7JnOul2HmLVuj9txREREpCIdOQgTH4fRfWF/Jgz9L4z4WiVNRLxGeaY+HgUutNa2BdoBA40x3UpvZIzxB54FJldoQhdd1rYOcRHBjNFS/SIiItWDtbBiLIzqDPNeg44j4IGF0PY60OwZEfEiZyxq1nHI82mg52LL2PQBYCywq+LiuSs4wJ+bu9Vn+s+7Wb/r0JnvICIiIt5r7wZ47yr4bCREJsId38Nl/4LQ2m4nExH5hXItJmKM8TfGLMEpYVOstfNK3Z4MDAVeO8Pj3GWMWWiMWbh79+5zjFy1buyWSlCAH2/9oFE1ERERn1RwBKY9Da90g6yFMOifcOc0SOnodjIRkVMqV1Gz1hZZa9sBKUAXY0yrUpu8CDxmrT3tWvbW2tHW2k7W2k7x8fHnkrfKxUUEc2W7uoxdtJXsvHy344iIiMjZWP+dU9BmPAvNr4D7F0DXu8DP3+1kIiKndVbL81trs4HpwMBSN3UCPjLGZAJXA68YY648/3jeYWSvdA4XFPHh/C1uRxEREZHyOLgNPrkF3hvmlLJbxsPVb0BkktvJRETKpTyrPsYbY2p5Pg4FLgLWlNzGWpturU2z1qYBnwH3Wmu/qPC0LmmWFEXPRrG8MzeTgqJit+OIiIjIqRQVwg+jnMVC1k6Gfr+He36ABn3dTiYiclbKM6JWB5hmjFkGLMA5Ru1rY8zdxpi7Kzee9xjZM53tB44wacUOt6OIiIhIWTbPg9F94NvfQf0ecO+P0OdRCAh2O5mIyFk74wmvrbXLgPZlXF/mwiHW2hHnH8v79GuaQFpsGG/MzuDytnXdjiMiIiLH5O2D7/4Ei9+BqGQY/i40v1zL7YuITzurY9RqMj8/w20901myJZvFm/e7HUdERESKi2Hxu/CfjvDT+9DjAbhvPrS4QiVNRCqVtZYjBaddR/G8nXFETU64umMKz337M2NmZ9DhBp1zRURExDU7V8LXv4UtP0K9bnDZC5DY0u1UIlLNWGvZlXOUtTtzWLvzEOt25rBu1yHW7sxhQIsknh/ettK+toraWQgPDuD6Lqm8MTuDbdmHqVsr1O1IIiIiNcvRHJj+DPz4KoREw5CXoe0N4KdJQiJy7qy17M45ytqdTglbt+tEMTt4pPD4drXDAmmcGMmQdnXp0TCuUjOpqJ2lW7rX5/VZG3l7bib/N6i523FERERqBmth9Zcw8XHI2QYdboWL/gxhMW4nExEfYq1l96GjrPMUspKjZAcOFxzfrlZYIE0SIrm8bV2aJEbSODGCxgmRxEUEYapoarWK2llKqR3GwFZJfDhvMw/2b0xYkHahiIhIpdqXARMehfVTILE1DH8b6nVxO5WIeLFjhWz9sUK2yylka3eeXMiiQwNpkhjBpW3q0CQhwlPKqraQnYpaxjm4vVc6E5bvYOzirdzcrb7bcURERKqnwqMw5yWY9Tz4BcAlf4cud4G/Xr6IiMNay55D+ScdO7Zu5yHW7sohO+9EIYsKCaBJYiSDW9ehSWLE8VGy+Ihg1wvZqegv3TnokFqbtinRvDk7gxu7pOLn550/XBEREZ+1YRp88zDs2wAtroSBf4conR5HpCbbc+jo8SJW8hiy/WUUskGtkmicEEmTxEiaJEYQH+m9hexUVNTOgTGGkb3SefCjJcxYu5t+zRLcjiQiIlI95OyAyU/AirFQOx1uGguNLnI7lYhUob2HnEU9nDJ2rJgdYl9u/vFtIj2FbGCrJBolRB4fJUvwwUJ2Kipq52hQqzo8HbWaMXMyVNRERETOV1EhLHwDpv7VmfLY9/+g50MQGOJ2MhGpJPty8z1FLOf4aovrdx1ib8lCFhxA48QIBrRIpHFiJI09x5ElRlWfQnYqKmrnKCjAj1u6p/HPyT+zdmcOTRIj3Y4kIiLim7IWwdcPwY5l0PBCGPwcxDZ0O5WIVJD9nkJ2YkEPp5DtOXSikEV4CtlFzRNpXOIYsqSokGpfyE5FRe083NAllX9/v44xszN4Zlgbt+OIiIj4lsP74bsnYdFbEJEIV78JLYdCDX1RJuLr9ufml1jQI+f49MXShaxRQgQXNks4vsJikxpeyE5FRe081A4P4qoOKYxdnMWjlzQlNiLY7UgiIiLez1pY+hF8+3s4vA+63eNMdQyJcjuZiJRDdl7+8RJW8nxkew4dPb5NeJA/jRIj6dc04fjoWJPESOpEq5CVl4raeRrZM40P52/mw/mbuf/Cxm7HERER8W67VjurOW6aAymd4dJxUEezUkS80YG8AtaetKCHU8h25/yykPVtGk+TxAjPCFkkdVXIzpuK2nlqnBhJ7ybxvDN3E3f1bkhQgJ/bkURERLxPfi7M+AfMHQVBEXD5S9D+FvDT86aI2w4cLvjFgh5rd+awq0QhCwvyp3FCBH2axB9f0KNxYgR1o0N1qqpKoqJWAUb2TGPEmwv4Zvk2hrZPcTuOiIiId1nzDUx8DA5sgXY3wcVPQnic26lEapwDhwtYv+tEITs2Srbz4IlCFhroT+PECC5ofGyELILGCZEk11Ihq2oqahWgd+N4GsaH88bsDK5sl6xhXhEREYD9m5yCtnYiJLSA2yZB/e5upxKpEay1LMjcz7crd/Czp5TtOHjk+O2hgf40SoigZ6O44yeFViHzLipqFcDPzzkB9u/GrWDhpv10TotxO5KIiIh7CvNh7n9gxj/B+MHFTzkLhvgHup1MpNrbvDePsYuz+PynLLbsO0xwgB+NEyPo0TD2+AqLjRMiSamtQubtVNQqyFXtU/jHpJ8ZMztDRU1ERGqujJnOYiF71kLzy2HgMxCtwwJEKtPBIwVMWLadsYuzWJC5H2OgR8NYfnNREwa2SiIsSC/5fZF+ahUkNMifG7qm8t8ZG9iyL496MWFuRxIREak6h3Y5y+0v+xhq1YcbPoUmA9xOJVJtFRVbZq3bzdjFW/l25Q6OFhbTID6cRy9pytD2ydStFep2RDlPKmoV6Jbu9fnfzI28/UMmv7+shdtxREREKl9xESwcA98/BQV5cMEjcMHDEKQ3LEUqw887cvh8cRbjftrKrpyjRIcGMrxTPa7qkEy7erW0VkI1oqJWgepEhzK4dR0+XrCFhy5uQkSwdq+IiFRj236Cr38L2xZDem+49AWI0zlFRSra3kNH+XLpNsYuzmLF1oME+Bn6Nk1gWIdkLmyeQHCAv9sRpRKoSVSwkb3S+XLpNj5buIURPdPdjiMiIlLxDmfD1L/CgtchIgGGvQGthoHeyRepMEcLi5i2ZhefLdrK9J93UVhsaVk3ij9e1oIr2tUlLiLY7YhSyVTUKli7erXokFqLN3/I5ObuafhrNR0REakurIXln8HkJyBvD3S5Ey78PYREu51MpFqw1rI06wCfL87iy6XbyM4rID4ymJG90rmqQzLNkqLcjihVSEWtEozslc79H/zE1DW7uLhFottxREREzt/utTDhYWdVx7od4MZPoG57t1OJVAvbDxxm3E9bGbsoiw27cwkO8GNAyySGdUimV6M4Avz93I4oLlBRqwQDWyZRNzqEMbMzVNRERMS35efBrOdgzr8hMAwufR463gZ+OiZG5Hzk5RcyeeUOxi7aypwNe7AWOtWvzd+vasClbeoQFaLzDtZ0KmqVIMDfj1t7pPH3iWtYte0gLepqmFpERHzQ2skw4RHI3gxtroMBTznHpInIOSkutszL2MfYxVlMXL6d3PwiUmqH8sCFjRnWIZn6seFuRxQvoqJWSa7rnMqL361jzJwMnrumrdtxREREyi97C0x6HNZ8DXFN4davIf0Ct1OJ+KzMPbmMXZzF54u3sjX7MBHBAVzapg7DOqTQOS0GP61pIGVQUask0WGBXN0xhY8XbOGxgc2Ij9TKPCIivsoYMxB4CfAHXrfWPlPq9r7AeCDDc9Xn1tq/lOe+XqWoAH58BaY/4ywc0v9P0P1+CAhyO5mIzzlwuIBvlm1n7OIsFm3ajzHQq1Ec/29gUwa0SCI0SNOH5fRU1CrRbT3TePfHTbw/bxMPXdTE7TgiInIOjDH+wMvAxUAWsMAY86W1dlWpTWdZay87x/u6b9MPzjnRdq+GJoNg0LNQu77bqUR8SmFRMbPW7eGzxVlMWbWT/MJiGiVE8NjAZgxtn0xSdIjbEcWHqKhVogbxEVzYLIH3ftzEPX0b6mSEIiK+qQuw3lq7EcAY8xEwBChP2Tqf+1aN3D0w5Y+w5H2IrgfXfQjNBrudSsSnrN5+kLGLsvhiyTb2HDpK7bBAru9cj2EdU2idHI3ROQblHKioVbKRPdO56Y15fLlkG9d0qud2HBEROXvJwJYSn2cBXcvYrrsxZimwDXjEWrvyLO6LMeYu4C6A1NTUCoh9BsXFsPht+O7PkH8Iev0Gej8KQVrMQKQ8duccZfySrXy+eCurth8k0N/Qr2kCwzqm0K9pAkEBWlJfzo+KWiXr2SiWpomRjJmTydUdU/SOioiI7ynrD7ct9flioL619pAxZjDwBdC4nPd1rrR2NDAaoFOnTmVuU2G2L4NvfgtZC6B+L2fJ/YRmlfolRaqDIwVFTF2zi7GLspi+djdFxZY2KdE8eUVLLm9bl5hwHc8pFUdFrZIZYxjZK43Hxi7nx4376N4w1u1IIiJydrKAklMiUnBGzY6z1h4s8fEEY8wrxpi48ty3Sh05CNOehvn/hdAYGPpfaHMt6E1EkVOy1vLTlmzGLsriq6XbOHikkMSoYO68oAHDOiTTODHS7YhSTamoVYEh7ZJ5dtLPvDE7Q0VNRMT3LAAaG2PSga3AdcANJTcwxiQBO6211hjTBfAD9gLZZ7pvlbAWVn4Ok56AQzuh023Q/48QWrvKo4j4iq3ZhxnnWVJ/455cQgL9uKRlEsM6pNCzURz+WlJfKpmKWhUICfTnxq6pjJq2nsw9uaTFaf6/iIivsNYWGmPuBybjLLE/xlq70hhzt+f214CrgXuMMYXAYeA6a60FyrxvlX4Dezc4J63eMBXqtIXrPoCUjlUaQcRX5B4tZNKKHYxdnMXcjXuxFrqkx3B3n4YMap1EZEig2xGlBlFRqyI3d6vPazM28NYPmfz5ipZuxxERkbNgrZ0ATCh13WslPh4FjCrvfatEwRGY/S/nEhAMg/4JnW8HP61ALFJScbHlx417+WxxFpNW7CAvv4jUmDAe6t+Eoe2TSY0Nczui1FAqalUkISqEy9vU5dOFW/jtgCZE6R0ZERGpLOu/g28egf0Z0OpquORvEJnkdioRr7Jh9yE+X5zFuMVb2XbgCJHBAQxpV5erOqTQqX5tLQAnrlNRq0K39Uzn85+28smCLdxxQQO344iISHU177/OyNnNX0DDfm6nEfEa2Xn5fLVsO2MXZbFkSzZ+Bi5oHM/jg5szoEUiIYEacRbvoaJWhVqnRNMlLYY352QyokcaAf46v4aIiFSCK1+F4EhnyqNIDVdQVMyMn3fz+U9ZfLdqF/lFxTRNjOSJwc24sl0yCVEhbkcUKZOKWhUb2Sudu99bxHerdzKwVR2344iISHUUHud2AhHXrdx2gLGLtvLl0q3sOZRPbHgQN3ZLZViHFFrWjdLURvF6KmpV7OIWiaTUDmXM7EwVNREREZEKtCvnCON/2sbYxVms2ZFDkL8f/ZsncFWHFPo2jSdQs5nEh6ioVTF/P8OIHmn89ZvVLM86QOuUaLcjiYiIiPisIwVFTFm1k88XZzFz3R6Kii3t6tXiqSEtubxtXWqFBbkdUeScnLGoGWNCgJlAsGf7z6y1fyq1zY3AY55PDwH3WGuXVnDWamN453r8a8paxszJ4F/XtnM7joiIiIhPsdayaNN+xi7eytfLtpFzpJA60SH8qncDruqQQqOECLcjipy38oyoHQUutNYeMsYEArONMROttT+W2CYD6GOt3W+MGQSMBrpWQt5qISokkOGd6/Hej5t4fFAzEnUQq4iIiMgZbdmXx7iftvL54iwy9+YRGujPoFZJDOuYQrcGsfj76bgzqT7OWNSstRZnlAwg0HOxpbb5ocSnPwIpFRWwuhrRI423fsjkvR838fCApm7HEREREfFKh44WMmG5s6T+vIx9AHRvEMt9/RoxqHUdIoJ1JI9UT+X6zTbG+AOLgEbAy9baeafZ/HZg4ike5y7gLoDU1NSzS1rN1I8N56Lmibw/bzP39Wuk83aIiIiIeBQVW37YsIexi7KYtHIHRwqKSY8L5+GLmzC0QzIptcPcjihS6cpV1Ky1RUA7Y0wtYJwxppW1dkXp7Ywx/XCKWq9TPM5onGmRdOrUyZa1TU0ysmc6U1bt5IuftnJdl5pdXEVERETW7zrE2MVZfPHTVrYfOEJUSABXdUhhWIcUOqTW0pL6UqOc1VixtTbbGDMdGAicVNSMMW2A14FB1tq9FZawGuvWIIYWdaIYMyeDazvX0x8fERERqXGKiy3jftrKO3MzWZp1AH8/Q58m8fz+0hb0b56gWUdSY5Vn1cd4oMBT0kKBi4BnS22TCnwO3GytXVspSashYwwje6XzyKdLmbN+L70a6wSlIiIiUnOs3ZnD78YtZ0HmfpomRvL7S5tzRbu6JERqoTWR8oyo1QHe9hyn5gd8Yq392hhzN4C19jXgj0As8IpnVKjQWtupkjJXK5e3rcMzE1fzxuyNKmoiIiJSIxzOL+LfU9fxv5kbiQgJ4B/D2nB1xxT8tGqjyHHlWfVxGdC+jOtfK/HxHcAdFRutZggO8OembvV58bt1bNh9iIbxOu+HiIiIVF/Tft7FH8evYMu+wwzrkMITg5sRGxHsdiwRr+PndgCBm7rVJ8jfj7fmZLodRURERKRS7Dx4hHvfX8Rtby4gyN+PD+/sxvPD26qkiZyCTjzhBeIighnSri6fLcrikQFNiQ4LdDuSiIiISIUoKra8OzeT575dS35RMQ9f3IS7+jQgOECLhIicjkbUvMRtPdM5XFDEhws2ux1FREREpEKs2HqAoa/M4c9fraJ9ai2+fag3D/RvrJImUg4aUfMSLepG0b1BLG//kMntvdIJ9FeHFhEREd+Uc6SA579dyztzM4kJD+bf17fn8jZ1dCoikbOgNuBFbu+VzvYDR5i8cofbUURERETOmrWWicu3c9ELM3h7biY3dq3P9w/34Yq2dVXSRM6SRtS8yIXNEqgfG8YbszO4rE1dt+OIiIiIlNuWfXn8cfwKpv28mxZ1onjtpo60T63tdiwRn6Wi5kX8/Ay39Ujjz1+tYvHm/XTQHzcRERHxcgVFxbw+K4OXvl+LnzH8/tLmjOiRRoAO4xA5L/of5GWu6VSPyJAA3tRS/SIiIuLlFmbu49J/z+LZSWvo3Tie737bhzsuaKCSJlIB9L/Iy4QHB3Bd53pMWL6d7QcOux1HRERE5Bey8/J5fOwyrn5tLoeOFPK/Wzox+pZO1K0V6nY0kWpDRc0L3dI9DWstb/+wye0oIiIiIsdZaxm7KIsLn5/Bp4uyuKt3A6b8tg8Xt0h0O5pItaNj1LxQvZgwLmmZxIfzN/Pr/o0IC9KPSURERNy1Yfchfj9uBXM37qV9ai3+dmVrWtSNcjuWSLWlETUvdXuvdA4cLuDzxVvdjiIiIiI12JGCIl749mcGvTiLldsO8LehrRh7dw+VNJFKpqEaL9Wxfm3apETz5pwMbuiSip+fzj0iIiIiVWvWut384YsVZO7NY0i7uvz+0hbERwa7HUukRtCImpcyxjCyZzobducyY91ut+OIiIhIDbIr5wi//vAnbn5jPgDv3d6Vl65rr5ImUoVU1LzY4NZ1SIgMZszsDLejiIiISA1QXGx578dN9H9+BpNW7ODX/Rsz6aHe9Goc53Y0kRpHUx+9WFCAH7f2SOOfk39m7c4cmiRGuh1JREREqqlV2w7yuy+W89PmbLo3iOWvQ1vRMD7C7VgiNZZG1Lzc9V1SCQ7w0wmwRUREpFLkHi3kb9+s4vJRs9m8N48Xhrflgzu7qqSJuEwjal4uJjyIqzok8/niLB69pCkx4UFuRxIREZFqYsqqnfxp/Aq2HTjC9V3q8djAZtQK02sNEW+gETUfcFvPdI4WFvPh/M1uRxEREZFqYFv2Ye58ZyF3vrOQyJBAPru7O3+/qo1KmogX0YiaD2iSGMkFjeN4Z24md17QgKAA9WsRERE5e4VFxbz1QyYvTFlLsbU8PqgZt/dKJ9Bfry1EvI3+V/qIkb3S2XnwKBNXbHc7ioiIiPignzbv5/JRc/jrN6vp1iCWKb/pw919GqqkiXgpjaj5iD6N42kQH84bszO4om1djNEJsEVEROTMDhwu4J+T1/D+vM0kRAbz6o0dGNgqSa8lRLycipqP8PMz3NYznT98sYJFm/bTKS3G7UgiIiLixay1fLl0G099vZp9uUcZ0SON317chMiQQLejiUg5aKzbhwzrkEx0aCBj5ugE2CIiInJqmXtyuWXMfB78aAl1a4Uw/r5e/OnylippIj5EI2o+JCwogOu7pDJ65gay9ueRUjvM7UgiIiLiRY4WFvHfGRsZNW09Qf5+PHlFS27qVh9/P01zFPE1GlHzMbd0r48xhrd/yHQ7ioiIiHiRuRv2MvilWbwwZS0XN0/k+4f7cGuPNJU0ER+lETUfU7dWKINb1+GjBVt48KImRATrRygiIlKT7T10lKcnrGHs4izqxYTy5m2d6dc0we1YInKeNKLmg0b2TCPnSCFjF2W5HUVERERcUlxs+XjBZvq/MIPxS7Zyb9+GfPtQH5U0kWpCwzE+qH1qbdqn1uLNORnc3K0+fprSICIiUqOs3ZnD78YtZ0HmfrqkxfDXoa1okhjpdiwRqUAaUfNRI3umk7k3j6lrdrkdRURERKrI4fwinp20hsEvzWLdrkP8Y1gbPrqrm0qaSDWkETUfNahVEnWjQxgzJ4OLWiS6HUdEREQq2bSfd/HH8SvYsu8wV3dM4YnBzYkJD3I7lohUEhU1HxXg78ctPdJ4ZuIaVm8/SPM6UW5HEhERkUqw8+ARnvxqJROW76BhfDgf3dWNbg1i3Y4lIpVMUx992HWd6xEa6M+Y2ToBtoiISHVTVGx5a04G/Z+fwferd/HIgCZMePAClTSRGkIjaj6sVlgQwzom88mCLB4b1Iy4iGC3I4mIiEgFWJ51gCfGLWf51gNc0DiOv17Zivqx4W7HEpEqpBE1H3dbz3Tyi4p5/8fNbkcRERGR85RzpIA/f7mSIS/PZvuBI/z7+va8M7KLSppIDaSi5uMaxkfQr2k87/64iaOFRW7HERGplowxA40xPxtj1htjHj/Ndp2NMUXGmKtLXJdpjFlujFlijFlYNYnF11hrmbB8Oxe9MIO352ZyY9f6fP9wH65oWxdjdBoekZpIRa0aGNkrnT2HjvLV0u1uRxERqXaMMf7Ay8AgoAVwvTGmxSm2exaYXMbD9LPWtrPWdqrUsOKTtuzLY+RbC7j3/cXEhgfz+T09eOrKVkSHBrodTURcpGPUqoFejeJokhjBmNkZDOuQrHfeREQqVhdgvbV2I4Ax5iNgCLCq1HYPAGOBzlUbT3xVQVExr8/K4KXv1+JnDL+/tDkjeqQR4K/30UVEI2rVgjGGkT3TWbX9IPMy9rkdR0SkukkGtpT4PMtz3XHGmGRgKPBaGfe3wLfGmEXGmLtO9UWMMXcZYxYaYxbu3r27AmKLN1uYuY9L/z2LZyetoXfjeL77bR/uuKCBSpqIHKe/BtXEle2TqR0WqKX6RUQqXlnTFGypz18EHrPWlnWwcE9rbQecqZP3GWN6l/VFrLWjrbWdrLWd4uPjzyuweK/svHweH7uMq1+bS+7RIv53SydG39KJurVC3Y4mIl5GUx+riZBAf27sWp+Xp69n095crQ4lIlJxsoB6JT5PAbaV2qYT8JFn6nkcMNgYU2it/cJauw3AWrvLGDMOZyrlzMqPLd7EWsvni7fytwmrOXC4gLt6N+DB/o0JD9ZLMREpm0bUqpGbu9cnwM/w1g+ZbkcREalOFgCNjTHpxpgg4Drgy5IbWGvTrbVp1to04DPgXmvtF8aYcGNMJIAxJhwYAKyo2vjitg27D3HD/+bx8KdLqR8bxtcP9OKJwc1V0kTktM74F8IYE4Lzzl+wZ/vPrLV/KrWNAV4CBgN5wAhr7eKKjyunkxgVwmVt6vLpwix+e3ETIkO0WpSIyPmy1hYaY+7HWc3RHxhjrV1pjLnbc3tZx6UdkwiM84y0BQAfWGsnVXZm8Q5HCop4Zdp6XpuxkZBAP/42tBXXd07Fz0+LfonImZXnrZyjwIXW2kPGmEBgtjFmorX2xxLbDAIaey5dgVc9/0oVG9kznXE/beWThVnc3ivd7TgiItWCtXYCMKHUdWUWNGvtiBIfbwTaVmo48Uqz1u3mD1+sIHNvHle2q8vvLm1BfGSw27FExIecsahZay1wyPNpoOdS+iDqIcA7nm1/NMbUMsbUsdbqxF5VrHVKNJ3TavPWDxmM6JGGv961ExERqTK7co7w169X8+XSbaTHhfPe7V3p1TjO7Vgi4oPKdYyaMcbfGLME2AVMsdbOK7XJGZcu9jyOlh6uAiN7prNl32GmrNrpdhQREZEaobjY8t6Pm+j//AwmrdjBg/0bM/HBC1TSROSclesoVs9yw+2MMbVw5tq3staWPBi6PEsXY60dDYwG6NSp0y9ul4oxoGUSKbVDGTMng4GtktyOIyIiUq2t2naQJ8YtZ8mWbLo3iOWvQ1vRMD7C7Vgi4uPOatVHa202MB0YWOqm8ixdLFXE388wokca8zP2sWLrAbfjiIiIVEu5Rwv52zeruHzUbLbsy+OF4W354M6uKmkiUiHOWNSMMfGekTSMMaHARcCaUpt9CdxiHN2AAzo+zV3DO9cjPMhfJ8AWERGpBFNW7eTiF2bwv1kZDO+UwvcP9+GqDil4VvgUETlv5Zn6WAd42xjjj1PsPrHWfl1qWeIJOEvzr8dZnv+2Ssor5RQVEsg1nerx/rxNPD6oGQlRIW5HEhER8Xnbsg/zpy9XMmXVTpomRvLZ9e3plBbjdiwRqYbKs+rjMqB9Gde/VuJjC9xXsdHkfI3okcbbczN578dN/HZAU7fjiIiI+LS1O3MY+vIciqzl8UHNuL1XOoH+Z3UUiYhIuemvSzWWFhdO/2aJvDdvM0cKityOIyIi4tNe+n4dxhi+fagPd/dpqJImIpVKf2GquZG90tiXm8/4JVvdjiIiIuKz1u86xITl27mle31SY8PcjiMiNYCKWjXXvUEszetEMWZ2Js4MVRERETlbr0xfT3CAH7f3Snc7iojUECpq1ZwxhpE90/h5Zw4/bNjrdhwRERGfs3lvHuOXbOPGrvWJjQh2O46I1BAqajXA5W3rEhcRxBtaql9EROSsvTZzA/7GcFfvBm5HEZEaREWtBggJ9OfGrvWZumYXG3cfcjuOiIiIz9h+4DCfLcxieOcUEnWqGxGpQipqNcRN3eoT5O/HWz9kuh1FRETEZ4yeuZFia/lV74ZuRxGRGkZFrYaIjwzminZ1+XRhFgfyCtyOIyIi4vV25xzlw/mbubJ9MvVitNKjiFQtFbUa5LaeaRwuKOKjBZvdjiIiIuL13pidwdHCYu7tq9E0Eal6Kmo1SMu60XRrEMPbP2RSWFTsdhwRERGvlZ2Xz7tzM7msTV0axEe4HUdEaiAVtRrm9l4N2HbgCJNX7nQ7ioiIiNd6c04muflF3NdPo2ki4g4VtRrmwmYJ1I8NY8wcLdUvIiJSlpwjBbw5J4MBLRJplhTldhwRqaFU1GoYfz/DiB5pLNq0nyVbst2OIyIi4nXe+3EzB48Ucv+FjdyOIiI1mIpaDXRNp3pEBgcwRifAFhEROcnh/CJen7WRPk3iaZNSy+04IlKDqajVQBHBAVzbuR4Tlm9n+4HDbscRERHxGh/O38ze3Hwe0GiaiLhMRa2GurVHGsXW8u7cTW5HERER8QpHC4v478wNdE2PoVNajNtxRKSGU1GroerFhDGgRRIfzN/M4fwit+OIiIi47rNFWew8eJQHLmzsdhQRERW1muz2C9LJzivg85+y3I4iIiLiqoKiYl6dvoF29WrRs1Gs23FERFTUarJO9WvTOjmaMbMzKC62bscRERFxzfgl28jaf5gHLmyEMcbtOCIiKmo1mTGGkb3S2LA7l1nr97gdR0RExBVFxZZXpq2neZ0oLmyW4HYcERFARa3Gu7R1XRIig3lDS/WLiEgNNWH5djbuydVomoh4FRW1Gi4owI9butdn5trdrNuZ43YcERGRKlVcbHl52noaJUQwsGWS23FERI5TUROu75JKcIAfb/6Q6XYUERGRKvX9ml2s2ZHDff0a4uen0TQR8R4qakJsRDBD2yfz+eIs9ufmux1HRESkSlhrGTV1HakxYVzepq7bcURETqKiJgDc1jOdIwXFfDB/s9tRREREqsSsdXtYmnWAe/o2JMBfL4lExLvor5IA0DQpkgsax/HO3EwKiordjiMiIlLpRk1dT53oEK7qkOx2FBGRX1BRk+NG9kxn58GjTFi+3e0oIiIilWrexr3Mz9zHr3o3IDjA3+04IiK/oKImx/VpEk+DuHDGzM7AWp0AW0REqq9R09YTFxHEdV1S3Y4iIlImFTU5zs/PcFvPNJZmHWDx5v1uxxEREakUS7ZkM2vdHu68oAEhgRpNExHvpKImJxnWMYWokAB+N24Fy7Ky3Y4jIiJS4UZNXU+tsEBu7Fbf7SgiIqekoiYnCQsK4F/XtmNvbj5DXp7D78YtJztPS/aLiEj1sGrbQb5bvZORPdOJCA5wO46IyCmpqMkv9G+eyNSH+zCyZzofLdhCv+em89H8zRQX67g1ERHxbS9PX09kcAC39khzO4qIyGmpqEmZIkMC+cNlLfjm171onBDJ458v56pXf2B51gG3o4mIiJyT9bsOMWH5dm7uXp/o0EC344iInJaKmpxWs6QoPv5VN/51bVuy9h/mipdn8/svNB1SRER8zyvT1xMc4MftvdLdjiIickYqanJGxhiGtk9h6iN9uK1HOh/O38KFz8/g4wWaDikiIr5h8948xi/Zxo1d6xMbEex2HBGRM1JRk3KLCgnkj5e34OsHetEwPpzHxjrTIVds1XRIERHxbq/O2IC/MdzVu4HbUUREykVFTc5a8zpRfPKr7rww3JkOefmo2fzhixUcyCtwO5qIiMgvbD9wmM8WbWF45xQSo0LcjiMiUi4qanJOjDFc1SGF7x/uw63d03h/3ib6PT+dTxZu0XRIERHxKv+dsRFr4Ve9G7odRUSk3FTU5LxEhwby5yta8vUDF9AgLpz/99kyrn5N0yFFRMQ77M45ykcLNjO0fTL1YsLcjiMiUm4qalIhWtR1pkM+d01bNu/L44pRs/njeE2HFBERd70xO4P8wmLu6avRNBHxLSpqUmH8/AxXd0zh+4f7ckv3NN77cRMXPj+dTzUdUkREXJCdl8+7czO5rE1dGsRHuB1HROSsqKhJhTs2HfKrB3qRFhfOo58t45r/zmXlNk2HFBGRqvPmnExy84u4r18jt6OIiJy1MxY1Y0w9Y8w0Y8xqY8xKY8yDZWwTbYz5yhiz1LPNbZUTV3xJy7rRfPqr7vzz6jZk7snl8v/M5k/jV3DgsKZDiohvMcYMNMb8bIxZb4x5/DTbdTbGFBljrj7b+0rFyjlSwJtzMhjQIpGmSZFuxxEROWvlGVErBB621jYHugH3GWNalNrmPmCVtbYt0Bd43hgTVKFJxSf5+Rmu6VSPqQ/35eZu9Xn3x030f346ny3K0nRIEfEJxhh/4GVgENACuL6M58Fj2z0LTD7b+0rFe/fHTRw8Usj9F2o0TUR80xmLmrV2u7V2sefjHGA1kFx6MyDSGGOACGAfTsETASA6LJAnh7Tiy/t7kRoTxiOfLmX4f+eyattBt6OJiJxJF2C9tXajtTYf+AgYUsZ2DwBjgV3ncF+pQHn5hbw+K4M+TeJpk1LL7TgiIufkrI5RM8akAe2BeaVuGgU0B7YBy4EHrbXFFRFQqpdWydF8dncP/nF1GzbuyeWy/8ziz1+u1HRIEfFmycCWEp9nUeoNS2NMMjAUeO1s7ysV78P5W9iXm88DGk0TER9W7qJmjInAeafwIWtt6WGQS4AlQF2gHTDKGBNVxmPcZYxZaIxZuHv37nMOLb7Nz88wvFM9pj3clxu71ueduZn0f346YxdlYa2mQ4qI1zFlXFf6j9WLwGPW2qJzuK+zoZ4jK8TRwiJGz9xAtwYxdEqLcTuOiMg5K1dRM8YE4pS09621n5exyW3A59axHsgAmpXeyFo72lrbyVrbKT4+/nxySzUQHRbIU1c60yHrxYTxsGc65Ortmg4pIl4lC6hX4vMUnBkkJXUCPjLGZAJXA68YY64s530BPUdWlM8WZbHz4FEeuLCx21FERM5LeVZ9NMAbwGpr7Qun2Gwz0N+zfSLQFNhYUSGlemuVHM3Yu3vwj2Ft2LA7l8v+M5snv1rJwSOaDikiXmEB0NgYk+5ZKOs64MuSG1hr0621adbaNOAz4F5r7Rflua9UnIKiYl6dvoH2qbXo0TDW7TgiIucloBzb9ARuBpYbY5Z4rnsCSAWw1r4GPAW8ZYxZjjPN4zFr7Z6KjyvVlZ+fYXjnegxomchz3/7MWz9k8tXS7TwxuBlD2yfjvF8gIlL1rLWFxpj7cVZz9AfGWGtXGmPu9txe+ri0M963KnLXROOXbCNr/2GevKKlnjdExOcZt44J6tSpk124cKErX1u83/KsA/xh/AqWbMmmS1oMf7myJc2SfnHYo4j4CGPMImttJ7dz+Ao9R569omLLxS/MIDjQnwm/7qWiJiI+4XTPj2e16qNIVWmdEs3n9/Tg2WGtWbcrh0v/PZu/fLVK0yFFRKRME5ZvZ+OeXB64sJFKmohUC75d1IpLL64l1Ymfn+HazqlMe6Qv13Wux5s/ZND/+Rl88dNWrQ4pIiLHFRdbRk1dT6OECAa2THI7johIhfDdorZrNbzSDbYtcTuJVLJaYUH8bWhrxt/Xk7rRITz08RKuHf0jP+/IcTuaiIh4ge9W7+TnnTnc168hfn4aTROR6sF3i5rxh4LD8NalsP57t9NIFWiTUotx9/bk71e1Zt3OHAb/exZPfb2KHE2HFBGpsay1jJq2ntSYMC5vU9ftOCIiFcZ3i1p8E7h9CtROgw+Gw9KP3U4kVcDPz3B9l1SmPtyXazvXY8ycDC58fgbjl2g6pIhITTRr3R6WZR3g3r4NCfD33Zc1IiKl+fZftKg6cNsESO0O4+6C2S+CXqzXCLXDg3h6aGu+uLcndaJDePCjJVw3+kfW7tR0SBGRmmTU1PXUiQ7hqg4pbkcREalQvl3UAEKi4aax0GoYfPcnmPiYFhmpQdrWc6ZDPj20NT/vzGHQS7P4q6ZDiojUCPM27mV+5j5+1bsBQQG+/5JGRKSk6vFXLSAYrnodut8P8/8Ln90GBUfcTiVVxN/PcEPXVKY93JfhnerxxhxndUhNhxQRqd5GTVtPXEQQ13VJdTuKiEiFqx5FDcDPDy75Gwz4G6waD+9dBYf3u51KqlDt8CD+flVrxt3bkyTPdMjr/6fpkCIi1dFPm/cza90e7rygASGB/m7HERGpcNWnqB3T434Y9gZsmQ9jBsGBrW4nkirWzjMd8m9DW7F6ew6DX5rF0xNWc+hoodvRRESkgrw8bT21wgK5sVt9t6OIiFSK6lfUAFpf7Ry3dnArvHEx7FzldiKpYv5+hhu71mfaI325umMKo2dupP/z0/ly6TZNhxQR8XErtx3gu9W7GNkznYjgALfjiIhUiupZ1AAa9HFWhCwugjcHQuYctxOJC2LCg3hmWBvG3duD+Mhgfv3hT9zwv3ms03RIERGf9cq0DUQGB3BrjzS3o4iIVJrqW9QAklrDHVMgIhHeHQorv3A7kbikfWptxt/Xi6eubMXKbQcY9NIs/q7pkCIiPmf9rhwmrNjOLT3qEx0a6HYcEZFKU72LGkCtVBg5Geq2g09HwLzRbicSl/j7GW7u5kyHHNYhhf/O3MhFz8/gK02HFBHxGa9M30BIgD8je6a7HUVEpFJV/6IGEBYDt4yHpoNh4qPw3Z91YuwaLDYimGevbsPYe3oQGxHEAx/+xI2vz2P9Lk2HFBHxZpv35jF+yTZu7JpKbESw23FERCpVzShqAIGhcO270GkkzP4XjLsbCvPdTiUu6li/Nl/e34unhrRkxdYDDHxxFn+fuJpcTYcUEfFKr87YgL8x3Nm7gdtRREQqXc0pagB+/nDpC9Dv97DsI/jwWjiqUZSazN/PcHP3NKY+0peh7ZP574yN9H9+Bt8s267pkCIiXmT7gcN8tmgLwzunkBgV4nYcEZFKV7OKGoAx0OdRGPIybJwBb10KOTvdTiUui4sI5p/XtGXsPT2ICQ/ivg8Wc/Mb81m/65Db0UREBPjvjI1YC7/q3dDtKCIiVaLmFbVj2t8E138Ee9Y551rbu8HtROIFOtavzVcP9OIvQ1qyNCubQS/N5JmJazQdUkTERbtzjvLh/M0MbZ9MvZgwt+OIiFSJmlvUAJoMgFu/hvxDTlnLWuR2IvEC/n6GW7qnMe2Rvgxpl8xrMzZw0QszmLBc0yFFRNzw+uyNFBQVc09fjaaJSM1Rs4saQEpHuH0KBEfC25fB2sluJxIvERcRzHPXtOWzu7tTKyyIe99fzC1j5rNht6ZDiohUley8fN6bu4nL2tSlQXyE23FERKqMihpAbEOnrMU1gQ+vh8XvuJ1IvEintBi+ur8nT17RkiVbshn44kyenbSGvHxNhxQRqWxvzskkN7+I+/o1cjuKiEiVUlE7JiIBRnwDDfrClw/A9Gd1rjU5LsDfj1t7pDH14b5c0TaZV6dv4KLnZzBR0yFFRCpNzpEC3pyTwSUtE2maFOl2HBGRKqWiVlJwBNzwMbS9HqY/DV8/BEUaNZET4iODeX54Wz69uztRoYHc45kOuVHTIUVEKty7P27i4JFC7u/X2O0oIiJVTkWtNP9AuPJVuOBhWPQWfHIz5Oe5nUq8TOe0GL5+oBd/urwFSzZnc8mLM/nnZE2HFBGpKHn5hbw+K4M+TeJpnRLtdhwRkSqnolYWY6D/H2Hwc/DzRHhnCOTtczuVeJkAfz9u65nO94/04fK2dXl5mjMdctIKTYcUETlfH87fwr7cfB64UMemiUjNpKJ2Ol3uhOHvwPal8MYA2L/J7UTihRIiQ3hheDs++ZUzHfLu9xZz65sLyNiT63Y0ERGfdKSgiNEzN9CtQQyd0mLcjiMi4goVtTNpcQXcMh5ydznnWtu+zO1E4qW6pDvTIf94WQt+2rSfS/41k+cm/8zh/CK3o4mI+JTPFmWx8+BRHrhQx6aJSM2lolYe9bvDyG/BLxDeHAwbp7udSLxUgL8fI3s50yEva1OHUdPWc9ELM5i0YoemQ4qIlENBUTGvTt9A+9Ra9GgY63YcERHXqKiVV0IzuP1bqFUP3rsaln3qdiLxYgmRIbxwbTs+vqsbEcEB3P3eIkZoOqSIyBl98dNWtmYf5oELG2GMcTuOiIhrVNTORnQy3DYR6nWFz++AOf/WudbktLo2iOXrX/fiD5e1YNGm/Vz8wgxufmMe78/bxO6co27HExHxKkXFllenb6BFnSj6NU1wO46IiKtU1M5WaC24aSy0uBKm/AEmPwHFxW6nEi8W6O/H7b3SmfpwH+64oAFb9uXxu3Er6PL0dwx/bS5jZmewLfuw2zFFRFw3Yfl2Nu7J1WiaiAgQ4HYAnxQYAle/CZPrwI+vQM52GPpfCAh2O5l4sYSoEB4f1IzHBjZlzY4cJq7YweQVO/jL16v4y9eraFuvFgNbJjGoVRJpceFuxxURqVLFxZZRU9fTKCGCS1omuR1HRMR1Kmrnys8PBv4dourAlD9C7h647n0I0Uk55fSMMTSvE0XzOlH89uImbNx9iIkrdjBpxQ6enbSGZyetoVlSJINa1WFgqySaJEbonWURqfa+W72Tn3fm8K9r2+Lnp795IiLGrZXoOnXqZBcuXOjK165wyz6BL+6FuCZw02cQVdftROKjsvbnMWnFDiav3MHCTfuxFhrEhTOwVRKDWtWhVXKUSpv4JGPMImttJ7dz+Ipq9RxZDtZahrw8h+y8AqY+3IcAfx2ZISI1w+meHzWiVhHaDIfwePj4Jnj9YucYtoRmbqcSH5RSO4w7LmjAHRc0YNfBI0xetZNJK7bz35kbeWX6BpJrhXpKWxIdUmvrXWcRqRZmrtvDsqwDPHNVa5U0EREPjahVpO1L4f1roPAoXP+Rc/41kQqwPzefKat3MmnFDmav20N+UTHxkcFc0jKRQa3q0DU9Ri9uxKtpRO3sVMvnyFOw1nLNa3PZmn2YGY/2IyhAf8tEpObQiFpVqdPWOdfae8Pg3Sth2OvQ/HK3U0k1UDs8iOGd6jG8Uz1yjhQwdc0uJq3YwdhFW3nvx83UCgvk4uaJDGqdRM9GcQQH+LsdWUSkXOZl7GPhpv08eUVLlTQRkRJU1Cpa7TQY+S18eC18fDMM/id0udPtVFKNRIYEMqRdMkPaJXM4v4gZa53SNmnFDj5dlEVkcAAXNk9gYMsk+jSNJyxI/81FxHu9PG09cRHBXNu5nttRRES8il7BVYbwWLjlS/hsJEx4xFm+/8I/gBaBkAoWGuTPwFZ1GNiqDkcLi/hh/V4mrtjOlFU7Gb9kGyGBfvRtksCg1kn0a5ZAVEig25FFRI77afN+Zq3bwxODmxESqJkAIiIlqahVlqAwuPY9+Oa3MOt5yNkBl78E/nqhLJUjOMCffs0S6NcsgcKiYuZn7HPO1bZyB5NW7iDI34+ejWIZ1KoOF7dIpHZ4kNuRRaSGe3naemqFBXJj1/puRxER8ToqapXJP8ApZ1HJMP1pOLQTrnkbgiPcTibVXIC/Hz0axdGjURxPXtGSn7bsZ+LyHUxcsYNpPy/Df5yha3oMg1olcUnLJBKiQtyOLCI1zMptB/hu9S5+e3ETwoP1ckREpDT9ZaxsxkDfxyAyCb5+CN6+DG74FCLi3U4mNYSfn6Fj/Rg61o/hd5c2Z8XWg0xauZ2JK3bwh/Er+eOXK+mYWpuBntJWLybM7cgiUgO8Mm0DkcEB3Nojze0oIiJe6YxFzRhTD3gHSAKKgdHW2pfK2K4v8CIQCOyx1vapyKA+r+OtEJEIn46ANzznWott6HYqqWGMMbROiaZ1SjSPDGjKul2HmLTCGWn76zer+es3q2mdHM3AVkkMbJVEw3iN/opIxVu/K4cJK7Zzb9+GRIfqkAARkbKc8Txqxpg6QB1r7WJjTCSwCLjSWruqxDa1gB+AgdbazcaYBGvtrtM9bk06R8xJshY651ozfnDjJ5Dc0e1EIgBk7sll0kpn9cglW7IBaJIY4SxW0jKJ5nUiMVoQR86RzqN2dqr7c+RvP17CxBU7mP1YP2Ijgt2OIyLimvM6j5q1djuw3fNxjjFmNZAMrCqx2Q3A59bazZ7tTlvSarSUTnD7FHhvKLx1GQx/Bxpf7HYqEdLiwrm7T0Pu7tOQbdmHmbzSGWn7z9R1/Pv7ddSPDXNG2lom0a5eLZU2ETknm/bmMn7pNm7rkaaSJiJyGmd1jJoxJg1oD8wrdVMTINAYMx2IBF6y1r5Txv3vAu4CSE1NPYe41URcI7j9O3j/avjgWrjiP9D+RrdTiRxXt1Yot/VM57ae6ezOOcqUVTuZuGI7b8zK4L8zNlInOoRLWiYxqFUSndJi8PdTaROR8nltxgb8/Qx39m7gdhQREa9W7qJmjIkAxgIPWWsPlvE4HYH+QCgw1xjzo7V2bcmNrLWjgdHgTOs4n+A+LzIRbpvgnBR7/L2Qsw0ueETnWhOvEx8ZzA1dU7mhayoH8gr4bvVOJq7YwQfzN/PWD5nERQRxcQuntHVvGEugv5/bkUXES20/cJjPFmVxXedUErXarIjIaZWrqBljAnFK2vvW2s/L2CQLZwGRXCDXGDMTaAusLWNbOSY4Em74BL68H6b+FQ5uh8H/BD+d9FO8U3RYIMM6pjCsYwqHjhYybc0uJq3cwfglW/lw/maiQgK4qEUig1rV4YLGcTqBrYic5L8zNmIt/KqPRtNERM6kPKs+GuANYLW19oVTbDYeGGWMCQCCgK7AvyosZXUWEARXvgaRdWDOi8651oa9DoGhbicTOa2I4AAub1uXy9vW5UhBETPX7mbSyh18t2onny/eSniQcwLuga2S6Nc0QedJEqnhducc5cP5mxnaPpmU2joNiIjImZTnlVNP4GZguTFmiee6J4BUAGvta9ba1caYScAynCX8X7fWrqiEvNWTnx9c/CRE1YWJj8E7V8L1H0JYjNvJRMolJNCfAS2TGNAyifzCYuZu3MukFdv5duVOvl62neAAP3o3iWdQqyT6N0skOkzLcYtvMcYMBF4C/HGe454pdfsQ4Cmc58BCnMMEZntuywRygCKgsKaufvn67I0UFBVzT1+dmkZEpDzOuDx/ZanuSw+fs5Xj4PO7oHa6c661WvXcTiRyzoqKLQsy9zFphbPs/46DRwjwM/RoFMegVkkMaJGoVd9qCF9ent8Y448zlf9inKn+C4DrS52mJgLItdZaY0wb4BNrbTPPbZlAJ2vtnvJ+zer2HLk/N59ez06lf/NE/n19e7fjiIh4jfNanl+qWMuhEB4PH97gnBj7xs8gqZXbqUTOib+foVuDWLo1iOWPl7VgaVb28RNs/9/ny/nduOV0TothUKskLmmVRJ1oTfkVr9QFWG+t3QhgjPkIGEKJ09RYaw+V2D4cqNkLZpXy5g+Z5OYXcV+/Rm5HERHxGVqezRul9YKRkwADbw6CjJluJxI5b35+hvaptfm/wc2Z8WhfJvz6Au7v14h9ufn8+atVdP/7VK58eQ7/nbGBzXvz3I4rUlIysKXE51me605ijBlqjFkDfAOMLHGTBb41xizynKamTMaYu4wxC40xC3fv3l1B0d138EgBb83J4JKWiTRNinQ7joiIz1BR81aJLeCOKc5xa+8NgxVj3U4kUmGMMbSoG8VvBzRlym/78N1v+/DoJU0pLC7m7xPX0Puf0xj80iz+/f061u3McTuuSFnnTfnFiJm1dpxnuuOVOMerHdPTWtsBGATcZ4zpXdYXsdaOttZ2stZ2io+Pr4DY3uHduZs4eKSQ+/s1djuKiIhP0dRHbxad4oysfXgDfDYScnZA9/vcTiVS4RolRNAooRH39WvEln15zjFtK3fwwpS1vDBlLQ3jwxnUqg4DWyXRsm4URucblKqVBZQ8YDgF2Haqja21M40xDY0xcdbaPdbabZ7rdxljxuFMpawRUyXy8gt5Y3YGfZvG0zol2u04IiI+RUXN24XWhpvHwed3wuQn4OA2uPgpZ6VIkWqoXkwYd/ZuwJ29G7Dz4BEmr9zBxOU7eGX6ekZNW0+9mFAGtkxiYKs6tK9XCz8/lTapdAuAxsaYdGArcB1wQ8kNjDGNgA2exUQ64JyqZq8xJhzws9bmeD4eAPylauO758P5W9iXm88DF+rYNBGRs6Wi5gsCQ+Cat2DS4zB3lDOyduUrEKDV8qR6S4wK4ZbuadzSPY19uflMWeUsRPLWD5n8b1YGiVHBXNIyiYGtkuhUP4agAL2BIRXPWltojLkfmIyzPP8Ya+1KY8zdnttfA4YBtxhjCoDDwLWe0pYIjPOMAgcAH1hrJ7nyjVSxIwVFjJ65gW4NYuhYX6ebERE5WypqvsLPHwb9wzlm7bs/Q+4uuPZ9CIlyO5lIlYgJD+Lazqlc2zmVA4cLmLZmFxNXbOeThVt4Z+4m/Awk1w4lLTac9LjwE//GhZNSO5RAf5U4OXfW2gnAhFLXvVbi42eBZ8u430agbaUH9EKfLcpi58GjvDC8ndtRRER8koqaLzEGev0GIuvA+PvgzcFw46cQVcftZCJVKjo0kCvbJ3Nl+2Ty8guZuXY3q7YdJGNvHhl7DvHT4mwOHS08vn2AnyGldihppQpcemw4ybVD8df0SZEKVVBUzKvTN9A+tRY9Gsa6HUdExCepqPmittdBeBx8fAu8McA5MXZ8E7dTibgiLCiAga3qMLDViTcsrLXsOZRP5t5cMvbkkrkn1/NxHvMz9pGXX3R820B/Q2pM2PFRuLS4E0WuTlSIjoETOQdf/LSVrdmHeerKllr8R0TkHKmo+apGF8Ft38D718CYAXD9x5Da1e1UIl7BGEN8ZDDxkcF0Tjv52BhrLbtyjh4vcBl7PUVuTx6z1u3haGHx8W2DA/yoHxt20ijcsY8To4L1AlSkDEXFllemb6BFnSj6NU1wO46IiM9SUfNlddvD7VPgvavgnSvg6jHQ7FK3U4l4NWMMiVEhJEaF0K3ByVOyiostOw4eOanAZezJY+OeXKb/vJv8ohMlLjTQn/qxYSdNo0yLCyctLoz4CJU4qbm+Wb6djD25vHpjB/0/EBE5Dypqvi4m3SlrHwyHj2+CS5+HTiPdTiXik/z8DHVrhVK3Vig9GsWddFtRsWVb9mEySxS4zL25/LwjhymrdlJYfOL8xxHBAc5IXIkClx7njMzFhAfpxatUW8XFlpenrqdRQgSXtExyO46IiE9TUasOwuPg1q/g0xHw9W/g4Hbo94Sz+IiIVAh/P0O9mDDqxYRxQeP4k24rLCpma/bhEsfD5ZGxJ5cVWw8wacUOikqUuMiQgFLHw52YWlkrLKiqvy2RCvXd6p38vDOHF69tp+M7RUTOk4padREUDtd9CF8/CDP/ATnb4LKXwF8/YpHKFuDvR/3YcOrHhkPTk2/LLywma3/e8cVMji1ssnjzfr5atg17osNRKyzwpNMLpMWdmFoZFRJYtd+UyFmy1jJq2npSY8K4rI1WIxYROV96FV+d+AfAFaMgKhlmPAuHdjknyg4KdzuZSI0VFOBHg/gIGsRH/OK2o4VFbNmXd7zAHTsubt7GvYz7aetJ28aGB5VYzCTspIVNwoP1p1zcN3PdHpZlHeCZq1oToPMWioicNz27VzfGONMeI5Pgm4fh7cvhhk+c6ZEi4lWCA/xplBBJo4TIX9x2pKCITZ4plCeOi8tl9vrdjF189KRt4yODPcfCnXxcXFpsOKFB/lX17UgNZq3lP9+vo050CFd1SHE7johItaCiVl11GgkRifDZyBPnWotJdzuViJRTSKA/TZMiaZr0yxKXl19I5p68X5wnbuqa3ew5lHXStklRIZ4plBEnHQ9XLyaMkECVOKkY8zL2sXDTfp68oiVBARpNExGpCCpq1VmzS+GWL+HDa+GNi+HGT50l/UXEp4UFBdCibhQt6kb94racIwUnRuJKTKecvHIH+3Lzj29nDNSNDvUcA3fyueLq1Q7Ti205K6OmricuIphrO9dzO4qISLWholbdpXaFkd/Ce8PgzUvh2neck2WLSLUUGRJIq+RoWiVH/+K2A3kFJc4Pd2JK5ZdLtnHwSOHx7fwMpNQ+No0yjAf6NyYuIrgqvw3xIYs372f2+j08MbiZRmlFRCqQilpNEN8Ebv8W3r8GPrjWWXCk3fVupxKRKhYdFki7sFq0q1frpOuttezPKzhpGuWxIrd4035+c3ETdwKLT3h56npqhQVyY9f6bkcREalWVNRqiqg6cNs3zkmxv7gbcrZDr9/oXGsigjGGmPAgYsKD6Fi/9km3WWt1gm45pZXbDvD9ml08fHETrT4qIlLBdBBCTRISDTeOhVZXw/dPwsT/B8VFbqcSES+mkian88q0DUQGB3BLjzS3o4iIVDt6+6umCQiCq/7nLN8/dxTk7HA+DwxxO5mIiPiQ9btymLBiO/f2bUh0qE7ILiJS0TSiVhP5+cElf4NLnobVX8K7Q+HwfrdTiYiID3ll2gZCAvwZ2VOnfhERqQwqajVZ9/vg6jGwdSGMGQgHss58HxERqfE27c1l/NJt3Ng1lVitCCoiUilU1Gq6VsOck2Ef3AavXww7V7mdSEREvNxrMzbg72e4s3cDt6OIiFRbKmoC6b3htomAdUbWMme7nUhERLzUtuzDfLYoi2s71SMxSsc3i4hUFhU1cSS1gtunOIuMvDsUfhgFOTvdTiUiIl5m9MyNWAu/6qPRNBGRyqSiJifUqgcjJ0G9rvDt7+D5pjBmEPz4GhzY6nY6ERFx2e6co3w4fzNXdUgmpXaY23FERKo1Lc8vJwuLgVu/gt1rYNV45zLpMeeS0gVaXAHNr4Da9d1OKiIiVez12RspKCrmnr6N3I4iIlLtqajJLxkDCc2dS9/HYfdaWD0eVn0J3/7eudRtDy2GOKUttqHbiUVEpJLtz83nvbmbuKxNXdLjwt2OIyJS7amoyZnFN4H4R6H3o7Bvo1PYVo2H7/7sXJJae0rbEGdbERGpdt78IZPc/CLu66fRNBGRqqCiJmcnpgH0esi5ZG8+Udqm/tW5xDd3SluLIc6InDFuJxYRkfN08EgBb83J4JKWiTRNinQ7johIjaCiJueuVir0uN+5HNgKa752StuMZ2HGMxDb2DmmrcUQSGqj0iYi4qPenbuJg0cKub9fY7ejiIjUGCpqUjGik6Hrr5xLzk5Y85Uz2jb7XzDreaiddmKkrW4HlTYRER+Rl1/IG7Mz6Ns0ntYp0W7HERGpMVTUpOJFJkLnO5xL7h5Y840z0jb3ZZjzEkTXcxYhaTEEUjqDn84SISLirT6Yt5l9ufk8cKGOTRMRqUoqalK5wuOg463OJW8f/DwRVn8JC/4HP74MkXVOlLbUbuDn73ZiERHxOFJQxOiZG+neIJaO9WPcjiMiUqOoqEnVCYuB9jc6lyMHYO1kZ6Rt8dsw/78QHg/NL3dKW/1e4K9fTxERN322KItdOUd58dp2bkcREalx9EpY3BESDW2GO5ejh2Ddt05pW/oRLBwDoTHQ/DJnyf/03hAQ5HZiEZEapaComFenb6B9ai26N4x1O46ISI2joibuC46AVlc5l/w8WP+dU9pWjIPF7zilrumlzkhbw34QEOx2YhGRau+Ln7ayNfswT13ZEqMFoEREqpyKmniXoDDPkv5XQMER2DjNKW1rvoGlH0BwFDQZ6JS2Rv0hMNTtxCIi1U5RseWV6RtoUSeKfk0T3I4jIlIjqaiJ9woMgaaDnEthPmTM8JS2r2H5JxAYDk0GOKWt8QAICnc7sYhItfDN8u1k7Mnl1Rs7aDRNRMQlZyxqxph6wDtAElAMjLbWvnSKbTsDPwLXWms/q8igUsMFBEHji53LZf+CzNlOaVv9FawcBwGh0Pgi55i2JpdASJTbiUVEfFJxseXlqetplBDBJS2T3I4jIlJjlWdErRB42Fq72BgTCSwyxkyx1q4quZExxh94FphcCTlFTvAPdI5Va9gPLn0eNs91StuqL53i5h8EDfs7I21NB0JobbcTi4j4jCmrd/LzzhxevLYdfn4aTRMRccsZi5q1djuw3fNxjjFmNZAMrCq16QPAWKBzRYcUOSU/f0jr5VwGPgtZ80+UtrUTwS8QGvTxlLZLIVwrl4mInIq1llFT11M/NozL2tRxO46ISI12VseoGWPSgPbAvFLXJwNDgQs5TVEzxtwF3AWQmpp6llFFzsDPzzlpdmo3uORp2LoYVn3hFLcvHwDzEKRf4JS2ZpdBhA6QFxEpacba3SzfeoBnh7UmwN/P7TgiIjVauYuaMSYCZ8TsIWvtwVI3vwg8Zq0tOt1Bx9ba0cBogE6dOtmzTitSXsZASkfncvFfYPtSWP0lrPwCvv4NfPMwpPZwSlvzyyFK7xyLSM12bDStbnQIQ9unuB1HRKTGK1dRM8YE4pS09621n5exSSfgI09JiwMGG2MKrbVfVFRQkXNmDNRt51wu/APsWuWZHjkeJj7qXOp1O1HaatVzO7GISJWbl7GPhZv28+QVLQkK0GiaiIjbyrPqowHeAFZba18oaxtrbXqJ7d8CvlZJE69kDCS2dC79noDdPzvHs60aD5P/z7kkd/SUtisgJv3MjykiUg2MmrqeuIhgru2sN6tERLxBeUbUegI3A8uNMUs81z0BpAJYa1+rnGgiVSC+KfR51Lns3XBipG3KH51Lnbae0jYE4hq5nVZEpFIs3ryf2ev38MTgZoQE+rsdR0REKN+qj7OBcq/Pa60dcT6BRFwT2xAu+K1z2Z/pLPW/ajx8/xfnktDSKW0thkBCM7fTiohUmJenrqdWWCA3dq3vdhQREfE4q1UfRWqM2mnQ4wHnciDLU9q+hOl/h+lPQ1xTaHGFU9oSWzlTKkVEfNCKrQf4fs0uHr64CeHBelkgIuIt9BdZ5EyiU6DbPc4lZ8eJkbZZz8PMf0JMgxMjbXXaqbSJiE95Zfp6IoMDuKVHmttRRESkBBU1kbMRmQRd7nQuh3bDz984pW3Ov2H2v6BWqrMISYsrnUVJ/LRymoh4r/W7cpi4Ygf39W1EdGig23FERKQEFTWRcxURDx1HOJe8ffDzBKe0zfsvzB0FUcme0nYF1OsKfjpAX0S8yyvTNhAS4M/IXlrhVkTE26ioiVSEsBhof5NzOZwNayc5x7QtHAPzXoWIROccbc0uhcTWEB6nKZIi4qpNe3MZv3Qbt/VIIyY8yO04IiJSioqaSEULrQVtr3MuR3Ng7WRnpO2n92HB6842IdEQ2whiGzv/xjVy/o1pCEFhrsYXkZrhtRkb8Pcz3Nm7gdtRRESkDCpqIpUpOBJaX+1c8nNh81zYsx72roM96yBzNiz76OT7RKWcKG6xjU98HF1P0ydFXGKMGQi8BPgDr1trnyl1+xDgKaAYKAQe8pze5oz3dcO27MN8tiiL6zqnkhgV4nYcEREpg4qaSFUJCodGFzmXkvJzYd9Gp7jt3XCixC37BI4ePLGdf7CzwmTJEhfbCOIaO1MvRaRSGGP8gZeBi4EsYIEx5ktr7aoSm30PfGmttcaYNsAnQLNy3rfKjZ65EWvhV300miYi4q1U1ETcFhQOSa2dS0nWQu5u2LveU+LWO5fdP8PPk6C44MS2oTEnSltswxMlLqYBBOrdcpHz1AVYb63dCGCM+QgYAhwvW9baQyW2Dwdsee9b1XblHOHD+Zu5qkMyKbU11VpExFupqIl4K2MgIsG51O9x8m1FhZC96ZclbsNUWPJ+yQeBWvU8Uyg95e3YJSpZpw8QKZ9kYEuJz7OArqU3MsYMBf4OJACXns19Pfe/C7gLIDU19bxDn8obszIoKCrmnr6NKu1riIjI+VNRE/FF/gGekbOG0OSSk287muOZQrm+RJFbBz/Ng/wSb/oHhHoeo1GJ0TjPx6G1qvTbEfFyZS3Ran9xhbXjgHHGmN44x6tdVN77eu4/GhgN0KlTpzK3OV/7c/N598dNXN62Lulx4ZXxJUREpIKoqIlUN8GRULedcynJWsjZ4SlwnuPh9qyDHctg9Vdgi05sGx5/8ujbsRJXOx0CtIy31DhZQL0Sn6cA2061sbV2pjGmoTEm7mzvW9nenJNBXn4R9/XTaJqIiLdTUROpKYyBqDrOJf2Ck28rzIf9mSVK3Hpndcq1kyH33RKP4Qe16nuKm+d4uGMlLrKOzg0n1dUCoLExJh3YClwH3FByA2NMI2CDZzGRDkAQsBfIPtN9q8rBIwW89UMmA1sm0SQx0o0IIiJyFlTURMQZJYtv4lxKO5xdYipliRKXMQsKD5/YLjD85OJW8tQCwXpRKL7LWltojLkfmIyzxP4Ya+1KY8zdnttfA4YBtxhjCoDDwLXWWguUeV83vo93527i4JFCjaaJiPgIFTUROb3QWpDS0bmUVFwMOdtKHAfnObVA1kJY8TknHYYTkXTyib2PrUpZuz74B1bldyNyTqy1E4AJpa57rcTHzwLPlve+VS0vv5A3ZmfQt2k8rVOi3YwiIiLlpKImIufGzw+iU5xLg74n31ZwBPZn/LLErfoSDu8r8RgBznFvZZW4iARNpRSpIB/M28y+3HweuFCjaSIivkJFTUQqXmAIJDR3LqXl7Su1ImWJUwsUHT2xXXDUiXPCnXR+uIbOuedEpFyOFBQxeuZGujeIpWP9GLfjiIhIOamoiUjVCouBsC5Qr8vJ1xcXwYGsk1ek3LseNs+F5Z+cvG1U8skn9o5rDFF1IaQWhNaGwFCNxol4fLooi105R3nx2nZuRxERkbOgoiYi3sHP3zlmrXZ9aHTRybfl58G+jScWNNnjGYVb8RkcOfDLx/IPdo6tC63tXEJKfFzm9Z5/Q6KdHCLVREFRMa9N30CH1Fp0bxjrdhwRETkLKmoi4v2CwiCplXMpyVrI2+uMvh3aCUey4fD+EhfP5weyYOcK5+OSJ/0uS0h0qWJ3qnJX6vrA0Mr4zkXOy7iftrI1+zB/vbIVRqPMIiI+RUVNRHyXMRAe51zKqzDfGYU7VuZOKnfZv7z+wJYT15c8KXhp/sHlG7U7PnpXS6N4UqmKii2vTt9Ay7pR9G0a73YcERE5SypqIlKzBARBRLxzORvWwtGcMxe7Y9dnb4Hty5zrTzuKZyAk6iymaZa4XqN4chrfLN9Oxp5cXr2xg0bTRER8kIqaiEh5GE+hComCWqlnd9/CfE+Ryz5FsStV+g5sOXHd6UbxAkJOM02z1qlLX3C0c3oFqbaKiy0vT11Po4QILmmZ5HYcERE5BypqIiKVLSDIOS9cRMLZ3e/YKF65pmlmQ/Ym2L7Uua4g9zQPbJwpl+U9/q5uB+eUC+Izpqzeyc87c3jx2nb4+Wk0TUTEF6moiYh4q5KjeLXrn919j4/ilWOa5uH9sH/Tiett8cmP9ZuVzonNxWeMnrmR+rFhXNamjttRRETkHKmoiYhUR+c6ildcDPk5Jxe78LN8DHHdyzd0YGv2YQL8NcVVRMRXqaiJiMgJfn6eUxREn/0onniNpOgQkqI1XVVExJfprTYREREREREvo6ImIiIiIiLiZVTUREREREREvIyKmoiIiIiIiJdRURMREREREfEyKmoiIiIiIiJeRkVNRERERETEy6ioiYiIiIiIeBkVNRERERERES+joiYiIiIiIuJlVNRERERERES8jIqaiIiIiIiIl1FRExERERER8TIqaiIiIiIiIl5GRU1ERERERMTLGGutO1/YmN3ApvN8mDhgTwXEqSq+lFdZK4cvZQXfyquslaOista31sZXwOPUCDXwOdKXsoJv5VXWyqGslceX8lZE1lM+P7pW1CqCMWahtbaT2znKy5fyKmvl8KWs4Ft5lbVy+FJWOZkv/ex8KSv4Vl5lrRzKWnl8KW9lZ9XURxERERERES+joiYiIiIiIuJlfL2ojXY7wFnypbzKWjl8KSv4Vl5lrRy+lFVO5ks/O1/KCr6VV1krh7JWHl/KW6lZffoYNRERERERkerI10fUREREREREqh0VNRERERERES/jE0XNGDPQGPOzMWa9MebxMm43xph/e25fZozp4EZOT5YzZe1rjDlgjFniufzRjZyeLGOMMbuMMStOcbs37dczZfWm/VrPGDPNGLPaGLPSGPNgGdt4xb4tZ1Zv2rchxpj5xpilnrxPlrGNt+zb8mT1mn3ryeNvjPnJGPN1Gbd5xX6VX9JzZOXQc2Tl0HNkpWXV82Mlcu350Vrr1RfAH9gANACCgKVAi1LbDAYmAgboBszz4qx9ga/d3q+eLL2BDsCKU9zuFfu1nFm9ab/WATp4Po4E1nrx72x5snrTvjVAhOfjQGAe0M1L9215snrNvvXk+S3wQVmZvGW/6vKLn4ueIysvr54jKyerniMrJ6ueHys3syvPj74wotYFWG+t3WitzQc+AoaU2mYI8I51/AjUMsbUqeqglC+r17DWzgT2nWYTb9mv5cnqNay12621iz0f5wCrgeRSm3nFvi1nVq/h2V+HPJ8Gei6lV0Tyln1bnqxewxiTAlwKvH6KTbxiv8ov6Dmykug5snLoObJy6Pmx8rj5/OgLRS0Z2FLi8yx++Z+kPNtUhfLm6O4Z7p1ojGlZNdHOibfs1/Lyuv1qjEkD2uO8W1SS1+3b02QFL9q3nukHS4BdwBRrrdfu23JkBe/Zty8C/w8oPsXtXrNf5SR6jnSPt+zX8vK6/arnyIql58dK8yIuPT/6QlEzZVxXunWXZ5uqUJ4ci4H61tq2wH+ALyo71Hnwlv1aHl63X40xEcBY4CFr7cHSN5dxF9f27RmyetW+tdYWWWvbASlAF2NMq1KbeM2+LUdWr9i3xpjLgF3W2kWn26yM67z170FNoudI93jLfi0Pr9uveo6seHp+rHhuPz/6QlHLAuqV+DwF2HYO21SFM+aw1h48NtxrrZ0ABBpj4qou4lnxlv16Rt62X40xgTh/1N+31n5exiZes2/PlNXb9u0x1tpsYDowsNRNXrNvjzlVVi/atz2BK4wxmTjT0S40xrxXahuv268C6DnSTd6yX8/I2/arniMrl54fK5Srz4++UNQWAI2NMenGmCDgOuDLUtt8CdziWXWlG3DAWru9qoNSjqzGmCRjjPF83AXnZ7C3ypOWj7fs1zPypv3qyfEGsNpa+8IpNvOKfVuerF62b+ONMbU8H4cCFwFrSm3mLfv2jFm9Zd9aa//PWptirU3D+bs11Vp7U6nNvGK/yi/oOdI93rJfz8ib9queIyuHnh8rh9vPjwEV8SCVyVpbaIy5H5iMs2LUGGvtSmPM3Z7bXwMm4Ky4sh7IA27z4qxXA/cYYwqBw8B11lpXhp2NMR/irKoTZ4zJAv6Ec0CnV+1XKFdWr9mvOO++3AwsN878a4AngFTwun1bnqzetG/rAG8bY/xx/mh/Yq392hv/HpQzqzft21/w0v0qJeg5svLoObLS6Dmycuj5sQpV1X41XvQ9i4iIiIiICL4x9VFERERERKRGUVETERERERHxMipqIiIiIiIiXkZFTURERERExMuoqImIiIiIiHgZFTWRs2CMKTLGLClxebwCHzvNGLOioh5PRESkquj5UaTief151ES8zGFrbTu3Q4iIiHgZPT+KVDCNqIlUAGNMpjHmWWPMfM+lkef6+saY740xyzz/pnquTzTGjDPGLPVcengeyt8Y8z9jzEpjzLfGmFDP9v+/nXtnrSKKwjD8LkOQgKigjaBgk0pQvOAPsLW0iGIlNqaJlZcfYK+EpLGwEAXLlAEREcRgIdhoKXYRkiJImiDyWWSLB80Bc5iQCbxPc/asGfaZXS3WvsxMVX1u/bzYpWFKkrQt5kdpdBZq0vZM/LW1Y2rg3vckF4E54FGLzQFPk5wGngOzLT4LvElyBjgHfGrxSWA+ySlgDbjS4veBs62fWzszNEmSRmZ+lDpWSXb7HaQ9o6rWkxzYIv4VuJTkS1WNA9+SHKmqVeBYkh8tvpzkaFWtAMeTbAz0cRJ4mWSyXd8DxpM8qKpFYB1YABaSrO/wUCVJ+m/mR6l7rqhJ3cmQ9rBntrIx0P7Jn3Okl4F54Dzwoao8XypJ2ivMj9IILNSk7kwN/C619jvgamtfB9629itgGqCqxqrq4LBOq2ofcCLJa+AucBj4Z9ZSkqSeMj9KI3DWQdqeiar6OHC9mOT3J4j3V9V7NidArrXYDPCkqu4AK8CNFr8NPK6qm2zODE4Dy0P+cwx4VlWHgAIeJlnraDySJHXB/Ch1zDNqUgfaHvwLSVZ3+10kSeoL86M0Orc+SpIkSVLPuKImSZIkST3jipokSZIk9YyFmiRJkiT1jIWaJEmSJPWMhZokSZIk9YyFmiRJkiT1zC832Hlh0bWpSAAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 1080x504 with 2 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "from helper_functions import plot_loss_curves\n",
        "\n",
        "# Check out the loss curves for FoodVision Big\n",
        "plot_loss_curves(effnetb2_food101_results)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SaVxuKwTQaQt"
      },
      "source": [
        "Nice!!!\n",
        "\n",
        "It looks like our regularization techniques (data augmentation and label smoothing) helped prevent our model from overfitting (the training loss is still higher than the test loss) this indicates our model has a bit more capacity to learn and could improve with further training."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vEDRE7K_QaQu"
      },
      "source": [
        "### 10.7 Saving and loading FoodVision Big\n",
        "\n",
        "Now we've trained our biggest model yet, let's save it so we can load it back in later."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7y1FqUuEQaQu",
        "outputId": "47c84650-972b-4bf6-e5aa-4e340aa2f0ae"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[INFO] Saving model to: models/09_pretrained_effnetb2_feature_extractor_food101_20_percent.pth\n"
          ]
        }
      ],
      "source": [
        "from going_modular.going_modular import utils\n",
        "\n",
        "# Create a model path\n",
        "effnetb2_food101_model_path = \"09_pretrained_effnetb2_feature_extractor_food101_20_percent.pth\"\n",
        "\n",
        "# Save FoodVision Big model\n",
        "utils.save_model(model=effnetb2_food101,\n",
        "                 target_dir=\"models\",\n",
        "                 model_name=effnetb2_food101_model_path)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jl18I6YRQaQu"
      },
      "source": [
        "Model saved!\n",
        "\n",
        "Before we move on, let's make sure we can load it back in.\n",
        "\n",
        "We'll do so by creating a model instance first with `create_effnetb2_model(num_classes=101)` (101 classes for all Food101 classes).\n",
        "\n",
        "And then loading the saved `state_dict()` with [`torch.nn.Module.load_state_dict()`](https://pytorch.org/docs/stable/generated/torch.nn.Module.html?highlight=load_state_dict#torch.nn.Module.load_state_dict) and [`torch.load()`](https://pytorch.org/docs/stable/generated/torch.load.html)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3jfLSaoZQaQu",
        "outputId": "886e35d6-ef15-49b2-b3df-8e08a62a7b34"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<All keys matched successfully>"
            ]
          },
          "execution_count": 75,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Create Food101 compatible EffNetB2 instance\n",
        "loaded_effnetb2_food101, effnetb2_transforms = create_effnetb2_model(num_classes=101)\n",
        "\n",
        "# Load the saved model's state_dict()\n",
        "loaded_effnetb2_food101.load_state_dict(torch.load(\"models/09_pretrained_effnetb2_feature_extractor_food101_20_percent.pth\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_xMCmCibQaQu"
      },
      "source": [
        "### 10.8 Checking FoodVision Big model size\n",
        "\n",
        "Our FoodVision Big model is capable of classifying 101 classes versus FoodVision Mini's 3 classes, a 33.6x increase!\n",
        "\n",
        "How does this effect the model size?\n",
        "\n",
        "Let's find out."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bKICxNe3QaQv",
        "outputId": "958c1a97-baa4-4e45-fc70-601dec01911b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Pretrained EffNetB2 feature extractor Food101 model size: 30 MB\n"
          ]
        }
      ],
      "source": [
        "from pathlib import Path\n",
        "\n",
        "# Get the model size in bytes then convert to megabytes\n",
        "pretrained_effnetb2_food101_model_size = Path(\"models\", effnetb2_food101_model_path).stat().st_size // (1024*1024) # division converts bytes to megabytes (roughly)\n",
        "print(f\"Pretrained EffNetB2 feature extractor Food101 model size: {pretrained_effnetb2_food101_model_size} MB\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z_GPr_6rQaQv"
      },
      "source": [
        "Hmm, it looks like the model size stayed largely the same (30 MB for FoodVision Big and 29 MB for FoodVision Mini) despite the large increase in the number of classes.\n",
        "\n",
        "This is because all the extra parameters for FoodVision Big are *only* in the last layer (the classifier head).\n",
        "\n",
        "All of the base layers are the same between FoodVision Big and FoodVision Mini.\n",
        "\n",
        "Going back up and comparing the model summaries will give more details.\n",
        "\n",
        "| **Model** | **Output shape (num classes)** | **Trainable parameters** | **Total parameters** | **Model size (MB)** |\n",
        "| ----- | ----- | ----- | ----- | ----- |\n",
        "| FoodVision Mini (EffNetB2 feature extractor) | 3 | 4,227 | 7,705,221 |  29 |\n",
        "| FoodVision Big (EffNetB2 feature extractor) | 101 | 142,309 | 7,843,303 | 30 |\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sp41_ja_QaQv"
      },
      "source": [
        "## 11. Turning our FoodVision Big model into a deployable app\n",
        "\n",
        "We've got a trained and saved EffNetB2 model on 20% of the Food101 dataset.\n",
        "\n",
        "And instead of letting our model live in a folder all its life, let's deploy it!\n",
        "\n",
        "We'll deploy our FoodVision Big model in the same way we deployed our FoodVision Mini model, as a Gradio demo on Hugging Face Spaces.\n",
        "\n",
        "To begin, let's create a `demos/foodvision_big/` directory to store our FoodVision Big demo files as well as a `demos/foodvision_big/examples` directory to hold an example image to test the demo with.\n",
        "\n",
        "When we're finished we'll have the following file structure:\n",
        "\n",
        "```\n",
        "demos/\n",
        "  foodvision_big/\n",
        "    09_pretrained_effnetb2_feature_extractor_food101_20_percent.pth\n",
        "    app.py\n",
        "    class_names.txt\n",
        "    examples/\n",
        "      example_1.jpg\n",
        "    model.py\n",
        "    requirements.txt\n",
        "```\n",
        "\n",
        "Where:\n",
        "* `09_pretrained_effnetb2_feature_extractor_food101_20_percent.pth` is our trained PyTorch model file.\n",
        "* `app.py` contains our FoodVision Big Gradio app.\n",
        "* `class_names.txt` contains all of the class names for FoodVision Big.\n",
        "* `examples/` contains example images to use with our Gradio app.\n",
        "* `model.py` contains the model defintion as well as any transforms assosciated with the model.\n",
        "* `requirements.txt` contains the dependencies to run our app such as `torch`, `torchvision` and `gradio`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wVD2KjIDQaQv"
      },
      "outputs": [],
      "source": [
        "from pathlib import Path\n",
        "\n",
        "# Create FoodVision Big demo path\n",
        "foodvision_big_demo_path = Path(\"demos/foodvision_big/\")\n",
        "\n",
        "# Make FoodVision Big demo directory\n",
        "foodvision_big_demo_path.mkdir(parents=True, exist_ok=True)\n",
        "\n",
        "# Make FoodVision Big demo examples directory\n",
        "(foodvision_big_demo_path / \"examples\").mkdir(parents=True, exist_ok=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_rakDkLZQaQv"
      },
      "source": [
        "### 11.1 Downloading an example image and moving it to the `examples` directory\n",
        "\n",
        "For our example image, we're going to use the faithful [`pizza-dad` image](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/images/04-pizza-dad.jpeg) (a photo of my dad eating pizza).\n",
        "\n",
        "So let's download it from the course GitHub via the `!wget` command and then we can move it to `demos/foodvision_big/examples` with the `!mv` command (short for \"move\").\n",
        "\n",
        "While we're here we'll move our trained Food101 EffNetB2 model from `models/09_pretrained_effnetb2_feature_extractor_food101_20_percent.pth` to `demos/foodvision_big` as well."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lUmRtY0ZQaQv",
        "outputId": "cdf63d7d-07f9-4f64-df07-79ca00177cea"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "--2022-08-25 14:24:41--  https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/04-pizza-dad.jpeg\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.110.133, 185.199.109.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 2874848 (2.7M) [image/jpeg]\n",
            "Saving to: '04-pizza-dad.jpeg’\n",
            "\n",
            "04-pizza-dad.jpeg   100%[===================>]   2.74M  7.85MB/s    in 0.3s    \n",
            "\n",
            "2022-08-25 14:24:43 (7.85 MB/s) - '04-pizza-dad.jpeg’ saved [2874848/2874848]\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# Download and move an example image\n",
        "!wget https://raw.githubusercontent.com/mrdbourke/pytorch-deep-learning/main/images/04-pizza-dad.jpeg\n",
        "!mv 04-pizza-dad.jpeg demos/foodvision_big/examples/04-pizza-dad.jpg\n",
        "\n",
        "# Move trained model to FoodVision Big demo folder (will error if model is already moved)\n",
        "!mv models/09_pretrained_effnetb2_feature_extractor_food101_20_percent.pth demos/foodvision_big"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hGBFh720QaQv"
      },
      "source": [
        "### 11.2 Saving Food101 class names to file (`class_names.txt`)\n",
        "\n",
        "Because there are so many classes in the Food101 dataset, instead of storing them as a list in our `app.py` file, let's saved them to a `.txt` file and read them in when necessary instead.\n",
        "\n",
        "We'll just remind ourselves what they look like first by checking out `food101_class_names`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dFgEyfSQQaQw",
        "outputId": "ffab9676-836e-4901-986a-b124a84ccfee"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['apple_pie',\n",
              " 'baby_back_ribs',\n",
              " 'baklava',\n",
              " 'beef_carpaccio',\n",
              " 'beef_tartare',\n",
              " 'beet_salad',\n",
              " 'beignets',\n",
              " 'bibimbap',\n",
              " 'bread_pudding',\n",
              " 'breakfast_burrito']"
            ]
          },
          "execution_count": 79,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Check out the first 10 Food101 class names\n",
        "food101_class_names[:10]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KVcK-ZhzQaQw"
      },
      "source": [
        "Wonderful, now we can write these to a text file by first creating a path to `demos/foodvision_big/class_names.txt` and then opening a file with Python's `open()` and then writing to it leaving a new line for each class.\n",
        "\n",
        "Ideally, we want our class names to be saved like:\n",
        "\n",
        "```\n",
        "apple_pie\n",
        "baby_back_ribs\n",
        "baklava\n",
        "beef_carpaccio\n",
        "beef_tartare\n",
        "...\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jBV16fxbQaQw",
        "outputId": "6908b8e0-a56e-4a69-9048-9acaa30fbe76"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[INFO] Saving Food101 class names to demos/foodvision_big/class_names.txt\n"
          ]
        }
      ],
      "source": [
        "# Create path to Food101 class names\n",
        "foodvision_big_class_names_path = foodvision_big_demo_path / \"class_names.txt\"\n",
        "\n",
        "# Write Food101 class names list to file\n",
        "with open(foodvision_big_class_names_path, \"w\") as f:\n",
        "    print(f\"[INFO] Saving Food101 class names to {foodvision_big_class_names_path}\")\n",
        "    f.write(\"\\n\".join(food101_class_names)) # leave a new line between each class"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mfwnpPXEQaQw"
      },
      "source": [
        "Excellent, now let's make sure we can read them in.\n",
        "\n",
        "To do so we'll use Python's [`open()`](https://www.w3schools.com/python/ref_func_open.asp) in read mode (`\"r\"`) and then use the [`readlines()`](https://www.w3schools.com/python/ref_file_readlines.asp) method to read each line of our `class_names.txt` file.\n",
        "\n",
        "And we can save the class names to a list by stripping the newline value of each of them with a list comprehension and [`strip()`](https://www.w3schools.com/python/ref_string_strip.asp)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "24fQ2BvOQaQw",
        "outputId": "43cc01f0-ee2b-434c-92b2-389f7c1548a0"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['apple_pie', 'baby_back_ribs', 'baklava', 'beef_carpaccio', 'beef_tartare']"
            ]
          },
          "execution_count": 81,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Open Food101 class names file and read each line into a list\n",
        "with open(foodvision_big_class_names_path, \"r\") as f:\n",
        "    food101_class_names_loaded = [food.strip() for food in  f.readlines()]\n",
        "\n",
        "# View the first 5 class names loaded back in\n",
        "food101_class_names_loaded[:5]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4TdjFeRvQaQw"
      },
      "source": [
        "### 11.3 Turning our FoodVision Big model into a Python script (`model.py`)\n",
        "\n",
        "Just like the FoodVision Mini demo, let's create a script that's capable of instantiating an EffNetB2 feature extractor model along with its necessary transforms."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "YL7gYaQ9QaQx",
        "outputId": "f94955d0-8f19-4167-9db1-f97f3d6f8323"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting demos/foodvision_big/model.py\n"
          ]
        }
      ],
      "source": [
        "%%writefile demos/foodvision_big/model.py\n",
        "import torch\n",
        "import torchvision\n",
        "\n",
        "from torch import nn\n",
        "\n",
        "\n",
        "def create_effnetb2_model(num_classes:int=3,\n",
        "                          seed:int=42):\n",
        "    \"\"\"Creates an EfficientNetB2 feature extractor model and transforms.\n",
        "\n",
        "    Args:\n",
        "        num_classes (int, optional): number of classes in the classifier head.\n",
        "            Defaults to 3.\n",
        "        seed (int, optional): random seed value. Defaults to 42.\n",
        "\n",
        "    Returns:\n",
        "        model (torch.nn.Module): EffNetB2 feature extractor model.\n",
        "        transforms (torchvision.transforms): EffNetB2 image transforms.\n",
        "    \"\"\"\n",
        "    # Create EffNetB2 pretrained weights, transforms and model\n",
        "    weights = torchvision.models.EfficientNet_B2_Weights.DEFAULT\n",
        "    transforms = weights.transforms()\n",
        "    model = torchvision.models.efficientnet_b2(weights=weights)\n",
        "\n",
        "    # Freeze all layers in base model\n",
        "    for param in model.parameters():\n",
        "        param.requires_grad = False\n",
        "\n",
        "    # Change classifier head with random seed for reproducibility\n",
        "    torch.manual_seed(seed)\n",
        "    model.classifier = nn.Sequential(\n",
        "        nn.Dropout(p=0.3, inplace=True),\n",
        "        nn.Linear(in_features=1408, out_features=num_classes),\n",
        "    )\n",
        "\n",
        "    return model, transforms"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bPBuCw9zQaQx"
      },
      "source": [
        "### 11.4 Turning our FoodVision Big Gradio app into a Python script (`app.py`)\n",
        "\n",
        "We've got a FoodVision Big `model.py` script, now let's create a FoodVision Big `app.py` script.\n",
        "\n",
        "This will again mostly be the same as the FoodVision Mini `app.py` script except we'll change:\n",
        "\n",
        "1. **Imports and class names setup** - The `class_names` variable will be a list for all of the Food101 classes rather than pizza, steak, sushi. We can access these via `demos/foodvision_big/class_names.txt`.\n",
        "2. **Model and transforms preparation** - The `model` will have `num_classes=101` rather than `num_classes=3`. We'll also be sure to load the weights from `\"09_pretrained_effnetb2_feature_extractor_food101_20_percent.pth\"` (our FoodVision Big model path).\n",
        "3. **Predict function** - This will stay the same as FoodVision Mini's `app.py`.\n",
        "4. **Gradio app** - The Gradio interace will have different `title`, `description` and `article` parameters to reflect the details of FoodVision Big.\n",
        "\n",
        "We'll also make sure to save it to `demos/foodvision_big/app.py` using the `%%writefile` magic command."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "c36ArsqaQaQx",
        "outputId": "4fdbad9f-ccb8-4534-d76d-a5f8496e7dca"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting demos/foodvision_big/app.py\n"
          ]
        }
      ],
      "source": [
        "%%writefile demos/foodvision_big/app.py\n",
        "### 1. Imports and class names setup ###\n",
        "import gradio as gr\n",
        "import os\n",
        "import torch\n",
        "\n",
        "from model import create_effnetb2_model\n",
        "from timeit import default_timer as timer\n",
        "from typing import Tuple, Dict\n",
        "\n",
        "# Setup class names\n",
        "with open(\"class_names.txt\", \"r\") as f: # reading them in from class_names.txt\n",
        "    class_names = [food_name.strip() for food_name in  f.readlines()]\n",
        "\n",
        "### 2. Model and transforms preparation ###\n",
        "\n",
        "# Create model\n",
        "effnetb2, effnetb2_transforms = create_effnetb2_model(\n",
        "    num_classes=101, # could also use len(class_names)\n",
        ")\n",
        "\n",
        "# Load saved weights\n",
        "effnetb2.load_state_dict(\n",
        "    torch.load(\n",
        "        f=\"09_pretrained_effnetb2_feature_extractor_food101_20_percent.pth\",\n",
        "        map_location=torch.device(\"cpu\"),  # load to CPU\n",
        "    )\n",
        ")\n",
        "\n",
        "### 3. Predict function ###\n",
        "\n",
        "# Create predict function\n",
        "def predict(img) -> Tuple[Dict, float]:\n",
        "    \"\"\"Transforms and performs a prediction on img and returns prediction and time taken.\n",
        "    \"\"\"\n",
        "    # Start the timer\n",
        "    start_time = timer()\n",
        "\n",
        "    # Transform the target image and add a batch dimension\n",
        "    img = effnetb2_transforms(img).unsqueeze(0)\n",
        "\n",
        "    # Put model into evaluation mode and turn on inference mode\n",
        "    effnetb2.eval()\n",
        "    with torch.inference_mode():\n",
        "        # Pass the transformed image through the model and turn the prediction logits into prediction probabilities\n",
        "        pred_probs = torch.softmax(effnetb2(img), dim=1)\n",
        "\n",
        "    # Create a prediction label and prediction probability dictionary for each prediction class (this is the required format for Gradio's output parameter)\n",
        "    pred_labels_and_probs = {class_names[i]: float(pred_probs[0][i]) for i in range(len(class_names))}\n",
        "\n",
        "    # Calculate the prediction time\n",
        "    pred_time = round(timer() - start_time, 5)\n",
        "\n",
        "    # Return the prediction dictionary and prediction time\n",
        "    return pred_labels_and_probs, pred_time\n",
        "\n",
        "### 4. Gradio app ###\n",
        "\n",
        "# Create title, description and article strings\n",
        "title = \"FoodVision Big 🍔👁\"\n",
        "description = \"An EfficientNetB2 feature extractor computer vision model to classify images of food into [101 different classes](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/extras/food101_class_names.txt).\"\n",
        "article = \"Created at [09. PyTorch Model Deployment](https://www.learnpytorch.io/09_pytorch_model_deployment/).\"\n",
        "\n",
        "# Create examples list from \"examples/\" directory\n",
        "example_list = [[\"examples/\" + example] for example in os.listdir(\"examples\")]\n",
        "\n",
        "# Create Gradio interface\n",
        "demo = gr.Interface(\n",
        "    fn=predict,\n",
        "    inputs=gr.Image(type=\"pil\"),\n",
        "    outputs=[\n",
        "        gr.Label(num_top_classes=5, label=\"Predictions\"),\n",
        "        gr.Number(label=\"Prediction time (s)\"),\n",
        "    ],\n",
        "    examples=example_list,\n",
        "    title=title,\n",
        "    description=description,\n",
        "    article=article,\n",
        ")\n",
        "\n",
        "# Launch the app!\n",
        "demo.launch()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PLD_6kS_QaQx"
      },
      "source": [
        "### 11.5 Creating a requirements file for FoodVision Big (`requirements.txt`)\n",
        "\n",
        "Now all we need is a `requirements.txt` file to tell our Hugging Face Space what dependencies our FoodVision Big app requires."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "a_WPUdacQaQy",
        "outputId": "fef48c22-d24d-4e05-9f05-130049cb446f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting demos/foodvision_big/requirements.txt\n"
          ]
        }
      ],
      "source": [
        "%%writefile demos/foodvision_big/requirements.txt\n",
        "torch==1.12.0\n",
        "torchvision==0.13.0\n",
        "gradio==3.1.4"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LcacloaaQaQy"
      },
      "source": [
        "### 11.6 Downloading our FoodVision Big app files\n",
        "\n",
        "We've got all the files we need to deploy our FoodVision Big app on Hugging Face, let's now zip them together and download them.\n",
        "\n",
        "We'll use the same process we used for the FoodVision Mini app above in [section 9.1: *Downloading our Foodvision Mini app files*](https://www.learnpytorch.io/09_pytorch_model_deployment/#91-downloading-our-foodvision-mini-app-files)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "weGN_lLVQaQy",
        "outputId": "6b17acd8-ce10-46b8-e29a-060974646c53"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "updating: 09_pretrained_effnetb2_feature_extractor_food101_20_percent.pth (deflated 8%)\n",
            "updating: app.py (deflated 54%)\n",
            "updating: class_names.txt (deflated 48%)\n",
            "updating: examples/ (stored 0%)\n",
            "updating: flagged/ (stored 0%)\n",
            "updating: model.py (deflated 56%)\n",
            "updating: requirements.txt (deflated 4%)\n",
            "updating: examples/04-pizza-dad.jpg (deflated 0%)\n",
            "Not running in Google Colab, can't use google.colab.files.download()\n"
          ]
        }
      ],
      "source": [
        "# Zip foodvision_big folder but exclude certain files\n",
        "!cd demos/foodvision_big && zip -r ../foodvision_big.zip * -x \"*.pyc\" \"*.ipynb\" \"*__pycache__*\" \"*ipynb_checkpoints*\"\n",
        "\n",
        "# Download the zipped FoodVision Big app (if running in Google Colab)\n",
        "try:\n",
        "    from google.colab import files\n",
        "    files.download(\"demos/foodvision_big.zip\")\n",
        "except:\n",
        "    print(\"Not running in Google Colab, can't use google.colab.files.download()\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rVr13jRcQaQy"
      },
      "source": [
        "### 11.7 Deploying our FoodVision Big app to HuggingFace Spaces\n",
        "\n",
        "B, E, A, Utiful!\n",
        "\n",
        "Time to bring our biggest model of the whole course to life!\n",
        "\n",
        "Let's deploy our FoodVision Big Gradio demo to Hugging Face Spaces so we can test it interactively and let others experience the magic of our machine learning efforts!\n",
        "\n",
        "> **Note:** There are [several ways to upload files to Hugging Face Spaces](https://huggingface.co/docs/hub/repositories-getting-started#getting-started-with-repositories). The following steps treat Hugging Face as a git repository to track files. However, you can also upload directly to Hugging Face Spaces via the [web interface](https://huggingface.co/docs/hub/repositories-getting-started#adding-files-to-a-repository-web-ui) or by the [`huggingface_hub` library](https://huggingface.co/docs/huggingface_hub/index).\n",
        "\n",
        "The good news is, we've already done the steps to do so with FoodVision Mini, so now all we have to do is customize them to suit FoodVision Big:\n",
        "\n",
        "1. [Sign up](https://huggingface.co/join) for a Hugging Face account.\n",
        "2. Start a new Hugging Face Space by going to your profile and then [clicking \"New Space\"](https://huggingface.co/new-space).\n",
        "    * **Note:** A Space in Hugging Face is also known as a \"code repository\" (a place to store your code/files) or \"repo\" for short.\n",
        "3. Give the Space a name, for example, mine is called `mrdbourke/foodvision_big`, you can see it here: https://huggingface.co/spaces/mrdbourke/foodvision_big\n",
        "4. Select a license (I used [MIT](https://opensource.org/licenses/MIT)).\n",
        "5. Select Gradio as the Space SDK (software development kit).\n",
        "   * **Note:** You can use other options such as Streamlit but since our app is built with Gradio, we'll stick with that.\n",
        "6. Choose whether your Space is it's public or private (I selected public since I'd like my Space to be available to others).\n",
        "7. Click \"Create Space\".\n",
        "8. Clone the repo locally by running: `git clone https://huggingface.co/spaces/[YOUR_USERNAME]/[YOUR_SPACE_NAME]` in terminal or command prompt.\n",
        "    * **Note:** You can also add files via uploading them under the \"Files and versions\" tab.\n",
        "9. Copy/move the contents of the downloaded `foodvision_big` folder to the cloned repo folder.\n",
        "10. To upload and track larger files (e.g. files over 10MB or in our case, our PyTorch model file) you'll need to [install Git LFS](https://git-lfs.github.com/) (which stands for \"git large file storage\").\n",
        "11. After you've installed Git LFS, you can activate it by running `git lfs install`.\n",
        "12. In the `foodvision_big` directory, track the files over 10MB with Git LFS with `git lfs track \"*.file_extension\"`.\n",
        "    * Track EffNetB2 PyTorch model file with `git lfs track \"09_pretrained_effnetb2_feature_extractor_food101_20_percent.pth\"`.\n",
        "    * **Note:** If you get any errors uploading images, you may have to track them with `git lfs` too, for example `git lfs track \"examples/04-pizza-dad.jpg\"`\n",
        "13. Track `.gitattributes` (automatically created when cloning from HuggingFace, this file will help ensure our larger files are tracked with Git LFS). You can see an example `.gitattributes` file on the [FoodVision Big Hugging Face Space](https://huggingface.co/spaces/mrdbourke/foodvision_big/blob/main/.gitattributes).\n",
        "    * `git add .gitattributes`\n",
        "14. Add the rest of the `foodvision_big` app files and commit them with:\n",
        "    * `git add *`\n",
        "    * `git commit -m \"first commit\"`\n",
        "15. Push (upload) the files to Hugging Face:\n",
        "    * `git push`\n",
        "16. Wait 3-5 minutes for the build to happen (future builds are faster) and your app to become live!\n",
        "\n",
        "If everything worked correctly, our FoodVision Big Gradio demo should be ready to classify!\n",
        "\n",
        "You can see my version here: https://huggingface.co/spaces/mrdbourke/foodvision_big/\n",
        "\n",
        "Or we can even embed our FoodVision Big Gradio demo right within our notebook as an [iframe](https://gradio.app/sharing_your_app/#embedding-with-iframes) with [`IPython.display.IFrame`](https://ipython.readthedocs.io/en/stable/api/generated/IPython.display.html#IPython.display.IFrame) and a link to our space in the format `https://hf.space/embed/[YOUR_USERNAME]/[YOUR_SPACE_NAME]/+`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "p3Bxp1n2QaQy",
        "outputId": "0ff117ce-98ce-4113-d644-71486a4a1720"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "        <iframe\n",
              "            width=\"900\"\n",
              "            height=\"750\"\n",
              "            src=\"https://hf.space/embed/mrdbourke/foodvision_big/+\"\n",
              "            frameborder=\"0\"\n",
              "            allowfullscreen\n",
              "            \n",
              "        ></iframe>\n",
              "        "
            ],
            "text/plain": [
              "<IPython.lib.display.IFrame at 0x7f145512baf0>"
            ]
          },
          "execution_count": 86,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# IPython is a library to help work with Python iteractively\n",
        "from IPython.display import IFrame\n",
        "\n",
        "# Embed FoodVision Big Gradio demo as an iFrame\n",
        "IFrame(src=\"https://hf.space/embed/mrdbourke/foodvision_big/+\", width=900, height=750)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F0ztwcrJQaQy"
      },
      "source": [
        "How cool is that!?!\n",
        "\n",
        "We've come a long way from building PyTorch models to predict a straight line... now we're building computer vision models accessible to people all around the world!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "v1RZfoNtQaQy"
      },
      "source": [
        "## Main takeaways\n",
        "\n",
        "* **Deployment is as important as training.** Once you’ve got a good working model, your first question should be: how can I deploy this and make it accessible to others? Deployment allows you to test your model in the real world rather than on private training and test sets.\n",
        "* **Three questions for machine learning model deployment:**\n",
        "    1. What’s the most ideal use case for the model (how well and how fast does it perform)?\n",
        "    2. Where’s the model going to go (is it on-device or on the cloud)?\n",
        "    3. How’s the model going to function (are predictions online or offline)?\n",
        "* **Deployment options are a plenty.** But best to start simple. One of the best current ways (I say current because these things are always changing) is to use Gradio to create a demo and host it on Hugging Face Spaces. Start simple and scale up when needed.\n",
        "* **Never stop experimenting.** Your machine learning model needs will likely change overtime so deploying a single model is not the last step. You might find the dataset changes, so you’ll have to update your model. Or new research gets released and there’s a better architecture to use.\n",
        "    * So deploying one model is an excellent step, but you'll likely want to update it over time.\n",
        "* **Machine learning model deployment is part of the engineering practice of MLOps (machine learning operations).** MLOps is an extension of DevOps (development operations) and involves all the engineering parts around training a model: data collection and storage, data preprocessing, model deployment, model monitoring, versioning and more. It’s a rapidly evolving field but there are some solid resources out there to learn more, many of which are in [PyTorch Extra Resources](https://www.learnpytorch.io/pytorch_extra_resources/#resources-for-machine-learning-and-deep-learning-engineering)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PpG23KZgQaQz"
      },
      "source": [
        "## Exercises\n",
        "\n",
        "All of the exercises are focused on practicing the code above.\n",
        "\n",
        "You should be able to complete them by referencing each section or by following the resource(s) linked.\n",
        "\n",
        "**Resources:**\n",
        "\n",
        "* [Exercise template notebook for 09](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/extras/exercises/09_pytorch_model_deployment_exercises.ipynb).\n",
        "* [Example solutions notebook for 09](https://github.com/mrdbourke/pytorch-deep-learning/blob/main/extras/solutions/09_pytorch_model_deployment_exercise_solutions.ipynb) try the exercises *before* looking at this.\n",
        "    * See a live [video walkthrough of the solutions on YouTube](https://youtu.be/jOX5ZCkWO-0) (errors and all).\n",
        "\n",
        "1. Make and time predictions with both feature extractor models on the test dataset using the GPU (`device=\"cuda\"`). Compare the model's prediction times on GPU vs CPU - does this close the gap between them? As in, does making predictions on the GPU make the ViT feature extractor prediction times closer to the EffNetB2 feature extractor prediction times?\n",
        "    * You'll find code to do these steps in [section 5. Making predictions with our trained models and timing them](https://www.learnpytorch.io/09_pytorch_model_deployment/#5-making-predictions-with-our-trained-models-and-timing-them) and [section 6. Comparing model results, prediction times and size](https://www.learnpytorch.io/09_pytorch_model_deployment/#6-comparing-model-results-prediction-times-and-size).\n",
        "2. The ViT feature extractor seems to have more learning capacity (due to more parameters) than EffNetB2, how does it go on the larger 20% split of the entire Food101 dataset?\n",
        "    * Train a ViT feature extractor on the 20% Food101 dataset for 5 epochs, just like we did with EffNetB2 in section [10. Creating FoodVision Big](https://www.learnpytorch.io/09_pytorch_model_deployment/#10-creating-foodvision-big).\n",
        "3. Make predictions across the 20% Food101 test dataset with the ViT feature extractor from exercise 2 and find the \"most wrong\" predictions.\n",
        "    * The predictions will be the ones with the highest prediction probability but with the wrong predicted label.\n",
        "    * Write a sentence or two about why you think the model got these predictions wrong.\n",
        "4. Evaluate the ViT feature extractor across the whole Food101 test dataset rather than just the 20% version, how does it perform?\n",
        "    * Does it beat the original Food101 paper's best result of 56.4% accuracy?\n",
        "5. Head to [Paperswithcode.com](https://paperswithcode.com/) and find the current best performing model on the Food101 dataset.\n",
        "    * What model architecture does it use?\n",
        "6. Write down 1-3 potential failure points of our deployed FoodVision models and what some potential solutions might be.\n",
        "    * For example, what happens if someone was to upload a photo that wasn't of food to our FoodVision Mini model?\n",
        "7. Pick any dataset from [`torchvision.datasets`](https://pytorch.org/vision/stable/datasets.html) and train a feature extractor model on it using a model from [`torchvision.models`](https://pytorch.org/vision/stable/models.html) (you could use one of the model's we've already created, e.g. EffNetB2 or ViT) for 5 epochs and then deploy your model as a Gradio app to Hugging Face Spaces.\n",
        "    * You may want to pick smaller dataset/make a smaller split of it so training doesn't take too long.\n",
        "    * I'd love to see your deployed models! So be sure to share them in Discord or on the [course GitHub Discussions page](https://github.com/mrdbourke/pytorch-deep-learning/discussions)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DXx5NYEqQaQz"
      },
      "source": [
        "## Extra-curriculum\n",
        "\n",
        "* Machine learning model deployment is generally an engineering challenge rather than a pure machine learning challenge, see the [PyTorch Extra Resources machine learning engineering section](https://www.learnpytorch.io/pytorch_extra_resources/#resources-for-machine-learning-and-deep-learning-engineering) for resources on learning more.\n",
        "    * Inside you'll find recommendations for resources such as Chip Huyen's book [*Designing Machine Learning Systems*](https://www.amazon.com/Designing-Machine-Learning-Systems-Production-Ready/dp/1098107969) (especially chapter 7 on model deployment) and Goku Mohandas's [Made with ML MLOps course](https://madewithml.com/#mlops).\n",
        "* As you start to build more and more of your own projects, you'll likely start using Git (and potentially GitHub) quite frequently. To learn more about both, I'd recommend the [*Git and GitHub for Beginners - Crash Course*](https://youtu.be/RGOj5yH7evk) video on the freeCodeCamp YouTube channel.\n",
        "* We've only scratched the surface with what's possible with Gradio. For more, I'd recommend checking out the [full documentation](https://gradio.app/docs/), especially:\n",
        "    * All of the different kinds of [input and output components](https://gradio.app/docs/#components).\n",
        "    * The [Gradio Blocks API](https://gradio.app/docs/#blocks) for more advanced workflows.\n",
        "    * The Hugging Face Course chapter on [how to use Gradio with Hugging Face](https://huggingface.co/course/chapter9/1).\n",
        "* Edge devices aren't limited to mobile phones, they include small computers like the Raspberry Pi and the PyTorch team have a [fantastic blog post tutorial](https://pytorch.org/tutorials/intermediate/realtime_rpi.html) on deploying a PyTorch model to one.\n",
        "* For a fanstastic guide on developing AI and ML-powered applications, see [Google's People + AI Guidebook](https://pair.withgoogle.com/guidebook). One of my favourites is the section on [setting the right expectations](https://pair.withgoogle.com/guidebook/patterns#set-the-right-expectations).\n",
        "    * I covered more of these kinds of resources, including guides from Apple, Microsoft and more in the [April 2021 edition of Machine Learning Monthly](https://zerotomastery.io/blog/machine-learning-monthly-april-2021/) (a monthly newsletter I send out with the latest and greatest of the ML field).\n",
        "* If you'd like to speed up your model's runtime on CPU, you should be aware of [TorchScript](https://pytorch.org/tutorials/beginner/Intro_to_TorchScript_tutorial.html), [ONNX](https://pytorch.org/docs/stable/onnx.html) (Open Neural Network Exchange) and [OpenVINO](https://docs.openvino.ai/latest/notebooks/102-pytorch-onnx-to-openvino-with-output.html). Going from pure PyTorch to ONNX/OpenVINO models I've seen a ~2x+ increase in performance.\n",
        "* For turning models into a deployable and scalable API, see the [TorchServe library](https://pytorch.org/serve/).\n",
        "* For a terrific example and rationale as to why deploying a machine learning model in the browser (a form of edge deployment) offers several benefits (no network transfer latency delay), see Jo Kristian Bergum's article on [*Moving ML Inference from the Cloud to the Edge*](https://bergum.medium.com/moving-ml-inference-from-the-cloud-to-the-edge-d6f98dbdb2e3)."
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.13"
    },
    "vscode": {
      "interpreter": {
        "hash": "3fbe1355223f7b2ffc113ba3ade6a2b520cadace5d5ec3e828c83ce02eb221bf"
      }
    },
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}