BERT-Text-Classification.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Modified from:\n",
    "\n",
    "# https://github.com/google-research/bert/blob/master/predicting_movie_reviews_with_bert_on_tf_hub.ipynb\n",
    "\n",
    "# Copyright 2019 Google Inc.\n",
    "\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "\n",
    "#     http://www.apache.org/licenses/LICENSE-2.0\n",
    "\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predicting Status of Environmental Issue with BERT on TF Hub\n",
    "\n",
    "If you’ve been following Natural Language Processing over the past year, you’ve probably heard of BERT: Bidirectional Encoder Representations from Transformers. It’s a neural network architecture designed by Google researchers that’s totally transformed what’s state-of-the-art for NLP tasks, like text classification, translation, summarization, and question answering.\n",
    "\n",
    "Now that BERT's been added to [TF Hub](https://www.tensorflow.org/hub) as a loadable module, it's easy(ish) to add into existing Tensorflow text pipelines. In an existing pipeline, BERT can replace text embedding layers like ELMO and GloVE. Alternatively, finetuning BERT can provide both an accuracy boost and faster training time in many cases.\n",
    "\n",
    "Here, we'll train a model to predict whether an environmental issue is resolved or unresolved using BERT in Tensorflow with tf hub\n",
    "\n",
    "Let's start by installing TensorFlow and TensorFlow Hub"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: tensorflow<2.0,>=1.15 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (1.15.5)\n",
      "Requirement already satisfied: gast==0.2.2 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (0.2.2)\n",
      "Requirement already satisfied: tensorflow-estimator==1.15.1 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (1.15.1)\n",
      "Requirement already satisfied: google-pasta>=0.1.6 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (0.2.0)\n",
      "Requirement already satisfied: six>=1.10.0 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (1.16.0)\n",
      "Requirement already satisfied: protobuf>=3.6.1 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (4.21.5)\n",
      "Requirement already satisfied: termcolor>=1.1.0 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (1.1.0)\n",
      "Requirement already satisfied: wrapt>=1.11.1 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (1.14.1)\n",
      "Requirement already satisfied: astor>=0.6.0 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (0.8.1)\n",
      "Requirement already satisfied: absl-py>=0.7.0 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (1.2.0)\n",
      "Requirement already satisfied: wheel>=0.26 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (0.37.1)\n",
      "Requirement already satisfied: keras-preprocessing>=1.0.5 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (1.1.2)\n",
      "Requirement already satisfied: numpy<1.19.0,>=1.16.0 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (1.18.5)\n",
      "Requirement already satisfied: h5py<=2.10.0 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (2.10.0)\n",
      "Requirement already satisfied: grpcio>=1.8.6 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (1.48.1)\n",
      "Requirement already satisfied: opt-einsum>=2.3.2 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (3.3.0)\n",
      "Requirement already satisfied: tensorboard<1.16.0,>=1.15.0 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (1.15.0)\n",
      "Requirement already satisfied: keras-applications>=1.0.8 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow<2.0,>=1.15) (1.0.8)\n",
      "Requirement already satisfied: werkzeug>=0.11.15 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow<2.0,>=1.15) (2.2.2)\n",
      "Requirement already satisfied: markdown>=2.6.8 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow<2.0,>=1.15) (3.4.1)\n",
      "Requirement already satisfied: setuptools>=41.0.0 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow<2.0,>=1.15) (63.4.1)\n",
      "Requirement already satisfied: importlib-metadata>=4.4 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow<2.0,>=1.15) (4.12.0)\n",
      "Requirement already satisfied: MarkupSafe>=2.1.1 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from werkzeug>=0.11.15->tensorboard<1.16.0,>=1.15.0->tensorflow<2.0,>=1.15) (2.1.1)\n",
      "Requirement already satisfied: typing-extensions>=3.6.4 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow<2.0,>=1.15) (4.3.0)\n",
      "Requirement already satisfied: zipp>=0.5 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<1.16.0,>=1.15.0->tensorflow<2.0,>=1.15) (3.8.1)\n",
      "Requirement already satisfied: bert-tensorflow==1.0.1 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (1.0.1)\n",
      "Requirement already satisfied: six in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from bert-tensorflow==1.0.1) (1.16.0)\n",
      "Requirement already satisfied: tensorflow-hub in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (0.12.0)\n",
      "Requirement already satisfied: numpy>=1.12.0 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow-hub) (1.18.5)\n",
      "Requirement already satisfied: protobuf>=3.8.0 in c:\\users\\t1nipun\\appdata\\local\\continuum\\anaconda3\\envs\\bertft\\lib\\site-packages (from tensorflow-hub) (4.21.5)\n"
     ]
    }
   ],
   "source": [
    "# install tf version 1 \n",
    "!pip install \"tensorflow>=1.15,<2.0\"\n",
    "# ! sign: It means run it as a shell command rather than a notebook command. It's the same as if you opened up a terminal and typed it without the !\n",
    "# install BERT python package\n",
    "!pip install bert-tensorflow==1.0.1\n",
    "!pip install --upgrade tensorflow-hub"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\bert\\optimization.py:87: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#from sklearn.model_selection import train_test_split\n",
    "import pandas as pd\n",
    "from datetime import datetime\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "## Tensorflow BERT helper functions\n",
    "import bert\n",
    "from bert import run_classifier\n",
    "from bert import optimization\n",
    "from bert import tokenization\n",
    "\n",
    "import tensorflow as tf\n",
    "import tensorflow_hub as hub\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensorflow version :  1.15.5\n",
      "tensorflow_hub version :  0.12.0\n"
     ]
    }
   ],
   "source": [
    "print(\"tensorflow version : \", tf.__version__)\n",
    "print(\"tensorflow_hub version : \", hub.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below, we'll set an output directory location to store our model output and checkpoints. This can be a local directory, in which case you'd set OUTPUT_DIR to the name of the directory you'd like to create."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "***** Model output directory: ./output *****\n",
      "***** Data directory: ./data/input *****\n"
     ]
    }
   ],
   "source": [
    "# Set the output directory for saving model file\n",
    "# Set the directory to read data\n",
    "\n",
    "OUTPUT_DIR = './output'#@param {type:\"string\"}\n",
    "DATA_INPUT_DIR = './data/input'\n",
    "DATA_OUTPUT_DIR = './data/output'\n",
    "print('***** Model output directory: {} *****'.format(OUTPUT_DIR))\n",
    "print('***** Data directory: {} *****'.format(DATA_INPUT_DIR))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead of using sklearn `train_test_split`, we have already segregated the training, validation and test dataset in three different csv files which can be found [here](https://github.com/nipun-goyal/BERT-Text-Classification/tree/main/data/input). In case of your dataset, you could either create three csv files, one each for train, validation and test set or else you could just have one file for train and validation set and then use `train_test_split` utility from sklearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1783, 6) (203, 6)\n"
     ]
    }
   ],
   "source": [
    "train = pd.read_csv(\"{}/train.csv\".format(DATA_INPUT_DIR))\n",
    "test = pd.read_csv(\"{}/testv1.csv\".format(DATA_INPUT_DIR))\n",
    "val = pd.read_csv(\"{}/dev.csv\".format(DATA_INPUT_DIR))\n",
    "print(train.shape, val.shape)\n",
    "## Labels\n",
    "# # Unresolved: 0, \n",
    "# Resolved: 1,"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As can be seen above, we have a sample of 1783 training examples to fine tune our model and 203 validation examples to evaluate how well the model performs on hold out dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "val['status_txt'] = val['status_txt'].astype(str)\n",
    "test['status_txt'] = test['status_txt'].astype(str)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's check the first ten rows of training dataset to see how the data is structured"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tableId</th>\n",
       "      <th>rowIndex</th>\n",
       "      <th>status_bin</th>\n",
       "      <th>status_txt</th>\n",
       "      <th>status</th>\n",
       "      <th>status_code</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1149</th>\n",
       "      <td>69579df1-eace-4925-9d25-c425bfc3ac28</td>\n",
       "      <td>75</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>None.</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1375</th>\n",
       "      <td>bdcc3529-ea0e-4131-9c0c-120b892452c4</td>\n",
       "      <td>51</td>\n",
       "      <td>Unresolved</td>\n",
       "      <td>Monitor in 2019 and control as required</td>\n",
       "      <td>Unresolved</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>479</th>\n",
       "      <td>3e1c53b4-5c01-46e2-bd72-5a338b5852f9</td>\n",
       "      <td>78</td>\n",
       "      <td>RESOLVED</td>\n",
       "      <td>No Further Action Recommended.</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1182</th>\n",
       "      <td>768b2aaf-8d86-4a30-bec5-7f4a4e646311</td>\n",
       "      <td>29</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>None.</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>235</th>\n",
       "      <td>0c200808-9563-467f-b74f-2d4b73c53fc4</td>\n",
       "      <td>415</td>\n",
       "      <td>U</td>\n",
       "      <td>Monitor in 2019. Add topsoil and recontour whe...</td>\n",
       "      <td>Unresolved</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>626</th>\n",
       "      <td>3e1c53b4-5c01-46e2-bd72-5a338b5852f9</td>\n",
       "      <td>226</td>\n",
       "      <td>RESOLVED</td>\n",
       "      <td>No Further Action Recommended.</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>126</th>\n",
       "      <td>0c200808-9563-467f-b74f-2d4b73c53fc4</td>\n",
       "      <td>232</td>\n",
       "      <td>U</td>\n",
       "      <td>Monitor in 2019. Repair subsidence when feasible.</td>\n",
       "      <td>Unresolved</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>408</th>\n",
       "      <td>3e1c53b4-5c01-46e2-bd72-5a338b5852f9</td>\n",
       "      <td>5</td>\n",
       "      <td>RESOLVED</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1020</th>\n",
       "      <td>4274c27b-ab24-4b6e-b72d-2c52e276758f</td>\n",
       "      <td>41</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>None.</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1372</th>\n",
       "      <td>bdcc3529-ea0e-4131-9c0c-120b892452c4</td>\n",
       "      <td>48</td>\n",
       "      <td>Unresolved</td>\n",
       "      <td>Monitor in 2019</td>\n",
       "      <td>Unresolved</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   tableId  rowIndex  status_bin  \\\n",
       "1149  69579df1-eace-4925-9d25-c425bfc3ac28        75    Resolved   \n",
       "1375  bdcc3529-ea0e-4131-9c0c-120b892452c4        51  Unresolved   \n",
       "479   3e1c53b4-5c01-46e2-bd72-5a338b5852f9        78    RESOLVED   \n",
       "1182  768b2aaf-8d86-4a30-bec5-7f4a4e646311        29    Resolved   \n",
       "235   0c200808-9563-467f-b74f-2d4b73c53fc4       415           U   \n",
       "626   3e1c53b4-5c01-46e2-bd72-5a338b5852f9       226    RESOLVED   \n",
       "126   0c200808-9563-467f-b74f-2d4b73c53fc4       232           U   \n",
       "408   3e1c53b4-5c01-46e2-bd72-5a338b5852f9         5    RESOLVED   \n",
       "1020  4274c27b-ab24-4b6e-b72d-2c52e276758f        41    Resolved   \n",
       "1372  bdcc3529-ea0e-4131-9c0c-120b892452c4        48  Unresolved   \n",
       "\n",
       "                                             status_txt      status  \\\n",
       "1149                                              None.    Resolved   \n",
       "1375            Monitor in 2019 and control as required  Unresolved   \n",
       "479                      No Further Action Recommended.    Resolved   \n",
       "1182                                              None.    Resolved   \n",
       "235   Monitor in 2019. Add topsoil and recontour whe...  Unresolved   \n",
       "626                      No Further Action Recommended.    Resolved   \n",
       "126   Monitor in 2019. Repair subsidence when feasible.  Unresolved   \n",
       "408                                            Resolved    Resolved   \n",
       "1020                                              None.    Resolved   \n",
       "1372                                    Monitor in 2019  Unresolved   \n",
       "\n",
       "      status_code  \n",
       "1149            1  \n",
       "1375            0  \n",
       "479             1  \n",
       "1182            1  \n",
       "235             0  \n",
       "626             1  \n",
       "126             0  \n",
       "408             1  \n",
       "1020            1  \n",
       "1372            0  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.sample(n = 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The columns tableId, rowIndex are just the unique identifiers for each row and have been kept here for the purpose of data mapping. You may not need to add unique identifer for your set of NLP problem. Let's now check each class distribution in both training as well as validation dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiUAAAHJCAYAAABaA0mPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAABXNUlEQVR4nO3dd1gUV/828HtpS11ApRoVREWwK0SxFxRrrI8xYkRje4zGFjXyRLDXqLHGksQajSb2aCyIYCWIELGDEESiIkYERKWf9w9f5ucKKBhgR/f+XBfX5Zw5M/ud2XY7e2ZGIYQQICIiItIwHU0XQERERAQwlBAREZFMMJQQERGRLDCUEBERkSwwlBAREZEsMJQQERGRLDCUEBERkSwwlBAREZEsMJQQERGRLDCUkFYbMmQIHBwc3mrZmTNnQqFQlG5BRERajKGEZEmhUBTrLzg4WNOlasSQIUPU9oOpqSmqV6+Ofv36Yc+ePcjLy3vrde/YsQPLly8vvWL/hWfPnmHmzJnFfp6Dg4OLfK0MGDCgbIvVUg8ePMDkyZNRu3ZtGBsbw8TEBE2aNMHcuXORkpKi6fIAyOs1Ta+n4L1vSI5++ukntemtW7ciICAA27ZtU2vv2LEjbGxs3vpxsrOzkZeXB6VSWeJlc3JykJOTA0NDw7d+/Lc1ZMgQ7Ny5Ez/88AMA4Pnz54iPj8dvv/2Gy5cvo23btjhw4ABUKlWJ1929e3dcvXoVt2/fLuWqS+6ff/6BlZUVZsyYgZkzZ76xf3BwMNq1a4dx48bB3d1dbZ6DgwNatmxZRpVqp7CwMHTt2hXp6ekYNGgQmjRpAgC4ePEidu7ciebNm+P48eMarlJer2l6PT1NF0BUmEGDBqlN//HHHwgICCjQ/qpnz57B2Ni42I+jr6//VvUBgJ6eHvT0NPcW0tPTK7A/5s6di4ULF8LX1xcjRozArl27NFSdZrVq1Qr9+vUrVt+cnBzk5eXBwMCgjKt6v6SkpKB3797Q1dXFn3/+idq1a6vNnzdvHr7//nsNVUfvKv58Q++stm3bom7duggPD0fr1q1hbGyM//3vfwCAAwcOoFu3brC3t4dSqYSTkxPmzJmD3NxctXW8Oqbk9u3bUCgUWLJkCTZs2AAnJycolUq4u7sjLCxMbdnCxpQoFAqMHTsW+/fvR926daFUKlGnTh0cPXq0QP3BwcFwc3ODoaEhnJycsH79+lIZpzJt2jR06tQJv/76K6Kjo6X24uyTtm3b4vDhw4iPj5d+9sjfP1lZWfD390eTJk1gbm4OExMTtGrVCkFBQQVq2LlzJ5o0aQIzMzOoVCrUq1cPK1asUOuTkpKCCRMmoEqVKlAqlahRowYWLVok/fR0+/ZtWFlZAQBmzZol1VOcIyZFefn5Xb58ufT8Xr9+HQBw8+ZN9OvXDxUqVIChoSHc3Nxw8ODBAuu5du0a2rdvDyMjI3zwwQeYO3cuNm7cCIVCofa/8aLqdXBwwJAhQ0q0P16t/02vz/zt6d+/P6ysrGBkZARnZ2d8/fXXAICgoCAoFArs27evwHI7duyAQqFASEhIkfty/fr1uHv3LpYtW1YgkACAjY0Npk+frtb23XffoU6dOlAqlbC3t8eYMWMK/MRT2L4BXrw227ZtK03n/1T3yy+/YN68efjggw9gaGiIDh06ICYmRm25ol7TJD88UkLvtEePHqFLly4YMGAABg0aJP2Us3nzZpiammLSpEkwNTXFyZMn4e/vj7S0NHzzzTdvXO+OHTvw5MkTjBo1CgqFAosXL0afPn3w119/vfHoytmzZ7F37158/vnnMDMzw8qVK9G3b1/cuXMHFStWBAD8+eef6Ny5M+zs7DBr1izk5uZi9uzZ0pfwv/Xpp5/i+PHjCAgIQK1atQAUb598/fXXSE1Nxd9//41vv/0WAGBqagoASEtLww8//IBPPvkEI0aMwJMnT/Djjz/Cy8sLFy5cQMOGDQEAAQEB+OSTT9ChQwcsWrQIAHDjxg2cO3cO48ePB/DiiFabNm1w9+5djBo1ClWrVsX58+fh6+uL+/fvY/ny5bCyssLatWsxevRo9O7dG3369AEA1K9f/43b/+TJE/zzzz9qbRUqVJD+vWnTJmRkZGDkyJFQKpWoUKECrl27hhYtWqBy5cqYNm0aTExM8Msvv6BXr17Ys2cPevfuDQBITExEu3btkJOTI/XbsGEDjIyM3uq5Ku7+eFlxXp+XL19Gq1atoK+vj5EjR8LBwQGxsbH47bffMG/ePLRt2xZVqlTB9u3bpW3Lt337djg5OcHDw6PImg8ePAgjI6NiH5GaOXMmZs2aBU9PT4wePRpRUVFYu3YtwsLCcO7cubc+arlw4ULo6Ohg8uTJSE1NxeLFi+Ht7Y3Q0FAAr39NkwwJonfAmDFjxKsv1zZt2ggAYt26dQX6P3v2rEDbqFGjhLGxscjIyJDafHx8RLVq1aTpuLg4AUBUrFhRJCcnS+0HDhwQAMRvv/0mtc2YMaNATQCEgYGBiImJkdoiIyMFALFq1SqprUePHsLY2FjcvXtXart165bQ09MrsM7C+Pj4CBMTkyLn//nnnwKAmDhxotRW3H3SrVs3tX2SLycnR2RmZqq1PX78WNjY2IjPPvtMahs/frxQqVQiJyenyPrmzJkjTExMRHR0tFr7tGnThK6urrhz544QQoiHDx8KAGLGjBlFrutlQUFBAkChf3FxcdLzq1KpRFJSktqyHTp0EPXq1VPbF3l5eaJ58+aiZs2aUtuECRMEABEaGiq1JSUlCXNzc+lx8hVVe7Vq1YSPj0+J90dJXp+tW7cWZmZmIj4+Xm2deXl50r99fX2FUqkUKSkpatuip6f3xn1uaWkpGjRo8No+L6/TwMBAdOrUSeTm5krtq1evFgDExo0bpbZX902+Nm3aiDZt2kjT+c+1i4uL2utyxYoVAoC4cuWK1FbUa5rkhz/f0DtNqVRi6NChBdpf/l9r/v+aW7VqhWfPnuHmzZtvXO/HH38MS0tLabpVq1YAgL/++uuNy3p6esLJyUmarl+/PlQqlbRsbm4uTpw4gV69esHe3l7qV6NGDXTp0uWN6y+O/P8JPnnyRGr7t/tEV1dXGneRl5eH5ORk5OTkwM3NDREREVI/CwsLPH36FAEBAUWu69dff0WrVq1gaWmJf/75R/rz9PREbm4uTp8+XeJtfpm/vz8CAgLU/mxtbaX5ffv2VTsqlZycjJMnT6J///7Svvnnn3/w6NEjeHl54datW7h79y4A4Pfff0ezZs3w4YcfSstbWVnB29v7rest6f540+vz4cOHOH36ND777DNUrVpVbdmXfx4cPHgwMjMzsXv3bqlt165dyMnJeeP4rbS0NJiZmRVr+06cOIGsrCxMmDABOjr/97UzYsQIqFQqHD58uFjrKczQoUPVxgOV5L1K8sOfb+idVrly5UIHKF67dg3Tp0/HyZMnkZaWpjYvNTX1jet99YM8/wvg8ePHJV42f/n8ZZOSkvD8+XPUqFGjQL/C2t5Geno6AKh9afzbfQIAW7ZswdKlS3Hz5k1kZ2dL7Y6OjtK/P//8c/zyyy/o0qULKleujE6dOqF///7o3Lmz1OfWrVu4fPlykT9XJSUlFaueotSrVw+enp5Fzn+5XgCIiYmBEAJ+fn7w8/MrsqbKlSsjPj4eTZs2LTDf2dn5rest6f540+sz/wu5bt26r33c2rVrw93dHdu3b8ewYcMAvPjpplmzZm98LapUKrXQ+zrx8fEACu4jAwMDVK9eXZr/Nv7Ne5Xkh6GE3mmF/Y6fkpKCNm3aQKVSYfbs2XBycoKhoSEiIiLw1VdfFesaHrq6uoW2i2KcQf9vli0tV69eBfB/Iac09slPP/2EIUOGoFevXpgyZQqsra2hq6uLBQsWIDY2VupnbW2NS5cu4dixYzhy5AiOHDmCTZs2YfDgwdiyZQuAF0daOnbsiKlTpxb6WPnjYMrKq6+b/O2fPHkyvLy8Cl2mtAIjgAIDrku6P0rzNTZ48GCMHz8ef//9NzIzM/HHH39g9erVb1yudu3auHTpErKyskr1zKWiBnrn5uYWut1yeL9R6WEoofdOcHAwHj16hL1796J169ZSe1xcnAar+j/W1tYwNDRUO0MgX2Ftb2Pbtm1QKBTo2LEjgJLtk6K+FHbv3o3q1atj7969an1mzJhRoK+BgQF69OiBHj16IC8vD59//jnWr18PPz8/1KhRA05OTkhPT3/t0YzX1VLaqlevDuDFKeJvqqlatWq4detWgfaoqKgCbZaWlgXOLsnKysL9+/fV2oq7P4orf3vyw+nrDBgwAJMmTcLPP/+M58+fQ19fHx9//PEbl+vRowdCQkKwZ88efPLJJ6/tW61aNQAv9lF+bcCLfREXF6e23YXtM+DF0ZaXly0JXnn53cExJfTeyf+f08v/U8rKysJ3332nqZLU6OrqwtPTE/v378e9e/ek9piYGBw5cuRfr3/hwoU4fvw4Pv74Y9SsWVN6TKB4+8TExKTQn3MKW0doaGiB00YfPXqkNq2joyOdMZOZmQkA6N+/P0JCQnDs2LECj5OSkoKcnBwAkK45U9ZXBrW2tkbbtm2xfv36AoEBeDFGI1/Xrl3xxx9/4MKFC2rzt2/fXmA5JyenAuNBNmzYUOBISXH3R3FZWVmhdevW2LhxI+7cuaM279UjCJUqVUKXLl3w008/Yfv27ejcuTMqVar0xsf473//Czs7O3z55Zdqp57nS0pKwty5cwG8GGdlYGCAlStXqj3+jz/+iNTUVHTr1k1qc3Jywh9//IGsrCyp7dChQ0hISCjexheiqNc0yQ+PlNB7p3nz5rC0tISPjw/GjRsHhUKBbdu2yepw7syZM3H8+HG0aNECo0ePRm5uLlavXo26devi0qVLxVpHTk6OdOXbjIwMxMfH4+DBg7h8+TLatWuHDRs2SH1Lsk+aNGmCXbt2YdKkSXB3d4epqSl69OiB7t27Y+/evejduze6deuGuLg4rFu3Dq6urtIYFgAYPnw4kpOT0b59e3zwwQeIj4/HqlWr0LBhQ7i4uAAApkyZgoMHD6J79+4YMmQImjRpgqdPn+LKlSvYvXs3bt++jUqVKsHIyAiurq7YtWsXatWqhQoVKqBu3bpvHCvxNtasWYOWLVuiXr16GDFiBKpXr44HDx4gJCQEf//9NyIjIwEAU6dOxbZt29C5c2eMHz9eOiW4WrVquHz5sto6hw8fjv/+97/o27cvOnbsiMjISBw7dqzAl35x90dJrFy5Ei1btkTjxo0xcuRIODo64vbt2zh8+HCB19jgwYOlU3vnzJlTrPVbWlpi37596Nq1Kxo2bKh2RdeIiAj8/PPP0inFVlZW8PX1xaxZs9C5c2d89NFHiIqKwnfffQd3d3e1QbXDhw/H7t270blzZ/Tv3x+xsbH46aef1AaPl1RRr2mSIc2c9ENUMkWdElynTp1C+587d040a9ZMGBkZCXt7ezF16lRx7NgxAUAEBQVJ/Yo6Jfibb74psE68cnpnUacEjxkzpsCyhZ3mGBgYKBo1aiQMDAyEk5OT+OGHH8SXX34pDA0Ni9gL/8fHx0ftdFdjY2Ph4OAg+vbtK3bv3q122mVJ90l6eroYOHCgsLCwEACk/ZOXlyfmz58vqlWrJpRKpWjUqJE4dOhQgX24e/du0alTJ2FtbS0MDAxE1apVxahRo8T9+/fV6nny5Inw9fUVNWrUEAYGBqJSpUqiefPmYsmSJSIrK0vqd/78edGkSRNhYGDwxtOD808T/fXXXwud/7rnVwghYmNjxeDBg4Wtra3Q19cXlStXFt27dxe7d+9W63f58mXRpk0bYWhoKCpXrizmzJkjfvzxxwKnBOfm5oqvvvpKVKpUSRgbGwsvLy8RExNT6OuhOPujJK9PIYS4evWq6N27t7CwsBCGhobC2dlZ+Pn5FVg2MzNTWFpaCnNzc/H8+fNC901R7t27JyZOnChq1aolDA0NhbGxsWjSpImYN2+eSE1NVeu7evVqUbt2baGvry9sbGzE6NGjxePHjwusc+nSpaJy5cpCqVSKFi1aiIsXLxZ5SvCrz3X+Ptq0aZPUVtRrmuSH974hkpFevXrh2rVrhY5ZIHnbvHkzhg4diri4uHfuiqE5OTmwt7dHjx498OOPP2q6HNJiHFNCpCHPnz9Xm7516xZ+//13tUtpE5WH/fv34+HDhxg8eLCmSyEtxzElRBpSvXp1DBkyRLpOw9q1a2FgYFDkaaFEpS00NBSXL1/GnDlz0KhRI7Rp00bTJZGWYygh0pDOnTvj559/RmJiIpRKJTw8PDB//nzpjBmisrZ27Vr89NNPaNiwITZv3qzpcojAMSVEREQkCxodU3L69Gn06NED9vb2UCgU2L9/v9p8IQT8/f1hZ2cHIyMjeHp6FhgAmJycDG9vb6hUKlhYWGDYsGFqpycC/3e3TENDQ1SpUgWLFy8u600jIiKiEtJoKHn69CkaNGiANWvWFDp/8eLFWLlyJdatW4fQ0FCYmJjAy8sLGRkZUh9vb29cu3YNAQEBOHToEE6fPo2RI0dK89PS0tCpUydUq1YN4eHh+OabbzBz5ky1azgQERGR5snm5xuFQoF9+/ahV69eAF4cJbG3t8eXX36JyZMnA3hx0zAbGxts3rwZAwYMwI0bN+Dq6oqwsDC4ubkBAI4ePYquXbvi77//hr29PdauXYuvv/4aiYmJ0v0Zpk2bhv379xfrzqjAi/tS3Lt3D2ZmZrxcMRERUQkIIfDkyRPY29ur3SW6MLId6BoXF4fExES1eyKYm5ujadOmCAkJwYABAxASEgILCwspkAAvLmeso6OD0NBQ9O7dGyEhIWjdurXaDaO8vLywaNEiPH78WO323/kyMzOly2EDwN27d+Hq6lpGW0pERPT+S0hIwAcffPDaPrINJYmJiQAAGxsbtXYbGxtpXmJiIqytrdXm6+npoUKFCmp9Xr1Nef46ExMTCw0lCxYswKxZswq0JyQkQKVSveUWERERaZ+0tDRUqVIFZmZmb+wr21CiSb6+vpg0aZI0nb9DVSoVQ4mMPXnyBH5+fti3bx+SkpLQqFEjrFixAu7u7gCAIUOGYMuWLWrLeHl54ejRo9K0g4MD4uPj1fosWLAA06ZNK/sNICJ6jxVn+INsQ4mtrS0A4MGDB7Czs5PaHzx4gIYNG0p9kpKS1JbLyclBcnKytLytrS0ePHig1id/Or/Pq5RKJZRKZalsB5Wf4cOH4+rVq9i2bRvs7e3x008/wdPTE9evX0flypUBvLg2yKZNm6RlCnueZ8+ejREjRkjTxUn3RET078n2MvOOjo6wtbVFYGCg1JaWlobQ0FDpzpMeHh5ISUlBeHi41OfkyZPIy8tD06ZNpT6nT59Gdna21CcgIADOzs6F/nRD76bnz59jz549WLx4MVq3bo0aNWpg5syZqFGjBtauXSv1UyqVsLW1lf4Kew2YmZmp9TExMSnPTSEi0loaDSXp6em4dOmSdBvtuLg4XLp0CXfu3IFCocCECRMwd+5cHDx4EFeuXMHgwYNhb28vnaHj4uKCzp07Y8SIEbhw4QLOnTuHsWPHYsCAAbC3twcADBw4EAYGBhg2bBiuXbuGXbt2YcWKFWo/z9C7LycnB7m5uTA0NFRrNzIywtmzZ6Xp4OBgWFtbw9nZGaNHj8ajR48KrGvhwoWoWLEiGjVqhG+++QY5OTllXj8REeGV+66Xs/xbT7/6l39L77y8POHn5ydsbGyEUqkUHTp0EFFRUWrrePTokfjkk0+EqampUKlUYujQoeLJkydqfSIjI0XLli2FUqkUlStXFgsXLixRnampqQJAgdtwk7x4eHiINm3aiLt374qcnByxbds2oaOjI2rVqiWEEOLnn38WBw4cEJcvXxb79u0TLi4uwt3dXeTk5EjrWLp0qQgKChKRkZFi7dq1wsLCQkycOFFTm0RE9M4ryXeobK5TImdpaWkwNzdHamoqB7rKWGxsLD777DOcPn0aurq6aNy4MWrVqoXw8HDcuHGjQP+//voLTk5OOHHiBDp06FDoOjdu3IhRo0YhPT2d44yIiN5CSb5DZTumhKiknJyccOrUKaSnpyMhIQEXLlxAdnY2qlevXmj/6tWro1KlSoiJiSlynU2bNkVOTg5u375dRlUTEVE+hhJ675iYmMDOzg6PHz/GsWPH0LNnz0L7/f3333j06JHa2V2vunTpEnR0dApcD4eIiEqfbE8JJiqpY8eOQQgBZ2dnxMTEYMqUKahduzaGDh2K9PR0zJo1C3379oWtrS1iY2MxdepU1KhRA15eXgCAkJAQhIaGol27djAzM0NISAgmTpyIQYMG8UwtIqJywFBC743U1FT4+vri77//RoUKFdC3b1/MmzcP+vr6yMnJweXLl7FlyxakpKTA3t4enTp1wpw5c6SxIkqlEjt37sTMmTORmZkJR0dHTJw4kWdqERGVEw50LQYOdCUiIno7HOhKRERE7xz+fEMFOEw7rOkSqBTdXthN0yUQERULj5QQERGRLDCUEBERkSwwlBAREZEsMJQQERGRLDCUEBERkSwwlBAREZEsMJQQERGRLDCUEBERkSwwlBAREZEsMJQQERGRLDCUEBERkSwwlBAREZEsMJQQERGRLDCUEBERkSwwlBAREZEsMJQQERGRLDCUEBERkSwwlBAREZEsMJQQERGRLDCUEBERkSwwlBAREZEsMJQQERGRLDCUEBERkSwwlBAREZEsMJQQERGRLDCUEBERkSwwlBARUZnLzc2Fn58fHB0dYWRkBCcnJ8yZMwdCCABAdnY2vvrqK9SrVw8mJiawt7fH4MGDce/ePbX1REdHo2fPnqhUqRJUKhVatmyJoKAgTWwSlQGGEiIiKnOLFi3C2rVrsXr1aty4cQOLFi3C4sWLsWrVKgDAs2fPEBERAT8/P0RERGDv3r2IiorCRx99pLae7t27IycnBydPnkR4eDgaNGiA7t27IzExURObRaVMIfJjKhUpLS0N5ubmSE1NhUql0nQ5Zc5h2mFNl0Cl6PbCbpougQjdu3eHjY0NfvzxR6mtb9++MDIywk8//VToMmFhYfjwww8RHx+PqlWr4p9//oGVlRVOnz6NVq1aAQCePHkClUqFgIAAeHp6lsu2UMmU5DuUR0qIiKjMNW/eHIGBgYiOjgYAREZG4uzZs+jSpUuRy6SmpkKhUMDCwgIAULFiRTg7O2Pr1q14+vQpcnJysH79elhbW6NJkyblsRlUxvQ0XQAREb3/pk2bhrS0NNSuXRu6urrIzc3FvHnz4O3tXWj/jIwMfPXVV/jkk0+k/10rFAqcOHECvXr1gpmZGXR0dGBtbY2jR4/C0tKyPDeHyghDCRERlblffvkF27dvx44dO1CnTh1cunQJEyZMgL29PXx8fNT6Zmdno3///hBCYO3atVK7EAJjxoyBtbU1zpw5AyMjI/zwww/o0aMHwsLCYGdnV96bRaWMoYSIiMrclClTMG3aNAwYMAAAUK9ePcTHx2PBggVqoSQ/kMTHx+PkyZNqYxBOnjyJQ4cO4fHjx1L7d999h4CAAGzZsgXTpk0r342iUsdQQkREZe7Zs2fQ0VEfxqirq4u8vDxpOj+Q3Lp1C0FBQahYsWKBdQAosB4dHR219dC7i6GEiIjKXI8ePTBv3jxUrVoVderUwZ9//olly5bhs88+A/AikPTr1w8RERE4dOgQcnNzpdN8K1SoAAMDA3h4eMDS0hI+Pj7w9/eHkZERvv/+e8TFxaFbN55l9j5gKCEiojK3atUq+Pn54fPPP0dSUhLs7e0xatQo+Pv7AwDu3r2LgwcPAgAaNmyotmxQUBDatm2LSpUq4ejRo/j666/Rvn17ZGdno06dOjhw4AAaNGhQ3ptEZYDXKSkGXqeE3mW8TgkRaRKvU0JERETvHP58Q0T0DuGRzPcLj2Sq45ESIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgVZh5Lc3Fz4+fnB0dERRkZGcHJywpw5c/DyWcxCCPj7+8POzg5GRkbw9PTErVu31NaTnJwMb29vqFQqWFhYYNiwYUhPTy/vzSEiIqLXkHUoWbRoEdauXYvVq1fjxo0bWLRoERYvXoxVq1ZJfRYvXoyVK1di3bp1CA0NhYmJCby8vJCRkSH18fb2xrVr1xAQEIBDhw7h9OnTGDlypCY2iYiIiIog61OCz58/j549e0qXD3ZwcMDPP/+MCxcuAHhxlGT58uWYPn06evbsCQDYunUrbGxssH//fgwYMAA3btzA0aNHERYWBjc3NwAvrizYtWtXLFmyBPb29prZOCIiIlIj6yMlzZs3R2BgIKKjowEAkZGROHv2LLp06QIAiIuLQ2JiIjw9PaVlzM3N0bRpU4SEhAAAQkJCYGFhIQUSAPD09ISOjg5CQ0MLfdzMzEykpaWp/REREVHZkvWRkmnTpiEtLQ21a9eGrq4ucnNzMW/ePHh7ewOAdLMmGxsbteVsbGykeYmJibC2tlabr6enhwoVKkh9XrVgwQLMmjWrtDeHiIiIXkPWR0p++eUXbN++HTt27EBERAS2bNmCJUuWYMuWLWX6uL6+vkhNTZX+EhISyvTxiIiISOZHSqZMmYJp06ZhwIABAIB69eohPj4eCxYsgI+PD2xtbQEADx48gJ2dnbTcgwcPpLtM2traIikpSW29OTk5SE5OlpZ/lVKphFKpLIMtIiIioqLI+kjJs2fPoKOjXqKuri7y8vIAAI6OjrC1tUVgYKA0Py0tDaGhofDw8AAAeHh4ICUlBeHh4VKfkydPIi8vD02bNi2HrSAiIqLikPWRkh49emDevHmoWrUq6tSpgz///BPLli3DZ599BgBQKBSYMGEC5s6di5o1a8LR0RF+fn6wt7dHr169AAAuLi7o3LkzRowYgXXr1iE7Oxtjx47FgAEDeOYNERGRjMg6lKxatQp+fn74/PPPkZSUBHt7e4waNQr+/v5Sn6lTp+Lp06cYOXIkUlJS0LJlSxw9ehSGhoZSn+3bt2Ps2LHo0KEDdHR00LdvX6xcuVITm0RERERFUIiXL49KhUpLS4O5uTlSU1OhUqk0XU6Z463R3y+8Nfr7he/P94s2vD9L8h0q6zElREREpD0YSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFhhKiIiISBYYSoiIiEgWGEqIiIhIFmQfSu7evYtBgwahYsWKMDIyQr169XDx4kVpvhAC/v7+sLOzg5GRETw9PXHr1i21dSQnJ8Pb2xsqlQoWFhYYNmwY0tPTy3tTiIiI6DVkHUoeP36MFi1aQF9fH0eOHMH169exdOlSWFpaSn0WL16MlStXYt26dQgNDYWJiQm8vLyQkZEh9fH29sa1a9cQEBCAQ4cO4fTp0xg5cqQmNomIiIiKoKfpAl5n0aJFqFKlCjZt2iS1OTo6Sv8WQmD58uWYPn06evbsCQDYunUrbGxssH//fgwYMAA3btzA0aNHERYWBjc3NwDAqlWr0LVrVyxZsgT29vYFHjczMxOZmZnSdFpaWlltIhEREf1/sj5ScvDgQbi5ueE///kPrK2t0ahRI3z//ffS/Li4OCQmJsLT01NqMzc3R9OmTRESEgIACAkJgYWFhRRIAMDT0xM6OjoIDQ0t9HEXLFgAc3Nz6a9KlSpltIVERESUT9ah5K+//sLatWtRs2ZNHDt2DKNHj8a4ceOwZcsWAEBiYiIAwMbGRm05GxsbaV5iYiKsra3V5uvp6aFChQpSn1f5+voiNTVV+ktISCjtTSMiIqJXyPrnm7y8PLi5uWH+/PkAgEaNGuHq1atYt24dfHx8yuxxlUollEplma2fiIiICpL1kRI7Ozu4urqqtbm4uODOnTsAAFtbWwDAgwcP1Po8ePBAmmdra4ukpCS1+Tk5OUhOTpb6EBERkebJOpS0aNECUVFRam3R0dGoVq0agBeDXm1tbREYGCjNT0tLQ2hoKDw8PAAAHh4eSElJQXh4uNTn5MmTyMvLQ9OmTcthK4iIiKg4ZP3zzcSJE9G8eXPMnz8f/fv3x4ULF7BhwwZs2LABAKBQKDBhwgTMnTsXNWvWhKOjI/z8/GBvb49evXoBeHFkpXPnzhgxYgTWrVuH7OxsjB07FgMGDCj0zBsiIiLSDFmHEnd3d+zbtw++vr6YPXs2HB0dsXz5cnh7e0t9pk6diqdPn2LkyJFISUlBy5YtcfToURgaGkp9tm/fjrFjx6JDhw7Q0dFB3759sXLlSk1sEhERERVBIYQQmi5C7tLS0mBubo7U1FSoVCpNl1PmHKYd1nQJVIpuL+ym6RKoFPH9+X7RhvdnSb5DZT2mhIiIiLQHQwkRERHJAkMJERERyQJDCREREckCQwkRERHJwluFkurVq+PRo0cF2lNSUlC9evV/XRQRERFpn7cKJbdv30Zubm6B9szMTNy9e/dfF0VERETap0QXTzt48KD072PHjsHc3Fyazs3NRWBgIBwcHEqtOCIiItIeJQol+ZduVygUBe7Sq6+vDwcHByxdurTUiiMiIiLtUaJQkpeXB+DFjfDCwsJQqVKlMimKiIiItM9b3fsmLi6utOsgIiIiLffWN+QLDAxEYGAgkpKSpCMo+TZu3PivCyMiIiLt8lahZNasWZg9ezbc3NxgZ2cHhUJR2nURERGRlnmrULJu3Tps3rwZn376aWnXQ0RERFrqra5TkpWVhebNm5d2LURERKTF3iqUDB8+HDt27CjtWoiIiEiLvdXPNxkZGdiwYQNOnDiB+vXrQ19fX23+smXLSqU4IiIi0h5vFUouX76Mhg0bAgCuXr2qNo+DXomIiOhtvFUoCQoKKu06iIiISMu91ZgSIiIiotL2VkdK2rVr99qfaU6ePPnWBREREZF2eqtQkj+eJF92djYuXbqEq1evFrhRHxEREVFxvFUo+fbbbwttnzlzJtLT0/9VQURERKSdSnVMyaBBg3jfGyIiInorpRpKQkJCYGhoWJqrJCIiIi3xVj/f9OnTR21aCIH79+/j4sWL8PPzK5XCiIiISLu8VSgxNzdXm9bR0YGzszNmz56NTp06lUphREREpF3eKpRs2rSptOsgIiIiLfdWoSRfeHg4bty4AQCoU6cOGjVqVCpFERERkfZ5q1CSlJSEAQMGIDg4GBYWFgCAlJQUtGvXDjt37oSVlVVp1khERERa4K3Ovvniiy/w5MkTXLt2DcnJyUhOTsbVq1eRlpaGcePGlXaNREREpAXe6kjJ0aNHceLECbi4uEhtrq6uWLNmDQe6EhER0Vt5qyMleXl50NfXL9Cur6+PvLy8f10UERERaZ+3CiXt27fH+PHjce/ePant7t27mDhxIjp06FBqxREREZH2eKtQsnr1aqSlpcHBwQFOTk5wcnKCo6Mj0tLSsGrVqtKukYiIiLTAW40pqVKlCiIiInDixAncvHkTAODi4gJPT89SLY6IiIi0R4mOlJw8eRKurq5IS0uDQqFAx44d8cUXX+CLL76Au7s76tSpgzNnzpRVrURERPQeK1EoWb58OUaMGAGVSlVgnrm5OUaNGoVly5aVWnFERESkPUoUSiIjI9G5c+ci53fq1Anh4eH/uigiIiLSPiUKJQ8ePCj0VOB8enp6ePjw4b8uioiIiLRPiUJJ5cqVcfXq1SLnX758GXZ2dv+6KCIiItI+JQolXbt2hZ+fHzIyMgrMe/78OWbMmIHu3buXWnFERESkPUp0SvD06dOxd+9e1KpVC2PHjoWzszMA4ObNm1izZg1yc3Px9ddfl0mhRERE9H4rUSixsbHB+fPnMXr0aPj6+kIIAQBQKBTw8vLCmjVrYGNjUyaFEhER0futxBdPq1atGn7//Xc8fvwYMTExEEKgZs2asLS0LIv6iIiISEu81RVdAcDS0hLu7u6lWQsRERFpsbe69w0RERFRaWMoISIiIllgKCEiIiJZYCghIiIiWXinQsnChQuhUCgwYcIEqS0jIwNjxoxBxYoVYWpqir59++LBgwdqy925cwfdunWDsbExrK2tMWXKFOTk5JRz9URERPQ670woCQsLw/r161G/fn219okTJ+K3337Dr7/+ilOnTuHevXvo06ePND83NxfdunVDVlYWzp8/jy1btmDz5s3w9/cv700gIiKi13gnQkl6ejq8vb3x/fffq10PJTU1FT/++COWLVuG9u3bo0mTJti0aRPOnz+PP/74AwBw/PhxXL9+HT/99BMaNmyILl26YM6cOVizZg2ysrI0tUlERET0incilIwZMwbdunWDp6enWnt4eDiys7PV2mvXro2qVasiJCQEABASEoJ69eqpXWnWy8sLaWlpuHbtWqGPl5mZibS0NLU/IiIiKltvffG08rJz505EREQgLCyswLzExEQYGBjAwsJCrd3GxgaJiYlSn1cvfZ8/nd/nVQsWLMCsWbNKoXoiIiIqLlkfKUlISMD48eOxfft2GBoaltvj+vr6IjU1VfpLSEgot8cmIiLSVrIOJeHh4UhKSkLjxo2hp6cHPT09nDp1CitXroSenh5sbGyQlZWFlJQUteUePHgAW1tbAICtrW2Bs3Hyp/P7vEqpVEKlUqn9ERERUdmSdSjp0KEDrly5gkuXLkl/bm5u8Pb2lv6tr6+PwMBAaZmoqCjcuXMHHh4eAAAPDw9cuXIFSUlJUp+AgACoVCq4urqW+zYRERFR4WQ9psTMzAx169ZVazMxMUHFihWl9mHDhmHSpEmoUKECVCoVvvjiC3h4eKBZs2YAgE6dOsHV1RWffvopFi9ejMTEREyfPh1jxoyBUqks920iIiKiwsk6lBTHt99+Cx0dHfTt2xeZmZnw8vLCd999J83X1dXFoUOHMHr0aHh4eMDExAQ+Pj6YPXu2BqsmIiKiV71zoSQ4OFht2tDQEGvWrMGaNWuKXKZatWr4/fffy7gyIiIi+jdkPaaEiIiItAdDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREckCQwkRERHJAkMJERERyQJDCREREcmCrEPJggUL4O7uDjMzM1hbW6NXr16IiopS65ORkYExY8agYsWKMDU1Rd++ffHgwQO1Pnfu3EG3bt1gbGwMa2trTJkyBTk5OeW5KURERPQGsg4lp06dwpgxY/DHH38gICAA2dnZ6NSpE54+fSr1mThxIn777Tf8+uuvOHXqFO7du4c+ffpI83Nzc9GtWzdkZWXh/Pnz2LJlCzZv3gx/f39NbBIREREVQSGEEJouorgePnwIa2trnDp1Cq1bt0ZqaiqsrKywY8cO9OvXDwBw8+ZNuLi4ICQkBM2aNcORI0fQvXt33Lt3DzY2NgCAdevW4auvvsLDhw9hYGDwxsdNS0uDubk5UlNToVKpynQb5cBh2mFNl0Cl6PbCbpougUoR35/vF214f5bkO1TWR0pelZqaCgCoUKECACA8PBzZ2dnw9PSU+tSuXRtVq1ZFSEgIACAkJAT16tWTAgkAeHl5IS0tDdeuXSv0cTIzM5GWlqb2R0RERGXrnQkleXl5mDBhAlq0aIG6desCABITE2FgYAALCwu1vjY2NkhMTJT6vBxI8ufnzyvMggULYG5uLv1VqVKllLeGiIiIXvXOhJIxY8bg6tWr2LlzZ5k/lq+vL1JTU6W/hISEMn9MIiIibaen6QKKY+zYsTh06BBOnz6NDz74QGq3tbVFVlYWUlJS1I6WPHjwALa2tlKfCxcuqK0v/+yc/D6vUiqVUCqVpbwVRERE9DqyPlIihMDYsWOxb98+nDx5Eo6OjmrzmzRpAn19fQQGBkptUVFRuHPnDjw8PAAAHh4euHLlCpKSkqQ+AQEBUKlUcHV1LZ8NISIiojeS9ZGSMWPGYMeOHThw4ADMzMykMSDm5uYwMjKCubk5hg0bhkmTJqFChQpQqVT44osv4OHhgWbNmgEAOnXqBFdXV3z66adYvHgxEhMTMX36dIwZM4ZHQ4iIiGRE1qFk7dq1AIC2bduqtW/atAlDhgwBAHz77bfQ0dFB3759kZmZCS8vL3z33XdSX11dXRw6dAijR4+Gh4cHTExM4OPjg9mzZ5fXZhAREVExyDqUFOcSKoaGhlizZg3WrFlTZJ9q1arh999/L83SiIiIqJTJekwJERERaQ+GEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBa0KJWvWrIGDgwMMDQ3RtGlTXLhwQdMlERER0f+nNaFk165dmDRpEmbMmIGIiAg0aNAAXl5eSEpK0nRpREREBC0KJcuWLcOIESMwdOhQuLq6Yt26dTA2NsbGjRs1XRoREREB0NN0AeUhKysL4eHh8PX1ldp0dHTg6emJkJCQAv0zMzORmZkpTaempgIA0tLSyr5YGcjLfKbpEqgUacvrVlvw/fl+0Yb3Z/42CiHe2FcrQsk///yD3Nxc2NjYqLXb2Njg5s2bBfovWLAAs2bNKtBepUqVMquRqKyYL9d0BURUFG16fz558gTm5uav7aMVoaSkfH19MWnSJGk6Ly8PycnJqFixIhQKhQYro9KSlpaGKlWqICEhASqVStPlENFL+P58vwgh8OTJE9jb27+xr1aEkkqVKkFXVxcPHjxQa3/w4AFsbW0L9FcqlVAqlWptFhYWZVkiaYhKpeKHHpFM8f35/njTEZJ8WjHQ1cDAAE2aNEFgYKDUlpeXh8DAQHh4eGiwMiIiIsqnFUdKAGDSpEnw8fGBm5sbPvzwQyxfvhxPnz7F0KFDNV0aERERQYtCyccff4yHDx/C398fiYmJaNiwIY4ePVpg8CtpB6VSiRkzZhT4mY6INI/vT+2lEMU5R4eIiIiojGnFmBIiIiKSP4YSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBa25eBppp4MHDxa770cffVSGlRDRq/r06VPsvnv37i3DSkguGErovdarVy+1aYVCgZevF/jyXZ9zc3PLqywigvpN2oQQ2LdvH8zNzeHm5gYACA8PR0pKSonCC73b+PMNvdfy8vKkv+PHj6Nhw4Y4cuQIUlJSkJKSgt9//x2NGzfG0aNHNV0qkdbZtGmT9GdjY4P+/fsjLi4Oe/fuxd69e/HXX39hwIABqFSpkqZLpXLCy8yT1qhbty7WrVuHli1bqrWfOXMGI0eOxI0bNzRUGRFZWVnh7NmzcHZ2VmuPiopC8+bN8ejRIw1VRuWJR0pIa8TGxsLCwqJAu7m5OW7fvl3u9RDR/8nJycHNmzcLtN+8eRN5eXkaqIg0gWNKSGu4u7tj0qRJ2LZtm3R36AcPHmDKlCn48MMPNVwdkXYbOnQohg0bhtjYWOn9GBoaioULF2Lo0KEaro7KC3++Ia0RExOD3r17Izo6GlWqVAEAJCQkoGbNmti/fz9q1Kih4QqJtFdeXh6WLFmCFStW4P79+wAAOzs7jB8/Hl9++SV0dXU1XCGVB4YS0ipCCAQEBEiHiV1cXODp6al2Fg4RaVZaWhoAQKVSabgSKm8MJaSVMjIyoFQqGUaIZCQnJwfBwcGIjY3FwIEDYWZmhnv37kGlUsHU1FTT5VE54EBX0hp5eXmYM2cOKleuDFNTU8TFxQEA/Pz88OOPP2q4OiLtFh8fj3r16qFnz54YM2YMHj58CABYtGgRJk+erOHqqLwwlJDWmDt3LjZv3ozFixfDwMBAaq9bty5++OEHDVZGROPHj4ebmxseP34MIyMjqb13794IDAzUYGVUnhhKSGts3boVGzZsgLe3t9qguQYNGhR6KiIRlZ8zZ85g+vTpav9hAAAHBwfcvXtXQ1VReWMoIa1x9+7dQs+wycvLQ3Z2tgYqIqJ8eXl5hd7q4e+//4aZmZkGKiJNYCghreHq6oozZ84UaN+9ezcaNWqkgYqIKF+nTp2wfPlyaVqhUCA9PR0zZsxA165dNVcYlStePI20hr+/P3x8fHD37l3k5eVh7969iIqKwtatW3Ho0CFNl0ek1ZYuXQovLy+4uroiIyMDAwcOxK1bt1CpUiX8/PPPmi6PyglPCSatcubMGcyePRuRkZFIT09H48aN4e/vj06dOmm6NCKtl5OTg507d+Ly5cvS+9Pb21tt4Cu93xhKiIhI4zIyMmBoaKjpMkjDOKaEtMbw4cMRHBys6TKIqBDW1tbw8fFBQEAAb8CnxRhKSGs8fPgQnTt3RpUqVTBlyhRcunRJ0yUR0f+3ZcsWPHv2DD179kTlypUxYcIEXLx4UdNlUTnjzzekVR4/foxff/0VO3bswJkzZ1C7dm14e3tj4MCBcHBw0HR5RFrvyZMn2L17N37++WecPHkS1atXx6BBg+Dv76/p0qgcMJSQ1vr777/x888/Y+PGjbh16xZycnI0XRIRveT69evw9vbG5cuXC72GCb1/+PMNaaXs7GxcvHgRoaGhuH37NmxsbDRdEhHhxYDXX375Bb169ULjxo2RnJyMKVOmaLosKie8TglplaCgIOzYsQN79uxBXl4e+vTpg0OHDqF9+/aaLo1Iqx07dgw7duzA/v37oaenh379+uH48eNo3bq1pkujcsSfb0hrVK5cGcnJyejcuTO8vb3Ro0cPKJVKTZdFRACMjY3RvXt3eHt7o2vXrtDX19d0SaQBDCWkNb7//nv85z//gYWFhaZLIaJXPHnyhPe4IYYSIiLSjLS0tGL3ValUZVgJyQVDCb3X+vTpU+y+e/fuLcNKiOhVOjo6UCgUr+0jhIBCoeDZN1qCA13pvWZubq7pEoioCEFBQZougWSGR0qIiIhIFnikhLTOw4cPERUVBQBwdnaGlZWVhisiIgBISUnBjz/+iBs3bgAA6tSpg88++4xHPLUIj5SQ1nj69Cm++OILbN26Vbrhl66uLgYPHoxVq1bB2NhYwxUSaa+LFy/Cy8sLRkZG+PDDDwEAYWFheP78OY4fP47GjRtruEIqDwwlpDVGjRqFEydOYPXq1WjRogUA4OzZsxg3bhw6duyItWvXarhCIu3VqlUr1KhRA99//z309F4cxM/JycHw4cPx119/4fTp0xqukMoDQwlpjUqVKmH37t1o27atWntQUBD69++Phw8faqYwIoKRkRH+/PNP1K5dW639+vXrcHNzw7NnzzRUGZUn3vuGtMazZ88KvceNtbU1P/CINEylUuHOnTsF2hMSEnhRNS3CUEJaw8PDAzNmzEBGRobU9vz5c8yaNQseHh4arIyIPv74YwwbNgy7du1CQkICEhISsHPnTgwfPhyffPKJpsujcsKfb0hrXL16FV5eXsjMzESDBg0AAJGRkTA0NMSxY8dQp04dDVdIpL2ysrIwZcoUrFu3Djk5OQAAfX19jB49GgsXLuR9qrQEQwlplWfPnmH79u24efMmAMDFxQXe3t4wMjLScGVEBLx4j8bGxgIAnJyceFaclmEoISIi2UlLS8PJkyfh7OwMFxcXTZdD5YRjSkhrbNmyBYcPH5amp06dCgsLCzRv3hzx8fEarIyI+vfvj9WrVwN4MdbLzc0N/fv3R/369bFnzx4NV0flhaGEtMb8+fOln2lCQkKwevVqLF68GJUqVcLEiRM1XB2Rdjt9+jRatWoFANi3bx+EEEhJScHKlSsxd+5cDVdH5YWhhLRGQkICatSoAQDYv38/+vXrh5EjR2LBggU4c+aMhqsj0m6pqamoUKECAODo0aPo27cvjI2N0a1bN9y6dUvD1VF5YSghrWFqaopHjx4BAI4fP46OHTsCAAwNDfH8+XNNlkak9apUqYKQkBA8ffoUR48eRadOnQAAjx8/hqGhoYaro/LCG/KR1ujYsSOGDx+ORo0aITo6Gl27dgUAXLt2DQ4ODpotjkjLTZgwAd7e3jA1NUXVqlWlKy+fPn0a9erV02xxVG54pIS0xpo1a+Dh4YGHDx9iz549qFixIgAgPDycF2ci0rDPP/8cISEh2LhxI86dOwcdnRdfT9WrV+eYEi3CU4KJiEg2srKyEBcXBycnJ+nGfKQ9eKSEtMqZM2cwaNAgNG/eHHfv3gUAbNu2DWfPntVwZUTa7dmzZxg2bBiMjY1Rp04d6T44X3zxBRYuXKjh6qi8MJSQ1tizZw+8vLxgZGSEiIgIZGZmAngx6n/+/Pkaro5Iu/n6+iIyMhLBwcFqA1s9PT2xa9cuDVZG5YmhhLTG3LlzsW7dOnz//ffQ19eX2lu0aIGIiAgNVkZE+/fvx+rVq9GyZUsoFAqpvU6dOtJl5+n9x1BCWiMqKgqtW7cu0G5ubo6UlJTyL4iIJA8fPoS1tXWB9qdPn6qFFHq/MZSQ1rC1tUVMTEyB9rNnz6J69eoaqIiI8rm5uandBiI/iPzwww/w8PDQVFlUzji0mbTGiBEjMH78eGzcuBEKhQL37t1DSEgIvvzyS/j7+2u6PCKtNn/+fHTp0gXXr19HTk4OVqxYgevXr+P8+fM4deqUpsujcsJTgklrCCEwf/58LFiwAM+ePQMAKJVKTJkyBb6+vtJ9cYhIM/766y8sWLAAkZGRSE9PR+PGjfHVV1/x4mlahKGEtE5WVhZiYmKQnp4OV1dXrF+/Ht988w0SExM1XRqRVsrOzsaoUaPg5+cHR0dHTZdDGsQxJfTey8zMhK+vL9zc3NCiRQv8/vvvcHV1xbVr1+Ds7IwVK1bwLsFEGqSvr489e/ZougySAR4poffeV199hfXr18PT0xPnz5/Hw4cPMXToUPzxxx/43//+h//85z/Q1dXVdJlEWs3HxwcNGzbkfxC0HAe60nvv119/xdatW/HRRx/h6tWrqF+/PnJychAZGclTDYlkombNmpg9ezbOnTuHJk2awMTERG3+uHHjNFQZlSceKaH3noGBAeLi4lC5cmUAgJGRES5cuMDBc0Qy8rqxJAqFAn/99Vc5VkOawiMl9N7Lzc2FgYGBNK2npwdTU1MNVkREr4qLi9N0CSQDDCX03hNCYMiQIVAqlQCAjIwM/Pe//y1weHjv3r2aKI+IiP4/hhJ67/n4+KhNDxo0SEOVEFFRcnNzsXnzZgQGBiIpKQl5eXlq80+ePKmhyqg8cUwJERFp3NixY7F582Z069YNdnZ2BQahf/vttxqqjMoTQwkREWlcpUqVsHXrVnTt2lXTpZAG8eJpRESkcQYGBqhRo4amyyANYyghIiKN+/LLL7FixQrw4L124883RESkcb1790ZQUBAqVKiAOnXqQF9fX20+z47TDjz7hoiINM7CwgK9e/fWdBmkYTxSQkRERLLAIyVERKQxlpaWhd6DytzcHLVq1cLkyZPRsWNHDVRGmsAjJUREpDFbtmwptD0lJQXh4eHYtWsXdu/ejR49epRzZaQJDCVERCRby5Ytw+7du3H+/HlNl0LlgKGEiIhkKzo6Gs2aNUNycrKmS6FywOuUEBGRbGVmZqrd5ZvebwwlREQkWz/++CMaNmyo6TKonPDsGyIi0phJkyYV2p6amoqIiAhER0fj9OnT5VwVaQpDCRERacyff/5ZaLtKpULHjh2xd+9eODo6lnNVpCkc6EpERESywDElREREJAsMJURERCQLDCVEREQkCwwlREREJAsMJVQsCoUC+/fvL/X1BgcHQ6FQICUlpdTXrSmbN2+GhYWFpsuQLQcHByxfvrzI+UOGDEGvXr3KrZ53QXm9T7jvi6e89lNZfe7KGUOJFlm3bh3MzMyQk5MjtaWnp0NfXx9t27ZV65v/IRgbG1tqj9+2bVtMmDCh1NYnVx9//DGio6M1XUaZK6sPzBUrVmDz5s2lvl4ikj+GEi3Srl07pKen4+LFi1LbmTNnYGtri9DQUGRkZEjtQUFBqFq1KpycnDRR6r+WnZ2tscc2MjKCtbV1kfOzsrLKsZp3j7m5+Tt1pInP5+uV5/4RQqj9p4vePQwlWsTZ2Rl2dnYIDg6W2oKDg9GzZ084Ojrijz/+UGtv166d2vL//PMPevfuDWNjY9SsWRMHDx5Um3/16lV06dIFpqamsLGxwaeffop//vkHwIvDnadOncKKFSugUCigUChw+/Ztadnw8HC4ubnB2NgYzZs3R1RUlNq6Dxw4gMaNG8PQ0BDVq1fHrFmz1D58FAoF1q5di48++ggmJiaYN29eofsgMzMTkydPRuXKlWFiYoKmTZuq7Y/8n16OHTsGFxcXmJqaonPnzrh//z4A4Pjx4zA0NCxwGH38+PFo37692jryzZw5Ew0bNsQPP/wAR0dHGBoaAgDu3LmDnj17wtTUFCqVCv3798eDBw8KLLdt2zY4ODjA3NwcAwYMwJMnT6Q+bdu2xRdffIEJEybA0tISNjY2+P777/H06VMMHToUZmZmqFGjBo4cOVLs5yp/vePGjcPUqVNRoUIF2NraYubMmdJ8BwcHAEDv3r2hUCik6djYWPTs2RM2NjYwNTWFu7s7Tpw4UehzUZRXD43v3r0b9erVg5GRESpWrAhPT088ffoUwIvX6YcffggTExNYWFigRYsWiI+PL3Q9ADBhwgS1o4J5eXlYsGABHB0dYWRkhAYNGmD37t2vrc/BwQFz5szB4MGDoVKpMHLkSADA2bNn0apVKxgZGaFKlSoYN26cVCcAfPfdd6hZsyYMDQ1hY2ODfv36SfMyMzMxbtw4WFtbw9DQEC1btkRYWFihj5+WlgYjI6MCz+m+fftgZmaGZ8+eAQASEhLQv39/WFhYoEKFCujZs6faey43NxeTJk2ChYUFKlasiKlTp6I4l63as2cP6tSpA6VSCQcHByxdurRY+6ew/fjqz3gNGzZUe50pFAr88MMPRX7u5B/RPXLkCJo0aQKlUomzZ8++8Xl9/PgxvL29YWVlBSMjI9SsWRObNm2S5l+5cgXt27eXXnMjR45Eenp6oduxYcMG2NvbIy8vT629Z8+e+Oyzz6TpN32G3bp1C61bt4ahoSFcXV0REBBQ6OO99wRplYEDB4pOnTpJ0+7u7uLXX38V//3vf4W/v78QQohnz54JpVIpNm/eLPUDID744AOxY8cOcevWLTFu3DhhamoqHj16JIQQ4vHjx8LKykr4+vqKGzduiIiICNGxY0fRrl07IYQQKSkpwsPDQ4wYMULcv39f3L9/X+Tk5IigoCABQDRt2lQEBweLa9euiVatWonmzZtLj3369GmhUqnE5s2bRWxsrDh+/LhwcHAQM2fOVKvP2tpabNy4UcTGxor4+PhCt3/48OGiefPm4vTp0yImJkZ88803QqlUiujoaCGEEJs2bRL6+vrC09NThIWFifDwcOHi4iIGDhwohBAiJydH2NjYiB9++EFa56ttmzZtEubm5tL8GTNmCBMTE9G5c2cREREhIiMjRW5urmjYsKFo2bKluHjxovjjjz9EkyZNRJs2bdSWMzU1FX369BFXrlwRp0+fFra2tuJ///uf1KdNmzbCzMxMzJkzR0RHR4s5c+YIXV1d0aVLF7FhwwYRHR0tRo8eLSpWrCiePn1arOcqf70qlUrMnDlTREdHiy1btgiFQiGOHz8uhBAiKSlJABCbNm0S9+/fF0lJSUIIIS5duiTWrVsnrly5IqKjo8X06dOFoaGh2vNRrVo18e233xb6/AghhI+Pj+jZs6cQQoh79+4JPT09sWzZMhEXFycuX74s1qxZI548eSKys7OFubm5mDx5soiJiRHXr18Xmzdvlh7r5fXkGz9+vNo+njt3rqhdu7Y4evSoiI2NFZs2bRJKpVIEBwcXWV+1atWESqUSS5YsETExMdKfiYmJ+Pbbb0V0dLQ4d+6caNSokRgyZIgQQoiwsDChq6srduzYIW7fvi0iIiLEihUrpHWOGzdO2Nvbi99//11cu3ZN+Pj4CEtLS+n9lf8+efz4sRBCiH79+olBgwap1dW3b1+pLSsrS7i4uIjPPvtMXL58WVy/fl0MHDhQODs7i8zMTCGEEIsWLRKWlpZiz5494vr162LYsGHCzMyswD572cWLF4WOjo6YPXu2iIqKEps2bRJGRkZi06ZNr90/Re3HV18HDRo0EDNmzJCm3/S5k79f6tevL44fPy5iYmLEo0eP3vi8jhkzRjRs2FCEhYWJuLg4ERAQIA4ePCiEECI9PV3Y2dlJ77vAwEDh6OgofHx8pLpefm0lJycLAwMDceLECWn+o0eP1Nre9BmWm5sr6tatKzp06CAuXbokTp06JRo1aiQAiH379hX5fLyPGEq0zPfffy9MTExEdna2SEtLE3p6eiIpKUns2LFDtG7dWgghRGBgoACg9kUCQEyfPl2aTk9PFwDEkSNHhBBCzJkzRy3sCCFEQkKCACCioqKEEC++6MaPH6/WJ/9D5eU39OHDhwUA8fz5cyGEEB06dBDz589XW27btm3Czs5Orb4JEya8dtvj4+OFrq6uuHv3rlp7hw4dhK+vrxDiRaAAoPZBumbNGmFjYyNNjx8/XrRv316aPnbsmFAqldIXRmGhRF9fX/riFkKI48ePC11dXXHnzh2p7dq1awKAuHDhgrScsbGxSEtLk/pMmTJFNG3aVJpu06aNaNmypTSdk5MjTExMxKeffiq13b9/XwAQISEhQojiP1cvr1eIFwH2q6++kqaL+4FZp04dsWrVKmm6JKEkPDxcABC3b98u0O/Ro0cCQJEB4k2hJCMjQxgbG4vz58+r9Rk2bJj45JNPiqyvWrVqolevXgWWGTlypFrbmTNnhI6Ojnj+/LnYs2ePUKlUas9lvvT0dKGvry+2b98utWVlZQl7e3uxePFiIUTBULJv3z5hamoqBc3U1FRhaGgovR+3bdsmnJ2dRV5enrTOzMxMYWRkJI4dOyaEEMLOzk5avxBCZGdniw8++OC1oWTgwIGiY8eOam1TpkwRrq6ur90/hSluKHnd507+ftm/f7/UpzjPa48ePcTQoUMLrWvDhg3C0tJSpKenS22HDx8WOjo6IjExUQhR8LXVs2dP8dlnn0nT69evF/b29iI3N1cI8ebPsGPHjgk9PT21z6YjR45oZSjhvW+0TNu2bfH06VOEhYXh8ePHqFWrFqysrNCmTRsMHToUGRkZCA4ORvXq1VG1alW1ZevXry/928TEBCqVCklJSQCAyMhIBAUFwdTUtMBjxsbGolatWq+t6+V129nZAQCSkpJQtWpVREZG4ty5c2o/yeTm5iIjIwPPnj2DsbExAMDNze21j3HlyhXk5uYWqCUzMxMVK1aUpo2NjdXG0tjZ2UnbCQDe3t5o1qwZ7t27B3t7e2zfvh3dunV77TiIatWqwcrKSpq+ceMGqlSpgipVqkhtrq6usLCwwI0bN+Du7g7gxSFuMzOzImsB1Pedrq4uKlasiHr16kltNjY2AFDi5+rl9Rb12K9KT0/HzJkzcfjwYdy/fx85OTl4/vw57ty589rlitKgQQN06NAB9erVg5eXFzp16oR+/frB0tISFSpUwJAhQ+Dl5YWOHTvC09MT/fv3l14/bxITE4Nnz56hY8eOau1ZWVlo1KjRa5d99bUWGRmJy5cvY/v27VKbEAJ5eXmIi4tDx44dUa1aNVSvXh2dO3dG586dpZ8kYmNjkZ2djRYtWkjL6uvr48MPP8SNGzcKffyuXbtCX18fBw8exIABA7Bnzx6oVCp4enpK9cTExKi9dgAgIyMDsbGxSE1Nxf3799G0aVNpnp6eHtzc3F77E86NGzfQs2dPtbYWLVpg+fLlyM3Nha6ubqH759943edOvpcfrzjP6+jRo9G3b19ERESgU6dO6NWrF5o3by5tY4MGDWBiYqK2jXl5eYiKipLeTy/z9vbGiBEj8N1330GpVGL79u0YMGAAdHRejJB402dY/ueBvb29NN/Dw6PE++p9wFCiZWrUqIEPPvgAQUFBePz4Mdq0aQMAsLe3R5UqVXD+/HkEBQVJ4yNepq+vrzatUCik31HT09PRo0cPLFq0qMByxfmSeHndCoUCANTWPWvWLPTp06fAcvnjMwCofYgUJj09Hbq6uggPD5c+PPO9/AVd2Ha+/EHt7u4OJycn7Ny5E6NHj8a+ffveeLbIm2oryuv2+ev6vGl/Fue5Ks5jv2ry5MkICAjAkiVLUKNGDRgZGaFfv35vPdhRV1cXAQEBOH/+PI4fP45Vq1bh66+/RmhoKBwdHbFp0yaMGzcOR48exa5duzB9+nQEBASgWbNm0NHRKfAF+/IA6PwxAocPH0blypXV+imVytfW9erzmZ6ejlGjRmHcuHEF+latWhUGBgaIiIhAcHAwjh8/Dn9/f8ycObPIcSNvYmBggH79+mHHjh0YMGAAduzYgY8//hh6enpSPU2aNFELSfleDsdlpTiv9zc9P/mK8zp8+fGK87x26dIF8fHx+P333xEQEIAOHTpgzJgxWLJkyRvrLkyPHj0ghMDhw4fh7u6OM2fO4Ntvv1WrqTifYcRQopXatWuH4OBgPH78GFOmTJHaW7dujSNHjuDChQsYPXp0idbZuHFj7NmzBw4ODtIH46sMDAyQm5tb4nobN26MqKgo1KhRo8TLvqxRo0bIzc1FUlISWrVq9a/W5e3tje3bt+ODDz6Ajo4OunXrVqLlXVxckJCQgISEBOloyfXr15GSkgJXV9d/VdubFOe5Kg59ff0Cz+e5c+cwZMgQ9O7dG8CLD+OXB1e+DYVCgRYtWqBFixbw9/dHtWrVsG/fPumW940aNUKjRo3g6+sLDw8P7NixA82aNYOVlRWuXr2qtq5Lly5JX3Kurq5QKpW4c+eOFM7fVuPGjXH9+vXXvkb19PTg6ekJT09PzJgxAxYWFjh58iS8vLxgYGCAc+fOoVq1agBefDmHhYW99hR6b29vdOzYEdeuXcPJkycxd+5ctXp27doFa2trqFSqQpe3s7NDaGgoWrduDQDIyclBeHg4GjduXORjuri44Ny5c2pt586dQ61atQoE/TexsrKSBpADLwbwxsXFlWgdhSnu82plZQUfHx/4+PigVatWmDJlCpYsWQIXFxds3rwZT58+lcLOuXPnoKOjA2dn50LXZWhoiD59+mD79u2IiYmBs7Oz2n5802dY/ufB/fv3pf8YvHzigTbh2TdaqF27djh79iwuXbqk9qZt06YN1q9fj6ysrAJn3rzJmDFjkJycjE8++QRhYWGIjY3FsWPHMHToUOmLy8HBAaGhobh9+zb++eefN/6vO5+/vz+2bt2KWbNm4dq1a7hx4wZ27tyJ6dOnl6jGWrVqwdvbG4MHD8bevXsRFxeHCxcuYMGCBTh8+HCJ1uXt7Y2IiAjMmzcP/fr1e+P/rF/l6emJevXqSeu5cOECBg8ejDZt2pTqoe/CFOe5Kg4HBwcEBgYiMTERjx8/BgDUrFkTe/fuxaVLlxAZGYmBAwcW+3kuTGhoKObPn4+LFy/izp072Lt3Lx4+fAgXFxfExcXB19cXISEhiI+Px/Hjx3Hr1i24uLgAANq3b4+LFy9i69atuHXrFmbMmKEWUszMzDB58mRMnDgRW7ZsQWxsLCIiIrBq1Sps2bKlRHV+9dVXOH/+PMaOHYtLly7h1q1bOHDgAMaOHQsAOHToEFauXIlLly4hPj4eW7duRV5eHpydnWFiYoLRo0djypQpOHr0KK5fv44RI0bg2bNnGDZsWJGP2bp1a9ja2sLb2xuOjo5qP8V4e3ujUqVK6NmzJ86cOYO4uDgEBwdj3Lhx+PvvvwG8OGNs4cKF2L9/P27evInPP//8jRdn+/LLLxEYGIg5c+YgOjoaW7ZswerVqzF58uQS7S/gxfOzbds2nDlzBleuXIGPj0+Jg01hivO8+vv748CBA4iJicG1a9dw6NAh6XXj7e0NQ0ND+Pj44OrVqwgKCsIXX3yBTz/9tNCfbvJ5e3vj8OHD2LhxI7y9vdXmvekzzNPTE7Vq1YKPjw8iIyNx5swZfP311/96X7yLGEq0ULt27fD8+XPUqFFD7U3Wpk0bPHnyRDp1uCTs7e1x7tw55ObmolOnTqhXrx4mTJgACwsL6XfVyZMnQ1dXF66urrCysir2OAMvLy8cOnQIx48fh7u7O5o1a4Zvv/1W+l9lSWzatAmDBw/Gl19+CWdnZ/Tq1QthYWEFxs+8SY0aNfDhhx/i8uXLBT6AikOhUODAgQOwtLRE69at4enpierVq2PXrl0lXldJFee5Ko6lS5ciICAAVapUkX6rX7ZsGSwtLdG8eXP06NEDXl5er/2f95uoVCqcPn0aXbt2Ra1atTB9+nQsXboUXbp0gbGxMW7evIm+ffuiVq1aGDlyJMaMGYNRo0YBePG68fPzw9SpU+Hu7o4nT55g8ODBauufM2cO/Pz8sGDBAri4uKBz5844fPgwHB0dS1Rn/fr1cerUKURHR6NVq1Zo1KgR/P39pTECFhYW2Lt3L9q3bw8XFxesW7cOP//8M+rUqQMAWLhwIfr27YtPP/0UjRs3RkxMDI4dOwZLS8siH1OhUOCTTz5BZGRkgdegsbExTp8+japVq6JPnz5wcXHBsGHDkJGRIR05+fLLL/Hpp5/Cx8cHHh4eMDMzk45wFaVx48b45ZdfsHPnTtStWxf+/v6YPXs2hgwZUqL9BQC+vr5o06YNunfvjm7duqFXr16ldl2kNz2vBgYG8PX1Rf369dG6dWvo6upi586dAF7su2PHjiE5ORnu7u7o168fOnTogNWrV7/2Mdu3b48KFSogKioKAwcOVJv3ps8wHR0d7Nu3D8+fP8eHH36I4cOHF3lZg/edQrxuVBMRERFROeGREiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKSBYYSIiIikgWGEiIiIpIFhhIiIiKShf8HmvBx8R/p/XQAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(6, 4))\n",
    "ax = train['status_code'].value_counts().plot(kind = 'bar')\n",
    "ax.set_title(\"Training Dataset Frequency Count\")\n",
    "ax.set_xlabel(\"Whether environmental issue resolved or unresolved\")\n",
    "ax.set_ylabel(\"Count\")\n",
    "ax.set_xticklabels(('Resolved', 'Unresolved'))\n",
    "\n",
    "for bars in ax.containers:\n",
    "    ax.bar_label(bars)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhwAAAHJCAYAAADO5eB9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAABUpklEQVR4nO3dd1gUV9sG8HtpS28WiqJgQ+zdgF1R7Bo1hrhGLFFfGxoTNSSKXaOJJRojpoglEo3dV6Oi2JVYwF5ACCoWwIiAiCCw5/vDl/lYAQVlWFzv33XtpXtmdubZ2dndmzNnZhVCCAEiIiIiGelpuwAiIiLSfQwcREREJDsGDiIiIpIdAwcRERHJjoGDiIiIZMfAQURERLJj4CAiIiLZMXAQERGR7Bg4iIiISHYMHPROuHXrFhQKBdasWSO1zZgxAwqFolCPVygUmDFjRrHW1LZtW7Rt27ZYl0lEpKsYOKjY9ezZE6ampnjy5EmB86hUKhgZGeHRo0clWFnRXbt2DTNmzMCtW7e0XYrkyJEjUCgU0k2pVMLOzg5t27bFvHnz8PDhwzdedml7vkFBQVi6dGmh53d2dtbYNrlv6enp8hX6nsrOzkZgYCDatm0LW1tbKJVKODs7Y8iQITh37py2ywNQ+vbp9xkDBxU7lUqFZ8+eYfv27flOT0tLw86dO9G5c2eUKVPmjdczdepUPHv27I0fXxjXrl3DzJkz8/2wCg4ORnBwsKzrfxVfX1+sX78eP//8MyZNmgRbW1tMnz4dbm5uOHTo0Bst81XPVxuKGjgAoEGDBli/fn2em5GRkTxFvqeePXuG7t27Y+jQoRBC4Ouvv8bKlSsxaNAghIaGolmzZrh79662yyx1+/T7zEDbBZDu6dmzJywsLBAUFIRBgwblmb5z5048ffoUKpXqrdZjYGAAAwPt7cLa/gJr1aoV+vXrp9F28eJFdOrUCX379sW1a9fg4OCgpeq0p0KFChg4cGCh509LS4OpqamMFemmSZMmYd++fViyZAkmTJigMW369OlYsmSJdgqj0ksQycDHx0cYGBiI+Pj4PNO6d+8uLCwsRFpamnj06JH44osvRJ06dYSZmZmwsLAQnTt3FhcuXNB4TExMjAAgAgMDpbbp06eLl3fh9PR0MWHCBFG2bFlhbm4uevToIWJjYwUAMX36dGm+W7duiVGjRokaNWoIY2NjYWtrK/r16ydiYmKkeQIDAwWAPLfDhw8LIYRo06aNaNOmjcb64+PjxdChQ0X58uWFUqkU9erVE2vWrMn3uXz33Xdi1apVokqVKsLIyEg0adJEnDlz5rXb9vDhwwKA2Lx5c77Tg4KCBADx9ddfF+vz3bFjh+jatatwcHAQRkZGokqVKmLWrFkiKytLY/2RkZGiT58+ws7OTiiVSlGhQgXx8ccfi6SkJI351q9fLxo1aiSMjY2FjY2N+Pjjj8WdO3ek6W3atMlTS+XKlV+5bSpXriy6detW4PQ2bdqI2rVri3PnzolWrVoJExMTMX78eCHEi33H399fVK1aVRgZGYmKFSuKSZMmifT0dI1lFHYf8/Hxybfe/PbbwmyP3PVfvXpVtG3bVpiYmAhHR0exYMGCPMt79uyZmD59uqhevbpQKpXC3t5efPjhhyIqKkqo1WpRuXJl0bNnz3wfZ2lpKUaMGFHgdoyNjRUGBgaiY8eOBc7zsvDwcNG5c2dhYWEhzMzMRPv27UVoaKjGPAVtm5x9M/f+mvNaHz9+XDRt2lQolUrh4uIi1q5dm+dxBe3TVLLYw0GyUKlUWLt2Lf7880+MHTtWak9MTMT+/fvxySefwMTEBFevXsWOHTvw0UcfwcXFBfHx8Vi1ahXatGmDa9euwdHRsUjr/eyzz/D7779jwIAB8PDwwKFDh9CtW7c88509exanTp2Ct7c3KlasiFu3bmHlypVo27Ytrl27BlNTU7Ru3Rq+vr5YtmwZvv76a7i5uQGA9O/Lnj17hrZt2yIqKgpjx46Fi4sLNm/ejMGDByMpKQnjx4/XmD8oKAhPnjzByJEjoVAosHDhQvTp0wf//PMPDA0Ni/S8c+vXrx+GDRuG4OBgzJ07t9ie75o1a2Bubo6JEyfC3Nwchw4dgr+/P1JSUvDdd98BAJ4/fw4vLy9kZGRg3LhxsLe3x71797B7924kJSXBysoKADB37lxMmzYN/fv3x2effYaHDx9i+fLlaN26Nc6fPw9ra2t88803SE5Oxt27d6W/ls3NzV/7/DMzM/Hvv/9qtJmamkq9GI8ePUKXLl3g7e2NgQMHws7ODmq1Gj179sSJEycwYsQIuLm54fLly1iyZAkiIyOxY8cOaVmF3ceKojDbI8fjx4/RuXNn9OnTB/3798eWLVswZcoU1K1bF126dAHwYmxF9+7dERISAm9vb4wfPx5PnjzBgQMHcOXKFVStWhUDBw7EwoULkZiYCFtbW2n5//3vf5GSkvLKXqK9e/ciKysLn376aaGe39WrV9GqVStYWlpi8uTJMDQ0xKpVq9C2bVscPXoUzZs3f6PtFhUVJe3vPj4+WL16NQYPHozGjRujdu3aRX4Pk8y0nXhIN2VlZQkHBwfh7u6u0R4QECAAiP379wshXvy1mJ2drTFPTEyMUCqVYtasWRpteE0Px4ULFwQAMXr0aI3lDRgwIM9fn2lpaXlqDg0NFQDEunXrpLbNmzcX+BfRyz0cS5cuFQDE77//LrU9f/5cuLu7C3Nzc5GSkqLxXMqUKSMSExOleXfu3CkAiP/+97951pXb63o4hBCifv36wsbGplifb37LGDlypDA1NZV6Ac6fP//a2m7duiX09fXF3LlzNdovX74sDAwMNNq7dev22l6N3CpXrpzvX7Q5r31Or0lAQIDG49avXy/09PTE8ePHNdpz9teTJ08KIYq2jxW2h6Mo2yOn/tyvWUZGhrC3txd9+/aV2lavXi0AiMWLF+dZv1qtFkIIERERIQCIlStXakzv2bOncHZ2lubLz+effy4AiPPnzxc4T269e/cWRkZGIjo6Wmq7f/++sLCwEK1bt5baitrDAUAcO3ZMaktISBBKpVJ88cUXUtur9mkqWRw0SrLQ19eHt7c3QkNDNQZrBQUFwc7ODh06dAAAKJVK6Om92A2zs7Px6NEjmJubw9XVFeHh4UVa519//QXgxWDK3F4+vgwAJiYm0v8zMzPx6NEjVKtWDdbW1kVeb+7129vb45NPPpHaDA0N4evri9TUVBw9elRj/o8//hg2NjbS/VatWgEA/vnnnzdaf27m5uYaZwkVx/PNvYwnT57g33//RatWrZCWloYbN24AgNSDsX//fqSlpeW7nG3btkGtVqN///74999/pZu9vT2qV6+Ow4cPF/n55ta8eXMcOHBA45Z7LJFSqcSQIUM0HrN582a4ubmhZs2aGjW1b98eAKSairKPFVZRt4e5ublG74ORkRGaNWumsd9s3boVZcuWxbhx4/KsL+dU8ho1aqB58+bYsGGDNC0xMRF79+6FSqV65SnnKSkpAAALC4vXPr/s7GwEBwejd+/eqFKlitTu4OCAAQMG4MSJE9LyiqpWrVrS+wYAypUrB1dX12J5D1Hx4yEVko1KpcKSJUsQFBSEr7/+Gnfv3sXx48fh6+sLfX19AIBarcYPP/yAn376CTExMcjOzpYeX9QzWG7fvg09PT1UrVpVo93V1TXPvM+ePcP8+fMRGBiIe/fuQQghTUtOTi7SenOvv3r16lKAypHTfXv79m2N9kqVKmnczwkfjx8/fqP155aamqrxZVAcz/fq1auYOnUqDh06lOcLImcZLi4umDhxIhYvXowNGzagVatW6NmzJwYOHCiFkZs3b0IIgerVq+e7nrc5nAQAZcuWhaenZ4HTK1SokGfA782bN3H9+nWUK1cu38ckJCQAKNo+VlhF3R4VK1bMEwZsbGxw6dIl6X50dDRcXV1fO6h60KBBGDt2LG7fvo3KlStj8+bNyMzMfO2hEktLSwB45anvOR4+fIi0tLR8t5GbmxvUajViY2NRu3bt1y7rZS+/h4AX26I43kNU/Bg4SDaNGzdGzZo18ccff+Drr7/GH3/8ASGExtkp8+bNw7Rp0zB06FDMnj0btra20NPTw4QJE6BWq2Wrbdy4cQgMDMSECRPg7u4OKysrKBQKeHt7y7re3HJC18tyh4E3kZmZicjISNSpU0dqe9vnm5SUhDZt2sDS0hKzZs1C1apVYWxsjPDwcEyZMkVjGYsWLcLgwYOxc+dOBAcHw9fXF/Pnz8fff/+NihUrQq1WQ6FQYO/evflug8KM03gbuXtqcqjVatStWxeLFy/O9zFOTk5FXk9BPQS5Q3XOuouyPYpzv/H29sbnn3+ODRs24Ouvv8bvv/+OJk2avDZA1axZEwBw+fJlNGjQoMjrLUhht1kOud5DJA8GDpKVSqXCtGnTcOnSJQQFBaF69epo2rSpNH3Lli1o164dfvvtN43HJSUloWzZskVaV+XKlaFWq6W/7nJERETkmXfLli3w8fHBokWLpLb09HQkJSVpzFfYK5nmrP/SpUtQq9UavRw5hxsqV65c6GW9jS1btuDZs2fw8vLSaHub53vkyBE8evQI27ZtQ+vWraX2mJiYfOevW7cu6tati6lTp+LUqVNo0aIFAgICMGfOHFStWhVCCLi4uKBGjRqvfC5F2f5vo2rVqrh48SI6dOjwynUWZR+zsbHJs32BvD1dRdkehVW1alWcPn0amZmZr+wxsrW1Rbdu3bBhwwaoVCqcPHmyUNc96dKlC/T19fH777+/tjekXLlyMDU1zXcb3bhxA3p6elKgy+nlS0pK0hgo+/I2K4qS2ofo9TiGg2SV05vh7++PCxcu5Ln2hr6+fp6/RjZv3ox79+4VeV05I/SXLVum0Z7fB2h+612+fHmev6TMzMwAIN8vjpd17doVcXFx2LRpk9SWlZWF5cuXw9zcHG3atCnM03grFy9exIQJE2BjY4MxY8ZI7W/7fHP+ksy9jOfPn+Onn37SmC8lJQVZWVkabXXr1oWenh4yMjIAAH369IG+vj5mzpyZpyYhhMbVZ83MzN74EFdR9O/fH/fu3cMvv/ySZ9qzZ8/w9OlTAEXbx6pWrYrk5GSNQx0PHjzIc0G8omyPwurbty/+/fdf/Pjjj3mmvbyOTz/9FNeuXcOkSZOksVev4+TkhOHDhyM4OBjLly/PM12tVmPRokW4e/cu9PX10alTJ+zcuVNjPFd8fDyCgoLQsmVL6RBNzqGqY8eOSfM9ffoUa9euLdTzzk9R3sMkL/ZwkKxcXFzg4eGBnTt3AkCewNG9e3fMmjULQ4YMgYeHBy5fvowNGzZoDC4rrAYNGuCTTz7BTz/9hOTkZHh4eCAkJARRUVF55u3evTvWr18PKysr1KpVC6GhoTh48GCecSMNGjSAvr4+FixYgOTkZCiVSrRv3x7ly5fPs8wRI0Zg1apVGDx4MMLCwuDs7IwtW7ZIfzUWZoBdURw/fhzp6enSYNuTJ09i165dsLKywvbt22Fvb19sz9fDwwM2Njbw8fGBr68vFAoF1q9fn+fL69ChQxg7diw++ugj1KhRA1lZWVi/fj309fXRt29fAC++VObMmQM/Pz/cunULvXv3hoWFBWJiYrB9+3aMGDECX375JYAXh+U2bdqEiRMnomnTpjA3N0ePHj2KdTsCL750//zzT/znP//B4cOH0aJFC2RnZ+PGjRv4888/sX//fjRp0qRI+5i3tzemTJmCDz/8EL6+vkhLS8PKlStRo0YNjYG6RdkehTVo0CCsW7cOEydOxJkzZ9CqVSs8ffoUBw8exOjRo9GrVy9p3m7duqFMmTLYvHkzunTpku++nZ9FixYhOjoavr6+2LZtG7p37w4bGxvcuXMHmzdvxo0bN6TwMmfOHBw4cAAtW7bE6NGjYWBggFWrViEjIwMLFy6UltmpUydUqlQJw4YNkwLQ6tWrUa5cOdy5c6dI2yBHUd7DJLOSPSmG3kcrVqwQAESzZs3yTEtPTxdffPGFcHBwECYmJqJFixYiNDQ0zymnhb3w17Nnz4Svr68oU6aMMDMzK/CiTI8fPxZDhgyRLt7k5eUlbty4ISpXrix8fHw0lvnLL7+IKlWqCH19/UJd+CtnuUZGRqJu3boaNed+Lt99912e7fFynfnJOS0252ZoaCjKlSsnWrduLebOnSsSEhLyPKY4nu/JkyfFBx98IF1savLkyWL//v0a8/zzzz9i6NChomrVqtIFxtq1aycOHjyYp6atW7eKli1bCjMzM2FmZiZq1qwpxowZIyIiIqR5UlNTxYABA4S1tXWxXvgrP8+fPxcLFiwQtWvXFkqlUtjY2IjGjRuLmTNniuTkZGm+wu5jQggRHBws6tSpI4yMjISrq6v4/fffCzz1szDbo6D68zsFNy0tTXzzzTfCxcVFGBoaCnt7e9GvXz+NU1NzjB49WgAQQUFBBW67/GRlZYlff/1VtGrVSlhZWQlDQ0NRuXJlMWTIkDynzIaHhwsvLy9hbm4uTE1NRbt27cSpU6fyLDMsLEw0b95cGBkZiUqVKonFixe/8sJfL8vvfVnQPk0lSyEER9cQEb0thUKB6dOnF/uvEpeEzz//HL/99hvi4uJ4mXeSDcdwEBG9x9LT0/H777+jb9++DBskK47hICJ6DyUkJODgwYPYsmULHj16lOfS+0TFjYGDiOg9dO3aNahUKpQvXx7Lli0r1utpEOWHYziIiIhIdhzDQURERLJj4CAiIiLZcQwHXlwV7/79+7CwsOBlcImIiIpACIEnT57A0dExz49X5sbAAeD+/ftv9ONMRERE9EJsbCwqVqxY4HQGDkC65HRsbKx0TX8iIiJ6vZSUFDg5Ob325xsYOPD/vyZoaWnJwFFKHTt2DN999x3CwsKkH8Dq3bu3NH3btm0ICAhAWFgYEhMTcf78+Tyn+cXFxWHSpEk4cOAAnjx5AldXV3zzzTfSb3wQEdGbe92QBA4apXfC06dPUb9+faxYsaLA6S1btsSCBQsKXMagQYMQERGBXbt24fLly+jTpw/69++P8+fPy1U2ERH9D3s46J3QpUsX6afB8/Ppp58CgMbPX7/s1KlTWLlyJZo1awYAmDp1KpYsWYKwsDA0bNiwWOslIiJN7OGg94aHhwc2bdqExMREqNVqbNy4Eenp6Wjbtq22SyMi0nlaDRzHjh1Djx494OjoCIVCgR07dkjTMjMzMWXKFNStWxdmZmZwdHTEoEGDcP/+fY1lJCYmQqVSwdLSEtbW1hg2bBhSU1NL+JnQu+DPP/9EZmYmypQpA6VSiZEjR2L79u2oVq2atksjItJ5Wg0crzoun5aWhvDwcEybNg3h4eHYtm0bIiIi0LNnT435VCoVrl69igMHDmD37t04duwYRowYUVJPgd4h06ZNQ1JSEg4ePIhz585h4sSJ6N+/Py5fvqzt0oiIdF6p+S0VhUKR58yDl509exbNmjXD7du3UalSJVy/fh21atXC2bNn0aRJEwDAvn370LVrV9y9exeOjo75LicjIwMZGRnS/ZxTepKTk3mWyjvgVfvKrVu34OLikucslejoaFSrVg1XrlxB7dq1pXZPT09Uq1YNAQEBJVA5EZHuSUlJgZWV1Wu/Q9+pMRzJyclQKBSwtrYGAISGhsLa2loKG8CLLxA9PT2cPn26wOXMnz8fVlZW0o0X/dJ9aWlpAJDnKnj6+vpQq9XaKImI6L3yzpylkp6ejilTpuCTTz6RElRcXBzKly+vMZ+BgQFsbW0RFxdX4LL8/PwwceJE6X5ODweVXqmpqYiKipLux8TE4MKFC7C1tUWlSpWQmJiIO3fuSGN8IiIiAAD29vawt7dHzZo1Ua1aNYwcORLff/89ypQpgx07dkiH4oiISF7vRA9HZmYm+vfvDyEEVq5c+dbLUyqV0kW+eLGvd8O5c+fQsGFD6fTViRMnomHDhvD39wcA7Nq1Cw0bNkS3bt0AAN7e3mjYsKF0qMTQ0BB//fUXypUrhx49eqBevXpYt24d1q5di65du2rnSRERvUdKfQ9HTti4ffs2Dh06pBEO7O3tkZCQoDF/VlYWEhMTYW9vX9Klkozatm2LVw03Gjx4MAYPHvzKZVSvXh1bt24t5sqIiKgwSnUPR07YuHnzJg4ePIgyZcpoTHd3d0dSUhLCwsKktkOHDkGtVqN58+YlXS4REREVQKs9HK86Lu/g4IB+/fohPDwcu3fvRnZ2tjQuw9bWFkZGRnBzc0Pnzp0xfPhwBAQEIDMzE2PHjoW3t3eBZ6i875y/2qPtEqgY3fq2m7ZLICIqFK0GjnPnzqFdu3bS/ZyBnD4+PpgxYwZ27doFAHl+hOvw4cPS1SE3bNiAsWPHokOHDtDT00Pfvn2xbNmyEqmfiIiICkergeN1x+ULc4kQW1tbBAUFFWdZREREVMxK9RgOIiIi0g0MHERERCQ7Bg4iIiKSHQMHERERyY6Bg4iIiGTHwEFERESyY+AgIiIi2TFwEBERkewYOIiIiEh2DBxEREQkOwYOIiIikh0DBxEREcmOgYOIiIhkx8BBREREsmPgICIiItkxcBAREZHsGDiIiIhIdgwcREREJDsGDiIiIpIdAwcRERHJjoGDiIiIZMfAQURERLJj4CAiIiLZMXAQERGR7Bg4iIiISHYMHERERCQ7Bg4iIiKSHQMHERERyY6Bg4iIiGTHwEFERESyY+AgIiIi2TFwEBERkewYOIiIiEh2DBxEREQkOwYOIiIikh0DBxEREcmOgYOIiIhkx8BBREREstNq4Dh27Bh69OgBR0dHKBQK7NixQ2O6EAL+/v5wcHCAiYkJPD09cfPmTY15EhMToVKpYGlpCWtrawwbNgypqakl+CyIiIjodbQaOJ4+fYr69etjxYoV+U5fuHAhli1bhoCAAJw+fRpmZmbw8vJCenq6NI9KpcLVq1dx4MAB7N69G8eOHcOIESNK6ikQERFRIRhoc+VdunRBly5d8p0mhMDSpUsxdepU9OrVCwCwbt062NnZYceOHfD29sb169exb98+nD17Fk2aNAEALF++HF27dsX3338PR0fHEnsuREREVLBSO4YjJiYGcXFx8PT0lNqsrKzQvHlzhIaGAgBCQ0NhbW0thQ0A8PT0hJ6eHk6fPl3gsjMyMpCSkqJxIyIiIvmU2sARFxcHALCzs9Not7Ozk6bFxcWhfPnyGtMNDAxga2srzZOf+fPnw8rKSro5OTkVc/VERESUW6kNHHLy8/NDcnKydIuNjdV2SURERDqt1AYOe3t7AEB8fLxGe3x8vDTN3t4eCQkJGtOzsrKQmJgozZMfpVIJS0tLjRsRERHJp9QGDhcXF9jb2yMkJERqS0lJwenTp+Hu7g4AcHd3R1JSEsLCwqR5Dh06BLVajebNm5d4zURERJQ/rZ6lkpqaiqioKOl+TEwMLly4AFtbW1SqVAkTJkzAnDlzUL16dbi4uGDatGlwdHRE7969AQBubm7o3Lkzhg8fjoCAAGRmZmLs2LHw9vbmGSpERESliFYDx7lz59CuXTvp/sSJEwEAPj4+WLNmDSZPnoynT59ixIgRSEpKQsuWLbFv3z4YGxtLj9mwYQPGjh2LDh06QE9PD3379sWyZctK/LkQERFRwRRCCKHtIrQtJSUFVlZWSE5O1vnxHM5f7dF2CVSMbn3bTdslENF7rrDfoaV2DAcRERHpDgYOIiIikh0DBxEREcmOgYOIiIhkx8BBREREsmPgICIiItkxcBAREZHsGDiIiIhIdgwcREREJDsGDiIiIpIdAwcRERHJjoGDiIiIZMfAQURERLJj4CAiIiLZMXAQERGR7Bg4iIiISHYMHERERCQ7Bg4iIiKSHQMHEREVu+zsbEybNg0uLi4wMTFB1apVMXv2bAghpHkGDx4MhUKhcevcubMWqyY5GWi7ACIi0j0LFizAypUrsXbtWtSuXRvnzp3DkCFDYGVlBV9fX2m+zp07IzAwULqvVCq1US6VAAYOIiIqdqdOnUKvXr3QrVs3AICzszP++OMPnDlzRmM+pVIJe3t7bZRIJYyHVIiIqNh5eHggJCQEkZGRAICLFy/ixIkT6NKli8Z8R44cQfny5eHq6opRo0bh0aNH2iiXSgB7OIiIqNh99dVXSElJQc2aNaGvr4/s7GzMnTsXKpVKmqdz587o06cPXFxcEB0dja+//hpdunRBaGgo9PX1tVg9yYGBg4iIit2ff/6JDRs2ICgoCLVr18aFCxcwYcIEODo6wsfHBwDg7e0tzV+3bl3Uq1cPVatWxZEjR9ChQwdtlU4yYeAgIqJiN2nSJHz11VdSqKhbty5u376N+fPnS4HjZVWqVEHZsmURFRXFwKGDOIaDiIiKXVpaGvT0NL9i9PX1oVarC3zM3bt38ejRIzg4OMhdHmkBeziIiKjY9ejRA3PnzkWlSpVQu3ZtnD9/HosXL8bQoUMBAKmpqZg5cyb69u0Le3t7REdHY/LkyahWrRq8vLy0XD3JgYGDiIiK3fLlyzFt2jSMHj0aCQkJcHR0xMiRI+Hv7w/gRW/HpUuXsHbtWiQlJcHR0RGdOnXC7NmzeS0OHaUQuS/79p5KSUmBlZUVkpOTYWlpqe1yZOX81R5tl0DF6Na33bRdAhG95wr7HcoxHERERCQ7HlIhIiol2AOpe9gL+f/Yw0FERESyY+AgIiIi2TFwEBERkewYOIiIiEh2DBxEREQkOwYOIiIikh0DBxEREcmOgYOIiIhkV6oDR3Z2NqZNmwYXFxeYmJigatWqmD17NnJfjV0IAX9/fzg4OMDExASenp64efOmFqsmIiKil5XqwLFgwQKsXLkSP/74I65fv44FCxZg4cKFWL58uTTPwoULsWzZMgQEBOD06dMwMzODl5cX0tPTtVg5ERER5VaqL21+6tQp9OrVC926vbg0rLOzM/744w+cOXMGwIvejaVLl2Lq1Kno1asXAGDdunWws7PDjh074O3trbXaiYiI6P+V6h4ODw8PhISEIDIyEgBw8eJFnDhxAl26dAEAxMTEIC4uDp6entJjrKys0Lx5c4SGhha43IyMDKSkpGjciIiISD6luofjq6++QkpKCmrWrAl9fX1kZ2dj7ty5UKlUAIC4uDgAgJ2dncbj7OzspGn5mT9/PmbOnClf4URERKShVPdw/Pnnn9iwYQOCgoIQHh6OtWvX4vvvv8fatWvfarl+fn5ITk6WbrGxscVUMREREeWnVPdwTJo0CV999ZU0FqNu3bq4ffs25s+fDx8fH9jb2wMA4uPj4eDgID0uPj4eDRo0KHC5SqUSSqVS1tqJiIjo/5XqHo60tDTo6WmWqK+vD7VaDQBwcXGBvb09QkJCpOkpKSk4ffo03N3dS7RWIiIiKlip7uHo0aMH5s6di0qVKqF27do4f/48Fi9ejKFDhwIAFAoFJkyYgDlz5qB69epwcXHBtGnT4OjoiN69e2u3eCIiIpKU6sCxfPlyTJs2DaNHj0ZCQgIcHR0xcuRI+Pv7S/NMnjwZT58+xYgRI5CUlISWLVti3759MDY21mLlRERElJtC5L5s53sqJSUFVlZWSE5OhqWlpbbLkZXzV3u0XQIVo1vfdtN2CVSM+P7UPe/De7Sw36GlegwHERER6QYGDiIiIpIdAwcRERHJjoGDiIiIZMfAQURERLJj4CAiIiLZMXAQERGR7Bg4iIiISHYMHERERCQ7Bg4iIiKSHQMHERERyY6Bg4iIiGTHwEFERESyY+AgIiIi2TFwEBERkewYOIiIiEh2DBxEREQkOwYOIiIikh0DBxEREcmOgYOIiIhk90aBo0qVKnj06FGe9qSkJFSpUuWtiyIiIiLd8kaB49atW8jOzs7TnpGRgXv37r11UURERKRbDIoy865du6T/79+/H1ZWVtL97OxshISEwNnZudiKIyIiIt1QpMDRu3dvAIBCoYCPj4/GNENDQzg7O2PRokXFVhwRERHphiIFDrVaDQBwcXHB2bNnUbZsWVmKIiIiIt1SpMCRIyYmprjrICIiIh32RoEDAEJCQhASEoKEhASp5yPH6tWr37owIiIi0h1vFDhmzpyJWbNmoUmTJnBwcIBCoSjuuoiIiEiHvFHgCAgIwJo1a/Dpp58Wdz1ERESkg97oOhzPnz+Hh4dHcddCREREOuqNAsdnn32GoKCg4q6FiIiIdNQbHVJJT0/Hzz//jIMHD6JevXowNDTUmL548eJiKY6IiIh0wxsFjkuXLqFBgwYAgCtXrmhM4wBSIiIietkbBY7Dhw8Xdx1ERESkw/jz9ERERCS7N+rhaNeu3SsPnRw6dOiNCyIiIiLd80aBI2f8Ro7MzExcuHABV65cyfOjbkRERERvFDiWLFmSb/uMGTOQmpr6VgURERGR7inWMRwDBw7k76gQERFRHsUaOEJDQ2FsbFyci8S9e/cwcOBAlClTBiYmJqhbty7OnTsnTRdCwN/fHw4ODjAxMYGnpydu3rxZrDUQERHR23mjQyp9+vTRuC+EwIMHD3Du3DlMmzatWAoDgMePH6NFixZo164d9u7di3LlyuHmzZuwsbGR5lm4cCGWLVuGtWvXwsXFBdOmTYOXlxeuXbtW7OGHiIiI3swbBQ4rKyuN+3p6enB1dcWsWbPQqVOnYikMABYsWAAnJycEBgZKbS4uLtL/hRBYunQppk6dil69egEA1q1bBzs7O+zYsQPe3t7FVgsRERG9uTcKHLkDgJx27doFLy8vfPTRRzh69CgqVKiA0aNHY/jw4QCAmJgYxMXFwdPTU3qMlZUVmjdvjtDQ0AIDR0ZGBjIyMqT7KSkp8j4RIiKi99xbjeEICwvD77//jt9//x3nz58vrpok//zzD1auXInq1atj//79GDVqFHx9fbF27VoAQFxcHADAzs5O43F2dnbStPzMnz8fVlZW0s3JyanYayciIqL/90Y9HAkJCfD29saRI0dgbW0NAEhKSkK7du2wceNGlCtXrliKU6vVaNKkCebNmwcAaNiwIa5cuYKAgIC3ut6Hn58fJk6cKN1PSUlh6CAiIpLRG/VwjBs3Dk+ePMHVq1eRmJiIxMREXLlyBSkpKfD19S224hwcHFCrVi2NNjc3N9y5cwcAYG9vDwCIj4/XmCc+Pl6alh+lUglLS0uNGxEREcnnjQLHvn378NNPP8HNzU1qq1WrFlasWIG9e/cWW3EtWrRARESERltkZCQqV64M4MUAUnt7e4SEhEjTU1JScPr0abi7uxdbHURERPR23uiQilqthqGhYZ52Q0NDqNXqty4qx+effw4PDw/MmzcP/fv3x5kzZ/Dzzz/j559/BgAoFApMmDABc+bMQfXq1aXTYh0dHdG7d+9iq4OIiIjezhv1cLRv3x7jx4/H/fv3pbZ79+7h888/R4cOHYqtuKZNm2L79u34448/UKdOHcyePRtLly6FSqWS5pk8eTLGjRuHESNGoGnTpkhNTcW+fft4DQ4iIqJSRCGEEEV9UGxsLHr27ImrV69Kgy1jY2NRp04d7Nq1CxUrViz2QuWUkpICKysrJCcn6/x4Duev9mi7BCpGt77tpu0SqBjx/al73of3aGG/Q9/okIqTkxPCw8Nx8OBB3LhxA8CLwZy5r4dBRERElKNIh1QOHTqEWrVqISUlBQqFAh07dsS4ceMwbtw4NG3aFLVr18bx48flqpWIiIjeUUUKHEuXLsXw4cPz7TKxsrLCyJEjsXjx4mIrjoiIiHRDkQLHxYsX0blz5wKnd+rUCWFhYW9dFBEREemWIgWO+Pj4fE+HzWFgYICHDx++dVFERESkW4oUOCpUqIArV64UOP3SpUtwcHB466KIiIhItxQpcHTt2hXTpk1Denp6nmnPnj3D9OnT0b1792IrjoiIiHRDkU6LnTp1KrZt24YaNWpg7NixcHV1BQDcuHEDK1asQHZ2Nr755htZCiUiIqJ3V5ECh52dHU6dOoVRo0bBz88POdcMUygU8PLywooVK/L8VDwRERFRkS/8VblyZfz11194/PgxoqKiIIRA9erVYWNjI0d9REREpAPe6EqjAGBjY4OmTZsWZy1ERESko97ox9uIiIiIioKBg4iIiGTHwEFERESyY+AgIiIi2TFwEBERkewYOIiIiEh2DBxEREQkOwYOIiIikh0DBxEREcmOgYOIiIhkx8BBREREsmPgICIiItkxcBAREZHsGDiIiIhIdgwcREREJDsGDiIiIpIdAwcRERHJjoGDiIiIZMfAQURERLJj4CAiIiLZMXAQERGR7Bg4iIiISHYMHERERCQ7Bg4iIiKSHQMHERERyY6Bg4iIiGTHwEFERESye6cCx7fffguFQoEJEyZIbenp6RgzZgzKlCkDc3Nz9O3bF/Hx8dorkoiIiPJ4ZwLH2bNnsWrVKtSrV0+j/fPPP8d///tfbN68GUePHsX9+/fRp08fLVVJRERE+XknAkdqaipUKhV++eUX2NjYSO3Jycn47bffsHjxYrRv3x6NGzdGYGAgTp06hb///luLFRMREVFu70TgGDNmDLp16wZPT0+N9rCwMGRmZmq016xZE5UqVUJoaGiBy8vIyEBKSorGjYiIiORjoO0CXmfjxo0IDw/H2bNn80yLi4uDkZERrK2tNdrt7OwQFxdX4DLnz5+PmTNnFnepREREVIBS3cMRGxuL8ePHY8OGDTA2Ni625fr5+SE5OVm6xcbGFtuyiYiIKK9SHTjCwsKQkJCARo0awcDAAAYGBjh69CiWLVsGAwMD2NnZ4fnz50hKStJ4XHx8POzt7QtcrlKphKWlpcaNiIiI5FOqD6l06NABly9f1mgbMmQIatasiSlTpsDJyQmGhoYICQlB3759AQARERG4c+cO3N3dtVEyERER5aNUBw4LCwvUqVNHo83MzAxlypSR2ocNG4aJEyfC1tYWlpaWGDduHNzd3fHBBx9oo2QiIiLKR6kOHIWxZMkS6OnpoW/fvsjIyICXlxd++uknbZdFREREubxzgePIkSMa942NjbFixQqsWLFCOwURERHRa5XqQaNERESkGxg4iIiISHYMHERERCQ7Bg4iIiKSHQMHERERyY6Bg4iIiGTHwEFERESyY+AgIiIi2TFwEBERkewYOIiIiEh2DBxEREQkOwYOIiIikh0DBxEREcmOgYOIiIhkx8BBREREsmPgICIiItkxcBAREZHsGDiIiIhIdgwcREREJDsGDiIiIpIdAwcRERHJjoGDiIiIZMfAQURERLJj4CAiIiLZMXAQERGR7Bg4iIiISHYMHERERCQ7Bg4iIiKSHQMHERERyY6Bg4iIiGTHwEFERESyY+AgIiIi2TFwEBERkewYOIiIiEh2DBxEREQkOwYOIiIikh0DBxEREcmOgYOIiIhkx8BBREREsivVgWP+/Plo2rQpLCwsUL58efTu3RsREREa86Snp2PMmDEoU6YMzM3N0bdvX8THx2upYiIiIspPqQ4cR48exZgxY/D333/jwIEDyMzMRKdOnfD06VNpns8//xz//e9/sXnzZhw9ehT3799Hnz59tFg1ERERvcxA2wW8yr59+zTur1mzBuXLl0dYWBhat26N5ORk/PbbbwgKCkL79u0BAIGBgXBzc8Pff/+NDz74QBtlExER0UtKdQ/Hy5KTkwEAtra2AICwsDBkZmbC09NTmqdmzZqoVKkSQkNDC1xORkYGUlJSNG5EREQkn3cmcKjVakyYMAEtWrRAnTp1AABxcXEwMjKCtbW1xrx2dnaIi4srcFnz58+HlZWVdHNycpKzdCIiovfeOxM4xowZgytXrmDjxo1vvSw/Pz8kJydLt9jY2GKokIiIiApSqsdw5Bg7dix2796NY8eOoWLFilK7vb09nj9/jqSkJI1ejvj4eNjb2xe4PKVSCaVSKWfJRERElEup7uEQQmDs2LHYvn07Dh06BBcXF43pjRs3hqGhIUJCQqS2iIgI3LlzB+7u7iVdLhERERWgVPdwjBkzBkFBQdi5cycsLCykcRlWVlYwMTGBlZUVhg0bhokTJ8LW1haWlpYYN24c3N3deYYKERFRKVKqA8fKlSsBAG3bttVoDwwMxODBgwEAS5YsgZ6eHvr27YuMjAx4eXnhp59+KuFKiYiI6FVKdeAQQrx2HmNjY6xYsQIrVqwogYqIiIjoTZTqMRxERESkGxg4iIiISHYMHERERCQ7Bg4iIiKSHQMHERERyY6Bg4iIiGTHwEFERESyY+AgIiIi2TFwEBERkewYOIiIiEh2DBxEREQkOwYOIiIikh0DBxEREcmOgYOIiIhkx8BBREREsmPgICIiItkxcBAREZHsGDiIiIhIdgwcREREJDsGDiIiIpIdAwcRERHJjoGDiIiIZMfAQURERLJj4CAiIiLZMXAQERGR7Bg4iIiISHYMHERERCQ7Bg4iIiKSHQMHERERyY6Bg4iIiGTHwEFERESyY+AgIiIi2TFwEBERkewYOIiIiEh2DBxEREQkOwYOIiIikh0DBxEREcmOgYOIiIhkx8BBREREstOZwLFixQo4OzvD2NgYzZs3x5kzZ7RdEhEREf2PTgSOTZs2YeLEiZg+fTrCw8NRv359eHl5ISEhQdulEREREXQkcCxevBjDhw/HkCFDUKtWLQQEBMDU1BSrV6/WdmlEREQEwEDbBbyt58+fIywsDH5+flKbnp4ePD09ERoamu9jMjIykJGRId1PTk4GAKSkpMhbbCmgzkjTdglUjN6HffZ9wven7nkf3qM5z1EI8cr53vnA8e+//yI7Oxt2dnYa7XZ2drhx40a+j5k/fz5mzpyZp93JyUmWGonkYrVU2xUQ0au8T+/RJ0+ewMrKqsDp73zgeBN+fn6YOHGidF+tViMxMRFlypSBQqHQYmVUHFJSUuDk5ITY2FhYWlpquxwiyoXvT90jhMCTJ0/g6Oj4yvne+cBRtmxZ6OvrIz4+XqM9Pj4e9vb2+T5GqVRCqVRqtFlbW8tVImmJpaUlP9CISim+P3XLq3o2crzzg0aNjIzQuHFjhISESG1qtRohISFwd3fXYmVERESU453v4QCAiRMnwsfHB02aNEGzZs2wdOlSPH36FEOGDNF2aURERAQdCRwff/wxHj58CH9/f8TFxaFBgwbYt29fnoGk9H5QKpWYPn16nsNmRKR9fH++vxTideexEBEREb2ld34MBxEREZV+DBxEREQkOwYOIiIikh0DBxEREcmOgYOIiIhkx8BBREREsmPgICIiItnpxIW/6P20a9euQs/bs2dPGSshovz06dOn0PNu27ZNxkqoNGDgoHdW7969Ne4rFArkvo5d7l/+zc7OLqmyiOh/cv+glxAC27dvh5WVFZo0aQIACAsLQ1JSUpGCCb27eEiF3llqtVq6BQcHo0GDBti7dy+SkpKQlJSEv/76C40aNcK+ffu0XSrReykwMFC62dnZoX///oiJicG2bduwbds2/PPPP/D29kbZsmW1XSqVAF7anHRCnTp1EBAQgJYtW2q0Hz9+HCNGjMD169e1VBkRAUC5cuVw4sQJuLq6arRHRETAw8MDjx490lJlVFLYw0E6ITo6GtbW1nnarayscOvWrRKvh4g0ZWVl4caNG3nab9y4AbVarYWKqKRxDAfphKZNm2LixIlYv3699CvB8fHxmDRpEpo1a6bl6ohoyJAhGDZsGKKjo6X35OnTp/Htt99iyJAhWq6OSgIPqZBOiIqKwocffojIyEg4OTkBAGJjY1G9enXs2LED1apV03KFRO83tVqN77//Hj/88AMePHgAAHBwcMD48ePxxRdfQF9fX8sVktwYOEhnCCFw4MABqdvWzc0Nnp6eGmerEJH2paSkAAAsLS21XAmVJAYO0jnp6elQKpUMGkSlTFZWFo4cOYLo6GgMGDAAFhYWuH//PiwtLWFubq7t8khmHDRKOkGtVmP27NmoUKECzM3NERMTAwCYNm0afvvtNy1XR0S3b99G3bp10atXL4wZMwYPHz4EACxYsABffvmllqujksDAQTphzpw5WLNmDRYuXAgjIyOpvU6dOvj111+1WBkRAcD48ePRpEkTPH78GCYmJlL7hx9+iJCQEC1WRiWFgYN0wrp16/Dzzz9DpVJpDD6rX79+vqfiEVHJOn78OKZOnarxBwEAODs74969e1qqikoSAwfphHv37uV7JoparUZmZqYWKiKi3NRqdb4/MXD37l1YWFhooSIqaQwcpBNq1aqF48eP52nfsmULGjZsqIWKiCi3Tp06YenSpdJ9hUKB1NRUTJ8+HV27dtVeYVRieOEv0gn+/v7w8fHBvXv3oFarsW3bNkRERGDdunXYvXu3tssjeu8tWrQIXl5eqFWrFtLT0zFgwADcvHkTZcuWxR9//KHt8qgE8LRY0hnHjx/HrFmzcPHiRaSmpqJRo0bw9/dHp06dtF0aEeHFabEbN27EpUuXpPeoSqXSGERKuouBg4iIZJeeng5jY2Ntl0FaxDEcpBM+++wzHDlyRNtlEFEBypcvDx8fHxw4cIA/1vaeYuAgnfDw4UN07twZTk5OmDRpEi5cuKDtkogol7Vr1yItLQ29evVChQoVMGHCBJw7d07bZVEJ4iEV0hmPHz/G5s2bERQUhOPHj6NmzZpQqVQYMGAAnJ2dtV0eEQF48uQJtmzZgj/++AOHDh1ClSpVMHDgQPj7+2u7NJIZAwfppLt37+KPP/7A6tWrcfPmTWRlZWm7JCJ6ybVr16BSqXDp0qV8r9FBuoWHVEjnZGZm4ty5czh9+jRu3boFOzs7bZdERP+Tnp6OP//8E71790ajRo2QmJiISZMmabssKgG8DgfpjMOHDyMoKAhbt26FWq1Gnz59sHv3brRv317bpRG99/bv34+goCDs2LEDBgYG6NevH4KDg9G6dWttl0YlhIdUSCdUqFABiYmJ6Ny5M1QqFXr06AGlUqntsojof0xNTdG9e3eoVCp07doVhoaG2i6JShgDB+mEX375BR999BGsra21XQoR5ePJkyf8zZT3HAMHERHJIiUlpdDzWlpaylgJlQYMHPTO6tOnT6Hn3bZtm4yVEFF+9PT0oFAoXjmPEAIKhYJnqbwHOGiU3llWVlbaLoGIXuHw4cPaLoFKEfZwEBERkezYw0E65eHDh4iIiAAAuLq6oly5clquiIhyJCUl4bfffsP169cBALVr18bQoUPZW/meYA8H6YSnT59i3LhxWLdunfTDUPr6+hg0aBCWL18OU1NTLVdI9H47d+4cvLy8YGJigmbNmgEAzp49i2fPniE4OBiNGjXScoUkNwYO0gkjR47EwYMH8eOPP6JFixYAgBMnTsDX1xcdO3bEypUrtVwh0futVatWqFatGn755RcYGLzoXM/KysJnn32Gf/75B8eOHdNyhSQ3Bg7SCWXLlsWWLVvQtm1bjfbDhw+jf//+ePjwoXYKIyIAgImJCc6fP4+aNWtqtF+7dg1NmjRBWlqaliqjksLfUiGdkJaWlu9vppQvX54fZESlgKWlJe7cuZOnPTY2lhcEe08wcJBOcHd3x/Tp05Geni61PXv2DDNnzoS7u7sWKyMiAPj4448xbNgwbNq0CbGxsYiNjcXGjRvx2Wef4ZNPPtF2eVQCeEiFdMKVK1fg5eWFjIwM1K9fHwBw8eJFGBsbY//+/ahdu7aWKyR6vz1//hyTJk1CQEAAsrKyAACGhoYYNWoUvv32W/720XuAgYN0RlpaGjZs2IAbN24AANzc3KBSqWBiYqLlyogoR1paGqKjowEAVatW5Rlk7xEGDiIiKnEpKSk4dOgQXF1d4ebmpu1yqARwDAfphLVr12LPnj3S/cmTJ8Pa2hoeHh64ffu2FisjIgDo378/fvzxRwAvxlc1adIE/fv3R7169bB161YtV0clgYGDdMK8efOkQyehoaH48ccfsXDhQpQtWxaff/65lqsjomPHjqFVq1YAgO3bt0MIgaSkJCxbtgxz5szRcnVUEhg4SCfExsaiWrVqAIAdO3agX79+GDFiBObPn4/jx49ruToiSk5Ohq2tLQBg37596Nu3L0xNTdGtWzfcvHlTy9VRSWDgIJ1gbm6OR48eAQCCg4PRsWNHAICxsTGePXumzdKICICTkxNCQ0Px9OlT7Nu3D506dQIAPH78GMbGxlqujkoCf7yNdELHjh3x2WefoWHDhoiMjETXrl0BAFevXoWzs7N2iyMiTJgwASqVCubm5qhUqZJ0VeBjx46hbt262i2OSgR7OEgnrFixAu7u7nj48CG2bt2KMmXKAADCwsJ4USGiUmD06NEIDQ3F6tWrcfLkSejpvfj6qVKlCsdwvCd4WiwREZWY58+fIyYmBlWrVpV+xI3eD+zhIJ1x/PhxDBw4EB4eHrh37x4AYP369Thx4oSWKyOitLQ0DBs2DKampqhdu7b0uyrjxo3Dt99+q+XqqCQwcJBO2Lp1K7y8vGBiYoLw8HBkZGQAeDEyft68eVqujoj8/Pxw8eJFHDlyRGOQqKenJzZt2qTFyqikMHCQTpgzZw4CAgLwyy+/wNDQUGpv0aIFwsPDtVgZEQEvTlf/8ccf0bJlSygUCqm9du3a0qXOSbcxcJBOiIiIQOvWrfO0W1lZISkpqeQLIiINDx8+RPny5fO0P336VCOAkO5i4CCdYG9vj6ioqDztJ06cQJUqVbRQERHl1qRJE42fH8gJGb/++ivc3d21VRaVIA4RJp0wfPhwjB8/HqtXr4ZCocD9+/cRGhqKL774Av7+/touj+i9N2/ePHTp0gXXrl1DVlYWfvjhB1y7dg2nTp3C0aNHtV0elQCeFks6QQiBefPmYf78+UhLSwMAKJVKTJo0CX5+fvyJeqJS4J9//sH8+fNx8eJFpKamolGjRpgyZQov/PWeYOAgnfL8+XNERUUhNTUVtWrVwqpVq/Ddd98hLi5O26URvbcyMzMxcuRITJs2DS4uLtouh7SEYzjonZaRkQE/Pz80adIELVq0wF9//YVatWrh6tWrcHV1xQ8//MBfiyXSMkNDQ/4EPbGHg95tU6ZMwapVq+Dp6YlTp07h4cOHGDJkCP7++298/fXX+Oijj6Cvr6/tMoneez4+PmjQoAH/AHiPcdAovdM2b96MdevWoWfPnrhy5Qrq1auHrKwsXLx4kafaEZUi1atXx6xZs3Dy5Ek0btwYZmZmGtN9fX21VBmVFPZw0DvNyMgIMTExqFChAgDAxMQEZ86c4SA0olLmVWM3FAoF/vnnnxKshrSBPRz0TsvOzoaRkZF038DAAObm5lqsiIjyExMTo+0SSMsYOOidJoTA4MGDoVQqAQDp6en4z3/+k6e7dtu2bdooj4iI/oeBg95pPj4+GvcHDhyopUqI6FWys7OxZs0ahISEICEhAWq1WmP6oUOHtFQZlRSO4SAiItmNHTsWa9asQbdu3eDg4JBnUPeSJUu0VBmVFAYOIiKSXdmyZbFu3Tp07dpV26WQlvDCX0REJDsjIyNUq1ZN22WQFjFwEBGR7L744gv88MMPYKf6+4uHVIiISHYffvghDh8+DFtbW9SuXRuGhoYa03kmme7jWSpERCQ7a2trfPjhh9oug7SIPRxEREQkO/ZwEBGRbGxsbPL9XSMrKyvUqFEDX375JTp27KiFyqiksYeDiIhks3bt2nzbk5KSEBYWhk2bNmHLli3o0aNHCVdGJY2Bg4iItGbx4sXYsmULTp06pe1SSGYMHEREpDWRkZH44IMPkJiYqO1SSGa8DgcREWlNRkaGxi8+k+5i4CAiIq357bff0KBBA22XQSWAZ6kQEZFsJk6cmG97cnIywsPDERkZiWPHjpVwVaQNDBxERCSb8+fP59tuaWmJjh07Ytu2bXBxcSnhqkgbOGiUiIiIZMcxHERERCQ7Bg4iIiKSHQMHERERyY6Bg4iIiGTHwEFQKBTYsWNHsS/3yJEjUCgUSEpKKvZla8uaNWtgbW2t7TJKLWdnZyxdurTA6YMHD0bv3r1LrJ53QUm9T7jtC6ektpNcn7ulGQOHjggICICFhQWysrKkttTUVBgaGqJt27Ya8+Z8wEVHRxfb+tu2bYsJEyYU2/JKq48//hiRkZHaLkN2cn0Y/vDDD1izZk2xL5eISj8GDh3Rrl07pKam4ty5c1Lb8ePHYW9vj9OnTyM9PV1qP3z4MCpVqoSqVatqo9S3lpmZqbV1m5iYoHz58gVOf/78eQlW8+6xsrJ6p3qI+Hq+WkluHyGExh9U9O5h4NARrq6ucHBwwJEjR6S2I0eOoFevXnBxccHff/+t0d6uXTuNx//777/48MMPYWpqiurVq2PXrl0a069cuYIuXbrA3NwcdnZ2+PTTT/Hvv/8CeNEFefToUfzwww9QKBRQKBS4deuW9NiwsDA0adIEpqam8PDwQEREhMayd+7ciUaNGsHY2BhVqlTBzJkzNT5YFAoFVq5ciZ49e8LMzAxz587NdxtkZGTgyy+/RIUKFWBmZobmzZtrbI+cwyH79++Hm5sbzM3N0blzZzx48AAAEBwcDGNj4zxd2+PHj0f79u01lpFjxowZaNCgAX799Ve4uLjA2NgYAHDnzh306tUL5ubmsLS0RP/+/REfH5/ncevXr4ezszOsrKzg7e2NJ0+eSPO0bdsW48aNw4QJE2BjYwM7Ozv88ssvePr0KYYMGQILCwtUq1YNe/fuLfRrlbNcX19fTJ48Gba2trC3t8eMGTOk6c7OzgCADz/8EAqFQrofHR2NXr16wc7ODubm5mjatCkOHjyY72tRkJe7q7ds2YK6devCxMQEZcqUgaenJ54+fQrgxX7arFkzmJmZwdraGi1atMDt27fzXQ4ATJgwQaM3T61WY/78+XBxcYGJiQnq16+PLVu2vLI+Z2dnzJ49G4MGDYKlpSVGjBgBADhx4gRatWoFExMTODk5wdfXV6oTAH766SdUr14dxsbGsLOzQ79+/aRpGRkZ8PX1Rfny5WFsbIyWLVvi7Nmz+a4/JSUFJiYmeV7T7du3w8LCAmlpaQCA2NhY9O/fH9bW1rC1tUWvXr003nPZ2dmYOHEirK2tUaZMGUyePBmFueTS1q1bUbt2bSiVSjg7O2PRokWF2j75bceXD601aNBAYz9TKBT49ddfC/zcyemJ3bt3Lxo3bgylUokTJ0689nV9/PgxVCoVypUrBxMTE1SvXh2BgYHS9MuXL6N9+/bSPjdixAikpqbm+zx+/vlnODo6Qq1Wa7T36tULQ4cOle6/7jPs5s2baN26NYyNjVGrVi0cOHAg3/XpPEE6Y8CAAaJTp07S/aZNm4rNmzeL//znP8Lf318IIURaWppQKpVizZo10nwARMWKFUVQUJC4efOm8PX1Febm5uLRo0dCCCEeP34sypUrJ/z8/MT169dFeHi46Nixo2jXrp0QQoikpCTh7u4uhg8fLh48eCAePHggsrKyxOHDhwUA0bx5c3HkyBFx9epV0apVK+Hh4SGt+9ixY8LS0lKsWbNGREdHi+DgYOHs7CxmzJihUV/58uXF6tWrRXR0tLh9+3a+z/+zzz4THh4e4tixYyIqKkp89913QqlUisjISCGEEIGBgcLQ0FB4enqKs2fPirCwMOHm5iYGDBgghBAiKytL2NnZiV9//VVa5sttgYGBwsrKSpo+ffp0YWZmJjp37izCw8PFxYsXRXZ2tmjQoIFo2bKlOHfunPj7779F48aNRZs2bTQeZ25uLvr06SMuX74sjh07Juzt7cXXX38tzdOmTRthYWEhZs+eLSIjI8Xs2bOFvr6+6NKli/j5559FZGSkGDVqlChTpox4+vRpoV6rnOVaWlqKGTNmiMjISLF27VqhUChEcHCwEEKIhIQEAUAEBgaKBw8eiISEBCGEEBcuXBABAQHi8uXLIjIyUkydOlUYGxtrvB6VK1cWS5Ysyff1EUIIHx8f0atXLyGEEPfv3xcGBgZi8eLFIiYmRly6dEmsWLFCPHnyRGRmZgorKyvx5ZdfiqioKHHt2jWxZs0aaV25l5Nj/PjxGtt4zpw5ombNmmLfvn0iOjpaBAYGCqVSKY4cOVJgfZUrVxaWlpbi+++/F1FRUdLNzMxMLFmyRERGRoqTJ0+Khg0bisGDBwshhDh79qzQ19cXQUFB4tatWyI8PFz88MMP0jJ9fX2Fo6Oj+Ouvv8TVq1eFj4+PsLGxkd5fOe+Tx48fCyGE6Nevnxg4cKBGXX379pXanj9/Ltzc3MTQoUPFpUuXxLVr18SAAQOEq6uryMjIEEIIsWDBAmFjYyO2bt0qrl27JoYNGyYsLCzybLPczp07J/T09MSsWbNERESECAwMFCYmJiIwMPCV26eg7fjyflC/fn0xffp06f7rPndytku9evVEcHCwiIqKEo8ePXrt6zpmzBjRoEEDcfbsWRETEyMOHDggdu3aJYQQIjU1VTg4OEjvu5CQEOHi4iJ8fHykunLvW4mJicLIyEgcPHhQmv7o0SONttd9hmVnZ4s6deqIDh06iAsXLoijR4+Khg0bCgBi+/btBb4euoiBQ4f88ssvwszMTGRmZoqUlBRhYGAgEhISRFBQkGjdurUQQoiQkBABQONLAoCYOnWqdD81NVUAEHv37hVCCDF79myNICOEELGxsQKAiIiIEEK8+BIbP368xjw5Hxi536x79uwRAMSzZ8+EEEJ06NBBzJs3T+Nx69evFw4ODhr1TZgw4ZXP/fbt20JfX1/cu3dPo71Dhw7Cz89PCPEiLADQ+JBcsWKFsLOzk+6PHz9etG/fXrq/f/9+oVQqpS+D/AKHoaGh9KUshBDBwcFCX19f3LlzR2q7evWqACDOnDkjPc7U1FSkpKRI80yaNEk0b95cut+mTRvRsmVL6X5WVpYwMzMTn376qdT24MEDAUCEhoYKIQr/WuVerhAvwumUKVOk+4X9MKxdu7ZYvny5dL8ogSMsLEwAELdu3coz36NHjwSAAsPB6wJHenq6MDU1FadOndKYZ9iwYeKTTz4psL7KlSuL3r1753nMiBEjNNqOHz8u9PT0xLNnz8TWrVuFpaWlxmuZIzU1VRgaGooNGzZIbc+fPxeOjo5i4cKFQoi8gWP79u3C3NxcCpHJycnC2NhYej+uX79euLq6CrVaLS0zIyNDmJiYiP379wshhHBwcJCWL4QQmZmZomLFiq8MHAMGDBAdO3bUaJs0aZKoVavWK7dPfgobOF71uZOzXXbs2CHNU5jXtUePHmLIkCH51vXzzz8LGxsbkZqaKrXt2bNH6Onpibi4OCFE3n2rV69eYujQodL9VatWCUdHR5GdnS2EeP1n2P79+4WBgYHGZ9PevXvfy8DB31LRIW3btsXTp09x9uxZPH78GDVq1EC5cuXQpk0bDBkyBOnp6Thy5AiqVKmCSpUqaTy2Xr160v/NzMxgaWmJhIQEAMDFixdx+PBhmJub51lndHQ0atSo8cq6ci/bwcEBAJCQkIBKlSrh4sWLOHnypMZhkuzsbKSnpyMtLQ2mpqYAgCZNmrxyHZcvX0Z2dnaeWjIyMlCmTBnpvqmpqcbYFQcHB+l5AoBKpcIHH3yA+/fvw9HRERs2bEC3bt1eOe6gcuXKKFeunHT/+vXrcHJygpOTk9RWq1YtWFtb4/r162jatCmAF93OFhYWBdYCaG47fX19lClTBnXr1pXa7OzsAKDIr1Xu5Ra07pelpqZixowZ2LNnDx48eICsrCw8e/YMd+7ceeXjClK/fn106NABdevWhZeXFzp16oR+/frBxsYGtra2GDx4MLy8vNCxY0d4enqif//+0v7zOlFRUUhLS0PHjh012p8/f46GDRu+8rEv72sXL17EpUuXsGHDBqlNCAG1Wo2YmBh07NgRlStXRpUqVdC5c2d07txZOkwQHR2NzMxMtGjRQnqsoaEhmjVrhuvXr+e7/q5du8LQ0BC7du2Ct7c3tm7dCktLS3h6ekr1REVFaew7AJCeno7o6GgkJyfjwYMHaN68uTTNwMAATZo0eeVhlevXr6NXr14abS1atMDSpUuRnZ0NfX39fLfP23jV506O3OsrzOs6atQo9O3bF+Hh4ejUqRN69+4NDw8P6TnWr18fZmZmGs9RrVYjIiJCej/lplKpMHz4cPz0009QKpXYsGEDvL29oaf3YkTC6z7Dcj4PHB0dpenu7u5F3la6gIFDh1SrVg0VK1bE4cOH8fjxY7Rp0wYA4OjoCCcnJ5w6dQqHDx+WxiPkZmhoqHFfoVBIxy1TU1PRo0cPLFiwIM/jCvMFkHvZCoUCADSWPXPmTPTp0yfP43LGQwDQ+IDIT2pqKvT19REWFiZ9MObI/eWb3/PM/SHctGlTVK1aFRs3bsSoUaOwffv2155V8braCvKqbf6qeV63PQvzWhVm3S/78ssvceDAAXz//feoVq0aTExM0K9fvzceOKivr48DBw7g1KlTCA4OxvLly/HNN9/g9OnTcHFxQWBgIHx9fbFv3z5s2rQJU6dOxYEDB/DBBx9AT08vz5dn7sHEOcfk9+zZgwoVKmjMp1QqX1nXy69namoqRo4cCV9f3zzzVqpUCUZGRggPD8eRI0cQHBwMf39/zJgxo8BxGq9jZGSEfv36ISgoCN7e3ggKCsLHH38MAwMDqZ7GjRtrBKAcuYOvXAqzv7/u9clRmP0w9/oK87p26dIFt2/fxl9//YUDBw6gQ4cOGDNmDL7//vvX1p2fHj16QAiBPXv2oGnTpjh+/DiWLFmiUVNhPsOIgUPntGvXDkeOHMHjx48xadIkqb1169bYu3cvzpw5g1GjRhVpmY0aNcLWrVvh7Owsfei9zMjICNnZ2UWut1GjRoiIiEC1atWK/NjcGjZsiOzsbCQkJKBVq1ZvtSyVSoUNGzagYsWK0NPTQ7du3Yr0eDc3N8TGxiI2Nlbq5bh27RqSkpJQq1att6rtdQrzWhWGoaFhntfz5MmTGDx4MD788EMALz5ocw9UfBMKhQItWrRAixYt4O/vj8qVK2P79u3ST5o3bNgQDRs2hJ+fH9zd3REUFIQPPvgA5cqVw5UrVzSWdeHCBekLrFatWlAqlbhz544UvN9Uo0aNcO3atVfuowYGBvD09ISnpyemT58Oa2trHDp0CF5eXjAyMsLJkydRuXJlAC++eM+ePfvK08hVKhU6duyIq1ev4tChQ5gzZ45GPZs2bUL58uVhaWmZ7+MdHBxw+vRptG7dGgCQlZWFsLAwNGrUqMB1urm54eTJkxptJ0+eRI0aNfKE+NcpV66cNBgbeDEYNiYmpkjLyE9hX9dy5crBx8cHPj4+aNWqFSZNmoTvv/8ebm5uWLNmDZ4+fSoFmZMnT0JPTw+urq75LsvY2Bh9+vTBhg0bEBUVBVdXV43t+LrPsJzPgwcPHkihP/cg/vcJz1LRMe3atcOJEydw4cIFjTdkmzZtsGrVKjx//jzPGSqvM2bMGCQmJuKTTz7B2bNnER0djf3792PIkCHSl5KzszNOnz6NW7du4d9//33tX8s5/P39sW7dOsycORNXr17F9evXsXHjRkydOrVINdaoUQMqlQqDBg3Ctm3bEBMTgzNnzmD+/PnYs2dPkZalUqkQHh6OuXPnol+/fq/9i/hlnp6eqFu3rrScM2fOYNCgQWjTpk2xdkfnpzCvVWE4OzsjJCQEcXFxePz4MQCgevXq2LZtGy5cuICLFy9iwIABhX6d83P69GnMmzcP586dw507d7Bt2zY8fPgQbm5uiImJgZ+fH0JDQ3H79m0EBwfj5s2bcHNzAwC0b98e586dw7p163Dz5k1Mnz5dI4BYWFjgyy+/xOeff461a9ciOjoa4eHhWL58OdauXVukOqdMmYJTp05h7NixuHDhAm7evImdO3di7NixAIDdu3dj2bJluHDhAm7fvo1169ZBrVbD1dUVZmZmGDVqFCZNmoR9+/bh2rVrGD58ONLS0jBs2LAC19m6dWvY29tDpVLBxcVF4/CISqVC2bJl0atXLxw/fhwxMTE4cuQIfH19cffuXQAvzqz69ttvsWPHDty4cQOjR49+7YXFvvjiC4SEhGD27NmIjIzE2rVr8eOPP+LLL78s0vYCXrw+69evx/Hjx3H58mX4+PgUObTkpzCvq7+/P3bu3ImoqChcvXoVu3fvlvYblUoFY2Nj+Pj44MqVKzh8+DDGjRuHTz/9NN/DKTlUKhX27NmD1atXQ6VSaUx73WeYp6cnatSoAR8fH1y8eBHHjx/HN99889bb4l3EwKFj2rVrh2fPnqFatWoab6A2bdrgyZMn0umzReHo6IiTJ08iOzsbnTp1Qt26dTFhwgRYW1tLxzG//PJL6Ovro1atWihXrlyhj+t7eXlh9+7dCA4ORtOmTfHBBx9gyZIl0l+DRREYGIhBgwbhiy++gKurK3r37o2zZ8/mGa/yOtWqVUOzZs1w6dKlPB8uhaFQKLBz507Y2NigdevW8PT0RJUqVbBp06YiL6uoCvNaFcaiRYtw4MABODk5ScfGFy9eDBsbG3h4eKBHjx7w8vJ65V/Mr2NpaYljx46ha9euqFGjBqZOnYpFixahS5cuMDU1xY0bN9C3b1/UqFEDI0aMwJgxYzBy5EgAL/abadOmYfLkyWjatCmePHmCQYMGaSx/9uzZmDZtGubPnw83Nzd07twZe/bsgYuLS5HqrFevHo4ePYrIyEi0atUKDRs2hL+/v3RM3traGtu2bUP79u3h5uaGgIAA/PHHH6hduzYA4Ntvv0Xfvn3x6aefolGjRoiKisL+/fthY2NT4DoVCgU++eQTXLx4Mc8+aGpqimPHjqFSpUro06cP3NzcMGzYMKSnp0s9Hl988QU+/fRT+Pj4wN3dHRYWFlLPVEEaNWqEP//8Exs3bkSdOnXg7++PWbNmYfDgwUXaXgDg5+eHNm3aoHv37ujWrRt69+5dbNf9ed3ramRkBD8/P9SrVw+tW7eGvr4+Nm7cCODFttu/fz8SExPRtGlT9OvXDx06dMCPP/74ynW2b98etra2iIiIwIABAzSmve4zTE9PD9u3b8ezZ8/QrFkzfPbZZwWe2q/rFOJVo4iIiIiIigF7OIiIiEh2DBxEREQkOwYOIiIikh0DBxEREcmOgYOIiIhkx8BBREREsmPgICIiItkxcBAREZHsGDiIiIhIdgwcREREJDsGDiIiIpLd/wEaaSlwa2r2KwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(6, 4))\n",
    "ax = val['status_code'].value_counts().plot(kind = 'bar')\n",
    "ax.set_title(\"Validation Dataset Frequency Count\")\n",
    "ax.set_xlabel(\"Whether environmental issue resolved or unresolved\")\n",
    "ax.set_ylabel(\"Count\")\n",
    "ax.set_xticklabels(('Resolved', 'Unresolved'))\n",
    "\n",
    "for bars in ax.containers:\n",
    "    ax.bar_label(bars)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As can be seen above, there is not vast difference in distribution for each class (resolved and unresolved) in the training as well as validation dataset\n",
    "\n",
    "Below, we define the data column and label column. In our case, the input data is the 'status_txt' column and label is the 'status_code' column (0, 1 for unresolved and resolved, respecitvely)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "DATA_COLUMN = 'status_txt'\n",
    "LABEL_COLUMN = 'status_code'\n",
    "# label_list is the list of labels, i.e. True, False or 0, 1 or 'dog', 'cat'\n",
    "label_list = [0,1] ## If we were doing multi class classification, we will just need to add more labels to this list here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Preprocessing\n",
    "\n",
    "We'll need to transform our data into a format BERT understands. This involves two steps. First, we create `InputExample` using the constructor provided in the BERT library.\n",
    "\n",
    "- `text_a` is the text we want to classify, which in this case, is the `status_txt` field in our Dataframe.\n",
    "- `text_b` is used if we're training a model to understand the relationship between sentences or question answering task (i.e. is `text_b` a translation of `text_a`? Is `text_b` an answer to the question asked by `text_a`?). This doesn't apply to our task, so we can leave `text_b` blank.\n",
    "- `label` is the label for our example, i.e. True, False"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use the InputExample class from BERT's run_classifier code to create examples from the data\n",
    "train_InputExamples = train.apply(lambda x: bert.run_classifier.InputExample(guid=None,\n",
    "                                                                   text_a = x[DATA_COLUMN], \n",
    "                                                                   text_b = None, \n",
    "                                                                   label = x[LABEL_COLUMN]), axis = 1) ## text_b is NONE since this is classification problem\n",
    "\n",
    "dev_InputExamples = val.apply(lambda x: bert.run_classifier.InputExample(guid=None, \n",
    "                                                                   text_a = x[DATA_COLUMN], \n",
    "                                                                   text_b = None, \n",
    "                                                                   label = x[LABEL_COLUMN]), axis = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we need to preprocess our data so that it matches the data BERT was trained on. For this, we'll need to do a couple of things (but don't worry--this is also included in the Python library):\n",
    "\n",
    "1. Lowercase our text (if we're using a BERT lowercase model)\n",
    "2. Tokenize it (i.e. \"sally says hi\" -> [\"sally\", \"says\", \"hi\"])\n",
    "3. Break words into WordPieces (i.e. \"calling\" -> [\"call\", \"##ing\"])\n",
    "4. Map our words to indexes using a vocab file that BERT provides\n",
    "5. Add special \"CLS\" and \"SEP\" tokens (see the readme)\n",
    "6. Append \"index\" and \"segment\" tokens to each input (see the BERT paper)\n",
    "\n",
    "However, we don't have to worry about most of these details.\n",
    "\n",
    "To start, we'll need to load a vocabulary file and lowercasing information directly from the BERT tf hub module:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download a Pre-trained BERT Model from TensorFlow Hub and Preprocess & Tokenize Text for BERT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saver not created because there are no variables in the graph to restore\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saver not created because there are no variables in the graph to restore\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\568325542.py:9: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\568325542.py:9: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\bert\\tokenization.py:125: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\bert\\tokenization.py:125: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# This is a path to an uncased (all lowercase) version of BERT. It uses L = 12 hidden layers (i.e., transformer blocks), a hidden size of H = 768, and A = 12 attention heads\n",
    "BERT_MODEL_HUB = \"https://tfhub.dev/google/bert_uncased_L-12_H-768_A-12/1\"\n",
    "\n",
    "def create_tokenizer_from_hub_module():\n",
    "  \"\"\"Get the vocab file and casing info from the Hub module.\"\"\"\n",
    "  with tf.Graph().as_default():\n",
    "    bert_module = hub.Module(BERT_MODEL_HUB)\n",
    "    tokenization_info = bert_module(signature=\"tokenization_info\", as_dict=True)\n",
    "    with tf.Session() as sess:\n",
    "      vocab_file, do_lower_case = sess.run([tokenization_info[\"vocab_file\"],\n",
    "                                            tokenization_info[\"do_lower_case\"]])\n",
    "      \n",
    "  return bert.tokenization.FullTokenizer(\n",
    "      vocab_file=vocab_file, do_lower_case=do_lower_case) ## this means we are working with uncased version\n",
    "\n",
    "tokenizer = create_tokenizer_from_hub_module()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As can be seen below that the BERT model we're using expects lowercase data (that's what stored in tokenization_info[\"do_lower_case\"]) and we also loaded BERT's vocab file. We also created a tokenizer, which breaks words into word pieces:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['this',\n",
       " 'here',\n",
       " \"'\",\n",
       " 's',\n",
       " 'an',\n",
       " 'example',\n",
       " 'of',\n",
       " 'using',\n",
       " 'the',\n",
       " 'bert',\n",
       " 'token',\n",
       " '##izer']"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.tokenize(\"This here's an example of using the BERT tokenizer\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how the first input example looks like after pre-processing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['monitor', 'in', '2019', '.']\n"
     ]
    }
   ],
   "source": [
    "print(tokenizer.tokenize(train_InputExamples.iloc[0].text_a))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using our tokenizer, we'll call `run_classifier.convert_examples_to_features` on our InputExamples to convert them into features BERT understands. What this function does is that it convert each token into a unique input ids (this is just indices in vocabulary corresponding to a particular token), prepend [CLS] and append [SEP] tokens, generate input mask with 1's for each token and padded 0's until max seq length size of 128 is reached and segment ids which is 0 in this case since we are working on text classification problem and not next sentence prediction. All (input ids, input mask and segment ids) are of length size 128 since we are using `MAX_SEQ_LENGTH` of 128 below.\n",
    "\n",
    "**NOTE:** You could try with different `MAX_SEQ_LENGTH` in case you are not constrained with computing power"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\bert\\run_classifier.py:774: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\bert\\run_classifier.py:774: The name tf.logging.info is deprecated. Please use tf.compat.v1.logging.info instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Writing example 0 of 1783\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Writing example 0 of 1783\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] monitor in 2019 . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] monitor in 2019 . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 8080 1999 10476 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 8080 1999 10476 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] monitor in 2019 . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] monitor in 2019 . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 8080 1999 10476 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 8080 1999 10476 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] none . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] none . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 3904 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 3904 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 1 (id = 1)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 1 (id = 1)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] none . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] none . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 3904 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 3904 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 1 (id = 1)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 1 (id = 1)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] none . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] none . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 3904 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 3904 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 1 (id = 1)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 1 (id = 1)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Writing example 0 of 203\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Writing example 0 of 203\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] monitor in 2018 . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] monitor in 2018 . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 8080 1999 2760 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 8080 1999 2760 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] monitor in 2018 and implement weed control as required . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] monitor in 2018 and implement weed control as required . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 8080 1999 2760 1998 10408 17901 2491 2004 3223 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 8080 1999 2760 1998 10408 17901 2491 2004 3223 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] remove erosion log . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] remove erosion log . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 6366 14173 8833 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 6366 14173 8833 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] monitor in 2018 and implement weed control as required . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] monitor in 2018 and implement weed control as required . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 8080 1999 2760 1998 10408 17901 2491 2004 3223 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 8080 1999 2760 1998 10408 17901 2491 2004 3223 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: None\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] remove sediment fence . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] remove sediment fence . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 6366 19671 8638 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 6366 19671 8638 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    }
   ],
   "source": [
    "#We'll set sequences to be at most 128 tokens long due to computational constraints\n",
    "MAX_SEQ_LENGTH = 128\n",
    "# Convert our train and test features to InputFeatures that BERT understands.\n",
    "train_features = bert.run_classifier.convert_examples_to_features(train_InputExamples, label_list, MAX_SEQ_LENGTH, tokenizer)\n",
    "test_features = bert.run_classifier.convert_examples_to_features(dev_InputExamples, label_list, MAX_SEQ_LENGTH, tokenizer)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sentence :  Monitor in 2019.\n",
      "------------------------------\n",
      "Tokens :  ['monitor', 'in', '2019', '.']\n",
      "------------------------------\n",
      "Input IDs :  [101, 8080, 1999, 10476, 1012, 102, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]\n",
      "------------------------------\n",
      "Input Masks :  [1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]\n",
      "------------------------------\n",
      "Segment IDs :  [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]\n"
     ]
    }
   ],
   "source": [
    "#Example on first observation in the training set\n",
    "print(\"Sentence : \", train_InputExamples.iloc[0].text_a)\n",
    "print(\"-\"*30)\n",
    "print(\"Tokens : \", tokenizer.tokenize(train_InputExamples.iloc[0].text_a))\n",
    "print(\"-\"*30)\n",
    "print(\"Input IDs : \", train_features[0].input_ids)\n",
    "print(\"-\"*30)\n",
    "print(\"Input Masks : \", train_features[0].input_mask)\n",
    "print(\"-\"*30)\n",
    "print(\"Segment IDs : \", train_features[0].segment_ids)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a model\n",
    "\n",
    "Now that we've prepared our data, let's focus on building a model. \n",
    "\n",
    "First, we load the BERT tf hub module again (this time to extract the computation graph). Next, it creates a single new layer that will be trained to adapt BERT to our task (i.e. classifying whether a text means the issue was unresolved or resolved)--> fine-tuning "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_model(is_predicting, input_ids, input_mask, segment_ids, labels,\n",
    "                 num_labels):\n",
    "  \"\"\"Creates a classification model.\"\"\"\n",
    "\n",
    "  bert_module = hub.Module(\n",
    "      BERT_MODEL_HUB,\n",
    "      trainable=True)\n",
    "  bert_inputs = dict(\n",
    "      input_ids=input_ids,\n",
    "      input_mask=input_mask,\n",
    "      segment_ids=segment_ids)\n",
    "  bert_outputs = bert_module(\n",
    "      inputs=bert_inputs,\n",
    "      signature=\"tokens\",\n",
    "      as_dict=True)\n",
    "\n",
    "  # Use \"pooled_output\" for classification tasks on an entire sentence. pooled_output will have shape of (batch_size, 768) with representations for entire\n",
    "  # input sentences i.e. contextualized sequence embeddings\n",
    "  \n",
    "  # Use \"sequence_outputs\" for token-level output. sequence_outputs will have shape of (batch_size, MAX_SEQ_LENGTH, 768) with representations for each input token\n",
    "  output_layer = bert_outputs[\"pooled_output\"]  \n",
    "\n",
    "  hidden_size = output_layer.shape[-1].value\n",
    "\n",
    "  # Create our own layer to tune for your data.\n",
    "  output_weights = tf.get_variable(\n",
    "      \"output_weights\", [num_labels, hidden_size],\n",
    "      initializer=tf.truncated_normal_initializer(stddev=0.02))\n",
    "\n",
    "  output_bias = tf.get_variable(\n",
    "      \"output_bias\", [num_labels], initializer=tf.zeros_initializer())\n",
    "\n",
    "  with tf.variable_scope(\"loss\"):\n",
    "\n",
    "    # Dropout helps prevent overfitting\n",
    "    output_layer = tf.nn.dropout(output_layer, keep_prob=0.9)\n",
    "\n",
    "    logits = tf.matmul(output_layer, output_weights, transpose_b=True)\n",
    "    logits = tf.nn.bias_add(logits, output_bias)\n",
    "    log_probs = tf.nn.log_softmax(logits, axis=-1)\n",
    "\n",
    "    # Convert labels into one-hot encoding\n",
    "    one_hot_labels = tf.one_hot(labels, depth=num_labels, dtype=tf.float32)\n",
    "\n",
    "    predicted_labels = tf.squeeze(tf.argmax(log_probs, axis=-1, output_type=tf.int32))\n",
    "    # If we're predicting, we want predicted labels and the probabiltiies.\n",
    "    if is_predicting:\n",
    "      return (predicted_labels, log_probs)\n",
    "\n",
    "    # If we're train/eval, compute loss between predicted and actual label\n",
    "    per_example_loss = -tf.reduce_sum(one_hot_labels * log_probs, axis=-1)\n",
    "    loss = tf.reduce_mean(per_example_loss)\n",
    "    return (loss, predicted_labels, log_probs)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we'll wrap our model function in a `model_fn_builder` function that adapts our model to work for training, evaluation, and prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# model_fn_builder actually creates our model function\n",
    "# using the passed parameters for num_labels, learning_rate, etc.\n",
    "def model_fn_builder(num_labels, learning_rate, num_train_steps,\n",
    "                     num_warmup_steps):\n",
    "  \"\"\"Returns `model_fn` closure for TPUEstimator.\"\"\"\n",
    "  def model_fn(features, labels, mode, params):  # pylint: disable=unused-argument\n",
    "    \"\"\"The `model_fn` for TPUEstimator.\"\"\"\n",
    "\n",
    "    input_ids = features[\"input_ids\"]\n",
    "    input_mask = features[\"input_mask\"]\n",
    "    segment_ids = features[\"segment_ids\"]\n",
    "    label_ids = features[\"label_ids\"]\n",
    "\n",
    "    is_predicting = (mode == tf.estimator.ModeKeys.PREDICT)\n",
    "    \n",
    "    # TRAIN and EVAL\n",
    "    if not is_predicting:\n",
    "\n",
    "      (loss, predicted_labels, log_probs) = create_model(\n",
    "        is_predicting, input_ids, input_mask, segment_ids, label_ids, num_labels)\n",
    "\n",
    "      train_op = bert.optimization.create_optimizer(\n",
    "          loss, learning_rate, num_train_steps, num_warmup_steps, use_tpu=False)\n",
    "\n",
    "      # Calculate evaluation metrics. \n",
    "      def metric_fn(label_ids, predicted_labels):\n",
    "        accuracy = tf.metrics.accuracy(label_ids, predicted_labels)\n",
    "        f1_score = tf.contrib.metrics.f1_score(\n",
    "            label_ids,\n",
    "            predicted_labels)\n",
    "        auc = tf.metrics.auc(\n",
    "            label_ids,\n",
    "            predicted_labels)\n",
    "        true_pos = tf.metrics.true_positives(\n",
    "            label_ids,\n",
    "            predicted_labels)\n",
    "        true_neg = tf.metrics.true_negatives(\n",
    "            label_ids,\n",
    "            predicted_labels)   \n",
    "        false_pos = tf.metrics.false_positives(\n",
    "            label_ids,\n",
    "            predicted_labels)  \n",
    "        false_neg = tf.metrics.false_negatives(\n",
    "            label_ids,\n",
    "            predicted_labels)\n",
    "    \n",
    "        return {\n",
    "            \"eval_accuracy\": accuracy, # free to change these metrics\n",
    "            \"f1_score\": f1_score,\n",
    "            \"auc\": auc,\n",
    "            \"true_positives\": true_pos,\n",
    "            \"true_negatives\": true_neg,\n",
    "            \"false_positives\": false_pos,\n",
    "            \"false_negatives\": false_neg \n",
    "        }\n",
    "\n",
    "      eval_metrics = metric_fn(label_ids, predicted_labels)\n",
    "\n",
    "      if mode == tf.estimator.ModeKeys.TRAIN:\n",
    "        return tf.estimator.EstimatorSpec(mode=mode,\n",
    "          loss=loss,\n",
    "          train_op=train_op)\n",
    "      else:\n",
    "          return tf.estimator.EstimatorSpec(mode=mode,\n",
    "            loss=loss,\n",
    "            eval_metric_ops=eval_metrics)\n",
    "    else:\n",
    "      (predicted_labels, log_probs) = create_model(\n",
    "        is_predicting, input_ids, input_mask, segment_ids, label_ids, num_labels)\n",
    "\n",
    "      predictions = {\n",
    "          'probabilities': log_probs,\n",
    "          'labels': predicted_labels\n",
    "      }\n",
    "      return tf.estimator.EstimatorSpec(mode, predictions=predictions)\n",
    "\n",
    "  # Return the actual model function in the closure\n",
    "  return model_fn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compute train and warmup steps from batch size\n",
    "# These hyperparameters are copied from this colab notebook (https://colab.sandbox.google.com/github/tensorflow/tpu/blob/master/tools/colab/bert_finetuning_with_cloud_tpus.ipynb)\n",
    "BATCH_SIZE = 32\n",
    "LEARNING_RATE = 2e-5\n",
    "NUM_TRAIN_EPOCHS = 3.0\n",
    "# Warmup is a period of time where hte learning rate \n",
    "# is small and gradually increases--usually helps training.\n",
    "WARMUP_PROPORTION = 0.1\n",
    "# Model configs\n",
    "SAVE_CHECKPOINTS_STEPS = 50 # original: 500\n",
    "SAVE_SUMMARY_STEPS = 10 # original: 100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compute # train and warmup steps from batch size\n",
    "num_train_steps = int(len(train_features) / BATCH_SIZE * NUM_TRAIN_EPOCHS)\n",
    "num_warmup_steps = int(num_train_steps * WARMUP_PROPORTION)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Specify output directory and number of checkpoint steps to save\n",
    "run_config = tf.estimator.RunConfig(\n",
    "    model_dir=OUTPUT_DIR,\n",
    "    save_summary_steps=SAVE_SUMMARY_STEPS,\n",
    "    save_checkpoints_steps=SAVE_CHECKPOINTS_STEPS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Using config: {'_model_dir': './output', '_tf_random_seed': None, '_save_summary_steps': 10, '_save_checkpoints_steps': 50, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true\n",
      "graph_options {\n",
      "  rewrite_options {\n",
      "    meta_optimizer_iterations: ONE\n",
      "  }\n",
      "}\n",
      ", '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x000001FEEAF41888>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Using config: {'_model_dir': './output', '_tf_random_seed': None, '_save_summary_steps': 10, '_save_checkpoints_steps': 50, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true\n",
      "graph_options {\n",
      "  rewrite_options {\n",
      "    meta_optimizer_iterations: ONE\n",
      "  }\n",
      "}\n",
      ", '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x000001FEEAF41888>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n"
     ]
    }
   ],
   "source": [
    "model_fn = model_fn_builder(\n",
    "  num_labels=len(label_list),\n",
    "  learning_rate=LEARNING_RATE,\n",
    "  num_train_steps=num_train_steps,\n",
    "  num_warmup_steps=num_warmup_steps)\n",
    "\n",
    "estimator = tf.estimator.Estimator(\n",
    "  model_fn=model_fn,\n",
    "  config=run_config,\n",
    "  params={\"batch_size\": BATCH_SIZE})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the model\n",
    "First, we create an input builder function that takes our training feature set (train_features) and produces a generator. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create an input function for training. drop_remainder = True for using TPUs.\n",
    "train_input_fn = bert.run_classifier.input_fn_builder(\n",
    "    features=train_features,\n",
    "    seq_length=MAX_SEQ_LENGTH,\n",
    "    is_training=True,\n",
    "    drop_remainder=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we begin training "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Beginning Training!\n",
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\tensorflow_core\\python\\training\\training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\tensorflow_core\\python\\training\\training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Calling model_fn.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Calling model_fn.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saver not created because there are no variables in the graph to restore\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saver not created because there are no variables in the graph to restore\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\1603645112.py:24: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\1603645112.py:24: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\1603645112.py:31: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\1603645112.py:31: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\1603645112.py:34: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\1603645112.py:34: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\bert\\optimization.py:27: The name tf.train.get_or_create_global_step is deprecated. Please use tf.compat.v1.train.get_or_create_global_step instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\bert\\optimization.py:27: The name tf.train.get_or_create_global_step is deprecated. Please use tf.compat.v1.train.get_or_create_global_step instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\bert\\optimization.py:32: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\bert\\optimization.py:32: The name tf.train.polynomial_decay is deprecated. Please use tf.compat.v1.train.polynomial_decay instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\bert\\optimization.py:70: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\bert\\optimization.py:70: The name tf.trainable_variables is deprecated. Please use tf.compat.v1.trainable_variables instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\tensorflow_core\\python\\ops\\math_grad.py:1375: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\tensorflow_core\\python\\ops\\math_grad.py:1375: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
      "c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\tensorflow_core\\python\\framework\\indexed_slices.py:424: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.\n",
      "  \"Converting sparse IndexedSlices to a dense Tensor of unknown shape. \"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\3405304385.py:27: The name tf.metrics.accuracy is deprecated. Please use tf.compat.v1.metrics.accuracy instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\3405304385.py:27: The name tf.metrics.accuracy is deprecated. Please use tf.compat.v1.metrics.accuracy instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:\n",
      "The TensorFlow contrib module will not be included in TensorFlow 2.0.\n",
      "For more information, please see:\n",
      "  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md\n",
      "  * https://github.com/tensorflow/addons\n",
      "  * https://github.com/tensorflow/io (for I/O related ops)\n",
      "If you depend on functionality not listed there, please file an issue.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:\n",
      "The TensorFlow contrib module will not be included in TensorFlow 2.0.\n",
      "For more information, please see:\n",
      "  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md\n",
      "  * https://github.com/tensorflow/addons\n",
      "  * https://github.com/tensorflow/io (for I/O related ops)\n",
      "If you depend on functionality not listed there, please file an issue.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\tensorflow_core\\contrib\\metrics\\python\\metrics\\classification.py:162: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Deprecated in favor of operator or tf.math.divide.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\tensorflow_core\\contrib\\metrics\\python\\metrics\\classification.py:162: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Deprecated in favor of operator or tf.math.divide.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\3405304385.py:31: The name tf.metrics.auc is deprecated. Please use tf.compat.v1.metrics.auc instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\3405304385.py:31: The name tf.metrics.auc is deprecated. Please use tf.compat.v1.metrics.auc instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\3405304385.py:34: The name tf.metrics.true_positives is deprecated. Please use tf.compat.v1.metrics.true_positives instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\3405304385.py:34: The name tf.metrics.true_positives is deprecated. Please use tf.compat.v1.metrics.true_positives instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\3405304385.py:37: The name tf.metrics.true_negatives is deprecated. Please use tf.compat.v1.metrics.true_negatives instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\3405304385.py:37: The name tf.metrics.true_negatives is deprecated. Please use tf.compat.v1.metrics.true_negatives instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\3405304385.py:40: The name tf.metrics.false_positives is deprecated. Please use tf.compat.v1.metrics.false_positives instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\3405304385.py:40: The name tf.metrics.false_positives is deprecated. Please use tf.compat.v1.metrics.false_positives instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\3405304385.py:43: The name tf.metrics.false_negatives is deprecated. Please use tf.compat.v1.metrics.false_negatives instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\t1nipun\\AppData\\Local\\Temp\\ipykernel_22640\\3405304385.py:43: The name tf.metrics.false_negatives is deprecated. Please use tf.compat.v1.metrics.false_negatives instead.\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Done calling model_fn.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Done calling model_fn.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Create CheckpointSaverHook.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Create CheckpointSaverHook.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Graph was finalized.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Graph was finalized.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Running local_init_op.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Running local_init_op.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Done running local_init_op.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Done running local_init_op.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving checkpoints for 0 into ./output\\model.ckpt.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving checkpoints for 0 into ./output\\model.ckpt.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.6484681, step = 1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.6484681, step = 1\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 3 vs previous value: 3. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 3 vs previous value: 3. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 9 vs previous value: 9. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 9 vs previous value: 9. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 11 vs previous value: 11. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 11 vs previous value: 11. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 15 vs previous value: 15. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 15 vs previous value: 15. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 17 vs previous value: 17. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 17 vs previous value: 17. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving checkpoints for 50 into ./output\\model.ckpt.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving checkpoints for 50 into ./output\\model.ckpt.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving checkpoints for 100 into ./output\\model.ckpt.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving checkpoints for 100 into ./output\\model.ckpt.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 0.0473643\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:global_step/sec: 0.0473643\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.013606116, step = 101 (2111.297 sec)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:loss = 0.013606116, step = 101 (2111.297 sec)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving checkpoints for 150 into ./output\\model.ckpt.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving checkpoints for 150 into ./output\\model.ckpt.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving checkpoints for 167 into ./output\\model.ckpt.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving checkpoints for 167 into ./output\\model.ckpt.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Loss for final step: 0.0042998344.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Loss for final step: 0.0042998344.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training took time  1:02:37.682604\n"
     ]
    }
   ],
   "source": [
    "print(f'Beginning Training!')\n",
    "current_time = datetime.now()\n",
    "estimator.train(input_fn=train_input_fn, max_steps=num_train_steps)\n",
    "print(\"Training took time \", datetime.now() - current_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate the BERT Text Classification Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's use our test data to see how well our model did:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_input_fn = run_classifier.input_fn_builder(\n",
    "    features=test_features,\n",
    "    seq_length=MAX_SEQ_LENGTH,\n",
    "    is_training=False,\n",
    "    drop_remainder=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Calling model_fn.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Calling model_fn.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saver not created because there are no variables in the graph to restore\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saver not created because there are no variables in the graph to restore\n",
      "c:\\Users\\t1nipun\\AppData\\Local\\Continuum\\anaconda3\\envs\\bertft\\lib\\site-packages\\tensorflow_core\\python\\framework\\indexed_slices.py:424: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.\n",
      "  \"Converting sparse IndexedSlices to a dense Tensor of unknown shape. \"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Done calling model_fn.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Done calling model_fn.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Starting evaluation at 2022-09-02T18:43:34Z\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Starting evaluation at 2022-09-02T18:43:34Z\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Graph was finalized.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Graph was finalized.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./output\\model.ckpt-167\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./output\\model.ckpt-167\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Running local_init_op.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Running local_init_op.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Done running local_init_op.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Done running local_init_op.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Finished evaluation at 2022-09-02-18:44:29\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Finished evaluation at 2022-09-02-18:44:29\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving dict for global step 167: auc = 0.8667996, eval_accuracy = 0.8719212, f1_score = 0.84523803, false_negatives = 14.0, false_positives = 12.0, global_step = 167, loss = 0.37102348, true_negatives = 106.0, true_positives = 71.0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving dict for global step 167: auc = 0.8667996, eval_accuracy = 0.8719212, f1_score = 0.84523803, false_negatives = 14.0, false_positives = 12.0, global_step = 167, loss = 0.37102348, true_negatives = 106.0, true_positives = 71.0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 167: ./output\\model.ckpt-167\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 167: ./output\\model.ckpt-167\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'auc': 0.8667996,\n",
       " 'eval_accuracy': 0.8719212,\n",
       " 'f1_score': 0.84523803,\n",
       " 'false_negatives': 14.0,\n",
       " 'false_positives': 12.0,\n",
       " 'loss': 0.37102348,\n",
       " 'true_negatives': 106.0,\n",
       " 'true_positives': 71.0,\n",
       " 'global_step': 167}"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "estimator.evaluate(input_fn=test_input_fn, steps=None) # around 2 min"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we can see the evaluation accuracy is pretty high. And the loss is small enough. \n",
    "\n",
    "### Prediction \n",
    "\n",
    "Now we can use this model to predict on our unlabelled dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "# A method to get predictions\n",
    "def prediction(sent):\n",
    "  #A list to map the actual labels to the predictions\n",
    "  labels = [\"Unresolved\", \"Resolved\"]\n",
    "\n",
    "  #Transforming the test data into BERT accepted form\n",
    "  input_examples = [run_classifier.InputExample(guid=\"\", text_a = x, text_b = None, label = 0) for x in sent] \n",
    "  \n",
    "  #Creating input features for Test data\n",
    "  input_features = run_classifier.convert_examples_to_features(input_examples, label_list, MAX_SEQ_LENGTH, tokenizer)\n",
    "\n",
    "  #Predicting the classes \n",
    "  predict_input_fn = run_classifier.input_fn_builder(features=input_features, seq_length=MAX_SEQ_LENGTH, is_training=False, drop_remainder=False)\n",
    "  predictions = estimator.predict(predict_input_fn)\n",
    "  return [(sentence, np.exp(prediction['probabilities']),prediction['labels'], labels[prediction['labels']]) for sentence, prediction in zip(sent, predictions)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "sentences = list(test['status_txt'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Writing example 0 of 9724\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Writing example 0 of 9724\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] june 22 , 2016 – intermittent weeds observed , recommend implementation of control measures . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] june 22 , 2016 – intermittent weeds observed , recommend implementation of control measures . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 2238 2570 1010 2355 1516 23852 20777 5159 1010 16755 7375 1997 2491 5761 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 2238 2570 1010 2355 1516 23852 20777 5159 1010 16755 7375 1997 2491 5761 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] july 13 , 2016 - vegetation continues to establish with desirable species throughout row . diverse mix of vegetation observed . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] july 13 , 2016 - vegetation continues to establish with desirable species throughout row . diverse mix of vegetation observed . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 2251 2410 1010 2355 1011 10072 4247 2000 5323 2007 16166 2427 2802 5216 1012 7578 4666 1997 10072 5159 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 2251 2410 1010 2355 1011 10072 4247 2000 5323 2007 16166 2427 2802 5216 1012 7578 4666 1997 10072 5159 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] july 13 , 2016 - cross drains are ve ##get ##ated and functioning . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] july 13 , 2016 - cross drains are ve ##get ##ated and functioning . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 2251 2410 1010 2355 1011 2892 18916 2024 2310 18150 4383 1998 12285 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 2251 2410 1010 2355 1011 2892 18916 2024 2310 18150 4383 1998 12285 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] july 13 , 2016 - wetlands stable and consistent with previous assessments . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] july 13 , 2016 - wetlands stable and consistent with previous assessments . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 2251 2410 1010 2355 1011 16630 6540 1998 8335 2007 3025 20794 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 2251 2410 1010 2355 1011 16630 6540 1998 8335 2007 3025 20794 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:*** Example ***\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:guid: \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] july 13 , 2016 - no erosion or sub ##side ##nce issues . [SEP]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:tokens: [CLS] july 13 , 2016 - no erosion or sub ##side ##nce issues . [SEP]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 2251 2410 1010 2355 1011 2053 14173 2030 4942 7363 5897 3314 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_ids: 101 2251 2410 1010 2355 1011 2053 14173 2030 4942 7363 5897 3314 1012 102 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:input_mask: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:segment_ids: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:label: 0 (id = 0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Calling model_fn.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Calling model_fn.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saver not created because there are no variables in the graph to restore\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Saver not created because there are no variables in the graph to restore\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Done calling model_fn.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Done calling model_fn.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Graph was finalized.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Graph was finalized.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./output\\model.ckpt-167\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./output\\model.ckpt-167\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Running local_init_op.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Running local_init_op.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Done running local_init_op.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Done running local_init_op.\n"
     ]
    }
   ],
   "source": [
    "pred = prediction(sentences) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('July 13, 2016   - Herbicide application conducted on valve pad and 2m perimeter.',\n",
       "  array([0.5679791, 0.4320209], dtype=float32),\n",
       "  0,\n",
       "  'Unresolved'),\n",
       " ('July 13, 2016   - Was not able to locate or identify any Dalmatian toadflax, 2015 issue resolved.',\n",
       "  array([0.01844416, 0.9815559 ], dtype=float32),\n",
       "  1,\n",
       "  'Resolved'),\n",
       " ('Sept 17, 2016 - 65-75% vegetation established compared to off ROW.',\n",
       "  array([0.13170545, 0.86829454], dtype=float32),\n",
       "  1,\n",
       "  'Resolved'),\n",
       " ('Sept 17, 2016  - Installed TC sign and marker posts at KP 0+040.',\n",
       "  array([0.58272207, 0.4172779 ], dtype=float32),\n",
       "  0,\n",
       "  'Unresolved'),\n",
       " ('Sept 17, 2016  - Re-installed marker posts at KP 0+300, issue resolved.',\n",
       "  array([0.06651572, 0.93348426], dtype=float32),\n",
       "  1,\n",
       "  'Resolved'),\n",
       " ('Sept 17, 2016  - Intermittent herbicide application conducted to control perennial sow thistle.',\n",
       "  array([0.44655612, 0.55344385], dtype=float32),\n",
       "  1,\n",
       "  'Resolved'),\n",
       " ('.', array([0.00109654, 0.9989034 ], dtype=float32), 1, 'Resolved'),\n",
       " ('Aug 23, 2016 - Species not observed at the pre-disturbance location. Habitat was altered by the construction of an adjacent foreign pipeline to the south of the ROW over the winter of 2014/2015.',\n",
       "  array([0.01325186, 0.9867481 ], dtype=float32),\n",
       "  1,\n",
       "  'Resolved'),\n",
       " ('Aug 23, 2016  - Species re-establishment is not considered likely, as conditions are no longer favorable to the presence of Reindeer lichen both on and off ROW. Issue resolved.',\n",
       "  array([0.00238181, 0.99761814], dtype=float32),\n",
       "  1,\n",
       "  'Resolved'),\n",
       " ('June 22, 2016 – Intermittent weeds observed, recommend implementation of control measures.',\n",
       "  array([0.95913225, 0.04086778], dtype=float32),\n",
       "  0,\n",
       "  'Unresolved'),\n",
       " ('July 13, 2016 - Vegetation continues to establish with desirable species throughout ROW. Diverse mix of vegetation observed.',\n",
       "  array([0.02238841, 0.97761154], dtype=float32),\n",
       "  1,\n",
       "  'Resolved'),\n",
       " ('July 13, 2016  - Cross drains are vegetated and functioning.',\n",
       "  array([0.01765602, 0.98234403], dtype=float32),\n",
       "  1,\n",
       "  'Resolved'),\n",
       " ('July 13, 2016   - Wetlands stable and consistent with previous assessments.',\n",
       "  array([0.57793415, 0.42206588], dtype=float32),\n",
       "  0,\n",
       "  'Unresolved'),\n",
       " ('July 13, 2016  - No erosion or subsidence issues.',\n",
       "  array([0.00327829, 0.9967217 ], dtype=float32),\n",
       "  1,\n",
       "  'Resolved'),\n",
       " ('July 13, 2016  - Intermittent herbicide application to control scentless chamomile and perennial sow thistle.',\n",
       "  array([0.5834243, 0.4165757], dtype=float32),\n",
       "  0,\n",
       "  'Unresolved'),\n",
       " ('July 13, 2016   - Herbicide application conducted on valve pad and 2m perimeter.',\n",
       "  array([0.6702178 , 0.32978213], dtype=float32),\n",
       "  0,\n",
       "  'Unresolved'),\n",
       " ('Sept 17, 2016 - 65-75% vegetation established compared to off ROW.',\n",
       "  array([0.19534285, 0.80465716], dtype=float32),\n",
       "  1,\n",
       "  'Resolved'),\n",
       " ('Sept 17, 2016  - Intermittent herbicide application conducted to control perennial sow thistle.',\n",
       "  array([0.44551834, 0.55448157], dtype=float32),\n",
       "  1,\n",
       "  'Resolved'),\n",
       " ('June 22, 2016 – Intermittent weeds observed, recommend implementation of control measures.',\n",
       "  array([0.9557093 , 0.04429072], dtype=float32),\n",
       "  0,\n",
       "  'Unresolved'),\n",
       " ('July 13, 2016 - Vegetation continues to establish with desirable species throughout ROW. Diverse mix of vegetation observed.',\n",
       "  array([0.0368394 , 0.96316063], dtype=float32),\n",
       "  1,\n",
       "  'Resolved')]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pred[20:40]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentence</th>\n",
       "      <th>probabilities</th>\n",
       "      <th>status_code_predicted</th>\n",
       "      <th>status_label_predicted</th>\n",
       "      <th>unresolved_probability</th>\n",
       "      <th>resolved_probability</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>June 22, 2016 – Intermittent weeds observed, r...</td>\n",
       "      <td>[0.9271113, 0.0728887]</td>\n",
       "      <td>0</td>\n",
       "      <td>Unresolved</td>\n",
       "      <td>0.927111</td>\n",
       "      <td>0.072889</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>July 13, 2016 - Vegetation continues to establ...</td>\n",
       "      <td>[0.025408609, 0.9745914]</td>\n",
       "      <td>1</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>0.025409</td>\n",
       "      <td>0.974591</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>July 13, 2016  - Cross drains are vegetated an...</td>\n",
       "      <td>[0.018914385, 0.98108566]</td>\n",
       "      <td>1</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>0.018914</td>\n",
       "      <td>0.981086</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>July 13, 2016   - Wetlands stable and consiste...</td>\n",
       "      <td>[0.59851533, 0.40148464]</td>\n",
       "      <td>0</td>\n",
       "      <td>Unresolved</td>\n",
       "      <td>0.598515</td>\n",
       "      <td>0.401485</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>July 13, 2016  - No erosion or subsidence issues.</td>\n",
       "      <td>[0.0044781854, 0.9955218]</td>\n",
       "      <td>1</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>0.004478</td>\n",
       "      <td>0.995522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9719</th>\n",
       "      <td>Aug 18, 2017 - Vegetation established at 65-80...</td>\n",
       "      <td>[0.0058666775, 0.9941333]</td>\n",
       "      <td>1</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>0.005867</td>\n",
       "      <td>0.994133</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9720</th>\n",
       "      <td>Aug 18, 2017  – Intermittent perennial sow thi...</td>\n",
       "      <td>[0.87230706, 0.1276929]</td>\n",
       "      <td>0</td>\n",
       "      <td>Unresolved</td>\n",
       "      <td>0.872307</td>\n",
       "      <td>0.127693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9721</th>\n",
       "      <td>Aug 23, 2017 – Intermittent herbicide applicat...</td>\n",
       "      <td>[0.025675168, 0.97432476]</td>\n",
       "      <td>1</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>0.025675</td>\n",
       "      <td>0.974325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9722</th>\n",
       "      <td>.</td>\n",
       "      <td>[0.001122542, 0.99887747]</td>\n",
       "      <td>1</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>0.001123</td>\n",
       "      <td>0.998877</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9723</th>\n",
       "      <td>Aug 18, 2017 - Rollback Access Control is pres...</td>\n",
       "      <td>[0.0937371, 0.90626293]</td>\n",
       "      <td>1</td>\n",
       "      <td>Resolved</td>\n",
       "      <td>0.093737</td>\n",
       "      <td>0.906263</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>9724 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               sentence  \\\n",
       "0     June 22, 2016 – Intermittent weeds observed, r...   \n",
       "1     July 13, 2016 - Vegetation continues to establ...   \n",
       "2     July 13, 2016  - Cross drains are vegetated an...   \n",
       "3     July 13, 2016   - Wetlands stable and consiste...   \n",
       "4     July 13, 2016  - No erosion or subsidence issues.   \n",
       "...                                                 ...   \n",
       "9719  Aug 18, 2017 - Vegetation established at 65-80...   \n",
       "9720  Aug 18, 2017  – Intermittent perennial sow thi...   \n",
       "9721  Aug 23, 2017 – Intermittent herbicide applicat...   \n",
       "9722                                                  .   \n",
       "9723  Aug 18, 2017 - Rollback Access Control is pres...   \n",
       "\n",
       "                  probabilities  status_code_predicted status_label_predicted  \\\n",
       "0        [0.9271113, 0.0728887]                      0             Unresolved   \n",
       "1      [0.025408609, 0.9745914]                      1               Resolved   \n",
       "2     [0.018914385, 0.98108566]                      1               Resolved   \n",
       "3      [0.59851533, 0.40148464]                      0             Unresolved   \n",
       "4     [0.0044781854, 0.9955218]                      1               Resolved   \n",
       "...                         ...                    ...                    ...   \n",
       "9719  [0.0058666775, 0.9941333]                      1               Resolved   \n",
       "9720    [0.87230706, 0.1276929]                      0             Unresolved   \n",
       "9721  [0.025675168, 0.97432476]                      1               Resolved   \n",
       "9722  [0.001122542, 0.99887747]                      1               Resolved   \n",
       "9723    [0.0937371, 0.90626293]                      1               Resolved   \n",
       "\n",
       "      unresolved_probability  resolved_probability  \n",
       "0                   0.927111              0.072889  \n",
       "1                   0.025409              0.974591  \n",
       "2                   0.018914              0.981086  \n",
       "3                   0.598515              0.401485  \n",
       "4                   0.004478              0.995522  \n",
       "...                      ...                   ...  \n",
       "9719                0.005867              0.994133  \n",
       "9720                0.872307              0.127693  \n",
       "9721                0.025675              0.974325  \n",
       "9722                0.001123              0.998877  \n",
       "9723                0.093737              0.906263  \n",
       "\n",
       "[9724 rows x 6 columns]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## Let's make a datframe containing our predictions\n",
    "pred_test_df = pd.DataFrame(pred, columns = ['sentence', 'probabilities', 'status_code_predicted', 'status_label_predicted'])\n",
    "## Breaking down resolved and unresolved probability in separate columns\n",
    "pred_test_df['unresolved_probability'], pred_test_df['resolved_probability'] = zip(*pred_test_df['probabilities'])\n",
    "pred_test_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred_test_df.to_csv(DATA_OUTPUT_DIR + '/predictions.csv', encoding = 'utf-8-sig', index = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGdCAYAAAAMm0nCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAnhElEQVR4nO3dfXRU5YHH8V9emEl4mQkvJkOWAFGOQBS0hBqmgF00yxSjq2vcSqWYVZDFDu6SnAVMZQHRFQ6+ICgvq6hhz0IR9oirRF7SUOAo4cVothEk1RI2tDhBFzMDFPJC7v7Rk1tGQJmQF574/Zwz5zT3PvfmuU/R+XpzJ0RZlmUJAADAINHtPQEAAIBIETAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjBPb3hNoLY2NjTp27Ji6deumqKio9p4OAAC4DJZl6eTJk0pOTlZ09KXvs3TYgDl27JhSUlLaexoAAKAZjh49qj59+lxyf4cNmG7dukn68wK4XK52ng0AALgcoVBIKSkp9vv4pXTYgGn6sZHL5SJgAAAwzHc9/sFDvAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADBOxAHzxz/+UT//+c/Vs2dPxcfHa8iQIfrwww/t/ZZlac6cOerdu7fi4+OVmZmpzz77LOwcJ06c0IQJE+RyuZSQkKBJkybp1KlTYWN++9vfavTo0YqLi1NKSooWLVrUzEsEAAAdTUQB8/XXX2vkyJHq1KmTNm/erIMHD+r5559X9+7d7TGLFi3S0qVLtXLlSu3du1ddunSRz+fT2bNn7TETJkzQgQMHVFRUpE2bNmnXrl2aMmWKvT8UCmns2LHq16+fSktL9eyzz2revHl65ZVXWuCSAQCA8awIzJo1yxo1atQl9zc2Nloej8d69tln7W01NTWW0+m0fvWrX1mWZVkHDx60JFn79++3x2zevNmKioqy/vjHP1qWZVnLly+3unfvbtXW1oZ974EDB172XIPBoCXJCgaDl30MAABoX5f7/h3RHZh33nlHw4cP19///d8rMTFRP/jBD/Tqq6/a+ysrKxUIBJSZmWlvc7vdysjIUElJiSSppKRECQkJGj58uD0mMzNT0dHR2rt3rz3m1ltvlcPhsMf4fD5VVFTo66+/bkamAQCAjiQ2ksGHDx/WihUrlJeXp1/+8pfav3+//umf/kkOh0M5OTkKBAKSpKSkpLDjkpKS7H2BQECJiYnhk4iNVY8ePcLGpKamXnCOpn3n/8iqSW1trWpra+2vQ6FQJJcWkf6PF7bauVvLkYVZ7T0FAABaTEQB09jYqOHDh+uZZ56RJP3gBz/QJ598opUrVyonJ6dVJni5FixYoCeffLJd5wAAANpGRD9C6t27t9LS0sK2DR48WFVVVZIkj8cjSaqurg4bU11dbe/zeDw6fvx42P6GhgadOHEibMzFznH+9/im/Px8BYNB+3X06NFILg0AABgkooAZOXKkKioqwrb97ne/U79+/SRJqamp8ng8Ki4utveHQiHt3btXXq9XkuT1elVTU6PS0lJ7zPbt29XY2KiMjAx7zK5du1RfX2+PKSoq0sCBAy/64yNJcjqdcrlcYS8AANAxRRQwubm52rNnj5555hl9/vnnWrt2rV555RX5/X5JUlRUlKZPn66nn35a77zzjsrLy/Xggw8qOTlZ99xzj6Q/37H5yU9+okceeUT79u3TBx98oGnTpmn8+PFKTk6WJD3wwANyOByaNGmSDhw4oDfffFNLlixRXl5ey149AAAwUkTPwPzwhz/Uxo0blZ+fr/nz5ys1NVUvvviiJkyYYI+ZOXOmTp8+rSlTpqimpkajRo3Sli1bFBcXZ49Zs2aNpk2bpttvv13R0dHKzs7W0qVL7f1ut1vbtm2T3+9Xenq6evXqpTlz5oT9rhgAAPD9FWVZltXek2gNoVBIbrdbwWCwxX+cxKeQAABoHZf7/s3fhQQAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADBORAEzb948RUVFhb0GDRpk7z979qz8fr969uyprl27Kjs7W9XV1WHnqKqqUlZWljp37qzExETNmDFDDQ0NYWN27NihYcOGyel0asCAASooKGj+FQIAgA4n4jswN9xwg7744gv79f7779v7cnNz9e6772rDhg3auXOnjh07pnvvvdfef+7cOWVlZamurk67d+/W6tWrVVBQoDlz5thjKisrlZWVpTFjxqisrEzTp0/X5MmTtXXr1iu8VAAA0FHERnxAbKw8Hs8F24PBoF577TWtXbtWt912myTpjTfe0ODBg7Vnzx6NGDFC27Zt08GDB/XrX/9aSUlJuvnmm/XUU09p1qxZmjdvnhwOh1auXKnU1FQ9//zzkqTBgwfr/fff1+LFi+Xz+a7wcgEAQEcQ8R2Yzz77TMnJybr22ms1YcIEVVVVSZJKS0tVX1+vzMxMe+ygQYPUt29flZSUSJJKSko0ZMgQJSUl2WN8Pp9CoZAOHDhgjzn/HE1jms5xKbW1tQqFQmEvAADQMUUUMBkZGSooKNCWLVu0YsUKVVZWavTo0Tp58qQCgYAcDocSEhLCjklKSlIgEJAkBQKBsHhp2t+079vGhEIhnTlz5pJzW7Bggdxut/1KSUmJ5NIAAIBBIvoR0rhx4+z/PXToUGVkZKhfv35av3694uPjW3xykcjPz1deXp79dSgUImIAAOigruhj1AkJCbr++uv1+eefy+PxqK6uTjU1NWFjqqur7WdmPB7PBZ9Kavr6u8a4XK5vjSSn0ymXyxX2AgAAHdMVBcypU6f0+9//Xr1791Z6ero6deqk4uJie39FRYWqqqrk9XolSV6vV+Xl5Tp+/Lg9pqioSC6XS2lpafaY88/RNKbpHAAAABEFzL/8y79o586dOnLkiHbv3q2/+7u/U0xMjH72s5/J7XZr0qRJysvL029+8xuVlpbqoYcektfr1YgRIyRJY8eOVVpamiZOnKj/+Z//0datWzV79mz5/X45nU5J0tSpU3X48GHNnDlThw4d0vLly7V+/Xrl5ua2/NUDAAAjRfQMzB/+8Af97Gc/0//93//pmmuu0ahRo7Rnzx5dc801kqTFixcrOjpa2dnZqq2tlc/n0/Lly+3jY2JitGnTJj366KPyer3q0qWLcnJyNH/+fHtMamqqCgsLlZubqyVLlqhPnz5atWoVH6EGAAC2KMuyrPaeRGsIhUJyu90KBoMt/jxM/8cLW/R8beHIwqz2ngIAAN/pct+/+buQAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxrmigFm4cKGioqI0ffp0e9vZs2fl9/vVs2dPde3aVdnZ2aqurg47rqqqSllZWercubMSExM1Y8YMNTQ0hI3ZsWOHhg0bJqfTqQEDBqigoOBKpgoAADqQZgfM/v379e///u8aOnRo2Pbc3Fy9++672rBhg3bu3Kljx47p3nvvtfefO3dOWVlZqqur0+7du7V69WoVFBRozpw59pjKykplZWVpzJgxKisr0/Tp0zV58mRt3bq1udMFAAAdSLMC5tSpU5owYYJeffVVde/e3d4eDAb12muv6YUXXtBtt92m9PR0vfHGG9q9e7f27NkjSdq2bZsOHjyo//zP/9TNN9+scePG6amnntKyZctUV1cnSVq5cqVSU1P1/PPPa/DgwZo2bZruu+8+LV68uAUuGQAAmK5ZAeP3+5WVlaXMzMyw7aWlpaqvrw/bPmjQIPXt21clJSWSpJKSEg0ZMkRJSUn2GJ/Pp1AopAMHDthjvnlun89nn+NiamtrFQqFwl4AAKBjio30gHXr1umjjz7S/v37L9gXCATkcDiUkJAQtj0pKUmBQMAec368NO1v2vdtY0KhkM6cOaP4+PgLvveCBQv05JNPRno5AADAQBHdgTl69Kj++Z//WWvWrFFcXFxrzalZ8vPzFQwG7dfRo0fbe0oAAKCVRBQwpaWlOn78uIYNG6bY2FjFxsZq586dWrp0qWJjY5WUlKS6ujrV1NSEHVddXS2PxyNJ8ng8F3wqqenr7xrjcrkuevdFkpxOp1wuV9gLAAB0TBEFzO23367y8nKVlZXZr+HDh2vChAn2/+7UqZOKi4vtYyoqKlRVVSWv1ytJ8nq9Ki8v1/Hjx+0xRUVFcrlcSktLs8ecf46mMU3nAAAA328RPQPTrVs33XjjjWHbunTpop49e9rbJ02apLy8PPXo0UMul0uPPfaYvF6vRowYIUkaO3as0tLSNHHiRC1atEiBQECzZ8+W3++X0+mUJE2dOlUvv/yyZs6cqYcffljbt2/X+vXrVVhY2BLXDAAADBfxQ7zfZfHixYqOjlZ2drZqa2vl8/m0fPlye39MTIw2bdqkRx99VF6vV126dFFOTo7mz59vj0lNTVVhYaFyc3O1ZMkS9enTR6tWrZLP52vp6QIAAANFWZZltfckWkMoFJLb7VYwGGzx52H6P27enaAjC7PaewoAAHyny33/5u9CAgAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYJyIAmbFihUaOnSoXC6XXC6XvF6vNm/ebO8/e/as/H6/evbsqa5duyo7O1vV1dVh56iqqlJWVpY6d+6sxMREzZgxQw0NDWFjduzYoWHDhsnpdGrAgAEqKCho/hUCAIAOJ6KA6dOnjxYuXKjS0lJ9+OGHuu2223T33XfrwIEDkqTc3Fy9++672rBhg3bu3Kljx47p3nvvtY8/d+6csrKyVFdXp927d2v16tUqKCjQnDlz7DGVlZXKysrSmDFjVFZWpunTp2vy5MnaunVrC10yAAAwXZRlWdaVnKBHjx569tlndd999+maa67R2rVrdd9990mSDh06pMGDB6ukpEQjRozQ5s2bdeedd+rYsWNKSkqSJK1cuVKzZs3Sl19+KYfDoVmzZqmwsFCffPKJ/T3Gjx+vmpoabdmy5bLnFQqF5Ha7FQwG5XK5ruQSL9D/8cIWPV9bOLIwq72nAADAd7rc9+9mPwNz7tw5rVu3TqdPn5bX61Vpaanq6+uVmZlpjxk0aJD69u2rkpISSVJJSYmGDBlix4sk+Xw+hUIh+y5OSUlJ2DmaxjSd41Jqa2sVCoXCXgAAoGOKOGDKy8vVtWtXOZ1OTZ06VRs3blRaWpoCgYAcDocSEhLCxiclJSkQCEiSAoFAWLw07W/a921jQqGQzpw5c8l5LViwQG63236lpKREemkAAMAQEQfMwIEDVVZWpr179+rRRx9VTk6ODh482Bpzi0h+fr6CwaD9Onr0aHtPCQAAtJLYSA9wOBwaMGCAJCk9PV379+/XkiVLdP/996uurk41NTVhd2Gqq6vl8XgkSR6PR/v27Qs7X9OnlM4f881PLlVXV8vlcik+Pv6S83I6nXI6nZFeDgAAMNAV/x6YxsZG1dbWKj09XZ06dVJxcbG9r6KiQlVVVfJ6vZIkr9er8vJyHT9+3B5TVFQkl8ultLQ0e8z552ga03QOAACAiO7A5Ofna9y4cerbt69OnjyptWvXaseOHdq6davcbrcmTZqkvLw89ejRQy6XS4899pi8Xq9GjBghSRo7dqzS0tI0ceJELVq0SIFAQLNnz5bf77fvnkydOlUvv/yyZs6cqYcffljbt2/X+vXrVVho3id/AABA64goYI4fP64HH3xQX3zxhdxut4YOHaqtW7fqb/7mbyRJixcvVnR0tLKzs1VbWyufz6fly5fbx8fExGjTpk169NFH5fV61aVLF+Xk5Gj+/Pn2mNTUVBUWFio3N1dLlixRnz59tGrVKvl8vha6ZAAAYLor/j0wVyt+D0w4fg8MAMAErf57YAAAANoLAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAME5se08AAIDvu/6PF7b3FCJ2ZGFWu35/7sAAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgEDAAAMA4BAwAAjEPAAAAA4xAwAADAOAQMAAAwDgEDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIxDwAAAAOMQMAAAwDgRBcyCBQv0wx/+UN26dVNiYqLuueceVVRUhI05e/as/H6/evbsqa5duyo7O1vV1dVhY6qqqpSVlaXOnTsrMTFRM2bMUENDQ9iYHTt2aNiwYXI6nRowYIAKCgqad4UAAKDDiShgdu7cKb/frz179qioqEj19fUaO3asTp8+bY/Jzc3Vu+++qw0bNmjnzp06duyY7r33Xnv/uXPnlJWVpbq6Ou3evVurV69WQUGB5syZY4+prKxUVlaWxowZo7KyMk2fPl2TJ0/W1q1bW+CSAQCA6aIsy7Kae/CXX36pxMRE7dy5U7feequCwaCuueYarV27Vvfdd58k6dChQxo8eLBKSko0YsQIbd68WXfeeaeOHTumpKQkSdLKlSs1a9Ysffnll3I4HJo1a5YKCwv1ySef2N9r/Pjxqqmp0ZYtWy5rbqFQSG63W8FgUC6Xq7mXeFH9Hy9s0fO1hSMLs9p7CgCAS+B95S8u9/37ip6BCQaDkqQePXpIkkpLS1VfX6/MzEx7zKBBg9S3b1+VlJRIkkpKSjRkyBA7XiTJ5/MpFArpwIED9pjzz9E0pukcF1NbW6tQKBT2AgAAHVOzA6axsVHTp0/XyJEjdeONN0qSAoGAHA6HEhISwsYmJSUpEAjYY86Pl6b9Tfu+bUwoFNKZM2cuOp8FCxbI7Xbbr5SUlOZeGgAAuMo1O2D8fr8++eQTrVu3riXn02z5+fkKBoP26+jRo+09JQAA0Epim3PQtGnTtGnTJu3atUt9+vSxt3s8HtXV1ammpibsLkx1dbU8Ho89Zt++fWHna/qU0vljvvnJperqarlcLsXHx190Tk6nU06nszmXAwAADBPRHRjLsjRt2jRt3LhR27dvV2pqatj+9PR0derUScXFxfa2iooKVVVVyev1SpK8Xq/Ky8t1/Phxe0xRUZFcLpfS0tLsMeefo2lM0zkAAMD3W0R3YPx+v9auXav//u//Vrdu3exnVtxut+Lj4+V2uzVp0iTl5eWpR48ecrlceuyxx+T1ejVixAhJ0tixY5WWlqaJEydq0aJFCgQCmj17tvx+v30HZerUqXr55Zc1c+ZMPfzww9q+fbvWr1+vwkLzntIGAAAtL6I7MCtWrFAwGNRf//Vfq3fv3vbrzTfftMcsXrxYd955p7Kzs3XrrbfK4/HorbfesvfHxMRo06ZNiomJkdfr1c9//nM9+OCDmj9/vj0mNTVVhYWFKioq0k033aTnn39eq1atks/na4FLBgAAprui3wNzNeP3wITj98AAwNWL95W/aJPfAwMAANAeCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYJ+KA2bVrl+666y4lJycrKipKb7/9dth+y7I0Z84c9e7dW/Hx8crMzNRnn30WNubEiROaMGGCXC6XEhISNGnSJJ06dSpszG9/+1uNHj1acXFxSklJ0aJFiyK/OgAA0CFFHDCnT5/WTTfdpGXLll10/6JFi7R06VKtXLlSe/fuVZcuXeTz+XT27Fl7zIQJE3TgwAEVFRVp06ZN2rVrl6ZMmWLvD4VCGjt2rPr166fS0lI9++yzmjdvnl555ZVmXCIAAOhoYiM9YNy4cRo3btxF91mWpRdffFGzZ8/W3XffLUn6j//4DyUlJentt9/W+PHj9emnn2rLli3av3+/hg8fLkl66aWXdMcdd+i5555TcnKy1qxZo7q6Or3++utyOBy64YYbVFZWphdeeCEsdAAAwPdTiz4DU1lZqUAgoMzMTHub2+1WRkaGSkpKJEklJSVKSEiw40WSMjMzFR0drb1799pjbr31VjkcDnuMz+dTRUWFvv7665acMgAAMFDEd2C+TSAQkCQlJSWFbU9KSrL3BQIBJSYmhk8iNlY9evQIG5OamnrBOZr2de/e/YLvXVtbq9raWvvrUCh0hVcDAACuVh3mU0gLFiyQ2+22XykpKe09JQAA0EpaNGA8Ho8kqbq6Omx7dXW1vc/j8ej48eNh+xsaGnTixImwMRc7x/nf45vy8/MVDAbt19GjR6/8ggAAwFWpRQMmNTVVHo9HxcXF9rZQKKS9e/fK6/VKkrxer2pqalRaWmqP2b59uxobG5WRkWGP2bVrl+rr6+0xRUVFGjhw4EV/fCRJTqdTLpcr7AUAADqmiAPm1KlTKisrU1lZmaQ/P7hbVlamqqoqRUVFafr06Xr66af1zjvvqLy8XA8++KCSk5N1zz33SJIGDx6sn/zkJ3rkkUe0b98+ffDBB5o2bZrGjx+v5ORkSdIDDzwgh8OhSZMm6cCBA3rzzTe1ZMkS5eXltdiFAwAAc0X8EO+HH36oMWPG2F83RUVOTo4KCgo0c+ZMnT59WlOmTFFNTY1GjRqlLVu2KC4uzj5mzZo1mjZtmm6//XZFR0crOztbS5cutfe73W5t27ZNfr9f6enp6tWrl+bMmcNHqAEAgCQpyrIsq70n0RpCoZDcbreCwWCL/zip/+OFLXq+tnBkYVZ7TwEAcAm8r/zF5b5/d5hPIQEAgO8PAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYBwCBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYJ7a9J4C20f/xwvaeQrMcWZjV3lMAAFyFuAMDAACMQ8AAAADjEDAAAMA4BAwAADAOAQMAAIzDp5CAFmbiJ774tBcA0xAwuKqZGAMAgNZHwAAAOhT+w+f7gWdgAACAcQgYAABgHH6EBMDIW+48eNw2TPyzge8HAgaAkUx8YyW6gJZDwABAGzExuoCr1VX9DMyyZcvUv39/xcXFKSMjQ/v27WvvKQEAgKvAVRswb775pvLy8jR37lx99NFHuummm+Tz+XT8+PH2nhoAAGhnV23AvPDCC3rkkUf00EMPKS0tTStXrlTnzp31+uuvt/fUAABAO7sqn4Gpq6tTaWmp8vPz7W3R0dHKzMxUSUnJRY+pra1VbW2t/XUwGJQkhUKhFp9fY+2fWvycAACYpDXeX88/r2VZ3zruqgyYr776SufOnVNSUlLY9qSkJB06dOiixyxYsEBPPvnkBdtTUlJaZY4AAHyfuV9s3fOfPHlSbrf7kvuvyoBpjvz8fOXl5dlfNzY26sSJE+rZs6eioqJa7PuEQiGlpKTo6NGjcrlcLXZeXIi1bjusddthrdsG69x2WnqtLcvSyZMnlZyc/K3jrsqA6dWrl2JiYlRdXR22vbq6Wh6P56LHOJ1OOZ3OsG0JCQmtNUW5XC7+oWgjrHXbYa3bDmvdNljnttOSa/1td16aXJUP8TocDqWnp6u4uNje1tjYqOLiYnm93nacGQAAuBpclXdgJCkvL085OTkaPny4brnlFr344os6ffq0HnroofaeGgAAaGdXbcDcf//9+vLLLzVnzhwFAgHdfPPN2rJlywUP9rY1p9OpuXPnXvDjKrQ81rrtsNZth7VuG6xz22mvtY6yvutzSgAAAFeZq/IZGAAAgG9DwAAAAOMQMAAAwDgEDAAAMA4BcxHLli1T//79FRcXp4yMDO3bt+9bx2/YsEGDBg1SXFychgwZovfee6+NZmq+SNb61Vdf1ejRo9W9e3d1795dmZmZ3/n/Df4i0j/XTdatW6eoqCjdc889rTvBDiLSda6pqZHf71fv3r3ldDp1/fXX8++QyxTpWr/44osaOHCg4uPjlZKSotzcXJ09e7aNZmuuXbt26a677lJycrKioqL09ttvf+cxO3bs0LBhw+R0OjVgwAAVFBS0/MQshFm3bp3lcDis119/3Tpw4ID1yCOPWAkJCVZ1dfVFx3/wwQdWTEyMtWjRIuvgwYPW7NmzrU6dOlnl5eVtPHPzRLrWDzzwgLVs2TLr448/tj799FPrH/7hHyy322394Q9/aOOZmyfStW5SWVlp/dVf/ZU1evRo6+67726byRos0nWura21hg8fbt1xxx3W+++/b1VWVlo7duywysrK2njm5ol0rdesWWM5nU5rzZo1VmVlpbV161ard+/eVm5ubhvP3Dzvvfee9cQTT1hvvfWWJcnauHHjt44/fPiw1blzZysvL886ePCg9dJLL1kxMTHWli1bWnReBMw33HLLLZbf77e/PnfunJWcnGwtWLDgouN/+tOfWllZWWHbMjIyrH/8x39s1Xl2BJGu9Tc1NDRY3bp1s1avXt1aU+wwmrPWDQ0N1o9+9CNr1apVVk5ODgFzGSJd5xUrVljXXnutVVdX11ZT7DAiXWu/32/ddtttYdvy8vKskSNHtuo8O5rLCZiZM2daN9xwQ9i2+++/3/L5fC06F36EdJ66ujqVlpYqMzPT3hYdHa3MzEyVlJRc9JiSkpKw8ZLk8/kuOR5/1py1/qY//elPqq+vV48ePVprmh1Cc9d6/vz5SkxM1KRJk9pimsZrzjq/88478nq98vv9SkpK0o033qhnnnlG586da6tpG6k5a/2jH/1IpaWl9o+ZDh8+rPfee0933HFHm8z5+6St3hev2t/E2x6++uornTt37oLf9puUlKRDhw5d9JhAIHDR8YFAoNXm2RE0Z62/adasWUpOTr7gHxSEa85av//++3rttddUVlbWBjPsGJqzzocPH9b27ds1YcIEvffee/r888/1i1/8QvX19Zo7d25bTNtIzVnrBx54QF999ZVGjRoly7LU0NCgqVOn6pe//GVbTPl75VLvi6FQSGfOnFF8fHyLfB/uwMBICxcu1Lp167Rx40bFxcW193Q6lJMnT2rixIl69dVX1atXr/aeTofW2NioxMREvfLKK0pPT9f999+vJ554QitXrmzvqXU4O3bs0DPPPKPly5fro48+0ltvvaXCwkI99dRT7T01NBN3YM7Tq1cvxcTEqLq6Omx7dXW1PB7PRY/xeDwRjcefNWetmzz33HNauHChfv3rX2vo0KGtOc0OIdK1/v3vf68jR47orrvusrc1NjZKkmJjY1VRUaHrrruudSdtoOb8me7du7c6deqkmJgYe9vgwYMVCARUV1cnh8PRqnM2VXPW+l//9V81ceJETZ48WZI0ZMgQnT59WlOmTNETTzyh6Gj+e76lXOp90eVytdjdF4k7MGEcDofS09NVXFxsb2tsbFRxcbG8Xu9Fj/F6vWHjJamoqOiS4/FnzVlrSVq0aJGeeuopbdmyRcOHD2+LqRov0rUeNGiQysvLVVZWZr/+9m//VmPGjFFZWZlSUlLacvrGaM6f6ZEjR+rzzz+3A1GSfve736l3797Ey7dozlr/6U9/uiBSmsLR4q8EbFFt9r7Yoo8EdwDr1q2znE6nVVBQYB08eNCaMmWKlZCQYAUCAcuyLGvixInW448/bo//4IMPrNjYWOu5556zPv30U2vu3Ll8jPoyRbrWCxcutBwOh/Vf//Vf1hdffGG/Tp482V6XYIxI1/qb+BTS5Yl0nauqqqxu3bpZ06ZNsyoqKqxNmzZZiYmJ1tNPP91el2CMSNd67ty5Vrdu3axf/epX1uHDh61t27ZZ1113nfXTn/60vS7BGCdPnrQ+/vhj6+OPP7YkWS+88IL18ccfW//7v/9rWZZlPf7449bEiRPt8U0fo54xY4b16aefWsuWLeNj1G3lpZdesvr27Ws5HA7rlltusfbs2WPv+/GPf2zl5OSEjV+/fr11/fXXWw6Hw7rhhhuswsLCNp6xuSJZ6379+lmSLnjNnTu37SduoEj/XJ+PgLl8ka7z7t27rYyMDMvpdFrXXnut9W//9m9WQ0NDG8/aTJGsdX19vTVv3jzruuuus+Li4qyUlBTrF7/4hfX111+3/cQN85vf/Oai/+5tWt+cnBzrxz/+8QXH3HzzzZbD4bCuvfZa64033mjxeUVZFvfOAACAWXgGBgAAGIeAAQAAxiFgAACAcQgYAABgHAIGAAAYh4ABAADGIWAAAIBxCBgAAGAcAgYAABiHgAEAAMYhYAAAgHEIGAAAYJz/B4AJbZ2R4tazAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(pred_test_df.unresolved_probability)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "status_label_predicted\n",
       "Resolved      AxesSubplot(0.125,0.11;0.352273x0.77)\n",
       "Unresolved    AxesSubplot(0.125,0.11;0.352273x0.77)\n",
       "Name: unresolved_probability, dtype: object"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGdCAYAAAAMm0nCAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAABn+klEQVR4nO3deXiTVfo38G+SJulCF7ZuUkoRZJMdKUWQrVAWGUBERISqgCO/omJHUGaQ1Zm6AS4gOCNQfBVBZhQdQKCABZGCbEU2GUCgILRstumapsnz/pHmaUOTtGmTJk/y/VxXrzTJydNzKE3v3uc+58gEQRBAREREJCFyV3eAiIiIyF4MYIiIiEhyGMAQERGR5DCAISIiIslhAENERESSwwCGiIiIJIcBDBEREUkOAxgiIiKSHB9Xd8BZDAYDrl+/jsDAQMhkMld3h8jrCIKA/Px8REZGQi6Xxt9KfN8gcr2avnd4bABz/fp1REVFubobRF7v6tWraNasmau7USN83yByH9W9d3hsABMYGAjA+A8QFBRktZ1Op8POnTsxZMgQKJXK+uqe03jSeDxpLIBnjacmY9FoNIiKihJ/FqWgpu8bROQ8NX3v8NgAxpT+DQoKqjaA8ff3R1BQkOR/qQCeNR5PGgvgWeOxZyxSmoqp6fsGETlfde8d0piYJiIiIqqEAQwRERFJDgMYIiIikhyPrYEhIiL3JwgCysrKoNfrXd0VqicKhQI+Pj51ro9jAENERC5RWlqKGzduoKioyNVdoXrm7++PiIgIqFSqWl+DAQwREdU7g8GAS5cuQaFQIDIyEiqVSlIr1qh2BEFAaWkpbt26hUuXLqF169a13uiSAQwREdW70tJSGAwGREVFwd/f39XdoXrk5+cHpVKJK1euoLS0FL6+vrW6Dot4iYjIZaRyzAQ5liO+7/yfQ0RERJLDAIaIiIgkhwEMERGRhDzzzDMYPXq007+OTCbD5s2bnf51aosBDBERkR2eeeYZyGQyyGQyKJVKxMTEYPbs2SgpKXF117wKVyEREUnctpM34KdSYECbUFd3xWsMHToUa9euhU6nw9GjR5GYmAiZTIa3337b1V3zGszAEFG927dvH0aOHInIyEiLaWrTX7f3frz77rtWr7lgwYIq7du2bevkkbhedl4J/u+LY3h27WH8UVjq6u7UmiAIKCotc8mHIAh291etViM8PBxRUVEYPXo04uPjkZaWBsC4x01KSgpiYmLg5+eHzp0749///rf42j/++AMTJ05E06ZN4efnh9atW2Pt2rXi8ydPnsTAgQPh5+eHxo0b4/nnn0dBQYHFfvzzn/9EZGQkDAaD2eOjRo3Cc889J97/9ttv0a1bN/j6+qJly5ZYuHAhysrKxOfPnz+PRx55BL6+vmjfvr04FnfGDEy5TUeuAXKFzTZPxTavp94QebbCwkJ07twZzz33HB577LEqz9+4ccPs/vfff48pU6Zg7NixNq/boUMH7Nq1S7zv4+P5b3GX7xSKn1/PK0bDgNrvbOpKxTo92s/b4ZKvfWZRAvxVtf+/curUKRw4cADR0dEAgJSUFHz++edYtWoVWrdujX379uHpp59G06ZN0a9fP7zxxhs4c+YMvv/+ezRp0gQXLlxAcXExAOPPRkJCAuLi4nD48GHcvHkTU6dOxYwZM5Camlrla48bNw4vvvgifvjhBwwaNAgAcPfuXWzfvh3btm0DAPz444+YPHkyPvzwQ/Tt2xcXL17E888/DwCYP38+DAYDHnvsMYSFheHQoUPIy8vDzJkza/3vUV88/6ebiNzOsGHDMGzYMKvPh4eHm93/9ttvMWDAALRs2dLmdX18fKq81tMVlVb8FZ1bpHNhT7zLli1b0KBBA5SVlUGr1UIul2P58uXQarX4xz/+gV27diEuLg4A0LJlS+zfvx+ffPIJ+vXrh6ysLHTt2hU9evQAALRo0UK87vr161FSUoLPPvsMAQEBAIDly5dj5MiRePvttxEWFmbWj4YNG2LYsGFYv369GMD8+9//RpMmTTBgwAAAwMKFC/H6668jMTFR7M/ixYsxe/ZszJ8/H7t27cKvv/6KHTt2IDIyEgDwj3/8w+bPqDtgAENEbi0nJwdbt27FunXrqm17/vx5REZGwtfXF3FxcUhJSUHz5tYzp1qtFlqtVryv0Wgc0uf6VKitOATxjyLpTiH5KRU4syjBZV/bXgMGDMDKlStRWFiIZcuWwcfHB2PHjsXp06dRVFSEwYMHm7UvLS1F165dAQDTp0/H2LFjcezYMQwZMgSjR49G7969AQBnz55F586dxeAFAB5++GEYDAacO3euSgADABMnTsS0adPw8ccfQ61W44svvsCTTz4pbhZ34sQJ/PTTT/j73/8uvkav16OkpARFRUU4e/YsoqKixOAFgBh8uTMGMETk1tatW4fAwECLU02VxcbGIjU1FW3atMGNGzewcOFC9O3bF6dOnUJgYKDF16SkpGDhwoXO6Ha98ZQMjEwmq9M0Tn0LCAhAq1atAABr1qxB586dsXr1ajz44IMAgK1bt+K+++4ze41arQZgzEBeuXIF27ZtQ1paGgYNGoSkpCS89957terLyJEjIQgCtm7dioceegg//vgjli1bJj5fUFCAhQsXWvwZqu02/u5AOv9biMgrrVmzBhMnTqz2jbZyurtTp06IjY1FdHQ0vvrqK0yZMsXia+bMmYPk5GTxvkajQVRUlGM6Xk8qZ2BKdHobLclZ5HI5/vrXvyI5ORn/+9//oFarkZWVhX79+ll9TdOmTZGYmIjExET07dsXs2bNwnvvvYd27dohNTUVhYWFYhbmp59+glwuR5s2bSxey9fXF4899hi++OILXLhwAW3atEG3bt3E57t164Zz586JAde92rVrh6tXr+LGjRuIiIgAABw8eLC2/xz1hgEMEbmtH3/8EefOncPGjRvtfm1ISAgeeOABXLhwwWobtVot/lUsVZUzMAxgXGfcuHGYNWsWPvnkE7z66qt45ZVXYDAY0KdPH+Tl5eGnn35CUFAQEhMTMW/ePHTv3h0dOnSAVqvFli1b0K5dOwDG6aD58+cjMTERCxYswK1bt/Diiy9i0qRJFqePTCZOnIhHH30Up0+fxtNPP2323Lx58/Doo4+iefPmePzxxyGXy3HixAmcOnUKb775JuLj4/HAAw8gMTER7777LjQaDf72t7859d/LEbiMmojc1urVq9G9e3d07tzZ7tcWFBTg4sWL4l+UnqrALANjsNGSnMnHxwczZszAO++8gzlz5uCNN95ASkoK2rVrh6FDh2Lr1q2IiYkBAKhUKsyZMwedOnXCI488AoVCgQ0bNgAA/P39sWPHDty9excPPfQQHn/8cQwaNAjLly+3+fUHDhyIRo0a4dy5c3jqqafMnktISMCWLVuwc+dOPPTQQ+jVqxeWLVsmrpqSy+X45ptvUFxcjJ49e2Lq1Klm9TLuihkYIqp3BQUFZpmRS5cuITMzE40aNRKLbjUaDTZt2oQlS5ZYvMagQYMwZswYzJgxAwDw6quvYuTIkYiOjsb169cxf/58KBQKTJgwwfkDciFmYOqfpeXMAPD666/j9ddfBwC8/PLLePnlly22mzt3LubOnWv1+h07dsSePXvs+vpyuRzXr1+3+pqEhAQkJFgvkn7ggQfw448/mj1Wm/1x6hMDGCKqd0eOHBGXeAIQ61ASExPFN+cNGzZAEASrAcjFixdx+/Zt8f61a9cwYcIE3LlzB02bNkWfPn1w8OBBNG3a1HkDcQNmNTBlDGDIezCAIaJ6179//2r/unv++efFzbYsuXz5stl9Uwre22grBS3FpZxCIu/BGhgiIgnT6SuCFmZgyJswgCEikrAyfUUmq7SMGRjyHgxgiIgkrLRSBqZyNobI0zGAISKSsMoZGAYw5E0YwBARSVjloEVX5t7LXokciQEMEZGE6QyVamCYgSEvwgCGiEjCdGWsgaGaS09Ph0wmQ25urlO/zjPPPIPRo0c79WswgCEikrAyAwOY+ta/f3/MnDmzyuOpqakICQmp9/54KwYwREQSpjMr4mUNjJSVlpa6uguSwgCGiEjCKmdduA+M+zBNobz33nuIiIhA48aNkZSUBJ1OJ7Zp0aIFFi9ejMmTJyMoKEjceXr//v3o27cv/Pz8EBUVhZdeegmFhYXi6z7++GO0bt0avr6+CAsLw+OPPy4+p9Vq8dJLLyE0NBS+vr7o06cPDh8+bLGPGo0Gfn5++P77780e/+abbxAYGIiioiIAwNWrV/HEE08gJCQEjRo1wqhRo8x2wtbr9UhOTkZISAgaN26M2bNn18s5SgxgiIgkTOcp+8AIAlBa6JoPJ/2y/eGHH3Dx4kX88MMPWLduHVJTU6scxPjee++hc+fOOH78ON544w1cvHgRQ4cOxdixY/HLL79g48aN2L9/v3ho6ZEjR/DSSy9h0aJFOHfuHLZv345HHnlEvN7s2bPxn//8B+vWrcOxY8fQqlUrJCQk4O7du1X6FxQUhEcffRTr1683e/yLL77A6NGj4e/vD51Oh4SEBAQGBuLHH3/ETz/9hAYNGmDo0KFixmjJkiVITU3FmjVrsH//fty9exfffPONg/81q+JZSEREEuYx+8DoioB/RLrma//1OqAKcPhlGzZsiOXLl0OhUKBt27YYMWIEdu/ejWnTpoltBg4ciL/85S/i/alTp2LixIlijU3r1q3x4Ycfol+/fli5ciWysrIQEBCARx99FIGBgYiOjkbXrl0BAIWFhVi5ciVSU1MxbNgwAMC//vUvpKWlYfXq1Zg1a1aVPk6cOBGTJk1CUVER/P39odFosHXrVjEA2bhxIwwGAz799FPIZDIAwNq1axESEoL09HQMGTIE77//PubMmYPHHnsMALBq1Srs2LHD4f+e92IGhohIwsx34mUNjDvp0KEDFAqFeD8iIgI3b940a9OjRw+z+ydOnEBqaioaNGggfiQkJMBgMODSpUsYPHgwoqOj0bJlS0yaNAlffPGFONVz8eJF6HQ6PPzww+L1lEolevbsibNnz1rs4/Dhw6FUKvHdd98BAP7zn/8gKCgI8fHxYn8uXLiAwMBAsT+NGjVCSUkJLl68iLy8PNy4cQOxsbHiNX18fKqMyxmYgSEikjCzGhgpZ2CU/sZMiKu+th2CgoKQl5dX5fHc3FwEBwdXXFapNHteJpPBYDD/HgUEmGd+CgoK8Oc//xkvvfRSles3b94cKpUKx44dQ3p6Onbu3Il58+ZhwYIFVutcqqNSqfD4449j/fr1ePLJJ7F+/XqMHz8ePj4+Yn+6d++OL774osprmzZtWquv6SgMYIiIJOzeKSRBEMRUv6TIZE6ZxnGGNm3aYOfOnVUeP3bsGB544IE6Xbtbt244c+YMWrVqZbWNj48P4uPjER8fj/nz5yMkJAR79uxBQkICVCoVfvrpJ0RHRwMAdDodDh8+bHHZt8nEiRMxePBgnD59Gnv27MGbb75p1p+NGzciNDQUQUFBFl8fERGBQ4cOibU4ZWVlOHr0KLp161aLf4Ga4xQSEZFECYKAsko78QoCoDdwGsnZpk+fjv/973946aWX8Msvv+DcuXNYunQpvvzyS7N6ltp47bXXcODAAcyYMQOZmZk4f/48vv32W7GId8uWLfjwww+RmZmJK1eu4LPPPoPBYECbNm0QEBCA6dOnY9asWdi+fTvOnDmDadOmoaioCFOmTLH6NR955BGEh4dj4sSJiImJMZsOmjhxIpo0aYJRo0bhxx9/xKVLl5Ceno6XXnoJ165dAwC8/PLLeOutt7B582b8+uuv+L//+z+nb5QHMANDRCRZlmpeygwCfBQWGpPDtGzZEvv27cPf/vY3xMfHo7S0FG3btsWmTZswdOjQOl27U6dO2Lt3L/72t7+hb9++EAQB999/P8aPHw8ACAkJwddff40FCxagpKQErVu3xpdffokOHToAAN566y0YDAZMmjQJ+fn56NGjB3bs2IGGDRta/ZoymQwTJkzAO++8g3nz5pk95+/vj3379uG1117DY489hvz8fNx3330YNGiQmJH5y1/+ghs3biAxMRFyuRzPPfccxowZY3GazZFkQn0s1nYBjUaD4OBg5OXlWU17Acb02rZt21AY2gmQ2/6pfyq2uaO76XCm8ZgKs6TMk8YCeNZ4ajKWmv4MuhOp9blQW4YO881Xe/yyYAiCfN3//1dJSQkuXbqEmJgY+Pr6uro7VM9sff9r+nPIKSQiIokqs5CB4WZ25C0YwBARSVTlVUcKubFwlzUw5C0YwBARSZTpIEeVQg6f8gCmjAEMeQkGMEREEqUrMwYrPgqZGMDouZkdeQmuQiIikihdeQZGqZCLh+fpDKyBIe9gVwZmwYIFkMlkZh9t27YVny8pKUFSUhIaN26MBg0aYOzYscjJyTG7RlZWFkaMGAF/f3+EhoZi1qxZKCsrM2uTnp6Obt26Qa1Wo1WrVlUOvyIioopdeJUKGXwUxrdzqdXAeOhCWKqGI77vdk8hdejQATdu3BA/9u/fLz73yiuv4L///S82bdqEvXv34vr16+LhToDxyO0RI0agtLQUBw4cEE/nrLzu/NKlSxgxYgQGDBiAzMxMzJw5E1OnTq2Xg6GIiKTEtArJRy4Xi3gtrUxyR6bl96ZzfMi7mL7vddlSwu4pJB8fH4SHh1d5PC8vD6tXr8b69esxcOBAAMYTK9u1a4eDBw+iV69e2LlzJ86cOYNdu3YhLCwMXbp0weLFi/Haa69hwYIFUKlUWLVqFWJiYrBkyRIAQLt27bB//34sW7YMCQkJtR4oEZGnMRXsKuQyyMvjFqlkYBQKBUJCQsTDDf39/aV5BALZRRAEFBUV4ebNmwgJCTE77NJedgcw58+fR2RkJHx9fREXF4eUlBQ0b94cR48ehU6nE0+wBIC2bduiefPmyMjIQK9evZCRkYGOHTsiLCxMbJOQkIDp06fj9OnT6Nq1KzIyMsyuYWpj6xwHANBqtdBqteJ9jUYDwLjhlk6ns/o68TmDvtqx27qOuzD1UQp9rY4njQXwrPHUZCyeME53p68UwJjCljIJ1cCY/hi+94Rm8nwhISEWkyH2sCuAiY2NRWpqKtq0aYMbN25g4cKF6Nu3L06dOoXs7GyoVCqEhISYvSYsLAzZ2dkAgOzsbLPgxfS86TlbbTQaDYqLi+Hn52exbykpKVi4cGGVx3fu3Al//+pPGg24fbraNtu2/VJtG3eRlpbm6i44jCeNBfCs8dgaC6cGnM8gmKaQZBAgvX1gZDIZIiIiEBoayoDXiyiVyjplXkzsCmCGDRsmft6pUyfExsYiOjoaX331ldXAor7MmTMHycnJ4n2NRoOoqCgMGTKk2qME0tLSUNikQ7VHCYzr0cxh/XUW03gGDx7sEdvVe8pYAM8aT03GYsqCkvOY6l3k8oqpF0vnI7k7hULhkF9o5F3qtIw6JCQEDzzwAC5cuIDBgwejtLQUubm5ZlmYnJwcMU0UHh6On3/+2ewaplVKldvcu3IpJycHQUFBNoMktVoNtVpd5XGlUlmzXxZyRbUBjJR+6dR43BLgSWMBPGs8tsbiKWN0Z5UzMCZSysAQ1UWdNrIrKCjAxYsXERERge7du0OpVGL37t3i8+fOnUNWVhbi4uIAAHFxcTh58qTZfGdaWhqCgoLQvn17sU3la5jamK5BRERGpiJeuUxWsQpJQjUwRHVhVwDz6quvYu/evbh8+TIOHDiAMWPGQKFQYMKECQgODsaUKVOQnJyMH374AUePHsWzzz6LuLg49OrVCwAwZMgQtG/fHpMmTcKJEyewY8cOzJ07F0lJSWL25IUXXsBvv/2G2bNn49dff8XHH3+Mr776Cq+88orjR09EJGGGSkW8Ut0Hhqi27JpCunbtGiZMmIA7d+6gadOm6NOnDw4ePIimTZsCAJYtWwa5XI6xY8dCq9UiISEBH3/8sfh6hUKBLVu2YPr06YiLi0NAQAASExOxaNEisU1MTAy2bt2KV155BR988AGaNWuGTz/9lEuoiYjuUXkZtYJnIZGXsSuA2bBhg83nfX19sWLFCqxYscJqm+joaGzbts3mdfr374/jx4/b0zUiIq+jtxTASLCIl6g2eJgjEZFEmYp4FXJZpdOoWQND3oEBDBGRRIlTSJWKeFkDQ96CAQwR1bt9+/Zh5MiRiIyMhEwmw+bNm82ef+aZZ6ocHDt06NBqr7tixQq0aNECvr6+iI2NrbJtg6cxK+JlDQx5GQYwRFTvCgsL0blzZ5v1ckOHDjU7OPbLL7+0ec2NGzciOTkZ8+fPx7Fjx9C5c2ckJCR49Db15kW8XIVE3qVOG9kREdXGsGHDzHb2tkStVtt1VsrSpUsxbdo0PPvsswCAVatWYevWrVizZg1ef/31OvXXXVXOwCgVzMCQd2EGhojcUnp6OkJDQ9GmTRtMnz4dd+7csdq2tLQUR48eNTsIVi6XIz4+HhkZGVZfp9VqodFozD6kRC9YWoXEIl7yDgxgiMjtDB06FJ999hl2796Nt99+G3v37sWwYcOg11s+Nf727dvQ6/UWD4I1HRRrSUpKCoKDg8WPqKgoh47D2SoX8fqwiJe8DKeQiMjtPPnkk+LnHTt2RKdOnXD//fcjPT0dgwYNctjXsXYIrFSIU0iKihoYTiGRt2AGhojcXsuWLdGkSRNcuHDB4vNNmjSBQqGweBCsrToatVqNoKAgsw8pYQaGvBkDGCJye9euXcOdO3cQERFh8XmVSoXu3bubHQRrMBiwe/dujz4ItnIRr0LBnXjJuzCAIaJ6V1BQgMzMTGRmZgIALl26hMzMTGRlZaGgoACzZs3CwYMHcfnyZezevRujRo1Cq1atzM5EGzRoEJYvXy7eT05Oxr/+9S+sW7cOZ8+exfTp01FYWCiuSvJElYt4ldyJl7wMa2CIqN4dOXIEAwYMEO+b6lASExOxcuVK/PLLL1i3bh1yc3MRGRmJIUOGYPHixeKp9QBw8eJF3L59W7w/fvx43Lp1C/PmzUN2dja6dOmC7du3Vyns9SR6s514WQND3oUBDBHVu/79+0MQrP+i3bFjR7XXuHz5cpXHZsyYgRkzZtSla5Kir1TE66NgDQx5F04hERFJlKWzkFgDQ96CAQwRkURZOgtJzxoY8hIMYIiIJMr8LCQeJUDehQEMEZFEGSqvQlKUF/FyCom8BAMYIiKJ0jMDQ16MAQwRkUTpLe7EyxoY8g4MYIiIJMoUwMiZgSEvxACGiEiiTMGKj5xnIZH3YQBDRCRRZmchle/Eq2MRL3kJBjBERBJV+Sykip14WQND3oEBDBGRRFkq4mUNDHkLBjBERBJlaRk1a2DIWzCAISKSKL3ZUQI8jZq8CwMYIiKJsriMWs8aGPIODGCIiCTKVMRrtoyaCRjyEgxgiIgkqnIRr4KrkMjLMIAhIpIo8xoY0xQSUzDkHRjAEBFJFFchkTdjAENEJFFmRbwyBjDkXRjAEBFJlL7yWUgKbmRH3oUBDBGRRJlWIcllFWchMQND3oIBDBGRROl5GjV5MQYwREQSZamIl1NI5C0YwBARSZSlZdTcB4a8BQMYIiKJYgaGvBkDGCIiiTIv4mUNDHkXBjBERBJlMBXxKhjAkPdhAENEJFGm6SK5TAYfLqMmL8MAhohIoiovo65cAyMIDGLI8zGAISKSKEurkACASRjyBgxgiIgkyqyIV1ERwJRxKTV5AQYwREQSVbmIt3IGhnUw5A0YwBARSVTlIl65TFblcSJPxgCGiEiiDBbOQqr8OJEnYwBDRCRRZRZ24q38OJEnYwBDRCRRYhGvXAYZd+MlL8MAhohIoirvAwOA5yGRV2EAQ0T1bt++fRg5ciQiIyMhk8mwefNm8TmdTofXXnsNHTt2REBAACIjIzF58mRcv37d5jUXLFgAmUxm9tG2bVsnj8S19JWKeIGKQEavZwBDno8BDBHVu8LCQnTu3BkrVqyo8lxRURGOHTuGN954A8eOHcPXX3+Nc+fO4U9/+lO11+3QoQNu3Lghfuzfv98Z3XcLlQt1q2ZguA8Meb46BTBvvfUWZDIZZs6cKT5WUlKCpKQkNG7cGA0aNMDYsWORk5Nj9rqsrCyMGDEC/v7+CA0NxaxZs1BWVmbWJj09Hd26dYNarUarVq2Qmppal64SkRsZNmwY3nzzTYwZM6bKc8HBwUhLS8MTTzyBNm3aoFevXli+fDmOHj2KrKwsm9f18fFBeHi4+NGkSRNnDcHlKk8Tye8JYAw8SoC8QK0DmMOHD+OTTz5Bp06dzB5/5ZVX8N///hebNm3C3r17cf36dTz22GPi83q9HiNGjEBpaSkOHDiAdevWITU1FfPmzRPbXLp0CSNGjMCAAQOQmZmJmTNnYurUqdixY0dtu0tEEpaXlweZTIaQkBCb7c6fP4/IyEi0bNkSEydOrDbg0Wq10Gg0Zh9SUTlIMWVgfFgDQ5UZ9K7ugVPVKoApKCjAxIkT8a9//QsNGzYUH8/Ly8Pq1auxdOlSDBw4EN27d8fatWtx4MABHDx4EACwc+dOnDlzBp9//jm6dOmCYcOGYfHixVixYgVKS0sBAKtWrUJMTAyWLFmCdu3aYcaMGXj88cexbNkyBwyZiKSkpKQEr732GiZMmICgoCCr7WJjY5Gamort27dj5cqVuHTpEvr27Yv8/Hyrr0lJSUFwcLD4ERUV5YwhOEXlIEVx7xQSa2Do8GpgUWMg80tX98RpfGrzoqSkJIwYMQLx8fF48803xcePHj0KnU6H+Ph48bG2bduiefPmyMjIQK9evZCRkYGOHTsiLCxMbJOQkIDp06fj9OnT6Nq1KzIyMsyuYWpTearqXlqtFlqtVrxv+ktKp9NBp9NZfZ34XA0iVVvXcRemPkqhr9XxpLEAnjWemozFEePU6XR44oknIAgCVq5cabPtsGHDxM87deqE2NhYREdH46uvvsKUKVMsvmbOnDlITk4W72s0GskEMZWXSlcU8cqrPEdeavciAAKw469Alwmu7o1T2B3AbNiwAceOHcPhw4erPJednQ2VSlUlzRsWFobs7GyxTeXgxfS86TlbbTQaDYqLi+Hn51fla6ekpGDhwoVVHt+5cyf8/f2rHVfA7dPVttm27Zdq27iLtLQ0V3fBYTxpLIBnjcfWWIqKiup0bVPwcuXKFezZs8dm9sWSkJAQPPDAA7hw4YLVNmq1Gmq1uk79dBXbRbwMYLxeSa7xtviuS7vhTHYFMFevXsXLL7+MtLQ0+Pr6OqtPtWLtL6khQ4bYfOPT6XRIS0tDYZMOgFxh82uM69HMYf11FtN4Bg8eDKVS6eru1IknjQXwrPHUZCx1qScxBS/nz5/HDz/8gMaNG9t9jYKCAly8eBGTJk2qdT/cmaUiXh9uZEcAoCs2v6/XAQppv+dYYlcAc/ToUdy8eRPdunUTH9Pr9di3bx+WL1+OHTt2oLS0FLm5uWZZmJycHISHhwMAwsPD8fPPP5td17RKqXKbe1cu5eTkICgoyGL2BbD+l5RSqazZLwu5otoARkq/dGo8bgnwpLEAnjUeW2OxNcaCggKzzMilS5eQmZmJRo0aISIiAo8//jiOHTuGLVu2QK/Xi9nZRo0aQaVSAQAGDRqEMWPGYMaMGQCAV199FSNHjkR0dDSuX7+O+fPnQ6FQYMIEz0yfm4p4K5+BJOcyagKA4j/M7xfeAoIiXdMXJ7KriHfQoEE4efIkMjMzxY8ePXpg4sSJ4udKpRK7d+8WX3Pu3DlkZWUhLi4OABAXF4eTJ0/i5s2bYpu0tDQEBQWhffv2YpvK1zC1MV2DiKTtyJEj6Nq1K7p27QoASE5ORteuXTFv3jz8/vvv+O6773Dt2jV06dIFERER4seBAwfEa1y8eBG3b98W71+7dg0TJkxAmzZt8MQTT6Bx48Y4ePAgmjZtWu/jqw/iSdSVAhhTMMP4xctVCWBuW24ncXZlYAIDA/Hggw+aPRYQEIDGjRuLj0+ZMgXJyclo1KgRgoKC8OKLLyIuLg69evUCAAwZMgTt27fHpEmT8M477yA7Oxtz585FUlKSmEF54YUXsHz5csyePRvPPfcc9uzZg6+++gpbt251xJiJyMX69+8PwcZeJbaeM7l8+bLZ/Q0bNtS1W5JiqoFRyCoCGG5kRwCqBjBa6yvxpKxWq5BsWbZsGeRyOcaOHQutVouEhAR8/PHH4vMKhQJbtmzB9OnTERcXh4CAACQmJmLRokVim5iYGGzduhWvvPIKPvjgAzRr1gyffvopEhISHN1dIiJJKjNUnUJiDQwBsBDASGd/I3vUOYBJT083u+/r64sVK1ZY3CLcJDo6Gtu2bbN53f79++P48eN17R4RkUfSW5hC4iokAuA1GRiehUREJEGmIl6FWQaG+8AQqgYwJXmu6YeTMYAhIpKge0+iBpiBoXLMwBARkbsyBTCKSu/i4mGODGC8m7bgnvueWQPDAIaISILEKSRmYOhepo3s5OVlrszAEBGRu7BUxFuxConLqL2artB469+k/H6J6/riRAxgiIgkyFIRLzMwBAAoLT+HLMAUwNTtXDJ3xQCGiEiC9OVJlspTSD4K7gNDqAhY/BsZb8uYgSEiIjdheR8Y41t6mZ4BjFcTA5jG5vc9DAMYIiIJslTEy514CUDFFJIYwDADQ0REbsJSBsa0J4y+BmdJkQdjBoaIiNyVpX1gmIEhAFUDGNbAEBGRu6gIYCrexhXlRbysgfFypn1gxAxMsev64kQMYIiIJEgv1sBUPMZ9YAiCAJSa9oFhAENERG7GYOA+MGRBWQmA8u8/AxgiInI3pgyMnKuQqLLKwYq4D0yxMTPjYRjAEBFJkN5iBkZu9hx5IdP0kUIFqAONnwsGQF/quj45CQMYIiIJsnyUgPGWU0hezLQCSekP+PhVetzzppEYwBARSZDpKAG5jBkYqsQUwKgCAIUSkCnKH2cAQ0REbsBSEa8Pi3jJtOuujy8gkxlvAY/cC4YBDBGRBFkq4lVwGTWVVQpgAMBHZbxlDQwREbkDWzvxMgPjxcq0xlsftfFWoTZ/3IMwgCEikiDLRbxcRu319PcEMMzAEBGROxEPc7Q4hcQAxmsxA0NERO7M0j4w3MiOxEDFFLiYAhkW8RIRkTsQp5AsLKNmDYwXE4t4TRkYTiEREZEbEfeBYQaGKhOnkEyrkDiFREREbsRyBoarkLyeWMRbnnlhBoaIiNyJWMRbOQOj4D4wXo8ZGCIicmeW9oHhKiSqUsQrZmAYwBARkRuwOIUkYwDj9e5dRi1mYDiFREREbsDSFBJrYKjKUQKmTAwzMERE5A70FjIwFTUwDGC8lqlY11TEa7plBoaIiNyBpdOoxX1g9AxgvBYzMERE5M64DwxZJBbxmjIwXIVEROQw+/btw8iRIxEZGQmZTIbNmzebPS8IAubNm4eIiAj4+fkhPj4e58+fr/a6K1asQIsWLeDr64vY2Fj8/PPPThqB65mWSlveB4bLqL2WtWXU3AeGiKjuCgsL0blzZ6xYscLi8++88w4+/PBDrFq1CocOHUJAQAASEhJQUmL9PJeNGzciOTkZ8+fPx7Fjx9C5c2ckJCTg5s2bzhqGS+ktnEZtysAwAePF7j2Nmoc5EhE5zrBhw/Dmm29izJgxVZ4TBAHvv/8+5s6di1GjRqFTp0747LPPcP369SqZmsqWLl2KadOm4dlnn0X79u2xatUq+Pv7Y82aNU4cieuYppAqBzByZmCoyjJq7sRLRAQA+O2335x6/UuXLiE7Oxvx8fHiY8HBwYiNjUVGRobF15SWluLo0aNmr5HL5YiPj7f6GgDQarXQaDRmH1JhqYhXrIFhEa/3slbEywwMEXm7Vq1aYfDgwUhPT7c5pVNb2dnZAICwsDCzx8PCwsTn7nX79m3o9Xq7XgMAKSkpCA4OFj+ioqLq2Pv6Y5pCkvMsJKrMtFxacc8yamZgiMjbHTt2DB07dsTatWsRFRWFP//5z5Itlp0zZw7y8vLEj6tXr7q6SzVmsHCUgE/5MmquQvJizMAQEVnWpUsXLF26FGvWrME///lP3LhxA3369MGDDz6IpUuX4tatW3W6fnh4OAAgJyfH7PGcnBzxuXs1adIECoXCrtcAgFqtRlBQkNmHVDADQxZV2ciO+8AQEZlRKBQYM2YMNm3ahLfffhsXLlzAq6++iqioKEyePNnm1I0tMTExCA8Px+7du8XHNBoNDh06hLi4OIuvUalU6N69u9lrDAYDdu/ebfU1Uqe3UQNjYADjvapkYLgTLxGRmQsXLuDFF19EREQEli5dildffRUXL15EWloarl+/jgkTJlh9bUFBATIzM5GZmQnAWLibmZmJrKwsyGQyzJw5E2+++Sa+++47nDx5EpMnT0ZkZCRGjx4tXmPQoEFYvny5eD85ORn/+te/sG7dOpw9exbTp09HYWEhnn32WWf9E7iUwcIyamZgyOphjh6YgfFxdQeISFpM00fnzp3DsGHD8Nlnn2H48OGQl9dfxMTEIDU1FS1atLB6jSNHjmDAgAHi/eTkZABAYmIiUlNTMXv2bBQWFuL5559Hbm4u+vTpg+3bt8PX11d8zcWLF3H79m3x/vjx43Hr1i3MmzcP2dnZ6NKlC7Zv316lsNdTiIc5WphCYg2MFxN34jXtA6M03up1rumPEzGAISK7rFy5EomJibjvvvvw9NNPQ6lUVmkTGhqK5cuXY/r06Rav0b9/fwiC9V+yMpkMixYtwqJFi6y2uXz5cpXHZsyYgRkzZlQ/CA9gaR8YH+4D490MBsBQHqjcO4XEVUhE5O3S0tIwa9YsNGrUyOxxQRCQlZUFwFiT8tRTT7mie17DYOE0akWlnXhZB+OFKk8TmYp4GcAQERndf//9ZlM3Jnfv3kVMTIwLeuSdxCkkswxMxVu63kaGizxUWaV9mcQMjGkKqaz+++NkDGCIyC7Wpn4KCgrMalTIuSqKeCseUygqghnWwXgh00ojmRyQl1eIeHAGhjUwRFQjpkJbmUyGhQsXIicnBz/88APkcjn0ej0OHTqELl26uLaTXsRSEa+PnAGMVzNlYBRqwPT/Qm7KwDCAISIvdfz4cQDGDMypU6eQn5+PvLw8yGQyqFQqdO7cGa+++qqLe+k9LO0DUzmY4VJqL3TvEmqAq5CIiH744QcAwLPPPov33nsP+/fvx/Dhwy2uQiLns1TEywyMl9NbCmDKp5AMDGCIyMutXbsWOp3nvRlKjaUiXrlcBpkMEAQupfZKFjMwlWpgBKFiaskD2FXEu3LlSnTq1Ek8MyQuLg7ff/+9+HxJSQmSkpLQuHFjNGjQAGPHjq1yNklWVhZGjBgBf39/hIaGYtasWSgrM6+OTk9PR7du3aBWq9GqVSukpqbWfoREVGePPfYYNBqN+Pm4cePw1ltvYdy4cXjsscfMPqh+6MsTLIp7fiH5cDM773XvJnZAxRQSABg8ayWSXQFMs2bN8NZbb+Ho0aM4cuQIBg4ciFGjRuH06dMAgFdeeQX//e9/sWnTJuzduxfXr183e0PT6/UYMWIESktLceDAAaxbtw6pqamYN2+e2ObSpUsYMWIEBgwYgMzMTMycORNTp07Fjh07HDRkIrJXcHAwZOW/KIODgxEcHAx/f3/x88ofVD8MFmpgKt8v0zOA8Tr3noMEmAcwHlbIa9cU0siRI83u//3vf8fKlStx8OBBNGvWDKtXr8b69esxcOBAAMZUc7t27XDw4EH06tULO3fuxJkzZ7Br1y6EhYWhS5cuWLx4MV577TUsWLAAKpUKq1atQkxMDJYsWQIAaNeuHfbv349ly5YhISHBQcMmInusXbvW7HOdTodt27axBsaFLE0hAaa9YAxijQx5EVtTSEB5ABNQr11yplrXwOj1emzatAmFhYWIi4vD0aNHodPpEB8fL7Zp27YtmjdvjoyMDPTq1QsZGRno2LGj2dkkCQkJmD59Ok6fPo2uXbsiIyPD7BqmNjNnzrTZH61WC622YhdCU7pbp9PZnK8XnzPoqx2zFOb9TX2UQl+r40ljATxnPMXFxSgtNf4lp9PpcOXKFXz77bdo164dBg8eLLaT+jjdnaUiXoAHOno1S0W88kq/5j1sMzu7A5iTJ08iLi4OJSUlaNCgAb755hu0b98emZmZUKlUCAkJMWsfFhaG7OxsAEB2dnaVg9VM96tro9FoUFxcDD8/P4v9SklJwcKFC6s8vnPnTvj7+1c7roDbp6tts23bL9W2cRdpaWmu7oLDeNJYAOmPZ/78+YiLi8PQoUPxzTffICkpCT4+PsjPz8ezzz6LYcOGAQCKiopc3FPPVpGBMX+cBzp6MUsZGJnMmIXRl3r3FBIAtGnTBpmZmcjLy8O///1vJCYmYu/evc7om13mzJkjbrQFGDMwUVFRGDJkCIKCgqy+TqfTIS0tDYVNOgByhc2vMa5HM4f111lM4xk8eLDkU/ueNBbAc8YzZcoUTJkyBb///jvy8vLQvHlzHD58GF9//TUWLlyIjz76CEBFFpScQ19dBoY1MN7HUhEvwADGRKVSoVWrVgCA7t274/Dhw/jggw8wfvx4lJaWIjc31ywLk5OTg/DwcABAeHg4fv75Z7PrmVYpVW5z78qlnJwcBAUFWc2+AIBarYZara7yuFKprNkvC7mi2gBGSr90ajxuCfCksQDSH09RUREaNmyI33//HXv27MHYsWOhVqvRp08fZGVliWOT8hilwNJGdgBXIXk1SxkYoGIaycM2s6vzWUgGgwFarRbdu3eHUqnE7t27xefOnTuHrKwsxMXFAQDi4uJw8uRJ3Lx5U2yTlpaGoKAgtG/fXmxT+RqmNqZrEJFrtWrVCt999x1u3bqFtLQ0DBkyBABw8+ZNm9lOcixrAUxFDQz3gfE6Yg3MPWeSeeh5SHZlYObMmYNhw4ahefPmyM/Px/r165Geno4dO3YgODgYU6ZMQXJyMho1aoSgoCC8+OKLiIuLQ69evQAAQ4YMQfv27TFp0iS88847yM7Oxty5c5GUlCRmT1544QUsX74cs2fPxnPPPYc9e/bgq6++wtatWx0/eiKy27x58/DUU09Br9dj4MCB4h8XO3fuRNeuXV3cO+9hbRm1KQPDVUheSFxGrTJ/3EN347UrgLl58yYmT56MGzduIDg4GJ06dcKOHTvElQfLli2DXC7H2LFjodVqkZCQgI8//lh8vUKhwJYtWzB9+nTExcUhICAAiYmJWLRokdgmJiYGW7duxSuvvIIPPvgAzZo1w6effsol1ERu4vHHH0dsbCz+/e9/4//+7//ExwcNGoQxY8a4sGfexVQDI2cNDJmUWcvAeOZ5SHYFMKtXr7b5vK+vL1asWIEVK1ZYbRMdHY1t27bZvE7//v3Fg+OIyP2Eh4ejZcuWkFdaAtOzZ08X9sj76MtniKpmYOTlzzOA8TpiEe+9GRjPPJGaZyERkV0KCwvx97//HV9//TX+8pe/wHBPrcVvv/3mop55F3EfmHsCGDn3gfFeVjMwrIEhIsLUqVOxd+9e9OrVCw8//DB8fPg24griPjA8C4lMLG1kB3AKiYgIAL7//nt8++23yM3N5VECLlTtWUgMYLyPtWXUYgbGswKYOi+jJiLv0rBhQzRs2NDV3fB61jayq8jAcBm117F0mCPgsVNIDGCIyC6LFy/GwoULzc4eo/pX/VEC9d0jcrmy8gDl3iJeD93IjlNIRGSXJUuW4OLFi9i2bRsWLVoElcr8zfLYsWMu6pl3sVbE66PgRnZey8syMAxgiMguo0ePhl6vx/nz59G6dWsoFLaP4CDnEHfirbIPDJdRey1TgFJlI7vyOjVv3siOiGj+/PnQ6XTYtm0bi3hdRBAEmOIT+b1FvOV3WcTrharNwHhWAMMaGCKyW25uLtLS0vC3v/0Nd+/eBWCcOvr9999d3DPvUDk2YQaGRNWuQuIUEhF5sV9++QXx8fHw8fHBrVu38MILL6BRo0b4+uuvkZWVhc8++8zVXfR4lYOTezMwPlxG7b3EnXjvDWBMRbyeFcAwA0NEdklOTsakSZOwcuVK+PpWpKqHDx+Offv2ubBn3qPyQY1V9oEpn0MyMIDxPuIUEveBISKq4vDhw5g2bVqVx++77z5kZ2e7oEfep3IGxto+MMzAeCGxiJcBDBFRFWq1GhqNpsrj//vf/9C0aVMX9Mj76IXKU0jmzym4kZ33slrE65mHOTKAISK7/OlPf8Lf//53lJWVAQBkMhmysrLw2muvYezYsS7unXcw2MjAmO4zA+OFrBXxyj3zLCQGMERklyVLlqCwsBCTJ09GcXEx+vXrh1atWiEwMBB///vfXd09r2A2hWRlIzu9ngGM1+Fp1ERE1gUHB+P777/HkiVLEBAQgOLiYnTr1g3x8fGu7prXME0hyWTGDFhlPMzRS+nLAEFv/PzeowQ8dAqJAQwR1ZjBYEBqair+85//4NSpUwgMDETLli0RHh4OQRCq/DIl5zCVt9w7fQQAPtwHxjuZ6l8A6xkYQ1n99acecAqJiGpEEAT86U9/wtSpU/H7778jOjoa7du3x5UrV/DMM89gzJgxru6i1zBlYO7dAwaoVMQrMIDxKpWzK16ykR0DGCKqkdTUVOzbtw+7d+/G4cOH8Ze//AVffPEFTpw4gV27dmHPnj3cxK6eGKycgwRULKNmBsbLmDIwch9Afs/5ZNzIjoi82Zdffom//vWvGDBgQJXnBg4ciNdffx1ffPGFw75eixYtIJPJqnwkJSVZbJ+amlqlbeWN9jyJqb7l3gLeyo+VsYjXu1hbQg1wHxgi8m6//PILhg4davX5YcOG4cSJEw77eocPH8aNGzfEj7S0NADAuHHjrL4mKCjI7DVXrlxxWH/cib4GAQz3gfEyZVY2sQM8dgqJRbxEVCN3795FWFiY1efDwsLwxx9/OOzr3bsp3ltvvYX7778f/fr1s/oamUyG8PBwh/XBXZmOErCZgeEUkncxZWDuPQcJqLQKiRkYIvJCer0ePj7W/+ZRKBTi5naOVlpais8//xzPPfeczZVOBQUFiI6ORlRUFEaNGoXTp0/bvK5Wq4VGozH7kAJTBkbOGhgysbaJHeCxU0jMwBBRjQiCgGeeeQZqtRoGgwE5OTlYu3Yt5OXLdrVardO+9ubNm5Gbm4tnnnnGaps2bdpgzZo16NSpE/Ly8vDee++hd+/eOH36NJo1a2bxNSkpKVi4cKGTeu08FVNIVZ9TcBm1d7JVAyPnPjBE5MUSExPFzw0GA/Lz8xEcHCwGMAAwefJkp3zt1atXY9iwYYiMjLTaJi4uDnFxceL93r17o127dvjkk0+wePFii6+ZM2cOkpOTxfsajQZRUVGO67iTiFNIzMCQiXiQo6rqcx46hcQAhohqZO3ateLnOp0O27Ztw/Dhw6FUKp36da9cuYJdu3bh66+/tut1SqUSXbt2xYULF6y2UavVUKstpNzdnDiFxBoYMqnJKiSDZwUwrIEhIre2du1ahIaGYsSIEXa9Tq/X4+TJk4iIiHBSz1zHVhGveBYSAxjvUqMaGM+aQmIAQ0Ruy2AwYO3atUhMTKxSQDx58mTMmTNHvL9o0SLs3LkTv/32G44dO4ann34aV65cwdSpU+u7206nt3GUgFw8jZrLqL2KKYDxolVInEIiIre1a9cuZGVl4bnnnqvyXFZWlln9zR9//IFp06YhOzsbDRs2RPfu3XHgwAG0b9++PrtcL2xNIbEGxkuJU0i2AhjPysAwgCEitzVkyBAIVs70SU9PN7u/bNkyLFu2rB565Xq2inhZA+OlxCkkWzvxelYAwykkIiKJsZmBYQ2Md9LbqoExZWB4GjUREbmQXuA+MHQPFvESEZG7q8lp1JxC8jI13cjOypSsFDGAISKSmJrsA8MMjJexeZijaa8mATDo661LzsYAhohIYmwW8cqYgfFKNg9zrLQ7rwdNIzGAISKSGNM+MBYzMGIRL/eB8So1qYEBPGo3XgYwREQSo6/RWUj12iVyNb2tZdSVjvvwoM3sGMAQEUmMWMRrswaGEYxXEYt4LRzmKJMB8vJt3ziFRERErmJ7J17j2zprYLyMrY3sAI9cSs0AhohIYiqmkKo+x1VIXsrWMmrAI89DYgBDRCQxNZlCKtMzgPEqpmXUCgtTSJUfZwBDRESuUmYjgOFhjl6q2gyMKYDR1k9/6gEDGCIiiTHtA+Mjr/oWzsMcvZStZdQAp5CIiMj1TNNDtjIwBg/aMp5qwNZhjkDFBncs4iUiIlcxTQ/52KyB4TJqr1JtBoarkIiIyMXKarCMmjUwXoarkIiIyN2ZNqmzmIEpX1utYwDjXWq6CqmMRbxEROQiptkhSzUwSq5C8k41zsBwComIiFzEVgbGR1ExhcQgxksYDBWHNFa7jJpTSERE5CIV+8BUfQtXVtqeV8dCXu9QeW8Xa0W8Pp63CsnH1R0gIjdyZG3N2nV+2rn9IJv0YgBT9TllpQd1egN8lYr66ha5imn6CKjBPjCeE8AwA0NEJDG2MzCVAxhOIXkFXXkAI1NUnDp9L2+fQkpJScFDDz2EwMBAhIaGYvTo0Th37pxZm5KSEiQlJaFx48Zo0KABxo4di5ycHLM2WVlZGDFiBPz9/REaGopZs2ahrKzMrE16ejq6desGtVqNVq1aITU1tXYjJCLyMNXtA2Mq7uUUkpfQFRlvlX6AzMIJnwCPEti7dy+SkpJw8OBBpKWlQafTYciQISgsLBTbvPLKK/jvf/+LTZs2Ye/evbh+/Toee+wx8Xm9Xo8RI0agtLQUBw4cwLp165Camop58+aJbS5duoQRI0ZgwIAByMzMxMyZMzF16lTs2LHDAUMmIpI2vY2zkICKOhgGMF7CNIWk9LPexgP3gbGrBmb79u1m91NTUxEaGoqjR4/ikUceQV5eHlavXo3169dj4MCBAIC1a9eiXbt2OHjwIHr16oWdO3fizJkz2LVrF8LCwtClSxcsXrwYr732GhYsWACVSoVVq1YhJiYGS5YsAQC0a9cO+/fvx7Jly5CQkOCgoRMRSZOtwxwB4zRSic7AKSRvYZpC8rEVwHjeTrx1KuLNy8sDADRq1AgAcPToUeh0OsTHx4tt2rZti+bNmyMjIwO9evVCRkYGOnbsiLCwMLFNQkICpk+fjtOnT6Nr167IyMgwu4apzcyZM632RavVQqutSI1pNBoAgE6ng05nPeIUnzPoqx2vreu4C1MfpdDX6njSWACJjKeGf7DXZCxuPU6JMy2jthbAqMrrYJiB8RKVp5CsYQBTwWAwYObMmXj44Yfx4IMPAgCys7OhUqkQEhJi1jYsLAzZ2dlim8rBi+l503O22mg0GhQXF8PPr+o3KSUlBQsXLqzy+M6dO+Hv71/teAJun662zbZtv1Tbxl2kpaW5ugsO40ljAdx9PI1q1uyGcQy2xlJUVOSIDpEFZTZqYICKQt7SMgYwXkGcQrKyBwzgkUW8tQ5gkpKScOrUKezfv9+R/am1OXPmIDk5Wbyv0WgQFRWFIUOGICgoyOrrdDod0tLSUNikAyC3vdxwXI9mDuuvs5jGM3jwYCiVSld3p048aSyARMZz/PMaNdM9OL7asZiyoOR4huqmkHxYA+NVxAyMjT/WmYExmjFjBrZs2YJ9+/ahWbOKX+rh4eEoLS1Fbm6uWRYmJycH4eHhYpuff/7Z7HqmVUqV29y7ciknJwdBQUEWsy8AoFaroVZXXf+uVCpr9stCrqg2gHHbXzoW1HjcEuBJYwHcfDw1Lesv77+tsbjtGD1ATWpgAC6j9hq6ao4RACqKeL31LCRBEDBjxgx888032LNnD2JiYsye7969O5RKJXbv3i0+du7cOWRlZSEuLg4AEBcXh5MnT+LmzZtim7S0NAQFBaF9+/Zim8rXMLUxXYOIyJvZWkYNsAbG69hVA+OlU0hJSUlYv349vv32WwQGBoo1K8HBwfDz80NwcDCmTJmC5ORkNGrUCEFBQXjxxRcRFxeHXr16AQCGDBmC9u3bY9KkSXjnnXeQnZ2NuXPnIikpScygvPDCC1i+fDlmz56N5557Dnv27MFXX32FrVu3Onj4RETSY2sjO6BSDQwDGO9Qo2XUnjeFZFcGZuXKlcjLy0P//v0REREhfmzcuFFss2zZMjz66KMYO3YsHnnkEYSHh+Prr78Wn1coFNiyZQsUCgXi4uLw9NNPY/LkyVi0aJHYJiYmBlu3bkVaWho6d+6MJUuW4NNPP+USaiIiVJ+BEfeBYRGvdzBlYGwto/bxvADGrgyMIFQ/n+rr64sVK1ZgxYoVVttER0dj27ZtNq/Tv39/HD9+3J7uERF5heo2svNhDYx30dmTgfGcKSSehUREJDHVBTCsgfEyYg1MTZZRe04GhgEMEZHElFWzkZ1pCok1MF5CrIGxtYyap1ETEZGLVV8DwwyMV6nRMmpmYIiI6sWCBQsgk8nMPtq2bWvzNZs2bULbtm3h6+uLjh07VltrJ1XV7gPjUx7AsIjXO3jpRnYMYIjIbXXo0AE3btwQP2zt/H3gwAFMmDABU6ZMwfHjxzF69GiMHj0ap06dqsce14/qduI11cCYAh3ycF56lAADGCJyWz4+PggPDxc/mjRpYrXtBx98gKFDh2LWrFlo164dFi9ejG7dumH58uX12OP6Uf1OvKyB8SrMwBARuZfz588jMjISLVu2xMSJE5GVlWW1rbVT7DMyMqy+RqvVQqPRmH1IQUUNjO2N7HRlzMB4BdbAEBG5j9jYWKSmpmL79u1YuXIlLl26hL59+yI/P99ie2un2Jt2DLckJSUFwcHB4kdUVJRDx+AsNT8LiRkYr1CjowRMZyExgCEicqphw4Zh3Lhx6NSpExISErBt2zbk5ubiq6++ctjXmDNnDvLy8sSPq1evOuzaziRmYBRWamB8GMB4FS89SqBWp1ETEdW3kJAQPPDAA7hw4YLF562dYm865d4Sa6fYuztTACOXsQaGAOiKjbe2jhIQ94FhES8RUb0qKCjAxYsXERERYfF5bzrFnvvAkBlTAGMrA+NTHqh7UAaGAQwRuaVXX30Ve/fuxeXLl3HgwAGMGTMGCoUCEyZMAABMnjwZc+bMEdu//PLL2L59O5YsWYJff/0VCxYswJEjRzBjxgxXDcFpqt+Jl0W8XkUMYGpYxFuDcw2lgFNIROSWrl27hgkTJuDOnTto2rQp+vTpg4MHD6Jp06YAgKysLMgrrcLp3bs31q9fj7lz5+Kvf/0rWrdujc2bN+PBBx901RCcptoaGGZgvIcgAGWmAKYGRwlAAAx6QCH9X//SHwEReaQNGzbYfD49Pb3KY+PGjcO4ceOc1CP3IR7maKUGxoc1MN5DXwoI5d/nmiyjBgC91iMCGE4hERFJDJdRk6i0sOJzVYD1dmYBjGfUwTCAISKSmOo2squYQvKMWgeywRTAKNSVpokskFfKuHjISiQGMEREEiNmYKzUwCh9jI8zA+MFTAGMrewLAMhkxiAHYAaGiIhco7oaGNMUUilPo/Z8YgDToPq2HraZHQMYIiIJEQShIoBhDQyVFhhvq8vAAB63mR0DGCIiCTFUKmuxtpEda2C8SE2nkICKDEyZ1nn9qUcMYIiIJMS0iR1gvQZGXX4WkrZMXy99IhcSAxgbe8CYiFNIzMAQEVE901dKwVjLwKiVCgBAiY5TSB5PnEKqSQ2MaQqJNTBERFTPyioFMNYOc2QGxovYM4XkYechMYAhIpIQQw0yML7MwHgPu2pgWMRLREQuUrkw19oqJFMGpkTHDIzHs2sKicuoiYjIRUxFvD5yGWRWppBMGRhtmQGCh5w8TFboioy39qxC0nMVEhER1bMyve2TqAHAV1nx1q7lZnaejVNIREQkBabN6ZRWzkECALWPQvycAYyHs2sjO+4DQ0RELmJahWQrA6NUyGAqj9GyDsaz2XOUgLgKiQEMERHVM1MGxkdh/e1bJpOJWRiuRPJwdi2j9jXeMgNDRET1zVQDo7SyAsnEVAfDvWA8XG32gSkrcV5/6hEDGCIiCTGtQlL62H775l4wXkKrMd7WaArJlIHhMmoiIqpnpn1grG1iZ8LdeL1ESXkA4xtcfVsxgGEGhoiI6pk4hWSjBgZgBsYrCAKgzTd+rg6qvr04hcQaGCIiqmc600Z2NlYhAdyN1yuUFgJC+ffXtyYBDDMwRETkIroy0068tt++1ZV24yUPVZJnvJUpAKV/9e2ZgSEiIlcx7QOjZAaGtJXqX6wcK2GGGRgiInIVcR+YajIwvszAeD6xgLcG00cAd+IlIiLXqclZSEDlIl5mYDyWKQNTkwJegBkYIiJyHXEfmGpWIYlTSFxG7blMNTA1WUINsAaGiIhcp6b7wPiZMjClDGA8limAYQaGiIjcXZm+ZhmYALUPAKBAywDGY2nt2MQOYAaGiIhcpyanUQNAA7UxA1NUWub0PpGL2FvEywwMERG5SsUUku23b3+VKQPDAMZj1bqIlxkYIiKqZxVTSNVlYIwBTBFrYDyX3RkYnkZNREQuoqvhFJJ/+RQSMzAerPgP461fw5q1ZwaGiIhcxd4i3kIGMJ6r6I7x1q9RzdqbMjB6BjBERFTPKo4SqCaAUXEKyeMV3zXe+jeuWfvKRbyC4Jw+1SMGMEREElJxlIDtKaQATiF5viJTAGNnBgYA9KWO7089YwBDRCQhFUcJ1DADwwDGM+l1FauQ7M3AAB5RyMsAhohIQsSjBKrNwJTXwJTqYTBIf7qA7mEq4IWs5hvZKZTG9oBHFPIygCEit5SSkoKHHnoIgYGBCA0NxejRo3Hu3Dmbr0lNTYVMJjP78PX1tfkaqdHVNANTPoUEAMU80NHziAW8DQG5wnZbE5nMo5ZS2x3A7Nu3DyNHjkRkZCRkMhk2b95s9rwgCJg3bx4iIiLg5+eH+Ph4nD9/3qzN3bt3MXHiRAQFBSEkJARTpkxBQUGBWZtffvkFffv2ha+vL6KiovDOO+/YPzoikqy9e/ciKSkJBw8eRFpaGnQ6HYYMGYLCwkKbrwsKCsKNGzfEjytXrtRTj+uHrob7wPgpFZCVN+FKJA9kb/2LiQcdJ2B3AFNYWIjOnTtjxYoVFp9/55138OGHH2LVqlU4dOgQAgICkJCQgJKSimhv4sSJOH36NNLS0rBlyxbs27cPzz//vPi8RqPBkCFDEB0djaNHj+Ldd9/FggUL8M9//rMWQyQiKdq+fTueeeYZdOjQAZ07d0ZqaiqysrJw9OhRm6+TyWQIDw8XP8LCwuqpx/WjtMwYwJhOm7ZGJpOJdTAs5PVApgxMTetfTEx1MLpix/bHBXzsfcGwYcMwbNgwi88JgoD3338fc+fOxahRowAAn332GcLCwrB582Y8+eSTOHv2LLZv347Dhw+jR48eAICPPvoIw4cPx3vvvYfIyEh88cUXKC0txZo1a6BSqdChQwdkZmZi6dKlZoEOEXmPvDzjybuNGtn+i7OgoADR0dEwGAzo1q0b/vGPf6BDhw4W22q1Wmi1FX+JajQax3XYSbTlAYyqmgAGAAJ9fVCgLYOmhAGMxzEtoa7pHjAmHnQekt0BjC2XLl1CdnY24uPjxceCg4MRGxuLjIwMPPnkk8jIyEBISIgYvABAfHw85HI5Dh06hDFjxiAjIwOPPPIIVCqV2CYhIQFvv/02/vjjDzRsWHXXQWtvRDqdDjqdzmqfxecM1c8R27qOuzD1UQp9rY4njQWQyHgMNWtWk7E4cpwGgwEzZ87Eww8/jAcffNBquzZt2mDNmjXo1KkT8vLy8N5776F37944ffo0mjVrVqV9SkoKFi5c6LB+1odSOwKYEH8VbuSVILdI+ktm6R5iBsbOAEYVYLwttT0VKwUODWCys7MBoErKNiwsTHwuOzsboaGh5p3w8UGjRo3M2sTExFS5huk5SwGMtTeinTt3wt/fv9q+B9w+XW2bbdt+qbaNu0hLS3N1FxzGk8YCuPt4avhmeMM4BltjKSoqckSHAABJSUk4deoU9u/fb7NdXFwc4uLixPu9e/dGu3bt8Mknn2Dx4sVV2s+ZMwfJycnifY1Gg6ioKIf12xnEAEZRfeFmQ38lACC3yI2DZqqd/BzjbYNQ2+3upSz/fahz3M+nqzg0gHEla29EQ4YMQVCQ9YOudDod0tLSUNikQ7WV3ON6VP0Lzt2YxjN48GAolUpXd6dOPGksgETGc/zzGjXTPTi+2rE4ajpmxowZYq2cpSyKLUqlEl27dsWFCxcsPq9Wq6FWqy0+5660+prVwABAQ39jFvsPZmA8T4HxD340CLfvdSpTAOOFNTC2hIcb/yFzcnIQEREhPp6Tk4MuXbqIbW7evGn2urKyMty9e1d8fXh4OHJycszamO6b2tzL2huRUqms2S8LuaLaAMZtf+lYUONxS4AnjQVw8/HUtKy/vP+2xlLXMQqCgBdffBHffPMN0tPTq2Rla0Kv1+PkyZMYPnx4nfriTuybQjJ+D/5gBsbzFJT/Hg20s0hd6TlTSA7dByYmJgbh4eHYvXu3+JhGo8GhQ4fEtG5cXBxyc3PNVhLs2bMHBoMBsbGxYpt9+/aZzaGnpaWhTZs2FqePiMjzJCUl4fPPP8f69esRGBiI7OxsZGdno7i44i/HyZMnY86cOeL9RYsWYefOnfjtt99w7NgxPP3007hy5QqmTp3qiiE4RWmZsV6vJgGMKQPDGhgPlG/KwNgbwPgZbz1gCsnuAKagoACZmZnIzMwEYCzczczMRFZWFmQyGWbOnIk333wT3333HU6ePInJkycjMjISo0ePBgC0a9cOQ4cOxbRp0/Dzzz/jp59+wowZM/Dkk08iMjISAPDUU09BpVJhypQpOH36NDZu3IgPPvjAbIqIiDzbypUrkZeXh/79+yMiIkL82Lhxo9gmKysLN27cEO//8ccfmDZtGtq1a4fhw4dDo9HgwIEDaN++vSuG4BSlevszMKyB8TCCABSYamDsDGBMU0gekIGxewrpyJEjGDBggHjfFFQkJiYiNTUVs2fPRmFhIZ5//nnk5uaiT58+2L59u9lumF988QVmzJiBQYMGQS6XY+zYsfjwww/F54ODg7Fz504kJSWhe/fuaNKkCebNm8cl1EReRKjBabnp6elm95ctW4Zly5Y5qUfuoaKIlzUwXkubX5FBCbSzBsY0heQBGRi7A5j+/fvbfGORyWRYtGgRFi1aZLVNo0aNsH79eptfp1OnTvjxxx/t7R4RkUer6UZ2ANAwgBkYj2TKvqgCK5ZF15QHFfHyLCQiIgmxZyM7UwbmToH0t42nSkz1L/YW8AIs4iUiItewZxVSeLBx6v5mvhZ6nkjtOTS/G28DI2y3s0TlOfvAMIAhIpIIg0FAWXkgUpMamNBAXyjkMpQZBGZhPEnuVeNtSLT9rzVtZFfKAIaIiOqJaQUSAKiV1e/Eq5DLEBpo3B/rep70z76hcrnlJ6yH1GLXaHEnXk4hERFRPTHVvwA1y8AAQET5NFJ2nvSLNqlcnikD09z+17KIl4iI6ltppQBGqZDV6DURwcaNy67nMgPjMXKzjLe1CWA4hURERPWt8iZ2MllNAxhjBub3XOn/xU0ADAYg75rx8+BaTCGZll1zComIiOqLuAdMDaePAKBl0wYAgAs3C5zSJ6pnBdmAvhSQKYCgSPtfzwwMERHVN3uWUJu0DmMA41FunzfeNmwBKGpxYKrKc3biZQBDRCQRxTrjQY6+NViBZNI61BjA/J5bjAJtmVP6RfXo9v+Mt03b1O71ykpnIdXguA53xgCGiEgiikqNAYi/quYBTIi/Ck3Ll1Kfy853Sr+oHpkCmCata/d6UwZG0ANl0t4biAEMEZFEFJcaMzB+dgQwANC5WTAA4HjWHw7vE9WzW+eMt01qmYFRNQBQXgCu1TikS67CAIaISCJMU0h+dkwhAUCPFo0AAIcv33V4n6ieiRmYB2r3erkc8A0yfl6S55g+uQgDGCIiiSgqz8DYM4UEAA+VBzA/X7rLM5GkLD8HyL8ByORAaLvaX8fXmJFjAENERPWiLlNIwX5K/FGkwxFmYaTr+jHjbZM2gLpB7a+jNgUwuXXukisxgCEikoiKKSQfu17no5BjULtQAMCO0zkO7xfVk9/LA5j7utXtOmIGhjUwRERUD2o7hQQAwx6MAAB8d+I6dJUOhSQJ+f2I8Taya92uwykkIiKqT8W1WEZt0r9NUzQNVON2gRa7zjALIzm6EuBKhvHz6N51uxaLeImIqD7VZiM7E6VCjid6NAMArP85y6H9onpw9SBQVgw0CAdC29ftWqYMDJdRExFRfajLFBIAPPlQc8hkwI/nb+PXbGn/8vI6/9tpvL1/AFDDgzytUjMDQ0RE9ai4jgFMVCN/DO9orIVZmX7RYf0iJ9PrgJObjJ+3H1X367EGhoiI6lNdppBMpve7HwDw3xPXceVOoUP6RU72v+1A4U3AvwnQKr7u1/MLMd4WS3tnZgYwREQSkV9iLOIN9LVvGXVlD94XjP5tmsIgAB//wCyM2zMYgL1vGz/vNrl2J1DfK8C4pB4FN+t+LRdiAENEJBGaYh0AIMivbr/EXhxoPAhw09GruHCTBzy6tZ//CWSfBFSBQO8XHXPNBk2Nt4W3HHM9F2EAQ0QkEXmmAMa3bgFM9+iGSOgQBoMAvL39nCO6Rs5w5QCQNs/4efx8wL+RY67bIMx4W3DTmOGRKAYwREQSoSkxBjDBdczAAMCshLaQy4C0Mzk85NEdXfoRWD8e0GuBNiOAh6Y67toB5RkYQS/pOhgGMEREEqAt06NEZ/xrua5TSADQKrQBxj8UBQCY9+1p7s7rLgwG4NA/gf832rhPS/TDwOOr6750ujKFEvBraPy8QLqbGjKAISKSAE2xsYBXJgMC1bUv4q3s1SFtEOKvxNkbGnz64yWHXJPspNcBxbnA3UvAyX8Da4YA388CDGXAg2OBp/8DKP0c/3VN00iF0i3kdcxPAREROVVuUSkAY/Ailzvmr/HGDdSYO6I9Xt10AkvTzuGhFg3Ro4WD6iyoKkEAbv8POJ8GXEgDsg4Zd9e9l9IfiF8A9HzesZmXygKaArd+lfRKJAYwdlh/qObbbz8V29yJPSEib3MrXwsAaBqoduh1x3a7D3t+zcG2k9mY9tkRfJrYA92jGcQ4TG6WsRj3ygHg4g9AnpXfIz6+QKP7gQcSgNg/A4Hhzu1XSPnvqD+uOPfrOBEDGCIiCbhV4JwARiaT4b1xnfF7bglOXM3FuFUZmBgbjRf634/7QpwwdeHpCm8DZzYDl38Crh4CNL+bP69QAy0eBloNNh4L0CDMmHFR+tZvPxvGGG//kO7UIQMYIiIJMGVgQgMd/4vOX+WD9VNjMefrk/juxHX8v4NX8OXPWRjV5T680K8lWocFOvxruozBYKz7KLoLlOQC2gJAHQgENAECIwB1A/uvWaYFzn0PnNhgnBoylFU8J/cBIroA0XFAi0eAFn0Alb+jRlN7jcoDmLsMYIiIyIlyNCUAHJ+BMQlQ++DDCV3xZM8oLN9zAQcu3sF/jl3Df45dQ/fohhjZKQI9WjRCy6YB8Ff5QG8QUFhahrwiHTQlOvirfNCsoR+UCutrQ3I0JTj1ex5u5mtRVKpHcWkZ/ijSIVtTgtyiUoQG+iKuZWMMbh+GhgEqxwys8A5w4kvg0j7g1llAcwMw6Ky39w0Ggu4r/4gEgpsZbwNCjfuw+DU0ruIpzjXWkPyWDpzdAmgrnSsU2Q1oMxxoHgvc1x1QBThmLI5kCmDuXHBtP+qAAQwRkQRcvlMEAGjeyLl/vfe+vwl6398EmVdzsSr9InaeycbRK3/g6JWK/UJUCjlKLSy7VipkaB0aiPaRQWgV2gBKhRy38rU4n5OPk+WBS3W+Of47fL6Rod8DTTG6632IbxcGv9ocXikIxl1s0+YBZSXmz8nkxkDEN8QYXJQWAAW3gNJ84wGHJXnAzTP2fb2g+4BO44HOTwJN29jf3/rWtK3x36HwJqC5bgzSJIYBDBGRBFy6bTx4MaZJ/fw13yUqBKsmdUeOpgTfZV7Hjxdu43jWH8gvKTMLXtQ+cgT5KZFfokOJzoAzNzQ4c0Nj8ZpyGdA6NBBRjfzgr/KBv0qBID8lwoJ8EeKnxJW7Rdh1Jgdnbmiw+9eb2P3rTfirFGgfEYT7mzZAsL8SDdQ+CPL1QZCfEtGNA9AhMqjq4Za6YmDLK8bMC2Ccwun0hDEzEhIFNAgHFBZ+/ZVojL/MNb+Xf1wH8q4Zb4tuG6ediu4aN4BTBwENWwDNegDtRgJRvQC5hHYmUQUATdsBN08Dvx9jAENE5GgrVqzAu+++i+zsbHTu3BkfffQRevbsabX9pk2b8MYbb+Dy5cto3bo13n77bQwfPrwee+x4JTq9eHJ0y6b1Ox0RFuSLaY+0xLRHWkIQBBRoy5BXrIOfUoEAtY8YPBgMAn7PLTYGMNc1uHKnEHoBaOivRMsmAejYLBjtIoLgr7L9ayd58AO4cDMfm49fx+bM33Htj2IcufIHjlyxvGOsj1yGDvcF45HWTdC3dVN0C86Hz6bJwI1MQKYAhiwGev1fzZYj+wYZP0Lb2vvPJE3NuhsDmEv7gHaPuro3dmMAQ0Rua+PGjUhOTsaqVasQGxuL999/HwkJCTh37hxCQ0OrtD9w4AAmTJiAlJQUPProo1i/fj1Gjx6NY8eO4cEHH3TBCBwj82oudHoBTQPVLl0ZJJPJEOirRKCFs5jkchmiGvkjqpE/EjrUbQlwq9BAvJrQBn8Z8gDO5eTjXHY+Lt8uQoFWhwJtGTQlZcgtKsW57HzcLijFiau5OHE1F8fTv8FHqhVoCA0Ev0aQjUsFWvarU1882gPDgGOfAWf/CyT8w3JWyo1Jq7dEVDtH1rq6B7WydOlSTJs2Dc8++ywAYNWqVdi6dSvWrFmD119/vUr7Dz74AEOHDsWsWbMAAIsXL0ZaWhqWL1+OVatW1WvfHenbzOsAgN73N4bMWRubuSGZTIa24UFoGx5k8XlBMGZ9Tpw8geCjH6N33n8hh4BThhaYp38dj9+Kwbhog83CYq92/0DAvzGQf91YLxT3f67ukV0YwBCRWyotLcXRo0cxZ84c8TG5XI74+HhkZGRYfE1GRgaSk5PNHktISMDmzZstttdqtdBqKwpLNRrLtRv3OnL5Lrb8cgOA8ZeoAGPNqACh/Lb8viBYfhzGO0KV11d9TWGpHvv+dwsA8ORDbrxBZm4WkLHCOBAIdtzinvuGGr9Wpteh2Z2LaHbnvNiNc80eR1LOWFzRCDj2zUmkfH8WXaJC0DRQDbWPAgo5oJDJIJPJnLbJbWV+SgVmD3XTKSmlL9DvNeD72cCOOcDZ74z7wyj9yqfcZMZCX9PnjhDZxVjo7AAMYIjILd2+fRt6vR5hYWFmj4eFheHXX3+1+Jrs7GyL7bOzsy22T0lJwcKFC+3u26/Z+Ug9cNnu19XFcw/HIO7+xvX6Ne1ScAs45KIsl0wOtOgL9JuNNi36YIdOj/WHsrBy70Xcytfix/O3XdMvGE8Od9sABjAeV5B3FTiwHMjKMH44U8cnGMAQEdXVnDlzzDI2Go0GUVFR1b7uwfuCkTTgfshg/CteBgAyGWTGG7PHZTKI0z6WnpOLz1V+PcQMgUwmQ6f7gtE5KsSxg3e0wHCg719g/KtdVs0tatjO1i2Mm8Q1jAbCOxk3oivnq1TguT4xSOzdAqd+z8P/coy1MqVlBhgEQfyoD74+tVgCXp9kMmDIm8BDU4HL+42nU5dpq2a9HCXMcbVoDGCIyC01adIECoUCOTk5Zo/n5OQgPNxykWh4eLhd7dVqNdRq+zeG6xIVgi7uHlDUt+D7gEHzXN0LMwq5DJ2jQtw/+HMHDVsYPySElU1E5JZUKhW6d++O3bt3i48ZDAbs3r0bcXFxFl8TFxdn1h4A0tLSrLYnIuliBoaI3FZycjISExPRo0cP9OzZE++//z4KCwvFVUmTJ0/Gfffdh5SUFADAyy+/jH79+mHJkiUYMWIENmzYgCNHjuCf//ynK4dBRE7AAMZJ1h+ycmT6PZ6KdeNVBUQuNn78eNy6dQvz5s1DdnY2unTpgu3bt4uFullZWZBX2v20d+/eWL9+PebOnYu//vWvaN26NTZv3izpPWCIyDIGMETk1mbMmIEZM2ZYfC49Pb3KY+PGjcO4ceOc3CsicjXWwBAREZHkMIAhIiIiyWEAQ0RERJLDGhgXY7EvERGR/RjASERNAx0Y9Ahw9DVriEGWC0j0kEYiorpiAOOhNh25BsjdfAtrsoxBCRFRtdw6gFmxYgXeffddZGdno3Pnzvjoo4/Qs2dPV3eLrHB0RseebJIkHP8cQCPjLavPiIjqxG0DmI0bNyI5ORmrVq1CbGws3n//fSQkJODcuXMIDQ11dfeoPrnqF36PZ2vWjhkTIqJ657YBzNKlSzFt2jRxy/BVq1Zh69atWLNmDV5//XUX9468AgMTIiK35ZYBTGlpKY4ePYo5c+aIj8nlcsTHxyMjI8Pia7RaLbRarXg/Ly8PAHD37l3odDqrX0un06GoqAhF+bmeUTNi0EPmKeMpH8ud/GIoZa7uTN3pBKDIQ8aju3PHOJY7d6BUKi22yc/PBwAIglCfXasTU181Go2Le0LkvUw/f9W9d7hlAHP79m3o9XrxvBOTsLAw/PrrrxZfk5KSgoULF1Z5PCYmxil9JPJuSTVumZ+fj+DgYCf2xXFMQVdUVJSLe0JE1b13uGUAUxtz5sxBcnKyeN9gMODu3bto3LgxZDLrf+5qNBpERUXh6tWrCAoKqo+uOpUnjceTxgJ41nhqMhZBEJCfn4/IyMh67l3tRUZG4urVqwgMDLT5vuFOPOn/laPw36QqKf2b1PS9wy0DmCZNmkChUCAnJ8fs8ZycHISHh1t8jVqthlqtNnssJCSkxl8zKCjI7b+p9vCk8XjSWADPGk91Y5FK5sVELpejWbNmru5GrXjS/ytH4b9JVVL5N6nJe4dbLuZUqVTo3r07du/eLT5mMBiwe/duxMXFubBnRERE5A7cMgMDAMnJyUhMTESPHj3Qs2dPvP/++ygsLBRXJREREZH3ctsAZvz48bh16xbmzZuH7OxsdOnSBdu3b69S2FtXarUa8+fPrzL9JFWeNB5PGgvgWePxpLFIHb8XVfHfpCpP/DeRCVJa40hEREQEN62BISIiIrKFAQwRERFJDgMYIiIikhwGMERERCQ5HhnArFixAi1atICvry9iY2Px888/22y/adMmtG3bFr6+vujYsSO2bdtm9rwgCJg3bx4iIiLg5+eH+Ph4nD9/3plDENkzln/961/o27cvGjZsiIYNGyI+Pr5K+2eeeQYymczsY+jQoc4ehsie8aSmplbpq6+vr1kbqXxv+vfvX2UsMpkMI0aMENu48nuzb98+jBw5EpGRkZDJZNi8eXO1r0lPT0e3bt2gVqvRqlUrpKamVmlj788iWebo9zRP4Oj3Eilz1s+v2xM8zIYNGwSVSiWsWbNGOH36tDBt2jQhJCREyMnJsdj+p59+EhQKhfDOO+8IZ86cEebOnSsolUrh5MmTYpu33npLCA4OFjZv3iycOHFC+NOf/iTExMQIxcXFbjWWp556SlixYoVw/Phx4ezZs8IzzzwjBAcHC9euXRPbJCYmCkOHDhVu3Lghfty9e9ep46jteNauXSsEBQWZ9TU7O9usjVS+N3fu3DEbx6lTpwSFQiGsXbtWbOPK7822bduEv/3tb8LXX38tABC++eYbm+1/++03wd/fX0hOThbOnDkjfPTRR4JCoRC2b98utrH334gsc8Z7mtQ5471Eypzx8ysFHhfA9OzZU0hKShLv6/V6ITIyUkhJSbHY/oknnhBGjBhh9lhsbKzw5z//WRAEQTAYDEJ4eLjw7rvvis/n5uYKarVa+PLLL50wggr2juVeZWVlQmBgoLBu3TrxscTERGHUqFGO7mqN2DuetWvXCsHBwVavJ+XvzbJly4TAwEChoKBAfMyV35vKavIGOHv2bKFDhw5mj40fP15ISEgQ79f134iMHP2e5gkc/V7iSRz18ysFHjWFVFpaiqNHjyI+Pl58TC6XIz4+HhkZGRZfk5GRYdYeABISEsT2ly5dQnZ2tlmb4OBgxMbGWr2mI9RmLPcqKiqCTqdDo0aNzB5PT09HaGgo2rRpg+nTp+POnTsO7bsltR1PQUEBoqOjERUVhVGjRuH06dPic1L+3qxevRpPPvkkAgICzB53xfemNqr7uXHEvxE55z1N6pzxXuJtPOX/iEcFMLdv34Zer6+yW29YWBiys7MtviY7O9tme9OtPdd0hNqM5V6vvfYaIiMjzf6jDh06FJ999hl2796Nt99+G3v37sWwYcOg1+sd2v971WY8bdq0wZo1a/Dtt9/i888/h8FgQO/evXHt2jUA0v3e/Pzzzzh16hSmTp1q9rirvje1Ye3nRqPRoLi42CH/f8k572lS54z3Em9T3c+vVLjtUQJUN2+99RY2bNiA9PR0s2K1J598Uvy8Y8eO6NSpE+6//36kp6dj0KBBruiqVXFxcWaHd/bu3Rvt2rXDJ598gsWLF7uwZ3WzevVqdOzYET179jR7XErfGyIp8dT3Em/nURmYJk2aQKFQICcnx+zxnJwchIeHW3xNeHi4zfamW3uu6Qi1GYvJe++9h7feegs7d+5Ep06dbLZt2bIlmjRpggsXLtS5z7bUZTwmSqUSXbt2Ffsqxe9NYWEhNmzYgClTplT7derre1Mb1n5ugoKC4Ofn55DvNznnPU3qnPFe4m2q+/mVCo8KYFQqFbp3747du3eLjxkMBuzevdss+q4sLi7OrD0ApKWlie1jYmIQHh5u1kaj0eDQoUNWr+kItRkLALzzzjtYvHgxtm/fjh49elT7da5du4Y7d+4gIiLCIf22prbjqUyv1+PkyZNiX6X2vQGMy1u1Wi2efvrpar9OfX1vaqO6nxtHfL/JOe9pUueM9xJv4zH/R1xdRexoGzZsENRqtZCamiqcOXNGeP7554WQkBBxydykSZOE119/XWz/008/CT4+PsJ7770nnD17Vpg/f77FZdQhISHCt99+K/zyyy/CqFGj6m2prj1jeeuttwSVSiX8+9//NlsumJ+fLwiCIOTn5wuvvvqqkJGRIVy6dEnYtWuX0K1bN6F169ZCSUmJU8dSm/EsXLhQ2LFjh3Dx4kXh6NGjwpNPPin4+voKp0+fNhuzFL43Jn369BHGjx9f5XFXf2/y8/OF48ePC8ePHxcACEuXLhWOHz8uXLlyRRAEQXj99deFSZMmie1NyzBnzZolnD17VlixYoXFZdS2/o2oZpzxniZ1zngvkTJn/PxKgccFMIIgCB999JHQvHlzQaVSCT179hQOHjwoPtevXz8hMTHRrP1XX30lPPDAA4JKpRI6dOggbN261ex5g8EgvPHGG0JYWJigVquFQYMGCefOnauPodg1lujoaAFAlY/58+cLgiAIRUVFwpAhQ4SmTZsKSqVSiI6OFqZNm1avv1DsGc/MmTPFtmFhYcLw4cOFY8eOmV1PKt8bQRCEX3/9VQAg7Ny5s8q1XP29+eGHHyz+3zGNITExUejXr1+V13Tp0kVQqVRCy5Ytzfa0MbH1b0Q15+j3NE/g6PcSKXPWz6+7kwmCINR31oeIiIioLjyqBoaIiIi8AwMYIiIikhwGMERERCQ5DGCIiIhIchjAEBERkeQwgCEiIiLJYQBDREREksMAhoiIiCSHAQwRERFJDgMYIiIikhwGMERERCQ5DGCIiIhIcv4/s8fmMQzNxQIAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, axs = plt.subplots(ncols=2)\n",
    "pred_test_df.groupby(\"status_label_predicted\").unresolved_probability.plot(kind='kde', ax=axs[1], legend = True)\n",
    "pred_test_df.groupby(\"status_label_predicted\").unresolved_probability.hist(alpha=0.4, ax=axs[0])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10.4 ('bertft')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "1bc61917cc361de09c4c15f43498a7bb290755df004990c135f9645bd69cef9b"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}