Understanding_and_Creating_NNs/3_layer_toy_network_XOR.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Nothing But NumPy: A 3-layer neural network\n",
    "\n",
    "Part of the blog [\"Nothing but NumPy: Understanding &amp; Creating neural networks with computational graphs from scratch\"](https://medium.com/@rafayak/nothing-but-numpy-understanding-creating-neural-networks-with-computational-graphs-from-scratch-6299901091b0)- by [Rafay Khan](https://twitter.com/RafayAK)\n",
    "\n",
    "In this notebook we'll create a 3-layer nueral network (i.e. two hidden and one output layer) and train it on the XOR data.\n",
    "\n",
    "\n",
    "First, let's import NumPy, our layers and helper functions.\n",
    "\n",
    "_Feel free to look into the helper functions in the `utils` directory_   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from util.utilities import *\n",
    "from Layers.LinearLayer import LinearLayer\n",
    "from Layers.ActivationLayer import SigmoidLayer\n",
    "# to show all the generated plots inline in the notebook\n",
    "%matplotlib inline "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The XOR data:\n",
    "\n",
    "\n",
    "![XOR_data_and_plot](architecture_imgs/XOR_data_n_plot.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This is our XOR gate data \n",
    "\n",
    "X = np.array([\n",
    "    [0, 0],\n",
    "    [0, 1],\n",
    "    [1, 0],\n",
    "    [1, 1]\n",
    "])\n",
    "\n",
    "Y = np.array([\n",
    "    [0],\n",
    "    [1],\n",
    "    [1],\n",
    "    [0]\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = X.T\n",
    "Y_train = Y.T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 3 layer neural network architecture looks as follows:\n",
    "\n",
    "\n",
    "![A 3 layer neural network](architecture_imgs/3_layer_NN.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define training constants\n",
    "learning_rate = 1\n",
    "number_of_epochs = 5000\n",
    "\n",
    "np.random.seed(48) # set seed value so that the results are reproduceable\n",
    "\n",
    "\n",
    "# Our network architecture has the shape: \n",
    "#               (input)--> [Linear->Sigmoid] -> [Linear->Sigmoid]->[Linear->Sigmoid] -->(output)  \n",
    "\n",
    "#------ LAYER-1 ----- define hidden layer that takes in training data \n",
    "Z1 = LinearLayer(input_shape=X_train.shape, n_out=5, ini_type='xavier')\n",
    "A1 = SigmoidLayer(Z1.Z.shape)\n",
    "\n",
    "#------ LAYER-2 ----- define output layer that take is values from hidden layer\n",
    "Z2= LinearLayer(input_shape=A1.A.shape, n_out=3, ini_type='xavier')\n",
    "A2= SigmoidLayer(Z2.Z.shape)\n",
    "\n",
    "\n",
    "#------ LAYER-3 ----- define output layer that take is values from 2nd hidden layer\n",
    "Z3= LinearLayer(input_shape=A2.A.shape, n_out=1, ini_type='xavier')\n",
    "A3= SigmoidLayer(Z3.Z.shape)\n",
    "\n",
    "# see what random weights and bias were selected and their shape \n",
    "# print(Z1.params)\n",
    "# print(Z2.params)\n",
    "# print(Z3.params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can start the training loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost at epoch#0: 0.12580048583996017\n",
      "Cost at epoch#100: 0.12488800620460391\n",
      "Cost at epoch#200: 0.1248544745100407\n",
      "Cost at epoch#300: 0.12481135430735724\n",
      "Cost at epoch#400: 0.12475501281371584\n",
      "Cost at epoch#500: 0.12468009633443186\n",
      "Cost at epoch#600: 0.12457849324582021\n",
      "Cost at epoch#700: 0.12443755028774649\n",
      "Cost at epoch#800: 0.1242369265778672\n",
      "Cost at epoch#900: 0.12394291469995844\n",
      "Cost at epoch#1000: 0.12349798488056588\n",
      "Cost at epoch#1100: 0.1228012603028108\n",
      "Cost at epoch#1200: 0.12167195753653542\n",
      "Cost at epoch#1300: 0.11978312534788918\n",
      "Cost at epoch#1400: 0.1165609953789445\n",
      "Cost at epoch#1500: 0.11114475840867588\n",
      "Cost at epoch#1600: 0.10280346921569668\n",
      "Cost at epoch#1700: 0.09162334961696444\n",
      "Cost at epoch#1800: 0.07586830923185872\n",
      "Cost at epoch#1900: 0.04926397861977669\n",
      "Cost at epoch#2000: 0.023658380552436506\n",
      "Cost at epoch#2100: 0.011797687706814072\n",
      "Cost at epoch#2200: 0.006918317173057669\n",
      "Cost at epoch#2300: 0.004605942643578608\n",
      "Cost at epoch#2400: 0.0033424679670456398\n",
      "Cost at epoch#2500: 0.002573629512576004\n",
      "Cost at epoch#2600: 0.002067199765072307\n",
      "Cost at epoch#2700: 0.0017132187740192127\n",
      "Cost at epoch#2800: 0.0014542544172051752\n",
      "Cost at epoch#2900: 0.0012578806759425266\n",
      "Cost at epoch#3000: 0.0011046096592161755\n",
      "Cost at epoch#3100: 0.000982117814959671\n",
      "Cost at epoch#3200: 0.0008822771448461744\n",
      "Cost at epoch#3300: 0.0007995342389536301\n",
      "Cost at epoch#3400: 0.0007299792994708615\n",
      "Cost at epoch#3500: 0.0006707879048694253\n",
      "Cost at epoch#3600: 0.000619873631648771\n",
      "Cost at epoch#3700: 0.0005756648105889975\n",
      "Cost at epoch#3800: 0.0005369569511452655\n",
      "Cost at epoch#3900: 0.0005028127248221174\n",
      "Cost at epoch#4000: 0.00047249266248723513\n",
      "Cost at epoch#4100: 0.00044540617279083554\n",
      "Cost at epoch#4200: 0.0004210763005753774\n",
      "Cost at epoch#4300: 0.0003991139591657726\n",
      "Cost at epoch#4400: 0.00037919881194499406\n",
      "Cost at epoch#4500: 0.0003610648967501852\n",
      "Cost at epoch#4600: 0.0003444896835554227\n",
      "Cost at epoch#4700: 0.00032928565137382447\n",
      "Cost at epoch#4800: 0.00031529373686577247\n",
      "Cost at epoch#4900: 0.00030237818967873963\n"
     ]
    }
   ],
   "source": [
    "costs = [] # initially empty list, this will store all the costs after a certian number of epochs\n",
    "\n",
    "# Start training\n",
    "for epoch in range(number_of_epochs):\n",
    "    \n",
    "    # ------------------------- forward-prop -------------------------\n",
    "    Z1.forward(X_train)\n",
    "    A1.forward(Z1.Z)\n",
    "    \n",
    "    Z2.forward(A1.A)\n",
    "    A2.forward(Z2.Z)\n",
    "    \n",
    "    Z3.forward(A2.A)\n",
    "    A3.forward(Z3.Z)\n",
    "    \n",
    "    # ---------------------- Compute Cost ----------------------------\n",
    "    cost, dA3 = compute_cost(Y=Y_train, Y_hat=A3.A)\n",
    "    \n",
    "    # print and store Costs every 100 iterations.\n",
    "    if (epoch % 100) == 0:\n",
    "        print(\"Cost at epoch#{}: {}\".format(epoch, cost))\n",
    "        costs.append(cost)\n",
    "    \n",
    "    # ------------------------- back-prop ----------------------------\n",
    "    A3.backward(dA3)\n",
    "    Z3.backward(A3.dZ)\n",
    "    \n",
    "    A2.backward(Z3.dA_prev)\n",
    "    Z2.backward(A2.dZ)\n",
    "    \n",
    "    A1.backward(Z2.dA_prev)\n",
    "    Z1.backward(A1.dZ)\n",
    "    \n",
    "    # ----------------------- Update weights and bias ----------------\n",
    "    Z3.update_params(learning_rate=learning_rate)\n",
    "    Z2.update_params(learning_rate=learning_rate)\n",
    "    Z1.update_params(learning_rate=learning_rate)\n",
    "\n",
    "# See what the final weights and bias are training \n",
    "# print(Z1.params)\n",
    "# print(Z2.params)\n",
    "# print(Z3.params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`predict` helper function returns three things:\n",
    "- `p`: predicted labels (output 1 if predictded output is greater than 0.5) \n",
    "- `probas`: raw probabilities (how sure the neural net thinks the output is 1, this is just Y_hat) \n",
    "- `accuracy`: the number of correct predictions from total predictions\n",
    "\n",
    "In the cell below we are not storing `probas`( pythonic way is to use `_` it its place)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The predicted outputs:\n",
      " [[ 0.  1.  1.  0.]]\n",
      "The accuracy of the model is: 100.0%\n"
     ]
    }
   ],
   "source": [
    "# see the ouptput predictions\n",
    "predicted_outputs, _, accuracy = predict(X=X_train, Y=Y_train, Zs=[Z1, Z2, Z3], As=[A1, A2, A3])\n",
    "\n",
    "print(\"The predicted outputs:\\n {}\".format(predicted_outputs))\n",
    "print(\"The accuracy of the model is: {}%\".format(accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The learning curve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEWCAYAAAB1xKBvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmcXHWd7vHPU70m3dm7Q3bSHQIKJEZsArggI4gBl+iIEnQUlBGXYRZ1xsHrvS7c6wzoOKADLig6iDqAqGMEhMEBdUTAdFgCYU1CICGEdPa90931vX+c00nRdJLqpKuru+p5v171qrP8TtX311mePud3FkUEZmZmB5MpdgFmZjY0ODDMzCwvDgwzM8uLA8PMzPLiwDAzs7w4MMzMLC8ODLODkPRrSecXuw6zYnNg2KAlaaWkM4pdR0ScFRHXFbsOAEm/lfSXA/A975X0R0k7Jf220N9nQ4MDw8qapMpi19BtMNUCbASuBC4rdiE2eDgwbEiS9DZJD0nanP4mPDtn3SWSlkvaJukxSe/KWXeBpHskXSFpI/DFdNkfJP2LpE2SnpF0Vs42e3+rz6Ntk6Tfp9/9G0lXS/rRfvpwmqTVkv5R0lrgB5LGSLpFUlv6+bdImpK2/zLwBuAqSdslXZUuf4WkOyVtlPSkpPce7s83In4TETcBaw73s6x0ODBsyJF0AvB94KPAOOA7wEJJNWmT5ST/sY4CvgT8SNLEnI84CVgBjAe+nLPsSaAB+ApwrSTtp4QDtf0J8Ke0ri8CHzhIdyYAY4EjgYtI/k3+IJ2fBuwCrgKIiM8B/wNcHBH1EXGxpDrgzvR7xwPnAd+UdFxvXybpm2nI9vZacpBarcw5MGwo+gjwnYi4PyK60vGFduBkgIj4aUSsiYhsRNwIPA3Mzdl+TUT8W0R0RsSudNmzEfHdiOgCrgMmAkfs5/t7bStpGnAi8PmI2BMRfwAWHqQvWeALEdEeEbsiYkNE/CwidkbENpJAe+MBtn8bsDIifpD25wHgZ8A5vTWOiE9ExOj9vGb3to1Zt8F0zNQsX0cC50v665xl1cAkAEkfBD4FTE/X1ZPsDXRb1ctnru2eiIid6Q5D/X6+f39tG4CNEbGzx3dNPUBf2iJid/eMpOHAFcA8YEy6eISkijSgejoSOEnS5pxllcD1B/hOs0PiwLChaBXw5Yj4cs8Vko4EvgucDtwbEV2SHgJyDy8V6hbNLwBjJQ3PCY0DhUVvtXwaOAY4KSLWSpoDPMi++nu2XwX8LiLenE+Bkr4N/MV+Vj8bEb0eyjIDH5Kywa9KUm3Oq5IkED4m6SQl6iS9VdIIoI7kP9U2AEkfAo4fiEIj4lmglWQgvVrSKcDb+/gxI0jGLTZLGgt8ocf6F4HmnPlbgKMlfUBSVfo6UdIr91Pjx9Lxj95ee8NCUoWkWpJfKjPpz76qj32xEuPAsMHuNpL/QLtfX4yIVpJxjKuATcAy4AKAiHgM+BpwL8l/rrOAewaw3vcDpwAbgP8H3EgyvpKvK4FhwHrgPuD2Huu/DpyTnkH1jXSc40xgAckZTWuBy4EaDs8HSH7e3yI5gWAXSVBbGZMfoGRWOJJuBJ6IiJ57CmZDjvcwzPpRejhohqSMpHnAfOA/i12XWX/woLdZ/5oA/JzkOozVwMcj4sHilmTWP3xIyszM8uJDUmZmlpeSOSTV0NAQ06dPL3YZZmZDyuLFi9dHRGM+bUsmMKZPn05ra2uxyzAzG1IkPZtvWx+SMjOzvDgwzMwsLw4MMzPLiwPDzMzy4sAwM7O8ODDMzCwvDgwzM8tL2QdGNhv8022Pc8uSNazdsvvgG5iZlamSuXDvUK3Zsovr732WXR3J0y8njx5Gy/QxtBw5htccOZbJo4ehDGQkMup+T6YrMiJ9PKeZWckr+8CYMmY4S754Jo+/sJVFKzex+NmN3Lt8A798aE1e2ysNkQqJTAYqJCoy3a8MlZl985UVoiqToSIjqipEZUWyvroyQ3VFhqqKTDJdmUzXVGaoraqgtirDsKqKvdO1VRXU11RSV1O5972upoKRtVXUVlUU+CdmZuWq7AMDoKoiw+wpo5k9ZTQXvr6JiGD1pl0sfnYTG3fsIRuRviAbQQR0ZYOubBARdEXQlU3WdS/vygad2SCbvndls3Rkg86uLF3ZoKMr6Mxm6egKtrd3sqczS0dXNn0P2juztHd20d6RZU9XNu++1FRmGDO8mtHDqxgzvJoxdcn7hJG1TBw9jEmjkveJo2odLmbWJw6MXkhi6tjhTB07vNilAEk47e7oSl6dWXbt6WR7exc72jvZ3t7JjvZOduzpYtvuDjbv7GDTjj1s2tnB5p17eGLttr3zPY2tq6a5oY5jJozgFRNGcMyEkRwzYQSjhvnRzWb2cg6MIaAio/Sw06H/ce3u6OKFLbt5YcsuXticvD+/eTfL123nVw+v4cf3d+5tO2lULXOmjea0o8fzxmMaOWJkbX90w8yGOAdGmaitqqCpoY6mhrqXrYsIXtiymyfXbuOJtdt4Yu1W7l+xkdseWQvAsRNHctoxjZx2zHhOmDaayoqyP7nOrCwV9Il76TONvw5UAN+LiMt6rD8VuBKYDSyIiJvT5XOAbwEjgS7gyxFx44G+q6WlJXx78/4TETyxdhu/fbKN3z65jtZnN9GVDRrqq/nw65v4wMlHMqLWh67MhjpJiyOiJa+2hQoMSRXAU8CbSZ5tvAg4LyIey2kznSQU/h5YmBMYRwMREU9LmgQsBl4ZEZv3930OjMLauruDe55ez38sWsXvn2pjZG0lF7x2Ohe8romxddXFLs/MDlFfAqOQh6TmAssiYkVa1A3AfGBvYETEynTdS04DioincqbXSFoHNAL7DQwrrJG1VZw1ayJnzZrIktWb+ebdy/nGXcv43h+e4X1zp/GRU5s91mFW4gp5MHoysCpnfnW6rE8kzQWqgeW9rLtIUquk1ra2tkMu1Ppm9pTRfPsDr+HOT57KW46bwA/+uJJTv3I3P1u8utilmVkBFTIwersEuk/HvyRNBK4HPhQRL7sYISKuiYiWiGhpbMzrkbTWj2YeMYIrzp3D3Z8+jVdPG82nf/owX771MbqyhRsXM7PiKWRgrAam5sxPAfK7fBqQNBK4FfjfEXFfP9dm/WjauOFcf+FJfPCUI/nu/zzDh/59EVt6ue7DzIa2QgbGImCmpCZJ1cACYGE+G6btfwH8MCJ+WsAarZ9UVWS4dP7x/POfz+Le5et55zfvYdm67cUuy8z6UcECIyI6gYuBO4DHgZsiYqmkSyW9A0DSiZJWA+8BviNpabr5e4FTgQskPZS+5hSqVus/582dxk8+cjJbd3Xwrqvv4e4n1hW7JDPrJwW9DmMg+bTaweX5zbv4yHWtPL52K1eddwJvnT2x2CWZWS/6clqtL9m1gpg8ehg/+/hrmTN1NJf8bAmrNu4sdklmdpgcGFYww6or+MaCVwPwNzc8SEcf7rprZoOPA8MKaurY4fzzu2fx4HObueLOpw6+gZkNWg4MK7i3zZ7EghOn8q3fLeeeZeuLXY6ZHSIHhg2Iz7/9WJob6vjkjQ+xYXt7scsxs0PgwLABMby6kn877wQ27+rgH25eQqmcnWdWThwYNmCOnTSSz539Su56Yh0/uGdlscsxsz5yYNiA+uApR3LGK4/gsl8/waPPbyl2OWbWBw4MG1CS+Oo5sxk1vIpLb3ns4BuY2aDhwLABN6aumo+9cQZ/emYji5/dWOxyzCxPDgwrivPmTmXM8Cq+effLHnNiZoOUA8OKYnh1JRe8ton/fmIdT6zdWuxyzCwPDgwrmvNfeyR11RV867feyzAbChwYVjSjh1fzvpOm8auH1/DcBt+c0Gywc2BYUf3lG5qpzGT4zu+9l2E22DkwrKiOGFnLu18zmZ8uXs26bbuLXY6ZHYADw4ruo6fOoLMry7V/eKbYpZjZATgwrOimN9Rx9qyJ/Pi+59iyq6PY5ZjZfjgwbFD4+Gkz2N7eyfX3rix2KWa2Hw4MGxSOmzSK045p5Pv3rGTXnq5il2NmvXBg2KDxidOOYuOOPdy46Llil2JmvXBg2KAxt2ksLUeO4d//uNLPyzAbhBwYNqi898SprNywkyWrfetzs8GmoIEhaZ6kJyUtk3RJL+tPlfSApE5J5/RYd76kp9PX+YWs0waPecdPoLoyw38+9HyxSzGzHgoWGJIqgKuBs4BjgfMkHduj2XPABcBPemw7FvgCcBIwF/iCpDGFqtUGj5G1VbzpmPH86uEX6Mr6sJTZYFLIPYy5wLKIWBERe4AbgPm5DSJiZUQsAbI9tn0LcGdEbIyITcCdwLwC1mqDyPw5k1i/vZ17l28odilmlqOQgTEZWJUzvzpd1m/bSrpIUquk1ra2tkMu1AaXP3vFeEbUVPqwlNkgU8jAUC/L8j3GkNe2EXFNRLREREtjY2OfirPBq7aqgnnHT+D2R9eyu8PXZJgNFoUMjNXA1Jz5KcCaAdjWSsD8OZPZ3t7JXU+sK3YpZpYqZGAsAmZKapJUDSwAFua57R3AmZLGpIPdZ6bLrEycMmMcjSNq+KUPS5kNGgULjIjoBC4m+Y/+ceCmiFgq6VJJ7wCQdKKk1cB7gO9IWppuuxH4vyShswi4NF1mZaIiI94+exJ3P9HmGxKaDRIqlStqW1paorW1tdhlWD96eNVm5l99D5e/exbnnjit2OWYlSRJiyOiJZ+2vtLbBq3ZU0YxfdxwfvmQh6/MBgMHhg1akpg/ZzL3rtjAi1v9ND6zYnNg2KD2jjmTiIBfPey9DLNic2DYoDajsZ5Zk0f5sJTZIODAsEFv/pxJPPL8Fpa3bS92KWZlzYFhg97bXzUJCe9lmBWZA8MGvSNG1nJK8zgWPvS8H6xkVkQODBsSzjp+Ais37OTZDTuLXYpZ2XJg2JBwyowGAO5b4VuemxWLA8OGhBmNdTTU1zgwzIrIgWFDgiRObh7LvSs2eBzDrEgcGDZknNw8jhe3trPS4xhmReHAsCHj5OZxgMcxzIrFgWFDhscxzIrLgWFDRvc4xn0exzArCgeGDSkexzArHgeGDSmnzPA4hlmxODBsSGluqKNxhMcxzIrBgWFDSjKOMc7jGGZF4MCwIefk5rEexzArAgeGDTm+HsOsOBwYNuR4HMOsOAoaGJLmSXpS0jJJl/SyvkbSjen6+yVNT5dXSbpO0iOSHpf02ULWaUOLxzHMiqNggSGpArgaOAs4FjhP0rE9ml0IbIqIo4ArgMvT5e8BaiJiFvAa4KPdYWIGHscwK4ZC7mHMBZZFxIqI2APcAMzv0WY+cF06fTNwuiQBAdRJqgSGAXuArQWs1YaY7nGMe5f7sJTZQClkYEwGVuXMr06X9domIjqBLcA4kvDYAbwAPAf8S0Rs7PkFki6S1Cqpta2trf97YIOWxzHMBl4hA0O9LOt5wHl/beYCXcAkoAn4tKTmlzWMuCYiWiKipbGx8XDrtSHE4xhmA6+QgbEamJozPwVYs7826eGnUcBG4H3A7RHRERHrgHuAlgLWakPQKc3jWLetnWfW7yh2KWZloZCBsQiYKalJUjWwAFjYo81C4Px0+hzgrkh+XXwOeJMSdcDJwBMFrNWGoJObxwJw34qXHa00swIoWGCkYxIXA3cAjwM3RcRSSZdKekfa7FpgnKRlwKeA7lNvrwbqgUdJgucHEbGkULXa0NTUUMd4j2OYDZjKQn54RNwG3NZj2edzpneTnELbc7vtvS03y9VzHCM5wc7MCsVXetuQdrLHMcwGjAPDhrTZU0YB8OTabUWuxKz0OTBsSJveUAfACu9hmBWcA8OGtPqaSsaPqPEhKbMB4MCwIa+poc6BYTYAHBg25DU3OjDMBoIDw4a8poY6Nu7Yw5adHcUuxaykOTBsyGtqqAfgmQ3eyzArJAeGDXlN6ZlSz6zfXuRKzEqbA8OGvGljh5MRPNPmPQyzQnJg2JBXXZlh6tjhvhbDrMAcGFYSpo/zmVJmhebAsJLQfS2GH6ZkVjh5BYak6/NZZlYszY117NzTxbpt7cUuxaxk5buHcVzujKQK4DX9X47Zoek+U2qFB77NCuaAgSHps5K2AbMlbU1f24B1wC8HpEKzPOw7tdaBYVYoBwyMiPjniBgBfDUiRqavERExLiI+O0A1mh3UpFHDqK7M+FoMswLK95DULemztZH0F5L+VdKRBazLrE8yGdHkM6XMCirfwPgWsFPSq4DPAM8CPyxYVWaHoKmhztdimBVQvoHRGcn5ivOBr0fE14ERhSvLrO+aGut4bsNOOruyxS7FrCTlGxjbJH0W+ABwa3qWVFXhyjLru6aGOjqzwepNu4pdillJyjcwzgXagQ9HxFpgMvDVglVldgiafaaUWUHlFRhpSPwYGCXpbcDuiPAYhg0qTX6+t1lB5Xul93uBPwHvAd4L3C/pnDy2myfpSUnLJF3Sy/oaSTem6++XND1n3WxJ90paKukRSbX5dsrK09i6akbWVvrUWrMCqcyz3eeAEyNiHYCkRuA3wM372yAd57gaeDOwGlgkaWFEPJbT7EJgU0QcJWkBcDlwrqRK4EfAByLiYUnjAD9OzQ5IEk2N9T4kZVYg+Y5hZLrDIrUhj23nAssiYkVE7AFuIDnLKtd84Lp0+mbgdEkCzgSWRMTDABGxISK68qzVylhzQx0r1+8sdhlmJSnfwLhd0h2SLpB0AXArcNtBtpkMrMqZX50u67VNRHQCW4BxwNFApN/5gKTP9PYFki6S1Cqpta2tLc+uWClraqjj+c272N3h3y/M+tvB7iV1lKTXRcQ/AN8BZgOvAu4FrjnIZ6uXZT3vPb2/NpXA64H3p+/vknT6yxpGXBMRLRHR0tjYeJByrBx0D3yv9PO9zfrdwfYwrgS2AUTEzyPiUxHxSZK9iysPsu1qYGrO/BRgzf7apOMWo4CN6fLfRcT6iNiZft8JB++Olbu9NyH0XWvN+t3BAmN6RCzpuTAiWoHpB9l2ETBTUpOkamABsLBHm4XA+en0OcBd6RXld5DcIXd4GiRvBB7D7CB8aq1Z4RzsLKkDnco67EAbRkSnpItJ/vOvAL4fEUslXQq0RsRC4FrgeknLSPYsFqTbbpL0ryShE8BtEXFrXj2yslZXU8kRI2t8ppRZARwsMBZJ+khEfDd3oaQLgcUH+/CIuI0eg+MR8fmc6d0k13b0tu2PSE6tNesTP9/brDAOFhh/B/xC0vvZFxAtQDXwrkIWZnaomhvruGPpi8Uuw6zkHDAwIuJF4LWS/gw4Pl18a0TcVfDKzA5RU0MdG3fsYfPOPYweXl3scsxKRl5XekfE3cDdBa7FrF80NdQDyU0IXz3NgWHWX/K9cM9syPDzvc0Kw4FhJWfa2OFk5MAw628ODCs51ZUZpo4d7msxzPqZA8NKUlNDna/2NutnDgwrSU0NybUYyY0DzKw/ODCsJDU31LGro4sXt7YXuxSzkuHAsJLUfWrtCj99z6zfODCsJDU1pjch9DiGWb9xYFhJmjiyltqqjE+tNetHDgwrSZmMaGqoZ0WbD0mZ9RcHhpWs5sY6lvuQlFm/cWBYyZrRWM/qTTv9fG+zfuLAsJI1o7GObMCzG3YWuxSzkuDAsJI1ozE9tdbjGGb9woFhJcvP9zbrXw4MK1l1NZVMGFnL8nXewzDrDw4MK2kzxtex3HsYZv3CgWElrbmhnhXrtvsmhGb9wIFhJW1GYx3b2jtp2+6bEJodroIGhqR5kp6UtEzSJb2sr5F0Y7r+fknTe6yfJmm7pL8vZJ1Wupr3ninlw1Jmh6tggSGpArgaOAs4FjhP0rE9ml0IbIqIo4ArgMt7rL8C+HWharTSN2N8EhjLfWqt2WEr5B7GXGBZRKyIiD3ADcD8Hm3mA9el0zcDp0sSgKR3AiuApQWs0Upc900IvYdhdvgKGRiTgVU586vTZb22iYhOYAswTlId8I/Alw70BZIuktQqqbWtra3fCrfS0X0TQu9hmB2+QgaGelnW81SV/bX5EnBFRBzwX3lEXBMRLRHR0tjYeIhlWqmb0VjnPQyzflBZwM9eDUzNmZ8CrNlPm9WSKoFRwEbgJOAcSV8BRgNZSbsj4qoC1mslqrmxnlsfeYHdHV3UVlUUuxyzIauQgbEImCmpCXgeWAC8r0ebhcD5wL3AOcBdkZww/4buBpK+CGx3WNihmtFYR6Q3ITxmwohil2M2ZBXskFQ6JnExcAfwOHBTRCyVdKmkd6TNriUZs1gGfAp42am3ZofLNyE06x+F3MMgIm4Dbuux7PM507uB9xzkM75YkOKsbHTfhNAD32aHx1d6W8mrq6lk4qhaD3ybHSYHhpWF5HGt3sMwOxwODCsLMxrrWdG2wzchNDsMDgwrC80Nvgmh2eFyYFhZ2HtPqXUexzA7VA4MKwt771q73uMYZofKgWFlofsmhN7DMDt0DgwrC5mMkqfveQ/D7JA5MKxsNPsmhGaHxYFhZWNGYz2rNu1kd0dXsUsxG5IcGFY2mnNuQmhmfefAsLLRfRNCX/FtdmgcGFY2mhuTmxD6rrVmh8aBYWVjeLVvQmh2OBwYVlZmNPr53maHyoFhZaX71FrfhNCs7xwYVlZmNNYnNyHc5psQmvWVA8PKSvfA93KPY5j1mQPDyopPrTU7dA4MKysTRtYyrKrCZ0qZHQIHhpWVTEYcP3kk963YUOxSzIYcB4aVnXnHT+SxF7byzHrvZZj1hQPDys7ZsyYAcNsjLxS5ErOhpaCBIWmepCclLZN0SS/rayTdmK6/X9L0dPmbJS2W9Ej6/qZC1mnlZeKoYZwwbTS3LnFgmPVFwQJDUgVwNXAWcCxwnqRjezS7ENgUEUcBVwCXp8vXA2+PiFnA+cD1harTytPZs5LDUit9WMosb4Xcw5gLLIuIFRGxB7gBmN+jzXzgunT6ZuB0SYqIByNiTbp8KVArqaaAtVqZOXvWRABu9WEps7wVMjAmA6ty5leny3ptExGdwBZgXI827wYejIiXXZor6SJJrZJa29ra+q1wK32TRg/j1dNGexzDrA8KGRjqZVnPG/gcsI2k40gOU320ty+IiGsioiUiWhobGw+5UCtPb501kaVrtvLsBh+WMstHIQNjNTA1Z34KsGZ/bSRVAqOAjen8FOAXwAcjYnkB67QydZYPS5n1SSEDYxEwU1KTpGpgAbCwR5uFJIPaAOcAd0VESBoN3Ap8NiLuKWCNVsYmp4elfLaUWX4KFhjpmMTFwB3A48BNEbFU0qWS3pE2uxYYJ2kZ8Cmg+9Tbi4GjgP8j6aH0Nb5QtVr58mEps/ypVJ4L0NLSEq2trcUuw4aY5zfv4nWX3cVn5h3DJ047qtjlmA04SYsjoiWftr7S28ra5NHDmDPVZ0uZ5cOBYWXvrbMm8ujzW3luw85il2I2qDkwrOydld5bymdLmR2YA8PK3pQxw3mVD0uZHZQDwwx466wJPPL8Fh+WMjsAB4YZcNbxyUV8tz3qvQyz/XFgmAFTxyaHpX7auordHV3FLsdsUHJgmKX+9vSjWN62g//1i0coleuTzPqTA8Ms9aZXHMEnzzianz/wPN+/Z2WxyzEbdBwYZjn++k1H8ZbjjuCfbnuce5atL3Y5ZoOKA8MsRyYjvvbeOcxorOOvfvKAz5oyy+HAMOuhvqaS736whWw2uOj6Vna0dxa7JLNBwYFh1osjx9Vx1ftO4KkXt/EPNz/sQXAzHBhm+3Xq0Y1cctYruO2RtVx117Jil2NWdA4MswP4yBuaeeecSXztzqe46IetHtOwsubAMDsASXzlnFfxmXnH8Idl6znjit/xtf96kp17PK5h5ceBYXYQ1ZUZPnHaUdz16dM4+/gJ/Ntdyzjja7/jliVrPLZhZcWBYZanCaNquXLBq/npx05h9PBqLv7Jg5x7zX386uE1bPeZVFYG/IhWs0PQlQ1uWPQcV/7madq2tVNdkeENMxt4y/ETOOOVRzC2rrrYJZrlpS+PaHVgmB2GrmzwwHObuP3Rtdz+6Fqe37yLjOCkpnGc3DyO4yeP5PjJoxg/ogZJxS7X7GUcGGZFEBEsXbOV2x9dy52PvchT67bR/c+rob4mCY9Jo5h5RD1Txgxn6thhNNY7SKy4HBhmg8CO9k4ef2Erjz6/hUfXJO9Pr9tOV3bfv7naqgxTxgxnyphhTB49jIb6GhpG1NBYX51M19cwrr6a+ppKB4sVRF8Co7LAhcwDvg5UAN+LiMt6rK8Bfgi8BtgAnBsRK9N1nwUuBLqAv4mIOwpZq1l/q6uppGX6WFqmj927bHdHF6s27mTVpp2s2rjrJdMPr9rMpp0dvX5WRUaMrK1k1LAqRg6r2vteX11JXU0l9TUVDK/ZNz2sqpJh1RUMq0pf1RlqqyqoraqgpjJDTWUFVRVyCFmfFCwwJFUAVwNvBlYDiyQtjIjHcppdCGyKiKMkLQAuB86VdCywADgOmAT8RtLREeEn29iQVltVwcwjRjDziBG9ru/syrJxxx7atrezfvse1m9rZ8OOdrbs6khfnWxNp5/ftIvt7Z3saO9kx56+/9OQ2BseNZUZqioye9+rKzNUVWjvdGVGVFZkqK7IUFkhKjPJ+u7pyoyoqBBVmQwVGe2dr5By5jPpPFRkMvveJTJKbvzY3V5KQrJCSahVZJLtJKXt97XJpNt3t8sIRLI+k853t89ICNJthDLsnVdOO7HvM5X+rByuhd3DmAssi4gVAJJuAOYDuYExH/hiOn0zcJWSP5X5wA0R0Q48I2lZ+nn3FrBes6KrrMgwfmQt40fW9mm7bDbY1dHFjvZOtrd3squji90dXezak2VXR1fy2tNJe2eWPZ1Z2juztHd0Je+dWdo7u9jTGezpyrKns4uOrmBP2nZ7eyedXUFHV5aOriyd2dg735XNec8GnV1ZsqVxlLtXuSGSSZLlJYGThEuynpz5nuEDue33BVx3e/a2O8D6nO945cSRXPW+Ewre/0IGxmRgVc78auCk/bWJiE5JW4Bx6fL7emw7uecXSLoIuAhg2rRp/Va42VCTyYi69JDU+CLXEhF0ZYPO7L73bDboyGbJZqErkvnu9V3ZIBv73rPBvulspO1J2qTLIpLPibR99/aRTmcjqaN7viudjpx13dtFQNC9vHtZ97Yq60ExAAAHsUlEQVT71tG9Ld3bJG15yeew9/O6h4cjYm/b3DZ0f06Pz+yeTzbuXhY52710GQHTxg4fkD/bQgZGb/tvPX/32F+bfLYlIq4BroFk0LuvBZpZ/5PSQ1UVxa7E+lshr/ReDUzNmZ8CrNlfG0mVwChgY57bmpnZACpkYCwCZkpqklRNMoi9sEebhcD56fQ5wF2RnOe7EFggqUZSEzAT+FMBazUzs4Mo2CGpdEziYuAOktNqvx8RSyVdCrRGxELgWuD6dFB7I0mokLa7iWSAvBP4K58hZWZWXL5wz8ysjPXlwj3frdbMzPLiwDAzs7w4MMzMLC8ODDMzy0vJDHpLagOePYyPaADW91M5Q4n7XV7c7/KST7+PjIjGfD6sZALjcElqzfdMgVLifpcX97u89He/fUjKzMzy4sAwM7O8ODD2uabYBRSJ+11e3O/y0q/99hiGmZnlxXsYZmaWFweGmZnlpewDQ9I8SU9KWibpkmLXc7gkfV/SOkmP5iwbK+lOSU+n72PS5ZL0jbTvSySdkLPN+Wn7pyWd39t3DSaSpkq6W9LjkpZK+tt0eUn3XVKtpD9Jejjt95fS5U2S7k/7cGP6iAHSRwbcmPb7fknTcz7rs+nyJyW9pTg96htJFZIelHRLOl/y/Za0UtIjkh6S1JouG5i/57H3cYTl9yK57fpyoBmoBh4Gji12XYfZp1OBE4BHc5Z9Bbgknb4EuDydPhv4NckTDk8G7k+XjwVWpO9j0ukxxe7bQfo9ETghnR4BPAUcW+p9T+uvT6ergPvT/twELEiXfxv4eDr9CeDb6fQC4MZ0+tj0738N0JT+u6godv/y6P+ngJ8At6TzJd9vYCXQ0GPZgPw9L/c9jLnAsohYERF7gBuA+UWu6bBExO9Jni2Saz5wXTp9HfDOnOU/jMR9wGhJE4G3AHdGxMaI2ATcCcwrfPWHLiJeiIgH0ultwOMkz4Ev6b6n9W9PZ6vSVwBvAm5Ol/fsd/fP42bgdElKl98QEe0R8QywjOTfx6AlaQrwVuB76bwog37vx4D8PS/3wJgMrMqZX50uKzVHRMQLkPzHCoxPl++v/0P655Iebng1yW/bJd/39LDMQ8A6kn/4y4HNEdGZNsntw97+peu3AOMYgv0GrgQ+A2TT+XGUR78D+C9JiyVdlC4bkL/nBXvi3hChXpaV03nG++v/kP25SKoHfgb8XURsTX6J7L1pL8uGZN8jeRrlHEmjgV8Ar+ytWfpeEv2W9DZgXUQslnRa9+JempZUv1Ovi4g1ksYDd0p64gBt+7Xf5b6HsRqYmjM/BVhTpFoK6cV0N5T0fV26fH/9H5I/F0lVJGHx44j4ebq4LPoOEBGbgd+SHKseLan7F8LcPuztX7p+FMkhzKHW79cB75C0kuRQ8ptI9jhKvd9ExJr0fR3JLwhzGaC/5+UeGIuAmemZFdUkg2ELi1xTISwEus+COB/4Zc7yD6ZnUpwMbEl3Z+8AzpQ0Jj3b4sx02aCVHo++Fng8Iv41Z1VJ911SY7pngaRhwBkk4zd3A+ekzXr2u/vncQ5wVySjoAuBBenZRE3ATOBPA9OLvouIz0bElIiYTvLv9q6IeD8l3m9JdZJGdE+T/P18lIH6e17sEf9iv0jOIniK5Ljv54pdTz/05z+AF4AOkt8iLiQ5VvvfwNPp+9i0rYCr074/ArTkfM6HSQYAlwEfKna/8uj360l2qZcAD6Wvs0u978Bs4MG0348Cn0+XN5P8x7cM+ClQky6vTeeXpeubcz7rc+nP40ngrGL3rQ8/g9PYd5ZUSfc77d/D6Wtp9/9ZA/X33LcGMTOzvJT7ISkzM8uTA8PMzPLiwDAzs7w4MMzMLC8ODDMzy4sDwwyQtD19ny7pff382f+rx/wf+/PzzQaKA8PspaYDfQoMSRUHafKSwIiI1/axJrNBwYFh9lKXAW9InzXwyfTGfl+VtCh9nsBHASSdpuT5Gz8huSAKSf+Z3hBuafdN4SRdBgxLP+/H6bLuvRmln/1o+nyDc3M++7eSbpb0hKQfp1eyI+kySY+ltfzLgP90rKyV+80HzXq6BPj7iHgbQPof/5aIOFFSDXCPpP9K284Fjo/kttgAH46IjektOhZJ+llEXCLp4oiY08t3/TkwB3gV0JBu8/t03auB40ju73MP8DpJjwHvAl4REdF9SxCzgeI9DLMDO5PkXjwPkdwufRzJ/YYA/pQTFgB/I+lh4D6SG7vN5MBeD/xHRHRFxIvA74ATcz57dURkSW5zMh3YCuwGvifpz4Gdh907sz5wYJgdmIC/jog56aspIrr3MHbsbZTcYvsM4JSIeBXJ/Z1q8/js/WnPme4CKiN5jsNckjvyvhO4vU89MTtMDgyzl9pG8ojXbncAH09vnY6ko9O7hPY0CtgUETslvYLkFuPdOrq37+H3wLnpOEkjyeN193un1PRZH6Mi4jbg70gOZ5kNGI9hmL3UEqAzPbT078DXSQ4HPZAOPLex7/GXuW4HPiZpCcldT+/LWXcNsETSA5HcgrvbL4BTSO48GsBnImJtGji9GQH8UlItyd7JJw+ti2aHxnerNTOzvPiQlJmZ5cWBYWZmeXFgmJlZXhwYZmaWFweGmZnlxYFhZmZ5cWCYmVle/j8HM+Er4D2FlAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_learning_curve(costs=costs, learning_rate=learning_rate, total_epochs=number_of_epochs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The decision boundary\n",
    "\n",
    "With more hidden layers our deeper neural network is able to learn a much more complex decision boundary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZoAAAEfCAYAAAB4ewAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzs3Xd0FHXbxvHvlvRCEnpLKBJa6EVqqDFURXqQqlIFadJ8QBEjVXwVHkABQUHpIAJBQARBqRI6hCIQegmQttlsks3O+0ceVgIBgimzSe7POZzDzs5srp0ke2VmfjOjURRFQQghhMgiWrUDCCGEyN2kaIQQQmQpKRohhBBZSopGCCFElpKiEUIIkaWkaIQQQmQpKRqR5caPH0/58uVT/fPz86NJkyaMGTOGixcvZunX79WrF82bN3+pZebOnUv58uW5ceNGFqWyja+ZGXJqbpF99GoHEHnHhAkT8PT0BCA+Pp6rV6+yYcMGtm/fzqJFi3j11Vez5OsOGjSI+Pj4l1omICAAb29vvLy8siSTEHmJFI3INi1btqREiRKppvXu3ZtOnToxYsQIdu7ciYuLS6Z/3YYNG770MhUqVKBChQqZnkWIvEh2nQlVFS1alHHjxvHw4UPWr1+vdhwhRBaQohGqa9WqFfb29vzxxx+pph87dox+/fpRo0YNatSowdtvv83JkyefWv7EiRP079+fOnXq8OqrrzJgwADOnz9vff7JYzSJiYl89tlntGjRwnqs6JNPPiE6Oto6T1rHHSIjI5k8eTKNGzfGz8+PwMBAFi5cSHJycqrlqlSpQnh4OAMHDqRGjRrUqVOHcePGERkZma71cfnyZXr37k3VqlVp2rQpX331FUlJSanmSW+WtI6dPDn9ZTJfu3aNYcOGWdf1//3f/5HWVazOnDnDsGHDaNCgAZUrV6Z+/fqMHj2aO3fuPLWufv31Vxo2bEiNGjVYtWoV5cuXZ+bMmU+95qxZs/Dz80v1fRI5g+w6E6pzcHDA29ubc+fOWaft27ePgQMHUqFCBYYPH05iYiIbNmzgrbfeYunSpdSuXRuAI0eO0LdvXwoVKsQ777yDo6Mjy5Yto3fv3qxfv/6pXXUAU6ZMYcuWLfTu3ZuSJUty8eJFfvzxR65evcqSJUvSzBgdHU337t25efMm3bt3p3Tp0uzbt4/Zs2dz9uxZvvzyS+u8FouF3r17U7t2bcaNG8epU6dYt24dJpOJr7766oXrY/jw4bz66quMGzeOw4cPM3/+fG7fvs306dNfOkt6pSfz/fv36d69O0lJSfTp0wdHR0dWrFhBTExMqtc6f/48PXr0wMfHhwEDBuDk5MTRo0f5+eefuXfvHsuXL7fOazabmThxIu+88w6JiYnUrVuXypUrs23bNsaOHZvqdX/55RcaN25Mvnz5Xvr9CXVJ0Qib4O7uzrVr14CUD72PP/6YKlWq8MMPP6DT6QDo2bMnHTp0IDg4mI0bNwIwY8YMPDw8WL9+vXWgQZMmTWjTpg0rVqx46sMKYPPmzXTq1IlRo0ZZpzk7O/PHH38QFxeX5nGiRYsWER4ezrx582jZsiUAb731Fp988gkrVqzgzTffpEmTJkDKh2ebNm0YP348AN27d+fu3bvs3LmT+Ph4nJycnrsumjRpYi2Lt956iwkTJrBhwwb69etH+fLlXypLeqUn87fffmvdxVm5cmUA3nzzTdq1a5fqtVasWIFGo2HZsmV4eHgA0K1bN5KSkggJCSEqKso63WKx0LNnTwYMGGBdvn379kyfPp2TJ09StWpVIGXr9ubNm3zwwQcv9b6EbZBdZ8ImmM1mNBoNAGfPnuX69eu0bNmS6OhoHj58yMOHDzGZTDRr1oywsDDu3LnDgwcPOHXqFO3bt7eWDEDp0qVZv349/fv3T/NrFSlShK1bt7JhwwbrX+MjRoxg/fr1zxyMsGvXLsqWLWv9YH9kyJAhAPz222+pprdu3TrV44oVK2I2m4mKinrhunjnnXdSPe7VqxcAe/bs+VdZ0utFmffu3UuVKlWsJQOQP39+2rZtm2q5yZMns2vXLmuZABgMBhwcHAAwGo2p5m/UqFGqx23atEGr1fLLL79Yp4WEhODs7EyzZs3+1XsT6pItGmEToqKirEOJH23ZzJw5M8199QC3b99Gp9OhKAo+Pj5PPV+pUqVnfq3JkyczYsQIJkyYwKRJk6hevToBAQF06tQJNze3NJe5ceMGjRs3fmp6wYIFcXd35+bNm6mmPzks2t7eHiDVMZRnKVOmTKrH3t7e1gz/Jkt6vSjzzZs3adGixQvzajQaIiMj+eabbzh//jzXrl3j1q1b1mM5Fosl1fz58+dP9bhw4cLUqVOH7du3M27cOCwWC9u2baNFixYv3BoUtkmKRqjOYDBw/fp1mjZtCvzzQTR8+HCqV6+e5jJlypThypUrAGi1L7dhXr9+fXbv3m39t2/fPqZNm8Z3333Hhg0b0jx35nm3bbJYLNjZ2aWa9mjr7N94ctlHX/vRLsSXzfKkZ5XdizJrNBoSEhKemv5knt9//50hQ4ZQqFAh6tWrh7+/P35+fvz555988803Ty2f1vevffv2TJw4kRMnTmAymYiIiHhqF53IOaRohOq2bduGoijWv5aLFy8OpBw3adCgQap5T548SXR0NI6OjhQtWhSAq1evPvWas2bNIl++fKn2/UPKiLOwsDCKFClC27Ztadu2LRaLhaVLlzJz5kxCQkKsu6oeV7x4cS5fvvzU9IiICAwGgzVLZrh58yblypWzPn5UqI+2bNKb5dEHeGJiYqr57t+//69ylShRgvDw8KemX79+PdXjTz/9FB8fH9avX4+zs7N1+ubNm9P9tQIDA5kyZQq7du0iPj4eDw+Pf3U+lLANcoxGqOrevXvMmTOHwoUL0759ewD8/PwoWLAgy5cvJy4uzjqvwWCw7vLS6XQULlyYChUqEBISgsFgsM53/fp1li1bluYHalRUFN26dUv1l7VWq6VKlSrW/6elWbNmXL58mZ07d6aavnDhQgDr1lhmWLNmTarHS5cuRaPRWIdopzdLwYIFAVKN5jMYDNZjPS/rtdde4+LFi+zdu9c6LTY2lp9//jnVfFFRURQrVixVydy+fZsdO3YA6dt96O7uTpMmTdizZw979uwhMDDwhVtqwnbJFo3INjt37rQetE9ISODy5cts3LiRhIQEFi1ahKOjIwB2dnZMmjSJESNG0LFjRzp37oyDgwNr167l1q1bfP755+j1KT+6EyZM4N1336VTp0506dIFrVbLDz/8gLu7e5qDAQoVKkT79u1ZsWIF8fHx1KhRg6ioKH744QcKFCjw1AHxRwYOHMiOHTsYMWIEQUFBlCpVioMHD7Jjxw5ee+21lx7l9TybN2/GYDBQtWpV9uzZw+7du3n33Xetx6LSm6Vly5YEBwczZcoUbt68ib29PWvWrElVAC+jX79+bNq0iWHDhtGnTx+8vLxYvXr1U7vO/P392bp1Kx999BFVqlThxo0brFmzxnoZoMf/eHiedu3aMXz4cACCg4P/VWZhG6RoRLaZNm2a9f8uLi4ULVqU5s2b079/f0qXLp1q3sDAQJYsWcKCBQuYP38+Wq2WcuXKsWDBglQjj+rVq8f333/PnDlzmDdvHg4ODtSpU4cxY8ZY/6J/0qeffkrJkiUJCQkhJCQEJycn6tevz8iRI595bTMPDw9Wr17Nl19+ydatW4mJiaFkyZKMHTuWvn37ZnzlPGbRokUEBwezZcsWChcuzIQJE1J9jfRm8fLyYtGiRcyePZs5c+bg6elJ165dKVOmDCNHjnzpXK6urqxYsYJZs2axevVqkpOTadOmDeXKlUtVBJMnT8bZ2Zldu3bx888/U6RIETp06EBAQABBQUEcPHjwuYM1HmnWrBmurq64urpaz5sSOZNGed6RRSGEUEliYiINGjSgW7dujBkzRu04IgPkGI0QwiaFhIQQGxtLx44d1Y4iMkh2nQkhbMqSJUs4evQoe/fupVmzZpQtW1btSCKDbHaL5sSJE2kOM126dClt27alV69e9OrVi8uXL2MymRg2bBg9evSgf//+PHz4UIXEQojMkJyczJ9//km1atVkEEAuYZPHaBYtWsSmTZtwcnJ6aqjnBx98QN++ffHz87NOW7p0KQaDgWHDhhESEsKxY8eYOHFidscWQgiRBpvcovH29mbu3LlpPnfmzBkWLlxIUFCQ9VyI0NBQ6yU5/P39OXDgQLZlFUII8Xw2eYwmMDDwmfcfb9u2LT169MDV1ZWhQ4eye/duDAaD9RpVLi4uxMbGprlsaGholmUWQojcrFatWv96WZssmmdRFIU+ffpYS6VJkyacPXsWV1dX60lgcXFxuLu7P/M1MrKysktYWBgVK1ZUO8ZzNW3aFKPRyOHDh9WO8kI5YX3aQsYbD+KoPnIjJlMSVSoW48Anrz01jy3kTA/Jmbky+ke6Te46exaDwUC7du2Ii4tDURQOHTqEn58fNWvWtF5WY+/evTmiTISwJdFxCdQY/TMmUxJ+FYvy50ctX7yQEOmUI7ZoNm/ejNFopFu3bowcOZLevXtjb29P/fr1adKkCXXr1mXcuHEEBQVhZ2fH7Nmz1Y4sRI6RkGimwvsbiTcmUrpUAf6c1BKdLkf9DSpsnM0WTYkSJawjzh5dbBGgQ4cOdOjQIdW8Tk5OzJkzJ1vzCZEbJCdbKPPeBmJj4ylcOB9HP2uFXq9TO5bIZeTPFiHyKEVR8H3/J6KjjRQq5M6ZL9pjZ2ezf3uKHEyKRog8qvrYLdyNiCWfhwuhM9rhKCUjsogUjRB5kP/H27l09QEuro7sn9oWTxd7tSOJXEyKRog8psknOzgadhtHJ3u2fhSIT4F/d38aIdJLtpWFyEM6zN5D6JlbODjasX1KK2r5eKodSeQBskUjRB4xYNEhdh66gp29nlXjWlDLJ+2bvAmR2aRohMgDPlp7ghW/hqHT65g7pDEBlYuoHUnkIVI0QuRy83dc4Iu1x9BqNUzqVZeeDXzUjiTyGCkaIXKx2SFhjP12PxqthkEdq/FB6/JqRxJ5kBSNELnUj/vCmbz8MCgwpGN1ZnatrnYkkUdJ0QiRC+06dYch/92LYlEIal2JGV2rqR1J5GFSNELkMmeuR9Jx+q8kJ1sIbPQKi/rVVTuSyOOkaITIRW49jMP/w62Yk5J5tXpJ1g5tqHYkIaRohMgtIg0JVB+1iYSEJCr6FuHX8c3RajVqxxJCikaI3CAh0YzfyI0YjQl4l/Ti4CevSckImyFFI0QOZzYnU2nkz0RHx1OokDsnZ7SVG5cJmyI/jULkcDXHh1gv93/mi/Zy4zJhc6RohMjBGn+8ncvXHuLs6siJz9vjZG+ndiQhniJFI0QO1X7W7xwLu42Dox17p7ahgLuj2pGESJPN3ibgxIkTfP755yxfvjzV9C1btvD999+j0+nw9fVl8uTJaLVaOnTogJubGwAlSpRg2rRpasQWIlv0/foAu/8Kx85ez7oJr1GhiLvakYR4JpssmkWLFrFp0yacnJxSTTeZTHz55Zds3rwZJycnRo0axe7du2nUqBHAU6UkRG705S/nWLfrPFqdlnlD/WlWsaDakYR4Lpvcdebt7c3cuXOfmm5vb8+qVausBWQ2m3FwcODcuXPEx8fz9ttv07t3b44fP57dkYXIFiv2hTPpu4NoNBqGvFmNHvW81Y4kxAvZ5BZNYGAgN27ceGq6VqulQIECQMrWi9FopGHDhly4cIF33nmHLl26EB4eTv/+/dm2bRt6/dNvLywsLMvzZ5TJZLL5nEajEYvFYvM5IWesz/RkDL1hZMi3p1EU6BpQnj5V7LP9feWEdQmS09bYZNE8j8ViYdasWVy5coW5c+ei0WgoXbo0Pj4+1v97eHgQERFB0aJFn1q+YsWKKqR+OWFhYTaf09nZGaPRaPM5IWeszxdlPHcrivc+3YTFotCq0Sss6V8vG9P9IyesS5CcmS00NDRDy9vkrrPn+eijj0hISGD+/PnWXWjr1q1j+vTpANy9exeDwUDBgrLfWuQO0XEJ1B+7hWSzhfo1vVn3fiO1IwnxUnLEFs3mzZsxGo34+fmxbt06ateuTZ8+fQDo3bs3nTt3ZsKECQQFBaHRaJg6dWqau82EyGmSzGYqvP8TSYlmKvoWYduYpmpHEuKl2eyncYkSJVizZg0A7du3t04/d+5cmvPPnj07W3IJkV0URaHcsI3ExpooWtSDPz9uKZeWETmS/NQKYaNqjgvh/gMDnl6uhE5tg4Odzf5dKMRzSdEIYYMCpu7kYvh9nJwdODyjLe4u9mpHEuJfk6IRwsZ0m/MHB47fwN5BT8ik1yiaz+nFCwlhw6RohLAhn/50ipA/L6HTa1kysjl1y+ZXO5IQGSZFI4SNWLTrb2asDEWj1TCqay061CymdiQhMoUcXRTCBpy6ZWTUotOggX7tqvJxh8pqRxIi08gWjRAqO38rigFLzqJYFNo38WVOzxpqRxIiU0nRCKGiyFgTjT7cijkpmdpVS/DjoPpqRxIi00nRCKGiqh9sIt6YiI93fnZ92AKtVqN2JCEynRSNECqp+sFmIiONeHq5sqbfK1IyIteSohFCBa2m7uLytQe4uDrw52et0cpvosjF5MdbiGzW7+sD/Hn8Gno7HT9PfA2f/C5qRxIiS0nRCJGNpm48w9pd59FqNSwb05x6ZeSETJH7SdEIkU12nrrNtJV/odHA2B51eL16cbUjCZEtpGjEMxmTjNyIefqW2s+TlJzElcgrWZQo57pwO4pOU39FUaBdk/JMfL2S2pFyvgsXsmcZkWFSNOKZev/Um8ZLG3M16mq65k9KTqL7+u7U/7Y+UaaoLE6Xc9yPjqfBuBCSky00ru3DyiFyrkyGbdsGFSvCwoXpX+ann6ByZfj++6zLJdIkRSOeaXyj8USZomj6fdMXls2jktkQtoEJjSbg4eiRTSltm9mcTNXRmzCZkihfrjBbRjdRO1Lu0KwZtGoFAwemr2x++gm6doXateHNN7M+n0hFikY8U+1itfm1168vLJvHS+bLwC8ZXm94Nie1XZVGbSImJp6CBd05ODlA7pCZWRwcYMMGaNPmxWXzeMls3w7u7tmXUwBSNOIFXlQ2UjLP1uSTHdy6E42rmxMnZrbDTu6QmbnSUzZSMjbBZovmxIkT9OrV66npu3btolOnTnTr1o01a9YAYDKZGDZsGD169KB///48fPgwu+Pmas8qG0VRpGSeod/X+wk9cwsHRzs2/ydA7pCZVZ5XNlIyNsMm/8RatGgRmzZtwskp9Z0Fk5KSmDZtGuvWrcPJyYmgoCCaNWvGli1b8PX1ZdiwYYSEhDB//nwmTpyoUvrc6VHZBCwPoOn3TSlkLsSlmEscCTsiJfOE8auOsXbXBXQ6Ld+ObEqdMl5qR8rdHpVNx44pZQO4JSTAqFFSMjbCJrdovL29mTt37lPTL126hLe3N/ny5cPe3p5atWpx5MgRQkNDady4MQD+/v4cOHAguyPnCY/KJiIugsM3DhOZECkl84S9YXf5708n0Ghgav8GdKgh58pkiye2bEq8/76UjA2xyS2awMBAbtx4+vwNg8GAm5ub9bGLiwsGgyHVdBcXF2JjY5/52mFhYZkfOJOZTCabzWlvsSefXT7iiAOgsr6yzWZ9JLvW5/XIBLrMPQkKtPEvR/OiSen+urb8PX+cred0b96c4lu3AnC3USMe3rwJN2+qnOrZbH19ZhabLJpncXV1JS4uzvo4Li4ONze3VNPj4uJwf85fMBUrVszynBkVFhZmkzkfHfi/ZbxFYdfC3Dfep/++/vze53d8PHzUjvdM2bE+H8TG03j6TyQnW2hYy4dV7zVE8xIXY7bV7/mTbDrnTz/B+PFQowZGrZbCn39O4XLlYMAAtZM9k02vz8eEhoZmaHmb3HX2LGXLluXq1atERUWRmJjIkSNHqFGjBjVr1mTPnj0A7N27l1q1aqmcNPd5cnRZhQIVKJ+vfLrPs8ntqo7ajCk+kXJlCxIy2v+lSkZkgscP/P/+O9e+/TZ9Q59FtsgRRbN582ZWr16NnZ0d48eP55133qF79+506tSJwoULExQUxMWLFwkKCmL16tUMHTpU7ci5yrOGMLvYuaTrPJvcrub4EKKjjXh6uXB4SiB6vU7tSHlLGqPLFHv79J9nI7Kcze46K1GihHX4cvv27a3TmzdvTvPmzVPN6+TkxJw5c7I1X17xovNknhyNZuu70TLbG5//zoXLETi7OLBrSms5Vya7PW8Icxqj0Wx5N1puliO2aIQ60nsyZnqvIJDb9F90kN8Oh6O307Hhw5aUK+SqdqS8JT3nybzMFQRElpGiEc/09qa3030y5pNlk9svqvnlL+dZ+es5tFoNC0c0pVG5gmpHylt27Ur/yZhPls3q1dmXUwA2vOtMqK9f9X68WvxVhtZN3zGv2sVqs7PXTrZf2k4+h3xZnE49hy5GMPG7A2g0GoZ1rkXXOiXVjpT31KkDgwdDcHD6zpN5VDbjxqVckFNkKyka8UzNSzeneenmL57xMbWK1aJWsdw76i/8XgytP9kOCrT2f4XPOvupHSlvcnODlz0u6+AAX36ZNXnEc8muMyHSKSExibrjQkhMNFOtcjFWDWmgdiQhcgQpGiHSqfzwnzHGJVCsmCd7/tMCrVZOlhEiPaRohEiHuh9u5f4DAx6eLhyb0VrOlRHiJUjRCPECnb7Yy9m/7+Hk7MCuyYG4OMgl/4V4GVI0QjzHgMWH2H7wcsq5Mv9piW9RuRKwEC9LikaIZ1jw60VW7AhDq9Xw3/f8aSznygjxr0jRCJGGI5fvM2bxPjQaGNSxGj0b5J3L6giR2aRohHjC1YgYWvxnKygQ2KgcM7tWVzuSEDmaFI0Qj4mKS6D2B1tITrZQp1oJVg+pr3YkIXI8KRoh/ich0UzF938iPj6R0qUK8Ou4Zuh08isiREbJb5EQ/1N59CZiY00UKZKPo5+1knNlhMgkUjRCAPUn/cKduzG4uTtxenZ7ua+MEJlIikbkea2n7+LU+bs4OtmzY3IgjlIyQmQqKRqRp/Wct58/jl7D3l7Phg9bUqWEh9qRhMh1pGhEnvXBiqNs3HMBvV7H4pFN8S9fSO1IQuRKNrmPwGKxMHnyZM6fP4+9vT3BwcH4+KScMBcWFsbUqVOt8x4/fpx58+ZRtWpVAgMD8fX1BaBly5b06dNHlfzC9o1YHsrizafQ6XVMe7cBHWuVUDuSELmWTRbNzp07SUxMZPXq1Rw/fpzp06ezYMECACpWrMjy5csB+OWXXyhUqBD+/v7s37+fdu3aMWnSJDWjixxgwqrjLN58Cq1Ww9R36jO4eVm1IwmRq9nkrrPQ0FAaN24MQPXq1Tl9+vRT8xiNRubOnct//vMfAE6fPs2ZM2fo2bMn77//Pvfu3cvWzCJn+O/288zdcByNVsOIrrV4r8UrakcSItezyS0ag8GAq6ur9bFOp8NsNqPX/xN33bp1tGrVCi8vLwDKlCmDn58fDRo0YNOmTQQHBzMnjVu9hoWFZf0byCCTyWTzOY1GIxaLxeZzwj/rc/2pGKavDwMNvNexGkEVdTaTPyd8z0FyZrackjOjbLJoXF1diYuLsz62WCypSgZg8+bNqYqkXr16ODk5ARAQEJBmyUDKrjdbFxYWZvM5nZ2dMRqNNp8TUtbn+vPJTF8fhkajYUCHqky3seuX5YTvOUjOzJZTcoaGhmZoeZvcdVazZk327t0LpBzsf3SA/5HY2FgSExMpWrSoddrEiRPZvn07AAcOHKBy5crZF1jYtO9Do5i+4ggaNLzXsQazu9dQO5IQeYpNbtEEBASwb98+unfvjqIoTJ06laVLl+Lt7U2LFi24cuUKxYsXT7XM6NGj+fDDD1m5ciVOTk4EBwerlF7YkjErjrJg03m0Oi1jutdm0huV1I4kRJ5jk0Wj1WqZMmVKqmlly/4zMqhq1arMnz8/1fMlS5a0jkYTAmDsimMs2HgSrVbDuB61+U97KRkh1GCTRSNERr278CCrdp5Dp9fyn6AajJWSEUI1UjQiV1EUheaf7uSv0zfR63VM6VePgBJmtWMJkadJ0Yhcw2hKotb4EK7fisLRyZ6V41sQULFwnhg+KoQtk6IRucLdKCO1xm4hKsqIh6cL+z9rg3cBF7VjCSGQohG5wMbQ67z71V5MpiS8S3oROq01TvZ2ascSQvyPTZ5Hk5W+2fW32hFEJpq49gS9Z+3CZEri1eolOTWznZSMEDYmzxXN6K//pPaErUTGxqsdRWSAxWKh/azdfLn2GBqNhn7tq/Dbhy3Q6fLcj7QQNi/P/VZ65Xfl3KV7vDJkA9M3nVE7jvgXTl6PpMSAtez+6ypOzg58PbIpc3vVUjuWEOIZ8lzRXJrTgdaNXyE52ULwD3/hN3oT525GqR1LpNP0TWfwH7+FmJh4KvkWIWzOmwTV9VY7lhC5iqLAyRtRvLcslBE/Hs3w6+W5wQB2dnrWDmvE3sDy9Jv7B+HXH1J71Eaq+Bbh1w+b4+pkr3ZEkYZbDw289tkuwq8/RK/X0a6JLysG10er1agdTYgcT1HggSGB4E1n2R92h+t3YjDEmlAUBZ1eR68KGTvhOc8VzSP+vgU5/39vMHLlcTbsvcSp83coNXAtw9+syn/eqIxWm+c29mzW2JXHWLTlNElJyRQu7M7SYY3w95XbLguREbci45n9yzl+P3WbWxEGjHEmkpMtAOj0Orw8XahWtgBDXvMF8+0Mfa08WzQAer2Oub1qMbtbVTrPPcCeI+HMWBnKvM1nWDC4IW/WKal2xDzt9PVI3pz1O7fvRGNnr6fba5X4pm8t9Hqd2tGEyHGMCUlM33qeXSdvc/lGJAaDCYu1WLS4uzvh6+1JvyZleKthaTSaf/YWhIZK0WSYvb0dm0b7s+OMLxN+OMqFKxH0mvUbhQq6sXJkE159pYDaEfOUiOh4Aqb+xt9X7gNQorgna0Y2pap3PpWTCZFzxMYlMnP7RX47cYOrt6MxGBJINicDoNVqcHd3onLp/AQ1KkWfRmVSFUtmk6J5zGuVi/DatDZ8uzecT1eFcu9+LC0+3ELxoh5M7VmDTnV81I6YqyUkJtF6xh5zqnxiAAAgAElEQVSOnruNOSkZLy9Xgpq/wrTO1eRYjBAvkGhOZuGeK6w/EM7fNyKJjTFhflQsOi1uro54F3GnZ+PS9G9WFnu77Pv4/1dfKSYmhqNHj+Lu7k6NGjVSNaHRaGTJkiUMHTo000Jmt3f8S/GOfyk+23SGhb+c4+btKPrM2s3YQu7Me/dVWlUv/uIXEel2OyqOdxceZt+JG5iTknF0sqdNw1dYNqCu7CYT4hkSk8x8tfMSvxy7wcVrD4mNNWFOSikWjUaDm5sjZUt40rmeN0MDfFU9x+yli+bixYv069ePhw8fYrFYqFSpEnPnzrXeiMxoNDJv3rwcXTSP/Of1yoxvW5HPt19gwZYz3I2IofPUX/H0cKF3S1+mdKoiJwhmwKW70fT7+hAnz9/FbE7GwdGO+tVKsn54A5wdZPSfEI8zJZr572+X2H7iJn9fjyIqykhSUsqVyTUaDa6ujhQr6UrXBqXp36w0Xm5OKif+x0sXzezZs6levTozZ87EYDDw2WefERQUxLJlyyhVqlQWRFSXTqdlXJsKjG1dgfm7L/J/G09z924MX607xtdbTtOgSnHmv12bkvld1Y6aIyiKwvI/LjN1wylu3o5GURQcHO1oWseHxe/UpYC7o9oRhbAJyckWlh24yqr94ZwPf0h0TDxJif/c8sLVzZFXfLx4vU5JPmhdASdH27300ksXzYkTJ1i2bBnOzs44Ozvz1VdfMW3aNHr16sWyZctwc3PLipyq02jgveblGNKsHHvO32PM8lD+Dn/A7r/CqXgknKKF89G7WTkmvemndlSbdOBCBB+tO8nx83eJj08EIF8+ZwLr+jC/Ty0c7eVwocjbkpLMLNl/jY2HrhEW/oCY2HgSEx4rFldHypfKT6vqxRnc4hUKezqrmPblvPRvd2Ji4lOjEyZMmICiKPTq1YvZs2dnWjhbpNFA0wqF+Ouz1tyOMjL8h2PsP3mT23eimbHyCP+34Tilinsy+DVf3m32SpaO5LB1V+7GMGHNSXYdvY4xLgFIGZ//SumC9GtRjvcDfMnDq0fkcUnmZH69amHs1r2EXX3Aw0gjiQlJQMrnjLOzA8ULu/N6HW+GBZajiEfOve3FSxdN6dKlOX36NK+88kqq6R9++CEWi4UhQ4ZkOJTFYmHy5MmcP38ee3t7goOD8fH5Z8RXcHAwR48excUlZcXPnz+fpKQkPvjgA0wmE4UKFWLatGk4OWXtPsqiHs6sGdoQi0Vh8/FbfLL2BFdvRnLhcgQjv45g7LcHKVnMg/Z1vBndqjxeeWC30M+h15nx8xkuXH2I6X9bLjq9jmLFPGlWrTjBnatQ0M1B5ZRCZD+LRWHD0Zv88MdlTl66T1R0vLVYAJxdHChZND+B1Usw4fUKeLrazjGWjHrpogkICGDLli106NDhqecmTpxIcnIyK1euzFConTt3kpiYyOrVqzl+/DjTp09nwYIF1ufPnDnD4sWL8fLysk4LDg6mXbt2dOzYkYULF7J69Wr69u2boRzppdVqeKNmcd6oWZykJDNzfrvEir2XuHozistXH/DV1Qd8tf4Ynh4ulC6WjyEB5ehUtyR2+py9u8hiUfjmtwtsOHydv29Gc/+BwXpmsZ29npIlPKng7cX0LlUpX1zOgRF5i9mczNrQm6zeH86pyw+IijaSYPqnWJyc7fEtU4AmfkUZ0uIVyhXNvb8jL/1JN3DgQAYOHPjM5z/++GM+/vjjDIUKDQ2lcePGAFSvXp3Tp09bn7NYLFy9epWPPvqI+/fv07lzZzp37kxoaKg1l7+/P1988UW2Fc3j7Oz0jG5VntGtymM2J7P4z3A2HLpG2JUHREfFcTQyjnfP3KK/VoOriwNFC7jyavlCDHutHL5F3dHrbHM4b2KSmZUHrrLr7D2O/h3B+XP3SLZYGP3NfiBl1IuLqyNFC7nSupY3E1+vgLPcF0bkIWZzMhuO3eS73y9z4XokUdHx1q16AEcne0oW96RFtWJ80LY8pQq6ExYWRsWKFVVMnT1eWDRffPEFo0aNyo4sVgaDAVfXf0Zx6XQ6zGYzer0eo9FIz5496devH8nJyfTu3Rs/Pz8MBoN1IIKLiwuxsbHZmjkter2OQU3LMqhpWQCijAlM3RLGnlN3uB1hICY2ngtX7nPhyn2WbzuLVqvBwdGOfG6OlCh4g8IeTrSqVoy6ZTypXNIzy/OazclsPX6LI+GRHLvykGv3YjHEJxEdayLBlISiKNZ5FUXBzk5HtUrFqF+hIO+1LIePl4ucWCnyjMQkM9vPRvDjn1cIvXCPyChj6mJxtMenhBf+fkX4uENlinjl3GMsGfXColm4cCHR0dF88skn2ZEHAFdXV+Li4qyPLRYL+v/tZnJycqJ3797W4y/16tXj3Llz1mUcHR2Ji4vD3d09zdcOCwvL+jfwHP2qONCvSsrxpuRkhYMRWkJO3CP8Tiz3o4wYjYncjYjlzt0YAEL+TLkjqEarQa/T4uiYcn5JPjcHnB1S1olep6F8UVfs9c8+p8eiKFy4E0d8QsoJXQrwIDqepCQLFouF+PhELIqS8sRjdHot9vZ2eHm5UKygGxVK5KNVZS8+v+KGolhY3C3lenCmiOucj8i01ZSpTCaT6t/3F8kJGSFv51SAUw80rD92n+OXHnD/gSHVMRYHRzvKeHtRo3Q+BjfwIp/jPx+vkXevEXk3e3LaohcWTYcOHVi9ejUGg4EZM2ZYP/AfFxoayqxZs1i1alWmhKpZsya7d++mTZs2HD9+HF9fX+tz4eHhjBw5kp9++gmLxcLRo0d58803qVmzJnv27KFjx47s3buXWrXSvhGWrW2m+gHvNqtgfZycbOFY2N8cuqfleHgk1yIM3IgwkJiUTEysCXNySjHcuJX6DqFn/375T3mtTouDQ8ruLQ9PFxwd7PByd6BgPide9S1IleL5aF2lKDptynGox0fQLXBxxmg02tz6TEtO2D2REzJC3sppNifz+4UH/Hf7OcKuRhIZbbSOnoT/FYtPfmq9UoDRrSrg5/Pyex1yyvoMDQ3N0PIvLJrp06fj4eHBd999R2xsLHPnzsXBIWXU0JUrV5g9eza//fZbhkI8KSAggH379tG9e3cURWHq1KksXboUb29vWrRoQfv27enatSt2dna88cYblCtXjsGDBzNu3DjWrFmDp6dnjh1mrdNpcdEl815z32fOYzYncy3KhGJJ2fzYd/EBoVcevPC1vQu48EbN4jyqi3xOegq45f6RcEKkh9mczIErD5m1OYyLNyK5/9BIvPGfYrF30FO8qAd1yxfik85+lCmU9l4T8bR0DQYYP348Hh4efPnll7z99tsEBwezbNky1q5di9lsxs/Pj9GjR2daKK1Wy5QpU1JNK1u2rPX//fv3p3///qmeL1CgAN9++22mZbBler2OMgX+2d9btpArvRvKBT+FeBkWi8Kh8Acs+PVvjly4x737hlTHWOzt9XiX8KRW2YLMDKpG0Tx8jCWj0j3qbNCgQbi5ufHpp5/Spk0bIOWcmuHDhxMYGJhlAYUQIjMkJ1s4dSuGaT+f5XT4Ax5ExmGINVmft7fXU7K4Jw0rFmbCG5UoW1i2WDJLuopGURR+/vlnlixZYn1csGBBfvzxRzw9s340lBBCvCxFgdM3o/hm1yX2nLrNvQexxBn+2RWmt9NRpLA7VcsU4JPOVaiSDSM786oXFs3OnTv58ssvuXTpEvb29gwYMAAvLy9mzJhB3759WbJkCfnz58+OrEII8UyKAmF3Yvhq23kOhN0l4uFJ633vIWWXc7Gi+ahaOj9TOlWhkhRLtnlh0QwdOhStVkuHDh0YMWIEhQsXBlKOiYwfP56goCCWLFlCiRIlsjysEEI87vrDOL7acYFdJ25z576BmBijdYi+Tq/Dy8uFqmUKMKyVL69VKaZu2DzshUXTsGFDxowZQ4UKFVJNb9euHW5ubgwfPpwePXrw7bffUq5cuSwLKoQQxsQkpm0+z+7Tt7lyI5LYx+97r9NSML8bFX28eLtuPjo1rZGnL2prS15YNM8bydWkSRMWL17MoEGD6NmzJ4cOHcrUcEKIvM1gTGJt6A2+33OJKzejiIqOT3Xfezc3J/zK5OetRqXo9dh978PCwqRkbEiGr+pYu3ZtfvjhB959993MyCOEyMPM5mTW/HWT5X9c5sL1hzx4GGe9PTEayOfuhE+RfAQ1Ks3gFq/Irb5ziEy5fHCFChVYsWJFZryUECIPURQ4eT2KL7adI/RiBHfuxaY6l8XZxYGy3l50etWH917zJZ+z3OI7J8q069R7e3tn1ksJIXKx2PgkZmw9z6ZD4dx7EIch9p/LKdnZ6ylbKj8NKhRmapeqeMqVK3KFnH1DFCGEzUtOtvD9/qusO3CVM1fuExVltN63SK9POZelsk9+grtUoYqP1wteTeREUjRCiEylKHA90sjUTWf5/cRNHkSmvmaYh4czPkXcGd2uEh3ryp6QvECKRgiRYUlJZpYduM7KfVc4e/k+sTHx1hMl7R30lCrpRbs63oxqXZ5C+XLPLYpF+kjRCCH+FWNCEtNCzrPxQDg370Rb782i0WookN8V35IeTO5clfrlCqobVKhOikYIkS6KArfi9Uz4v70cuxhBdLTROvTYwcGO8mUL4l+pCMFdquLiKLfxFv+QohFCPJOiwP6/I5i28Swn/r5HVJQxZZeYBtxcnShVOh9jO1Tmzdol1Y4qbJgUjRAiFYtFYdf5CKZvPE3YlftER6dcP0yjAS9PFxr5FWP2W9Up4in3ZxHpI0UjhCA52cLGY7eYu+0c58IfYIhJObdFo9VQML8bLaoXZ1ZQde7cuJIjbj0sbIsUjRB52Mnr0Xyy4RQHT99K2XIh5RpiRQq706qWNzODquHs8M/xljtqBRU5mhSNEHnMw7hEPlp/iq2HrxIREYuiKGg0UKyoBx3rl2JKJz/s7eSjQWQe+WkSIg9QFPh2z2Xmbz/H5WsPrKPFvDxdaFSlGJ/3qEYxL1eVU4rcyuaKxmKxMHnyZM6fP4+9vT3BwcH4+PhYn//uu+8ICQkBUm5TMHToUBRFwd/fn1KlSgFQvXp1Ro8erUZ8IWyKKcnMBytP8PP+y0Q+jAPAwdGOWpWK8nFHP/wrFlE5ocgLbK5odu7cSWJiIqtXr+b48eNMnz6dBQsWAHD9+nU2bdrE2rVr0Wg09OjRg5YtW+Lk5ETlypX5+uuvVU4vhG04dzOGYcuOcOzcHUzxiWg0ULRwPsa+WYV3m5WVe7WIbGVzRRMaGkrjxo2BlC2T06dPW58rUqQIixcvRqdLuQeF2WzGwcGBM2fOcPfuXXr16oWjoyMTJkygTJkyquQXQk0rD11j2vqTXL3xkGSzBZ1OS5XyRVgysB4VS3ioHU/kUTZXNAaDAVfXf/YV63Q6zGYzer0eOzs7vLy8UBSFmTNnUqlSJUqXLs39+/cZMGAArVu35siRI4wZM4b169en+fphYWHZ9Vb+NZPJZPM5jUYjFovF5nNCzlifGc146C5M23iBW7ejUBQFZ2cHmtYpwqSAguh1Ooi9TVjYbdVzZhfJaVtsrmhcXV2Ji4uzPrZYLOj1/8RMSEjgww8/xMXFhY8//hgAPz8/61ZO7dq1uXv37v9G0jy9eyAnnAMQFhZm8zmdnZ0xGo02nxNyxvr8NxkVBUJO3mbkkkPcvhMFCnh4uDCqgx+j2mTN+80J6xIkZ2YLDQ3N0PI2VzQ1a9Zk9+7dtGnThuPHj+Pr62t9TlEUhgwZwquvvsqAAQOs0//73//i4eFB//79OXfuHMWKFZN90CJXuxIRR4fPf+dSeAQoUKCAK5O61uCdpmXVjibEU2yuaAICAti3bx/du3dHURSmTp3K0qVL8fb2xmKxcPjwYRITE/njjz8AGDVqFAMGDGDMmDHs2bMHnU7HtGnTVH4XQmSNSEMinef8wbGwOyQmJOHp6cKw9pUZ266S2tGEeCabKxqtVsuUKVNSTStb9p+/0k6dOpXmcgsXLszSXEKo7fNfzjNzzVGMcQno9TqCAiqyqP+rascS4oVsrmiEEKndfBjH65/v4cKle2g0GprVKcXPo/3RarVqRxMiXaRohLBhX+24wKc/HsEUn4inhws/jPSnScXCascS4qVI0Qhhg5KSzPT4+iDb913CYlHo0KQcy4c0kEEuIkeSohHCxtyPMdHgo23cuhWFg4MdHwbVZHQWDVcWIjtI0QhhQ/668pB2n+4gzmDC09OZk5+3x9PNSe1YQmSIFI0QNmLdkesMnvsH8fGJ+PkWZv+UQDngL3IFKRohbMDvN2DCd79jTkqmSS1vQsY1VzuSEJlGikYIlW09eYvxS/4iOdlC5+bl+W5QfbUjCZGpZLtcCBUduRrJWzN2kZxs4XX/V6RkRK4kRSOESuITk3j90x0kJZmp61eUFUMbqR1JiCwhRSOEChQFGn+yk5iYePLnd2VeF2+1IwmRZaRohFDBkO9DOXfxLno7HWf/73W14wiRpaRohMhmG4/d5MdtKXeOXT6qOS6O9ionEiJrSdEIkY2Sky28t2AfFotCtxblaV+ruNqRhMhyUjTi2c6dg19+ebllrl+HdeuyJk8u8NY3B4mOMpIvnzPfDpQRZv9WsiWZBX8twGwxp3sZRVH45sg3mMymLEwm0iJFI55t7Fh44w3YsiV981+/Dk2bwoABEBWVpdFyoltR8ew4cBmATePlhMyM2Pb3NoZsHULPDT3TVTaKojB823AGhQxi+Ynl2ZBQPE6KRjzbsmVQrRp07PjisnlUMvfvw7Zt4OGRLRFzks5f/kFigplypQpQq2wBtePkaG192zKz5UxWn1n9wrJ5VDJzD89lVL1RvFvz3WxMKkCuDCCex8MDfv0VAgJSymbDBmjX7un5Hi+ZX3+FunWzPaqtizUmEfb3PdDAgU9bqR0nVxjTcAwAY3eOBeCHjj+g16b+SHuyZD5/7XO51YIKpGjE872obKRk0qXrvP0kJZrxKeGJo4P82mWW55WNlIztsMmfeIvFwuTJkzl//jz29vYEBwfj4+NjfX7NmjWsWrUKvV7P4MGDadasGQ8fPuSDDz7AZDJRqFAhpk2bhpOTXF49U6RVNoAmMVFKJh3M5mRCz94CYOuEFiqnyX3SKhspGdtik0Wzc+dOEhMTWb16NcePH2f69OksWLAAgIiICJYvX8769etJSEigR48eNGzYkPnz59OuXTs6duzIwoULWb16NX379lX3jeQmT5ZNqVI4XrsGDg5SMi8w7/crGOMSKJDfFZ+CrmrHyZUeLxsFBZ1Jx8pLK6VkbIRNFk1oaCiNGzcGoHr16pw+fdr63MmTJ6lRowb29vbY29vj7e3NuXPnCA0NZeDAgQD4+/vzxRdfpFk0TZs2zY63kCFGoxFnZ2e1Y6TNwQGA4xcvAtC0cuWU0Wk2TO31efjSAxISktAWcqNp0/+mOY/aGdPL1nOWiS7DmqVrACjuXpzQ3aE0m9ZM5VTPZuvr85HZs2dnaHmbLBqDwYCr6z9/+el0OsxmM3q9HoPBgJubm/U5FxcXDAZDqukuLi7Exsam+dpGozFrw2cCi8Viszk1iYk4WizWxwmxsSTrdComejE116cCJCWa0aChgLPmmTls+Xv+OFvPaTAZrP+PT4wnLi7OprdmbH19ZhabLBpXV1fi4uKsjy0WC3q9Ps3n4uLicHNzs053dHQkLi4Od3f3NF/78OHDWRs+E4SFhVGxog3eI/7RgX8XF5qWKoXl77/ZGx7+7NFoNkLN9fnfXZcI+/oPihZ25/Dcjs+cz2a/50+w1ZyPjsn8dfgvRtUbhcaoYfbJ2QRUDkhzNJqtsNX1+aTQ0NAMLW+T59HUrFmTvXv3AnD8+HF8fX2tz1WtWpXQ0FASEhKIjY3l0qVL+Pr6UrNmTfbs2QPA3r17qVWrlirZc60nR5d5emIqXz7959nkUUt+PQ9Al4alVU6Se6V14P+dCu+k+zwbkfVssuYDAgLYt28f3bt3R1EUpk6dytKlS/H29qZFixb06tWLHj16oCgKI0eOxMHBgcGDBzNu3DjWrFmDp6dnhvcpisc8awizXg87drz4PJs8SlHgdkTKLtxJb1ZROU3u9LzRZek5z0ZkD5tc61qtlilTpqSaVrZsWev/u3btSteuXVM9X6BAAb799ttsyZenvOg8mfSe1JkHnbsbi8FgwsXFASc5dybTpWcIs5SNbZA1Lp4tvSdjStmk6a8rkSgWBe8i+dSOkuu8zHkyUjbqs8ljNMJGDB6c/pMxH5VNtWrQs6dcVBNY9WfKBTRf9ZXrmmW2zRc2v9TJmGMajrEes/n2qOz5yG5S6+LZFi2Cmzehdu30zf+obE6flotqAlduxwDwutxzJtO1923PlqAttCnXJt3Dl8c0HEPlQpUJLBuYxenEk6RoxLMVLZry72V4eECjRlmTJ4eJjUtAq9UQUKWY2lFyHY1GQ1vfti+9XJtybbIgjXgR2XUmRBZRFNDb6Wz6hEEhsoMUjRBZ4FZkPIbYeHRa+RUTQn4LhMgCVyPjSU62UCi/XERTCCkaIbKQi5McBhVCikYIIUSWkqIRQgiRpaRohBBCZCkpGiGEEFlKikYIIUSWkqIRQgiRpaRohBBCZCkpGiGEEFlKikaILKQoaicQQn1SNEJkAQ9nOzQaDdduR6sdRQjVSdEIkQUqFHbDxdWRJHOy2lGEUJ3NXYjJZDIxZswYHjx4gIuLCzNmzMDLyyvVPDNmzODo0aOYzWa6detG165diYqKIjAwEF9fXwBatmxJnz591HgLQqDRgF6vJTFR7SRCqM/mimblypX4+voybNgwQkJCmD9/PhMnTrQ+f/DgQa5du8bq1atJTEykbdu2BAYGcvbsWdq1a8ekSZNUTC/EP7RaDYkJZq7cjaF0YXe14wihGpvbdRYaGkrjxo0B8Pf358CBA6mer1GjBlOnTrU+Tk5ORq/Xc/r0ac6cOUPPnj15//33uXfvXrbmFuJJJf5XLsevy3EakbepukWzdu1avv/++1TT8ufPj5ubGwAuLi7Exsamet7BwQEHBweSkpIYP3483bp1w8XFhTJlyuDn50eDBg3YtGkTwcHBzJkz56mvGRYWlnVvKJOYTCabz2k0GrFYLDafE9Rbn56uDgCs//MiFVwMz503J3zPQXJmtpySM6NULZouXbrQpUuXVNOGDh1KXFwcAHFxcbi7P73LITo6mvfff5+6desycOBAAOrVq4eTkxMAAQEBaZYMQMWKFTPzLWSJsLAwm8/p7OyM0Wi0+Zyg3vrsGefGnsNXuHDP9MKvnxO+5yA5M1tOyRkaGpqh5W1u11nNmjXZs2cPAHv37qVWrVqpnjeZTPTt25dOnTrx3nvvWadPnDiR7du3A3DgwAEqV66cfaGFSEOlIu5oNHAz4vlbM0LkdjY3GCAoKIhx48YRFBSEnZ0ds2fPBmDmzJm0atWKo0ePcv36ddauXcvatWsBmDp1KqNHj+bDDz9k5cqVODk5ERwcrObbEAK/oq44OTtgNCaoHUUIVdlc0Tg5OaW522vs2LEAVK1alb59+6a57PLly7MymhAvRafT4uLsQERcAqevReLn7al2JCFUYXO7zoTITUoVSznGuPrgVZWTCKEeKRohstDrdX0A2HjomspJhFCPFI0QWahb7RLodFpu34998cxC5FJSNEJkoWKeTnh4OGOKT+Ly3Ri14wihCikaIbKYd9F8AMzdfkHlJEKoQ4pGiCzWt3k5ADYdkgEBIm+SohEii/WoUxwHRzvuR8apHUUIVUjRCJHFnBzs8PRwJtls4bs9l9SOI0S2k6IRIht08X8FgC82n1E5iRDZT4pGiGww8rVy6PQ6rsutnUUeJEUjRDYo5O5IgfyuJCUls3LfFbXjCJGtpGiEyCYdGpUG4JO1J1ROIkT2kqIRIptMbl8Zewc9t+5EoyiK2nGEyDZSNEJkEzdnOwoVdMdiUZj2swwKEHmHFI0Q2ej99pUAmLtFikbkHVI0QmSjQU3K4p7PmdiYeI5eeaB2HCGyhRSNENlIq9XQsGpxAPp/c1DlNEJkDykaIbLZ3N410eq0XAi/T0KiWe04QmQ5KRohslmRfE6UK10QxaLw9kLZqhG5n17tAE8ymUyMGTOGBw8e4OLiwowZM/Dy8ko1z6BBg4iKisLOzg4HBwcWL17M1atXGT9+PBqNhnLlyvHxxx+j1UqPCts0q1dtXv84hC37L6O811DtOEJkKZv7JF65ciW+vr6sWLGCDh06MH/+/KfmuXbtGitXrmT58uUsXrwYgGnTpjFixAhWrFiBoij89ttv2R1diHRrXrEgxYp5kmy2MGHVcbXjCJGlbK5oQkNDady4MQD+/v4cOHAg1fP3798nJiaGQYMGERQUxO7duwE4c+YMdevWtS63f//+7A0uxEua2LU6AEt3nFM5iRBZS9VdZ2vXruX7779PNS1//vy4ubkB4OLiQmxs6nutJyUl8fbbb9O7d2+io6MJCgqiatWqKIqCRqN55nKPhIaGZsE7yXy2nnP27NmA7ed8xBZzVnaAPeOqAGA0Gm0yY1okZ+bKKTkzQtWi6dKlC126dEk1bejQocTFpdwgKi4uDnd391TPFyhQgO7du6PX68mfPz8VK1bkypUrqY7HpLUcQK1atbLgXQghhHgem9t1VrNmTfbs2QPA3r17nyqH/fv3M2LECCClUC5evEiZMmWoVKkShw4dsi5Xu3bt7A0uhBAiTRrFxq7uFx8fz7hx44iIiMDOzo7Zs2dTsGBBZs6cSatWrahatSqfffYZJ06cQKvV8u6779KyZUuuXLnCpEmTSEpKokyZMgQHB6PT6dR+O0IIkefZXNFkVE4ZHp2enDNmzODo0aOYzWa6detG165diYqKIjAwEF9fXwBatmxJnz59MjWbxWJh8uTJnD9/Hnt7e4KDg/Hx8bE+v2bNGlatWoVer2fw4ME0a9aMhw8f8t2F/5AAAAl1SURBVMEHH2AymShUqBDTpk3DyckpU3O9bM7vvvuOkJAQAJo0acLQoUNRFAV/f39KlSoFQPXq1Rk9erSqOYODgzl69CguLi4AzJ8/n6SkJJtan2FhYUydOtU67/Hjx5k3bx5Vq1bN8p/HtJw4cYLPP/+c5cuXp5q+a9cu5s2bh16vp1OnTnTt2jVdv2vZnXPLli18//336HQ6fH19mTx5Mlqtlg4dOliPUZcoUYJp06apmnPp0qWsW7fOur4++eQTihUr9vLrU8lllixZosyZM0dRFEXZsmWL8umnnz41T+vWrRWLxZJq2sCBA5WDBw8qiqIokyZNUnbs2KFqzgMHDihDhgxRFEVREhISlJYtWypRUVHKvn37lClTpmRptu3btyvjxo1TFEVRjh07pgwaNMj63L1795R27dopCQkJSkxMjPX/n376qbJ+/XpFURTlm2++UZYuXZqlGV+U89q1a8qbb76pmM1mJTk5WenWrZsSFhamhIeHKwMHDszybOnNqSiK0r17d+XBgwepptna+nzc1q1blVGjRimKomTLz+OTFi5cqLRr107p0qVLqumJiYnW35OEhASlY8eOyr1799L1mZCdOePj45UWLVooRqNRURRFGTlypLJz507FZDIpb7zxRrZkS09ORVGU0aNHK6dOnUo17d+sT5s7RpNROWV49Ity1qhRI9VfkMnJyej1ek6fPs2ZM2fo2bMn77//Pvfu3cvSbNWrV+f06dPW506ePEmNGjWwt7fHzc0Nb29vzp0799T7yY7h5c/LWaRIERYvXoxOp0Or1WI2m3FwcODMmTPcvXuXXr160b9///9v7+5CmmoDOID/1ZyWE0vsy4uJSq5yRJaEWeGFlJMCCamlxSyllAqDpWUaRmmhlhA5yr4Q8qZkfWkXFkQkREhLXrqoJaxcmSR5Ubit5sfOexEe3pW9vvl6zlny/13psw3+e3y2/9n2eIY3b94omtPr9cLhcKCyshLbtm2DxWL56Tb+MJ9j3G43GhoaUFFRAQCyrMcfaTQaNDQ0/DRut9uh0WgQEREBlUqFlStXwmq1TvhYkzunSqXC9evXxVeoY2vTZrPh69ev4q7av/6S5/+rfpUT+P68eOnSJeTk5ODixYsAJn7uGo/fnRngdyixPVqunCEhIQgJCcHw8DDKyspgMBgQFhaGuLg46HQ6pKamorW1FdXV1Th37tyUZQUAp9MJtVot/h4UFISRkRHMmDEDTqdTzD2W3el0+oxP9fxNJmdwcDAiIyMhCALq6uqwdOlSxMbGYmBgAHv27EFmZiasVitKS0tx8+ZNxXK63W7s2LEDu3btwujoKIxGI3Q6nd/N5xiLxQK9Xi++VSLHevxRRkYGent7x83vL2vz33IGBgYiKioKANDc3Ay32401a9agu7sbBQUF2LJlC3p6erB79260t7f7zL+cOQFg48aNyM3NhVqtxv79+/Ho0aNJzecfXTRyb4+WMycAfPnyBcXFxVi1ahUKCwsBACkpKeKR0Pr16yV5UKvVajEb8P2oe2yx/3iZy+VCeHi4OB4aGjrl8zeZnADg8XhQXl6OsLAwHDt2DACg0+nETSLJycno7+/3OciQO+fMmTNhNBrFv2lKSgpsNptfzicAtLW1+aw5OdbjfzXR2hwbk2MuJ+L1enH69Gm8ffsWDQ0NCAgIQGxsLGJiYsSfZ8+ejU+fPmHhwoWKZBQEAXl5eWKppKWl4eXLl5Oaz2n31tmfsj16opzfvn3Dzp07kZ2djX379onjR48exf379wEAT58+RWJioiTZOjo6AHz/0Hfsg14AWLZsGZ4/fw6Px4PBwUHY7XYkJCRMeH+k8G85BUHA3r17odVqceLECbFczGaz+OrSZrMhOjpa0pKZKGdPTw9yc3MxOjqK4eFhdHV1ITEx0e/mEwAGBwcxNDTk88Qnx3r8r+Lj4+FwOPD582cMDQ3BarUiKSlJkbmcSGVlJTweD86fPy8WtcViQU1NDQCgv78fTqcTc+fOVSyj0+nEpk2b4HK5IAgCOjs7odPpJjWf027X2Z+yPXqinF1dXTCbzViyZIl4m7HPbMrLywF8Pxqurq7GvHnzpjTb2O6j7u5uCIKAU6dOoaOjAxqNBunp6WhpacGNGzcgCAIKCwuRkZGBgYEBHD58GC6XC3PmzEF9fT1mzZo1pbl+J6fX64XJZMLy5cvF65tMJsTFxaG0tBRutxtBQUGorKxEfHy8YjnT09Nx+fJltLe3Izg4GFlZWcjJyfG7+UxPT8eLFy/Q2Njoc/7B9+/fS74ex9Pb2wuTyYSWlha0tbXB7XbDYDCIu84EQUB2dja2b9/+y8eaHMbLqdPpkJ2djeTkZPEgx2g0Ii0tDUeOHEFfXx8CAgJQUlKCFStWKJbTYDDgzp07aG5uhkqlwurVq1FcXDyp+Zx2RUNERP5l2r11RkRE/oVFQ0REkmLREBGRpFg0REQkKRYNERFJikVDRESSYtEQEZGkWDREMsjPz4dWq8WDBw98xgVBQFlZGbRaLc6cOaNQOiJpsWiIZHDo0CEEBgbi7NmzGB0dFcdra2tx+/ZtbN26FSUlJQomJJIOi4ZIBosXL0ZWVhbsdjvu3r0LAGhsbERTUxMyMzNx/PhxhRMSSYenoCGSycePH7FhwwZERUUhPz8fVVVVWLt2LS5cuACVSqV0PCLJ8BUNkUwWLFiAvLw8fPjwAVVVVUhKSoLZbP6pZJ49e4aioiKsW7cOWq0Wt27dUigx0dRg0RDJ6J/frX7y5EnxFPH/5Ha7kZCQgIqKCoSGhsoZj0gSLBoimdy7dw+1tbXiKdWvXbs27vXS0tJgMpmg1+t9vpCP6E/FVUwkg8ePH6OsrAyLFi1Ca2sr4uLiYLFYYLfblY5GJDkWDZHErFYriouLMX/+fFy9ehWRkZE4cOAARkZGUF9fr3Q8IsmxaIgkZLPZUFRUhPDwcDQ1NYnfPqnX66HT6fDw4UNYrVaFUxJJi0VDJBGHw4GCggIEBATgypUr0Gg0PpcfPHgQAFBXV6dEPCLZzFA6ANF0FRMTgydPnvzy8tTUVLx+/VrGRETKYNEQ+RmXy4V3794BALxeL/r6+vDq1StEREQgOjpa4XREv49nBiDyM52dnTAajT+Nb968GTU1NQokIvp/WDRERCQpbgYgIiJJsWiIiEhSLBoiIpIUi4aIiCTFoiEiIkmxaIiISFIsGiIikhSLhoiIJMWiISIiSf0Nms9hJ83lHtcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_decision_boundary(lambda x:predict_dec(Zs=[Z1, Z2, Z3], As=[A1, A2, A3], X=x.T), X_train.T, Y_train.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The shaded decision boundary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZoAAAEfCAYAAAB4ewAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XlYVPXiBvCXbQBZVNzNIPEKlugFtNwCrlcIU7ypXGVJcEsxrytqqLmgkoKG1yS13MjlqiBaLpR6TS+Umf2EcMfr466ZSwYywzrM+f1hnOsICChnzmF4P8/j8zBngXfG4bx8z/nOjIkgCAKIiIgkYip3ACIiMm4sGiIikhSLhoiIJMWiISIiSbFoiIhIUiwaIiKSFIuGJDdz5ky4urrq/XNzc4OPjw9mzJiBS5cuSfrzw8LC8Ne//rVG+yQkJMDV1RW3bt2SKJUyfmZtqKu5yXDM5Q5A9cesWbPQuHFjAEBBQQGuX7+O3bt34+DBg1i3bh26desmyc8dN24cCgoKarSPn58fHB0d4eDgIEkmovqERUMG4+vrizZt2ugtCw8PR2BgIKZMmYLDhw/Dxsam1n9ur169arxPhw4d0KFDh1rPQlQf8dQZyapVq1aIiorCw4cPsWvXLrnjEJEEWDQku759+0KlUuG7777TW/7zzz9j5MiR8PDwgIeHB0aNGoXTp0+X2//UqVMYM2YMXn/9dXTr1g1jx47FxYsXxfVPX6MpLi7GRx99hD59+ojXihYsWIDc3Fxxm4quO/z++++Ijo6Gl5cX3Nzc4O/vj7Vr16K0tFRvv06dOuHatWuIiIiAh4cHXn/9dURFReH333+v1uNx5coVhIeHo3PnzvjLX/6CTz75BCUlJXrbVDdLRddOnl5ek8w3btzAxIkTxcf6n//8Jyp6F6tz585h4sSJ6NmzJzp27IgePXpg2rRp+PXXX8s9Vv/+97/Rq1cveHh4YMeOHXB1dcXSpUvLfc9ly5bBzc1N7/+J6gaeOiPZWVpawtHREdnZ2eKyY8eOISIiAh06dMDkyZNRXFyM3bt3491330ViYiK6du0KADh58iRGjBiB5s2bY/To0bCyssLmzZsRHh6OXbt2lTtVBwALFy7E/v37ER4ejpdffhmXLl3Cv/71L1y/fh0bN26sMGNubi6Cg4Nx+/ZtBAcHo23btjh27Bji4+Nx/vx5rFixQtxWp9MhPDwcXbt2RVRUFM6cOYOUlBQUFhbik08+qfLxmDx5Mrp164aoqCj89NNPWL16Ne7cuYPY2NgaZ6mu6mR+8OABgoODUVJSguHDh8PKygrbtm3Do0eP9L7XxYsXERoaCicnJ4wdOxbW1tbIzMzEnj17cO/ePWzZskXcVqvVYs6cORg9ejSKi4vxxhtvoGPHjjhw4AA++OADve/7zTffwMvLCw0bNqzx/SN5sWhIEezt7XHjxg0Ajw968+fPR6dOnbB161aYmZkBAIYNG4aBAwciJiYGX331FQAgLi4OjRo1wq5du8SJBj4+PujXrx+2bdtW7mAFAPv27UNgYCAiIyPFZQ0aNMB3330HjUZT4XWidevW4dq1a1i1ahV8fX0BAO+++y4WLFiAbdu2YdCgQfDx8QHw+ODZr18/zJw5EwAQHByMu3fv4vDhwygoKIC1tfUzHwsfHx+xLN59913MmjULu3fvxsiRI+Hq6lqjLNVVncwbNmwQT3F27NgRADBo0CAEBATofa9t27bBxMQEmzdvRqNGjQAAQUFBKCkpQWpqKnJycsTlOp0Ow4YNw9ixY8X9BwwYgNjYWJw+fRqdO3cG8Hh0e/v2bUyfPr1G94uUgafOSBG0Wi1MTEwAAOfPn8fNmzfh6+uL3NxcPHz4EA8fPkRhYSF69+6NCxcu4Ndff8Vvv/2GM2fOYMCAAWLJAEDbtm2xa9cujBkzpsKf1bJlS3z99dfYvXu3+Nf4lClTsGvXrkonIxw5cgTt2rUTD+xlxo8fDwD49ttv9Za//fbberdfffVVaLVa5OTkVPlYjB49Wu92WFgYACAtLe25slRXVZnT09PRqVMnsWQAoEmTJujfv7/eftHR0Thy5IhYJgCgVqthaWkJAMjPz9fb/s0339S73a9fP5iamuKbb74Rl6WmpqJBgwbo3bv3c903khdHNKQIOTk54lTispHN0qVLKzxXDwB37tyBmZkZBEGAk5NTufWvvfZapT8rOjoaU6ZMwaxZszB37ly4u7vDz88PgYGBsLOzq3CfW7duwcvLq9zyZs2awd7eHrdv39Zb/vS0aJVKBQB611Aq4+zsrHfb0dFRzPA8Waqrqsy3b99Gnz59qsxrYmKC33//HZ9//jkuXryIGzdu4JdffhGv5eh0Or3tmzRpone7RYsWeP3113Hw4EFERUVBp9PhwIED6NOnT5WjQVImFg3JTq1W4+bNm/jLX/4C4H8HosmTJ8Pd3b3CfZydnXH16lUAgKlpzQbmPXr0wNGjR8V/x44dw5IlS/DFF19g9+7dFb525lkf26TT6WBhYaG3rGx09jye3rfsZ5edQqxplqdVVnZVZTYxMUFRUVG55U/n+c9//oPx48ejefPm6N69O7y9veHm5obvv/8en3/+ebn9K/r/GzBgAObMmYNTp06hsLAQ9+/fL3eKjuoOFg3J7sCBAxAEQfxr+aWXXgLw+LpJz5499bY9ffo0cnNzYWVlhVatWgEArl+/Xu57Llu2DA0bNtQ79w88nnF24cIFtGzZEv3790f//v2h0+mQmJiIpUuXIjU1VTxV9aSXXnoJV65cKbf8/v37UKvVYpbacPv2bbRv3168XVaoZSOb6mYpO4AXFxfrbffgwYPnytWmTRtcu3at3PKbN2/q3V60aBGcnJywa9cuNGjQQFy+b9++av8sf39/LFy4EEeOHEFBQQEaNWr0XK+HImXgNRqS1b1797By5Uq0aNECAwYMAAC4ubmhWbNm2LJlCzQajbitWq0WT3mZmZmhRYsW6NChA1JTU6FWq8Xtbt68ic2bN1d4QM3JyUFQUJDeX9ampqbo1KmT+HVFevfujStXruDw4cN6y9euXQsA4misNiQnJ+vdTkxMhImJiThFu7pZmjVrBgB6s/nUarV4raem3nrrLVy6dAnp6enisry8POzZs0dvu5ycHLRu3VqvZO7cuYNDhw4BqN7pQ3t7e/j4+CAtLQ1paWnw9/evcqRGysURDRnM4cOHxYv2RUVFuHLlCr766isUFRVh3bp1sLKyAgBYWFhg7ty5mDJlCgYPHoy///3vsLS0xM6dO/HLL7/g448/hrn546furFmz8N577yEwMBBDhgyBqakptm7dCnt7+wonAzRv3hwDBgzAtm3bUFBQAA8PD+Tk5GDr1q1o2rRpuQviZSIiInDo0CFMmTIFISEheOWVV/Djjz/i0KFDeOutt2o8y+tZ9u3bB7Vajc6dOyMtLQ1Hjx7Fe++9J16Lqm4WX19fxMTEYOHChbh9+zZUKhWSk5P1CqAmRo4cib1792LixIkYPnw4HBwckJSUVO7Umbe3N77++mvMmzcPnTp1wq1bt5CcnCy+DdCTfzw8S0BAACZPngwAiImJea7MpAwsGjKYJUuWiF/b2NigVatW+Otf/4oxY8agbdu2etv6+/tj48aNWLNmDVavXg1TU1O0b98ea9as0Zt51L17d2zatAkrV67EqlWrYGlpiddffx0zZswQ/6J/2qJFi/Dyyy8jNTUVqampsLa2Ro8ePTB16tRK39usUaNGSEpKwooVK/D111/j0aNHePnll/HBBx9gxIgRL/7gPGHdunWIiYnB/v370aJFC8yaNUvvZ1Q3i4ODA9atW4f4+HisXLkSjRs3xtChQ+Hs7IypU6fWOJetrS22bduGZcuWISkpCaWlpejXrx/at2+vVwTR0dFo0KABjhw5gj179qBly5YYOHAg/Pz8EBISgh9//PGZkzXK9O7dG7a2trC1tRVfN0V1k4nwrCuLREQyKS4uRs+ePREUFIQZM2bIHYdeAK/REJEipaamIi8vD4MHD5Y7Cr0gnjojIkXZuHEjMjMzkZ6ejt69e6Ndu3ZyR6IXpNgRzalTpyqcZpqYmIj+/fsjLCwMYWFhuHLlCgoLCzFx4kSEhoZizJgxePjwoQyJiag2lJaW4vvvv8ef//xnTgIwEoq8RrNu3Trs3bsX1tbW5aZ6Tp8+HSNGjICbm5u4LDExEWq1GhMnTkRqaip+/vlnzJkzx9CxiYioAooc0Tg6OiIhIaHCdefOncPatWsREhIivhYiIyNDfEsOb29vHD9+3GBZiYjo2RR5jcbf37/Szx/v378/QkNDYWtriwkTJuDo0aNQq9Xie1TZ2NggLy+vwn0zMjIky0xEZMy6dOny3PsqsmgqIwgChg8fLpaKj48Pzp8/D1tbW/FFYBqNBvb29pV+D23b53uxGul7/2/DAQBr9m6SOQlJwTnrjtwRSEFuvOBnACny1Fll1Go1AgICoNFoIAgCTpw4ATc3N3h6eopvq5Genv5CzUtU37FkqLbViRHNvn37kJ+fj6CgIEydOhXh4eFQqVTo0aMHfHx88MYbbyAqKgohISGwsLBAfHy83JGJiOgPipx1JpWMjAyeOqslPHVmnDiaoYrcaNiw/lyjISJpsGBISnXqGg0REdU9LBqieo6jGZIai4aoHmPJkCGwaIjqKZYMGQqLhqgeYsmQIbFoiOoZlgwZGouGqB5hyZAcWDRE9QRLhuTCoiGqB1gyJCcWDZGRY8mQ3Fg0REaMJUNKwKIhMlIsGVIKFg0REUmKRUNkhDiaISVh0RAZGZYMKQ2LhsiIsGRIiVg0REaCJUNKxaIhMgIsGVIyxX6U86lTp/Dxxx9jy5Ytesv379+PTZs2wczMDC4uLoiOjoapqSkGDhwIOzs7AECbNm2wZMkSOWITGRxLhpROkUWzbt067N27F9bW1nrLCwsLsWLFCuzbtw/W1taIjIzE0aNH8eabbwJAuVIiMnYsGaoLFHnqzNHREQkJCeWWq1Qq7NixQywgrVYLS0tLZGdno6CgAKNGjUJ4eDiysrIMHZnI4FgyVFcosmj8/f1hbl5+sGVqaoqmTZsCeDx6yc/PR69evWBlZYXRo0djw4YNWLBgAaZPnw6tVmvo2EQGw5KhukSRp86eRafTYdmyZbh69SoSEhJgYmKCtm3bwsnJSfy6UaNGuH//Plq1aiV3XKJax5KhukaRI5pnmTdvHoqKirB69WrxFFpKSgpiY2MBAHfv3oVarUazZs3kjEkkCZYM1UV1YkSzb98+5Ofnw83NDSkpKejatSuGDx8OAAgPD8ff//53zJo1CyEhITAxMcHixYsrPPVGVJexZKiuUuzRuE2bNkhOTgYADBgwQFyenZ1d4fbx8fEGyUVERDVT506dEdVHHM1QXcaiIVI4lgzVdSwaIgVjyZAxYNEQKRRLhowFi4ZIgVgyZExYNEQKw5IhY8OiIVIQlgwZIxYNkUKwZMhYsWiIiEhSLBoiBeBohowZi4ZIZiwZMnYsGiIZsWSoPmDREMmEJUP1BYuGSAYsGapPWDRUqcKSAtxT/1qjfbSlJfgl95ZEiYwDS6Z2mN28aZB96MWxaKhSC/49CxG7wnDn0e1qba8tLcGcg9PxXkoI8ooeSZyubmLJ1A6Ln35C41GjYLV/f7X3UX3/PRqPHg3LgwclTEYVYdFQpcK7vAd1UR7GfzmiyrIpK5n/XP43hncZCztLewOlrDtYMrWnxN0dxa+/Drt//rNaZaP6/nvYL1wIrYsLit980wAJ6UksGqrUqy3csPKd9VWWzZMlM9VrFoLcwwyclOodlQqPoqNR1K1blWXzZMnkxsVBsLExYFACWDRUharKhiVTPRzNSKAaZcOSUQbFFs2pU6cQFlb+oHXkyBEEBgYiKCgIycnJAIDCwkJMnDgRoaGhGDNmDB4+fGjouEatsrIRBIElUw0sGQk9o2xYMsphIgiCIHeIp61btw579+6FtbW1WCYAUFJSgn79+iElJQXW1tYICQnBZ599hv3790OtVmPixIlITU3Fzz//jDlz5pT7vhkZGdC2bWDIu2JULtw9i0l73oOtpR0aJzngdu4t5Ab/zpJ5BpaMgRQXwz46GpYnTiBv6lToGjViydSiGw0bokuXLs+9vyJHNI6OjkhISCi3/PLly3B0dETDhg2hUqnQpUsXnDx5EhkZGfDy8gIAeHt74/jx44aOXC+UjWxyCh7i/K9nkFvAknkWlowBPTWyaTh/PktGQRRZNP7+/jA3Ny+3XK1Ww87OTrxtY2MDtVqtt9zGxgZ5eXkGy1rftG/qioZWjcXb3s5/lTGNcrFkZKBSocjXV7xZ5O3NklEIRRZNZWxtbaHRaMTbGo0GdnZ2ess1Gg3s7Tm1VgplF/5/zfsFDg2awMzUrFpTn+sblow8VN9/D7vYWJT86U8o7tQJtp9/XqPX2ZB06lTRtGvXDtevX0dOTg6Ki4tx8uRJeHh4wNPTE2lpaQCA9PT0FzqXSBV7enaZk4Mz/tTUtdqvs6kvWDLy0Lvwv3w5cpcurdbUZzKMOlE0+/btQ1JSEiwsLDBz5kyMHj0awcHBCAwMRIsWLRASEoJLly4hJCQESUlJmDBhgtyRjUplU5gbqGyq9TobIilVOLusBq+zIekpctaZVDjrrOYqK5n3/zYcALBm7ya92WirB32BVvYvyRlZNhzNGF6VU5ifmo1WGBAgT9A6zihnnZEyVPfFmNV9BwFjxpIxvGq9ToYjG0Vg0VClYr6dU+0XYz5dNvXpTTVZMoZn8fPP1X+dzFNlY3n0qOGCEgCg/Bxioj8EvDoIHVt0xpA/v1ut7V9t4YaEgRvw4/XvYauyq3oHI8CSkYfW1RUFf/sb8keOrN4U5j/KxmbdOhS7u0sfkPSwaKhSXV/ujq4vd6/RPh2ad0SH5h0lSqQsLBn5CA0aQFPTST8qFTT/+Ic0geiZeOqM6DmwZIiqj0VDRESSYtEQ1RBHM0Q1w6IhqgGWDFHNsWiIqoklQ/R8WDRE1cCSIXp+LBqiKrBkiF4Mi4boGVgyRC+ORUNUCZYMUe1g0RARkaRYNEQV4GiGqPawaIiewpIhql0sGqInsGSIah+LhugPLBkiabBoiMCSIZKSIj+PRqfTITo6GhcvXoRKpUJMTAycnJwAABcuXMDixYvFbbOysrBq1Sp07twZ/v7+cHFxAQD4+vpi+PDhsuSnuoUlQyQtRRbN4cOHUVxcjKSkJGRlZSE2NhZr1qwBALz66qvYsmULAOCbb75B8+bN4e3tjR9++AEBAQGYO3eunNGpjmHJEElPkafOMjIy4OXlBQBwd3fH2bNny22Tn5+PhIQEfPjhhwCAs2fP4ty5cxg2bBgmTZqEe/fuGTQz1T0sGSLDUGTRqNVq2NrairfNzMyg1Wr1tklJSUHfvn3h4OAAAHB2dsakSZOwdetW+Pr6IiYmxqCZqW5hyRAZjiKLxtbWFhqNRryt0+lgbq5/lm/fvn0YMmSIeLt79+7o1q0bAMDPzw/nz583TFiqc1gyRIalyKLx9PREeno6gMcX+8su8JfJy8tDcXExWrVqJS6bM2cODh48CAA4fvw4OnbsaLjAVGewZIgMT5GTAfz8/HDs2DEEBwdDEAQsXrwYiYmJcHR0RJ8+fXD16lW89NJLevtMmzYNs2fPxvbt22Ftbc1TZ1QOS4ZIHiaCIAhyhzCUjIwMaNs2kDuGUXj/b4+njq/Zu0nmJNXDkiF6fjcaNkSXLl2ee39Fnjojqk0sGSJ5sWjIqLFkiOTHoiGjxZIhUgYWDRkllgyRcrBoyOiwZIiURZHTm4meBwuGSJk4oiGjwJIhUi4WDdV5LBkiZat3p87KDkpX3FtVsSUpHQuGqG6od0VTxjnrDsumDmPJEEnn6WOj+dX8F/p+9bZoAI5u6iqWDFHtkvoYWK+LpgwLp25gwRDVDkMf61g0T2DhKBdLhuj5yX1MY9FUgNdvlIMFQ1RzSjt+PVfRPHr0CJmZmbC3t4eHhwdMTEzEdfn5+di4cSMmTJhQayHlwNGNvFgwRNWn9ONUjYvm0qVLGDlyJB4+fAidTofXXnsNCQkJ4geR5efnY9WqVXW+aMqwcAyLBUNUtbp2PKrxCzbj4+Ph7u6OkydPIj09HS+//DJCQkJw7do1CeIph3PWHR4EJcTHl6hyV9xb6f2ra2o8ojl16hQ2b96MBg0aoEGDBvjkk0+wZMkShIWFYfPmzbCzs5Mip2I8eTCsi//hSsNyIaqYMR1falw0xcXFetdkAGDWrFkQBAFhYWGIj4+vtXBKx9Nqz4flQlSeMR9Halw0bdu2xdmzZ/GnP/1Jb/ns2bOh0+kwfvz4Fw6l0+kQHR2NixcvQqVSISYmBk5OTuL6mJgYZGZmwsbGBgCwevVqlJSUYPr06SgsLETz5s2xZMkSWFtbv3CW6uAop2osF6L/qW/HiRoXjZ+fH/bv34+BAweWWzdnzhyUlpZi+/btLxTq8OHDKC4uRlJSErKyshAbG4s1a9aI68+dO4f169fDwcFBXBYTE4OAgAAMHjwYa9euRVJSEkaMGPFCOZ4HS+d/WC5E/1Ofjwc1ngwQERGB9evXV7p+/vz5yM7OfqFQGRkZ8PLyAgC4u7vj7Nmz4jqdTofr169j3rx5CA4ORkpKSrl9vL298cMPP7xQhtpQdoG7Ph1w6+N9JqpIXb+AX5uqHNEsX74ckZGRhsgiUqvVsLW1FW+bmZlBq9XC3Nwc+fn5GDZsGEaOHInS0lKEh4fDzc0NarVanIhgY2ODvLw8g2auytMHXmN44lmri1koRH8wht9pqVRZNGvXrkVubi4WLFhgiDwAAFtbW2g0GvG2TqeDufnjqNbW1ggPDxevv3Tv3h3Z2dniPlZWVtBoNLC3tzdY3udR0QFa6U/UJzNbq4tlTEIkP6X/vipJlUUzcOBAJCUlQa1WIy4uTjzgPykjIwPLli3Djh07aiWUp6cnjh49in79+iErKwsuLi7iumvXrmHq1Kn48ssvodPpkJmZiUGDBsHT0xNpaWkYPHgw0tPT0aVLl1rJYkhVjQ6kfmJzdEJUORbL86uyaGJjY9GoUSN88cUXyMvLQ0JCAiwtLQEAV69eRXx8PL799ttaDeXn54djx44hODgYgiBg8eLFSExMhKOjI/r06YMBAwZg6NChsLCwwDvvvIP27dvj/fffR1RUFJKTk9G4cWOjnGbNIiAyHBZL7TERBEGozoafffYZVqxYgS5duiAmJgabN2/Gzp07odVq4ebmhmnTpqFHjx5S530hGRkZcMzNlTuGURj4x3W7r5YvlzkJUe1gsVTO/Gr+C50lqvb05nHjxsHOzg6LFi1Cv379ADx+Tc3kyZPh7+//3AGIiOTAYjGcahWNIAjYs2cPNm7cKN5u1qwZ/vWvf6Fx48aSBiQiqg0sFvlUWTSHDx/GihUrcPnyZahUKowdOxYODg6Ii4vDiBEjsHHjRjRp0sQQWYmIqo3FohxVFs2ECRNgamqKgQMHYsqUKWjRogUAoGnTppg5cyZCQkKwceNGtGnTRvKwRESVYbEoV5VF06tXL8yYMQMdOnTQWx4QEAA7OztMnjwZoaGh2LBhA9q3by9ZUCKiJ7FY6o4q34Jmw4YN5UqmjI+PD9avXy++Wp+ISCp8S5e6q8bvdfa0rl27YuvWrbCwsKiNPEREIhaLcajxuzdXpEOHDti2bVttfCsiqsdYKMapVooGABwdHWvrWxFRPcFiqR9qrWiIiKrCYqmfWDREJCmWC7FoiKhWsVjoaSwaInphLBd6FhYNEdUYi4VqgkVDRNXCcqHnxaIhokqxXKg2sGiISA/LhWobi4aIWC4kKRYNUT3FciFDYdEQ1TMsGDI0xRWNTqdDdHQ0Ll68CJVKhZiYGDg5OYnrv/jiC6SmpgJ4/DEFEyZMgCAI8Pb2xiuvvAIAcHd3x7Rp0+SIT6RILBeSk+KK5vDhwyguLkZSUhKysrIQGxuLNWvWAABu3ryJvXv3YufOnTAxMUFoaCh8fX1hbW2Njh074rPPPpM5PZGysGBICRRXNBkZGfDy8gLweGRy9uxZcV3Lli2xfv16mJmZAQC0Wi0sLS1x7tw53L17F2FhYbCyssKsWbPg7OwsS34iubFcSGle+IPPaptarYatra1428zMDFqtFgBgYWEBBwcHCIKAuLg4vPbaa2jbti2aNWuGsWPHYsuWLYiIiMCMGTPkik8kK5YMKZHiRjS2trbQaDTibZ1OB3Pz/8UsKirC7NmzYWNjg/nz5wMA3NzcxFFO165dcffuXQiCABMTE8OGJ5IBy4WUTnEjGk9PT6SnpwMAsrKy4OLiIq4TBAHjx4+Hq6srFi5cKJbLp59+ik2bNgEAsrOz0bp1a5YMGT1+xDHVFYob0fj5+eHYsWMIDg6GIAhYvHgxEhMT4ejoCJ1Oh59++gnFxcX47rvvAACRkZEYO3YsZsyYgbS0NJiZmWHJkiUy3wsiabFgqC5RXNGYmppi4cKFesvatWsnfn3mzJkK91u7dq2kuYiUgAVDdZHiTp0RUcVYMlRXKW5EQ0T6WDBU13FEQ6RgLBkyBiwaIoViyZCx4KkzIoVhwZCx4YiGSEFYMmSMWDRECsGSIWPFoiFSAJYMGTMWDZHMWDJk7Fg0RDJiyVB9wKIhkglLhuoLFg0REUmKRUMkA45mqD5h0RAZGEuG6hsWDVXK7MYNqE6cqNE+pvfuQZWWJlEiosdKdaXYdWYHtDpttfcRBAFfnk1CkbZIwmRUERYNVcpm7VrYz5sH1fHj1dre9N49NIqMhN0//wkTtVridHUTRzO148fr32PZfxYi+lBUtcpGEAQsT1+MuKML8E32XgMkpCexaKhSeTNnQtuuHeyjo6ssm7KSMXn0CLmxsRBsbQ2UkuqjXm19MKHXNBy+9E2VZVNWMjtP/wsh7sPxTse/GzApASwaegbB1ha5S5dWWTZ6JbN0KbQdOhg4ad3A0UztGuY5usqyebpkJr35AUxMTGRIW7+xaOiZqioblgzJ6Vllw5JRDkUWjU6nw7x58xAUFISwsDBcv35db31ycjIGDx6MoUOH4ujRowCAhw8fYtSoUQgNDcWUKVNQUFAgR3QsDSN/AAAR4ElEQVSjVGnZFBezZKqJoxnpVFQ2LBllUeTn0Rw+fBjFxcVISkpCVlYWYmNjsWbNGgDA/fv3sWXLFuzatQtFRUUIDQ1Fr169sHr1agQEBGDw4MFYu3YtkpKSMGLECHnviBEpK5uGH3wA++homLZoAdP792FiYcGSIdkN8xwNAPj0WDwEQUAj68bYdWY7S0YhFFk0GRkZ8PLyAgC4u7vj7Nmz4rrTp0/Dw8MDKpUKKpUKjo6OyM7ORkZGBiIiIgAA3t7eWL58eYVFMzAy0iD3wWhZWMAcwJnbtwEA3k5OENaulTeTwhXYquSOUG+0zmuDb3MPAACa2TZH9qHzGL9shLyhjMC6T9a80P6KPHWmVqth+8SsJTMzM2i1WnGdnZ2duM7GxgZqtVpvuY2NDfLy8gwbur4oLYWJIPzvdkmJfFmIniQAJaXF4s0SnRYChGfsQIaiyBGNra0tNBqNeFun08Hc3LzCdRqNBnZ2duJyKysraDQa2NvbV/i9v1q+XNrwRky88G9lBe8WLWB25w7+c+sWHr33Hop79JA7nmLx+oz0yq7J/Hz6/xDiPhxNbJri02PxaNq+G6LfioO5qSIPdXXH1fwX2l2RIxpPT0+kp6cDALKysuDi4iKu69y5MzIyMlBUVIS8vDxcvnwZLi4u8PT0RNofr0hPT09Hly5dZMlurJ6eXSbY2UHbrl21X2dDJJWKLvxXZ+ozGY4ia97Pzw/Hjh1DcHAwBEHA4sWLkZiYCEdHR/Tp0wdhYWEIDQ2FIAiYOnUqLC0t8f777yMqKgrJyclo3Lgx4uPj5b4bRqPSKcxmZsj96CNxgsCj6GiObJ7C0Yy0njW77MkJAgA4spGRiSAI9eYkZkZGBhxzc+WOUadUVjJlkyq+Wr4cJmo1Gn7wAcwvX2bZPIVFI53qTmHemrkBnx6Lh2/7t1k2z8n8av4LnSVS5KkzUobqvhizuu8gQFRbavI6GZ5Gkx+Lhiplu2JFtV+M+WTZ2C1ZwjfVJEl9d/VojV6M+WTZ7Du/y0ApqQzHkFQp9bRpMH3wAFpX12ptX1Y2Zlev8k01wdNmUvJq2xvxAWvQ8xXvar8Yc5jnaDg7/AndHN+UOB09jUVDldI1aQJdkyY12kewtYW2UyeJEhE9ZmJigl5tfWq8X89Xar4PvTieOiMiIkmxaIiISFIsGiIikhSLhoiIJMWiISIiSbFoiIhIUiwaIiKSFIuGiIgkxaIhIiJJsWiIiEhSLBoiIpIUi4aIiCTFoiEiIkmxaIiISFIsGiIikpTiPo+msLAQM2bMwG+//QYbGxvExcXBwcFBb5u4uDhkZmZCq9UiKCgIQ4cORU5ODvz9/eHi4gIA8PX1xfDhw+W4C0RE9ATFFc327dvh4uKCiRMnIjU1FatXr8acOXPE9T/++CNu3LiBpKQkFBcXo3///vD398f58+cREBCAuXPnypieiIieprhTZxkZGfDy8gIAeHt74/jx43rrPTw8sHjxYvF2aWkpzM3NcfbsWZw7dw7Dhg3DpEmTcO/ePYPmJnqac9YduSMQKYKsI5qdO3di06ZNesuaNGkCOzs7AICNjQ3y8vL01ltaWsLS0hIlJSWYOXMmgoKCYGNjA2dnZ7i5uaFnz57Yu3cvYmJisHLlSoPdFyIiqpisRTNkyBAMGTJEb9mECROg0WgAABqNBvb29uX2y83NxaRJk/DGG28gIiICANC9e3dYW1sDAPz8/FgyREQKobhTZ56enkhLSwMApKeno0uXLnrrCwsLMWLECAQGBuIf//iHuHzOnDk4ePAgAOD48ePo2LGj4UITEVGlFDcZICQkBFFRUQgJCYGFhQXi4+MBAEuXLkXfvn2RmZmJmzdvYufOndi5cycAYPHixZg2bRpmz56N7du3w9raGjExMXLeDSIi+oOJIAiC3CEMJSMjA465uXLHMAoDIyMBAF8tXy5zEmW74t5K7ghEL8z8an65s0s1obhTZ0TGhDPPiFg0REQkMRYNERFJikVDRESSYtEQSYzXaai+Y9EQEZGkWDRERCQpFg0REUmKRUNkALxOQ/UZi4aIiCTFoiEiIkmxaIgMhKfPqL5i0RARkaRYNEREJCkWDZEB8fQZ1UcsGiIikhSLhsjAOKqh+oZFQ0REkmLREMmAoxqqT8zlDvC0wsJCzJgxA7/99htsbGwQFxcHBwcHvW3GjRuHnJwcWFhYwNLSEuvXr8f169cxc+ZMmJiYoH379pg/fz5MTdmjRERyU9yRePv27XBxccG2bdswcOBArF69utw2N27cwPbt27FlyxasX78eALBkyRJMmTIF27ZtgyAI+Pbbbw0dnYiIKqC4osnIyICXlxcAwNvbG8ePH9db/+DBAzx69Ajjxo1DSEgIjh49CgA4d+4c3njjDXG/H374wbDBiWqIp8+ovpD11NnOnTuxadMmvWVNmjSBnZ0dAMDGxgZ5eXl660tKSjBq1CiEh4cjNzcXISEh6Ny5MwRBgImJSaX7lbnRsKEE96T+WblhAwDghsw56jrzq/lyRyCSnKxFM2TIEAwZMkRv2YQJE6DRaAAAGo0G9vb2euubNm2K4OBgmJubo0mTJnj11Vdx9epVvesxFe0HAF26dJHgXhAR0bMo7tSZp6cn0tLSAADp6enlyuGHH37AlClTADwulEuXLsHZ2RmvvfYaTpw4Ie7XtWtXwwYnIqIKmQiCIMgd4kkFBQWIiorC/fv3YWFhgfj4eDRr1gxLly5F37590blzZ3z00Uc4deoUTE1N8d5778HX1xdXr17F3LlzUVJSAmdnZ8TExMDMzEzuu0NEVO8prmheVF2ZHl2dnHFxccjMzIRWq0VQUBCGDh2KnJwc+Pv7w8XFBQDg6+uL4cOH12o2nU6H6OhoXLx4ESqVCjExMXBychLXJycnY8eOHTA3N8f777+P3r174+HDh5g+fToKCwvRvHlzLFmyBNbW1rWaq6Y5v/jiC6SmpgIAfHx8MGHCBAiCAG9vb7zyyisAAHd3d0ybNk3WnDExMcjMzISNjQ0AYPXq1SgpKVHU43nhwgUsXrxY3DYrKwurVq1C586dJX8+VuTUqVP4+OOPsWXLFr3lR44cwapVq2Bubo7AwEAMHTq0Wr9rhs65f/9+bNq0CWZmZnBxcUF0dDRMTU0xcOBA8Rp1mzZtsGTJEllzJiYmIiUlRXy8FixYgNatW9f88RSMzMaNG4WVK1cKgiAI+/fvFxYtWlRum7ffflvQ6XR6yyIiIoQff/xREARBmDt3rnDo0CFZcx4/flwYP368IAiCUFRUJPj6+go5OTnCsWPHhIULF0qa7eDBg0JUVJQgCILw888/C+PGjRPX3bt3TwgICBCKioqER48eiV8vWrRI2LVrlyAIgvD5558LiYmJkmasKueNGzeEQYMGCVqtVigtLRWCgoKECxcuCNeuXRMiIiIkz1bdnIIgCMHBwcJvv/2mt0xpj+eTvv76ayEyMlIQBMEgz8enrV27VggICBCGDBmit7y4uFj8PSkqKhIGDx4s3Lt3r1rHBEPmLCgoEPr06SPk5+cLgiAIU6dOFQ4fPiwUFhYK77zzjkGyVSenIAjCtGnThDNnzugte57HU3HXaF5UXZkeXVVODw8Pvb8gS0tLYW5ujrNnz+LcuXMYNmwYJk2ahHv37kmazd3dHWfPnhXXnT59Gh4eHlCpVLCzs4OjoyOys7PL3R9DTC9/Vs6WLVti/fr1MDMzg6mpKbRaLSwtLXHu3DncvXsXYWFhGDNmDK5cuSJrTp1Oh+vXr2PevHkIDg5GSkpKuX2U8HiWyc/PR0JCAj788EMAMMjz8WmOjo5ISEgot/zy5ctwdHREw4YNoVKp0KVLF5w8ebLK3zVD51SpVNixY4c4Qi17bmZnZ6OgoECcVZuVlSVrTuDxcXHt2rUICQnB559/DqDqY1dFFPfOADUhx/RoQ+W0tLSEpaUlSkpKMHPmTAQFBcHGxgbOzs5wc3NDz549sXfvXsTExGDlypW1lhUA1Go1bG1txdtmZmbQarUwNzeHWq0Wc5dlV6vVestr+/F7npwWFhZwcHCAIAhYunQpXnvtNbRt2xYPHjzA2LFj8fbbb+PkyZOYMWMGdu3aJVvO/Px8DBs2DCNHjkRpaSnCw8Ph5uamuMezTEpKCvr27SueKjHE8/Fp/v7+uHXrVoX5lfLcfFZOU1NTNG3aFACwZcsW5Ofno1evXvjvf/+L0aNHY8iQIbh27RrGjBmDAwcO6D3+hswJAP3790doaChsbW0xYcIEHD169LkezzpdNIaeHm3InACQm5uLSZMm4Y033kBERAQAoHv37uJfQn5+fpL8Utva2orZgMd/dZc92Z9ep9FoYGdnJy63srKq9cfveXICQFFREWbPng0bGxvMnz8fAODm5iZOEunatSvu3r2r90eGoXNaW1sjPDxc/D/t3r07srOzFfl4AsC+ffv0nnOGeD5WV1XPzbJlhngsq6LT6bBs2TJcvXoVCQkJMDExQdu2beHk5CR+3ahRI9y/fx+tWrWSJaMgCBg+fLhYKj4+Pjh//vxzPZ5Gd+qsrkyPripnYWEhRowYgcDAQPzjH/8Ql8+ZMwcHDx4EABw/fhwdO3aUJFt6ejqAxxd9yy70AkDnzp2RkZGBoqIi5OXl4fLly3Bxcany/kjhWTkFQcD48ePh6uqKhQsXiuXy6aefiqPL7OxstG7dWtKSqSrntWvXEBoaitLSUpSUlCAzMxMdO3ZU3OMJAHl5eSguLtY78Bni+Vhd7dq1w/Xr15GTk4Pi4mKcPHkSHh4esjyWVZk3bx6KioqwevVqsahTUlIQGxsLALh79y7UajWaNWsmW0a1Wo2AgABoNBoIgoATJ07Azc3tuR5Po5t1VlemR1eVMzMzE59++ileffVVcZ+yazazZ88G8Piv4ZiYGDRv3rxWs5XNPvrvf/8LQRCwePFipKenw9HREX369EFycjKSkpIgCAIiIiLg7++PBw8eICoqChqNBo0bN0Z8fDwaNGhQq7lqklOn0yEyMhLu7u7i9pGRkXB2dsaMGTOQn58PMzMzzJs3D+3atZMtZ58+fbBu3TocOHAAFhYWeOeddxASEqK4x7NPnz44ffo0PvvsM733H7x586bkz8eK3Lp1C5GRkUhOTsa+ffuQn5+PoKAgcdaZIAgIDAzEu+++W+nvmiFUlNPNzQ2BgYHo2rWr+EdOeHg4fHx8MGvWLPzyyy8wMTHB9OnT4enpKVvOoKAgfPXVV9iyZQtUKhV69OiBSZMmPdfjaXRFQ0REymJ0p86IiEhZWDRERCQpFg0REUmKRUNERJJi0RARkaRYNEREJCkWDRERSYpFQ2QAo0aNgqurKw4dOqS3XBAEzJw5E66urvj4449lSkckLRYNkQF88MEHMDU1xYoVK1BaWiouj4uLw5dffomhQ4di+vTpMiYkkg6LhsgAOnTogHfeeQeXL1/Gnj17AACfffYZEhMT8fbbb2PBggUyJySSDt+ChshAfv31V7z11lto2rQpRo0ahUWLFuHNN9/EmjVroFKp5I5HJBmOaIgMpGXLlhg+fDhu376NRYsWwcPDA59++mm5kvm///s/jBs3Dl5eXnB1dcXu3btlSkxUO1g0RAb05Gerf/TRR+JbxD8pPz8fLi4u+PDDD2FlZWXIeESSYNEQGcj+/fsRFxcnvqX65s2bK9zOx8cHkZGR6Nu3r94H8hHVVXwWExlAWloaZs6cifbt22Pv3r1wdnZGSkoKLl++LHc0IsmxaIgkdvLkSUyaNAktWrTAhg0b4ODggMmTJ0Or1SI+Pl7ueESSY9EQSSg7Oxvjxo2DnZ0dEhMTxU+f7Nu3L9zc3PDtt9/i5MmTMqckkhaLhkgi169fx+jRo2FiYoL169fD0dFRb/20adMAAEuXLpUjHpHBmMsdgMhYOTk54dixY5Wu79mzJy5evGjARETyYNEQKYxGo8GNGzcAADqdDr/88gsuXLiAhg0bonXr1jKnI6o5vjMAkcKcOHEC4eHh5ZYPGjQIsbGxMiQiejEsGiIikhQnAxARkaRYNEREJCkWDRERSYpFQ0REkmLREBGRpFg0REQkKRYNERFJikVDRESSYtEQEZGk/h8Tj1bKQqgVRAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_decision_boundary_shaded(lambda x:predict_dec(Zs=[Z1, Z2, Z3], As=[A1, A2, A3], X=x.T), X_train.T, Y_train.T)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bonus \n",
    "\n",
    "Try to reinitialize the weights to `ini_type=plain` and see what happens."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}