Exploratory data analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "9b690f2c8129ad9445c544337375b93246c26668"
   },
   "source": [
    "# <center> Exploratory data analysis with Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "8000f32601aff93ebda3e8ca0baea376988a5649"
   },
   "source": [
    "## 1. Demonstration of main Pandas methods\n",
    "\n",
    "**[Pandas](http://pandas.pydata.org)** is a Python library that provides extensive means for data analysis. Data scientists often work with data stored in table formats like `.csv`, `.tsv`, or `.xlsx`. Pandas makes it very convenient to load, process, and analyze such tabular data using SQL-like queries. In conjunction with other useful library `Matplotlib` and `Seaborn`, `Pandas` provides a wide range of opportunities for visual analysis of tabular data.\n",
    "\n",
    "The main data structures in `Pandas` are implemented with **DataFrame** classes. `DataFrames` are great for representing real data: rows correspond to instances (examples, observations, etc.), and columns correspond to features of these instances.\n",
    "\n",
    "To run each cell we press **Shift+Enter**. Running the first cell should print \"Executing the first cell of my class.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Executing the first cell of my class\n"
     ]
    }
   ],
   "source": [
    "print (\"Executing the first cell of my class\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we import python libraries `numpy` and `pandas`. To do so, we execute the below cell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "_uuid": "e855067a1d281ca6634772513ef918aa1c9224f4"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "pd.set_option(\"display.precision\", 2)\n",
    "pd.set_option('display.max_columns', None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "e3ae8b8d09f0dd881831253957ca96d086038f7b"
   },
   "source": [
    "In this assignment, we'll demonstrate the main methods in action by analyzing a [dataset](https://bigml.com/user/francisco/gallery/dataset/5163ad540c0b5e5b22000383) on the churn rate of telecom operator clients. Let's read the data (using `read_csv`), and take a look at the first 5 lines using the `head` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "_uuid": "f274f88e14b4df01f8f60bd8be5e0cea903a7dd7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>account length</th>\n",
       "      <th>area code</th>\n",
       "      <th>phone number</th>\n",
       "      <th>international plan</th>\n",
       "      <th>voice mail plan</th>\n",
       "      <th>number vmail messages</th>\n",
       "      <th>total day minutes</th>\n",
       "      <th>total day calls</th>\n",
       "      <th>total day charge</th>\n",
       "      <th>total eve minutes</th>\n",
       "      <th>total eve calls</th>\n",
       "      <th>total eve charge</th>\n",
       "      <th>total night minutes</th>\n",
       "      <th>total night calls</th>\n",
       "      <th>total night charge</th>\n",
       "      <th>total intl minutes</th>\n",
       "      <th>total intl calls</th>\n",
       "      <th>total intl charge</th>\n",
       "      <th>customer service calls</th>\n",
       "      <th>churn</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>KS</td>\n",
       "      <td>128</td>\n",
       "      <td>415</td>\n",
       "      <td>382-4657</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>25</td>\n",
       "      <td>265.1</td>\n",
       "      <td>110</td>\n",
       "      <td>45.07</td>\n",
       "      <td>197.4</td>\n",
       "      <td>99</td>\n",
       "      <td>16.78</td>\n",
       "      <td>244.7</td>\n",
       "      <td>91</td>\n",
       "      <td>11.01</td>\n",
       "      <td>10.0</td>\n",
       "      <td>3</td>\n",
       "      <td>2.70</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>OH</td>\n",
       "      <td>107</td>\n",
       "      <td>415</td>\n",
       "      <td>371-7191</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>26</td>\n",
       "      <td>161.6</td>\n",
       "      <td>123</td>\n",
       "      <td>27.47</td>\n",
       "      <td>195.5</td>\n",
       "      <td>103</td>\n",
       "      <td>16.62</td>\n",
       "      <td>254.4</td>\n",
       "      <td>103</td>\n",
       "      <td>11.45</td>\n",
       "      <td>13.7</td>\n",
       "      <td>3</td>\n",
       "      <td>3.70</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NJ</td>\n",
       "      <td>137</td>\n",
       "      <td>415</td>\n",
       "      <td>358-1921</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>243.4</td>\n",
       "      <td>114</td>\n",
       "      <td>41.38</td>\n",
       "      <td>121.2</td>\n",
       "      <td>110</td>\n",
       "      <td>10.30</td>\n",
       "      <td>162.6</td>\n",
       "      <td>104</td>\n",
       "      <td>7.32</td>\n",
       "      <td>12.2</td>\n",
       "      <td>5</td>\n",
       "      <td>3.29</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>OH</td>\n",
       "      <td>84</td>\n",
       "      <td>408</td>\n",
       "      <td>375-9999</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>299.4</td>\n",
       "      <td>71</td>\n",
       "      <td>50.90</td>\n",
       "      <td>61.9</td>\n",
       "      <td>88</td>\n",
       "      <td>5.26</td>\n",
       "      <td>196.9</td>\n",
       "      <td>89</td>\n",
       "      <td>8.86</td>\n",
       "      <td>6.6</td>\n",
       "      <td>7</td>\n",
       "      <td>1.78</td>\n",
       "      <td>2</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>OK</td>\n",
       "      <td>75</td>\n",
       "      <td>415</td>\n",
       "      <td>330-6626</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>166.7</td>\n",
       "      <td>113</td>\n",
       "      <td>28.34</td>\n",
       "      <td>148.3</td>\n",
       "      <td>122</td>\n",
       "      <td>12.61</td>\n",
       "      <td>186.9</td>\n",
       "      <td>121</td>\n",
       "      <td>8.41</td>\n",
       "      <td>10.1</td>\n",
       "      <td>3</td>\n",
       "      <td>2.73</td>\n",
       "      <td>3</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  state  account length  area code phone number international plan  \\\n",
       "0    KS             128        415     382-4657                 no   \n",
       "1    OH             107        415     371-7191                 no   \n",
       "2    NJ             137        415     358-1921                 no   \n",
       "3    OH              84        408     375-9999                yes   \n",
       "4    OK              75        415     330-6626                yes   \n",
       "\n",
       "  voice mail plan  number vmail messages  total day minutes  total day calls  \\\n",
       "0             yes                     25              265.1              110   \n",
       "1             yes                     26              161.6              123   \n",
       "2              no                      0              243.4              114   \n",
       "3              no                      0              299.4               71   \n",
       "4              no                      0              166.7              113   \n",
       "\n",
       "   total day charge  total eve minutes  total eve calls  total eve charge  \\\n",
       "0             45.07              197.4               99             16.78   \n",
       "1             27.47              195.5              103             16.62   \n",
       "2             41.38              121.2              110             10.30   \n",
       "3             50.90               61.9               88              5.26   \n",
       "4             28.34              148.3              122             12.61   \n",
       "\n",
       "   total night minutes  total night calls  total night charge  \\\n",
       "0                244.7                 91               11.01   \n",
       "1                254.4                103               11.45   \n",
       "2                162.6                104                7.32   \n",
       "3                196.9                 89                8.86   \n",
       "4                186.9                121                8.41   \n",
       "\n",
       "   total intl minutes  total intl calls  total intl charge  \\\n",
       "0                10.0                 3               2.70   \n",
       "1                13.7                 3               3.70   \n",
       "2                12.2                 5               3.29   \n",
       "3                 6.6                 7               1.78   \n",
       "4                10.1                 3               2.73   \n",
       "\n",
       "   customer service calls  churn  \n",
       "0                       1  False  \n",
       "1                       1  False  \n",
       "2                       0  False  \n",
       "3                       2  False  \n",
       "4                       3  False  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('telecom_churn.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "a2f8fc5742d83ec5a110fa56fd967497013bec39"
   },
   "source": [
    "Recall that each row corresponds to one client, an **instance**, and columns are **features** of this instance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "d1a98d3e208e59ad0825687ba732f32714fd5e5c"
   },
   "source": [
    "Let’s have a look at data dimensionality, feature names, and feature types."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "_uuid": "85347476c8a704b43ddc6b5ec54ac0059b2b740a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3333, 21)\n"
     ]
    }
   ],
   "source": [
    "print(df.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "26e01159ec9dfc2634bbb3be4ccc3ec82c1b01e4"
   },
   "source": [
    "From the output, we can see that the table contains 3333 rows and 21 columns.\n",
    "\n",
    "Now let's try printing out column names using `columns`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "_uuid": "216df4e9a25d0baa8b506e1ec72df488d78367eb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['state', 'account length', 'area code', 'phone number', 'international plan', 'voice mail plan', 'number vmail messages', 'total day minutes', 'total day calls', 'total day charge', 'total eve minutes', 'total eve calls', 'total eve charge', 'total night minutes', 'total night calls', 'total night charge', 'total intl minutes', 'total intl calls', 'total intl charge', 'customer service calls', 'churn']\n"
     ]
    }
   ],
   "source": [
    "print(list(df.columns)) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "60f8c02d4e9b4ef82f9721fc2fb274ca50e0d6e4"
   },
   "source": [
    "We can use the `info()` method to output some general information about the dataframe. Note that the 'object' data type refers to categorical variables in this data set. '3333 non-null' means that all 3333 values of the corresponding variable are not missing.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "_uuid": "2bcfb060dcbda4fe1720eaf3c2e5c003a891dd77"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 3333 entries, 0 to 3332\n",
      "Data columns (total 21 columns):\n",
      " #   Column                  Non-Null Count  Dtype  \n",
      "---  ------                  --------------  -----  \n",
      " 0   state                   3333 non-null   object \n",
      " 1   account length          3333 non-null   int64  \n",
      " 2   area code               3333 non-null   int64  \n",
      " 3   phone number            3333 non-null   object \n",
      " 4   international plan      3333 non-null   object \n",
      " 5   voice mail plan         3333 non-null   object \n",
      " 6   number vmail messages   3333 non-null   int64  \n",
      " 7   total day minutes       3333 non-null   float64\n",
      " 8   total day calls         3333 non-null   int64  \n",
      " 9   total day charge        3333 non-null   float64\n",
      " 10  total eve minutes       3333 non-null   float64\n",
      " 11  total eve calls         3333 non-null   int64  \n",
      " 12  total eve charge        3333 non-null   float64\n",
      " 13  total night minutes     3333 non-null   float64\n",
      " 14  total night calls       3333 non-null   int64  \n",
      " 15  total night charge      3333 non-null   float64\n",
      " 16  total intl minutes      3333 non-null   float64\n",
      " 17  total intl calls        3333 non-null   int64  \n",
      " 18  total intl charge       3333 non-null   float64\n",
      " 19  customer service calls  3333 non-null   int64  \n",
      " 20  churn                   3333 non-null   bool   \n",
      "dtypes: bool(1), float64(8), int64(8), object(4)\n",
      "memory usage: 524.2+ KB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "print(df.info())   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "6fac47f7e7bcf060164b1881d8e199973834cd38"
   },
   "source": [
    "`bool`, `int64`, `float64` and `object` are the data types of our features. We see that one feature is logical (`bool`), 3 features are of type `object`, and 16 features are numeric. With this same method, we can easily see if there are any missing values. Here, there are none because each column contains 3333 observations, the same number of rows we saw before with `shape`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "11030a8bd020c6a3a04beb73531e9676160bcab8"
   },
   "source": [
    "The `describe` method shows basic statistical characteristics of each numerical feature (`int64` and `float64` types): number of non-missing values, mean, standard deviation, range, median, 0.25 and 0.75 quartiles."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "_uuid": "24e1d394f938980403a736421b4cf855ddb8ef8c"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>account length</th>\n",
       "      <th>area code</th>\n",
       "      <th>number vmail messages</th>\n",
       "      <th>total day minutes</th>\n",
       "      <th>total day calls</th>\n",
       "      <th>total day charge</th>\n",
       "      <th>total eve minutes</th>\n",
       "      <th>total eve calls</th>\n",
       "      <th>total eve charge</th>\n",
       "      <th>total night minutes</th>\n",
       "      <th>total night calls</th>\n",
       "      <th>total night charge</th>\n",
       "      <th>total intl minutes</th>\n",
       "      <th>total intl calls</th>\n",
       "      <th>total intl charge</th>\n",
       "      <th>customer service calls</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "      <td>3333.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>101.06</td>\n",
       "      <td>437.18</td>\n",
       "      <td>8.10</td>\n",
       "      <td>179.78</td>\n",
       "      <td>100.44</td>\n",
       "      <td>30.56</td>\n",
       "      <td>200.98</td>\n",
       "      <td>100.11</td>\n",
       "      <td>17.08</td>\n",
       "      <td>200.87</td>\n",
       "      <td>100.11</td>\n",
       "      <td>9.04</td>\n",
       "      <td>10.24</td>\n",
       "      <td>4.48</td>\n",
       "      <td>2.76</td>\n",
       "      <td>1.56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>39.82</td>\n",
       "      <td>42.37</td>\n",
       "      <td>13.69</td>\n",
       "      <td>54.47</td>\n",
       "      <td>20.07</td>\n",
       "      <td>9.26</td>\n",
       "      <td>50.71</td>\n",
       "      <td>19.92</td>\n",
       "      <td>4.31</td>\n",
       "      <td>50.57</td>\n",
       "      <td>19.57</td>\n",
       "      <td>2.28</td>\n",
       "      <td>2.79</td>\n",
       "      <td>2.46</td>\n",
       "      <td>0.75</td>\n",
       "      <td>1.32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1.00</td>\n",
       "      <td>408.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>23.20</td>\n",
       "      <td>33.00</td>\n",
       "      <td>1.04</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>74.00</td>\n",
       "      <td>408.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>143.70</td>\n",
       "      <td>87.00</td>\n",
       "      <td>24.43</td>\n",
       "      <td>166.60</td>\n",
       "      <td>87.00</td>\n",
       "      <td>14.16</td>\n",
       "      <td>167.00</td>\n",
       "      <td>87.00</td>\n",
       "      <td>7.52</td>\n",
       "      <td>8.50</td>\n",
       "      <td>3.00</td>\n",
       "      <td>2.30</td>\n",
       "      <td>1.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>101.00</td>\n",
       "      <td>415.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>179.40</td>\n",
       "      <td>101.00</td>\n",
       "      <td>30.50</td>\n",
       "      <td>201.40</td>\n",
       "      <td>100.00</td>\n",
       "      <td>17.12</td>\n",
       "      <td>201.20</td>\n",
       "      <td>100.00</td>\n",
       "      <td>9.05</td>\n",
       "      <td>10.30</td>\n",
       "      <td>4.00</td>\n",
       "      <td>2.78</td>\n",
       "      <td>1.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>127.00</td>\n",
       "      <td>510.00</td>\n",
       "      <td>20.00</td>\n",
       "      <td>216.40</td>\n",
       "      <td>114.00</td>\n",
       "      <td>36.79</td>\n",
       "      <td>235.30</td>\n",
       "      <td>114.00</td>\n",
       "      <td>20.00</td>\n",
       "      <td>235.30</td>\n",
       "      <td>113.00</td>\n",
       "      <td>10.59</td>\n",
       "      <td>12.10</td>\n",
       "      <td>6.00</td>\n",
       "      <td>3.27</td>\n",
       "      <td>2.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>243.00</td>\n",
       "      <td>510.00</td>\n",
       "      <td>51.00</td>\n",
       "      <td>350.80</td>\n",
       "      <td>165.00</td>\n",
       "      <td>59.64</td>\n",
       "      <td>363.70</td>\n",
       "      <td>170.00</td>\n",
       "      <td>30.91</td>\n",
       "      <td>395.00</td>\n",
       "      <td>175.00</td>\n",
       "      <td>17.77</td>\n",
       "      <td>20.00</td>\n",
       "      <td>20.00</td>\n",
       "      <td>5.40</td>\n",
       "      <td>9.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       account length  area code  number vmail messages  total day minutes  \\\n",
       "count         3333.00    3333.00                3333.00            3333.00   \n",
       "mean           101.06     437.18                   8.10             179.78   \n",
       "std             39.82      42.37                  13.69              54.47   \n",
       "min              1.00     408.00                   0.00               0.00   \n",
       "25%             74.00     408.00                   0.00             143.70   \n",
       "50%            101.00     415.00                   0.00             179.40   \n",
       "75%            127.00     510.00                  20.00             216.40   \n",
       "max            243.00     510.00                  51.00             350.80   \n",
       "\n",
       "       total day calls  total day charge  total eve minutes  total eve calls  \\\n",
       "count          3333.00           3333.00            3333.00          3333.00   \n",
       "mean            100.44             30.56             200.98           100.11   \n",
       "std              20.07              9.26              50.71            19.92   \n",
       "min               0.00              0.00               0.00             0.00   \n",
       "25%              87.00             24.43             166.60            87.00   \n",
       "50%             101.00             30.50             201.40           100.00   \n",
       "75%             114.00             36.79             235.30           114.00   \n",
       "max             165.00             59.64             363.70           170.00   \n",
       "\n",
       "       total eve charge  total night minutes  total night calls  \\\n",
       "count           3333.00              3333.00            3333.00   \n",
       "mean              17.08               200.87             100.11   \n",
       "std                4.31                50.57              19.57   \n",
       "min                0.00                23.20              33.00   \n",
       "25%               14.16               167.00              87.00   \n",
       "50%               17.12               201.20             100.00   \n",
       "75%               20.00               235.30             113.00   \n",
       "max               30.91               395.00             175.00   \n",
       "\n",
       "       total night charge  total intl minutes  total intl calls  \\\n",
       "count             3333.00             3333.00           3333.00   \n",
       "mean                 9.04               10.24              4.48   \n",
       "std                  2.28                2.79              2.46   \n",
       "min                  1.04                0.00              0.00   \n",
       "25%                  7.52                8.50              3.00   \n",
       "50%                  9.05               10.30              4.00   \n",
       "75%                 10.59               12.10              6.00   \n",
       "max                 17.77               20.00             20.00   \n",
       "\n",
       "       total intl charge  customer service calls  \n",
       "count            3333.00                 3333.00  \n",
       "mean                2.76                    1.56  \n",
       "std                 0.75                    1.32  \n",
       "min                 0.00                    0.00  \n",
       "25%                 2.30                    1.00  \n",
       "50%                 2.78                    1.00  \n",
       "75%                 3.27                    2.00  \n",
       "max                 5.40                    9.00  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "2314370435c6c44da57d4c97ad01ace128047fd5"
   },
   "source": [
    "In order to see statistics on non-numerical features, one has to explicitly indicate data types of interest in the `include` parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "_uuid": "c1bd67bc0b915dbac429152c649a7c118770d824"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>phone number</th>\n",
       "      <th>international plan</th>\n",
       "      <th>voice mail plan</th>\n",
       "      <th>churn</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>3333</td>\n",
       "      <td>3333</td>\n",
       "      <td>3333</td>\n",
       "      <td>3333</td>\n",
       "      <td>3333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>51</td>\n",
       "      <td>3333</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>WV</td>\n",
       "      <td>382-4657</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>106</td>\n",
       "      <td>1</td>\n",
       "      <td>3010</td>\n",
       "      <td>2411</td>\n",
       "      <td>2850</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       state phone number international plan voice mail plan  churn\n",
       "count   3333         3333               3333            3333   3333\n",
       "unique    51         3333                  2               2      2\n",
       "top       WV     382-4657                 no              no  False\n",
       "freq     106            1               3010            2411   2850"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe(include=['object', 'bool'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "7acac4f33f2229215aa7818b10f4dfb1fafd96af"
   },
   "source": [
    "For categorical (type `object`) and boolean (type `bool`) features we can use the `value_counts` method. Let's have a look at the distribution of `churn`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "_uuid": "eef4f419ac223f320d1841635ffbb816c807fe54"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False    2850\n",
       "True      483\n",
       "Name: churn, dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['churn'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "78e28e7dbdd84116fe5c30a56f6c649befadd832"
   },
   "source": [
    "2850 users out of 3333 are *loyal*; their `Churn` value is `0`. To calculate fractions, pass `normalize=True` to the `value_counts` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "_uuid": "b516f5e583ac75e0379048738386ece5b208f0e7"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False    0.86\n",
       "True     0.14\n",
       "Name: churn, dtype: float64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['churn'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "7f54ceacde8fe8b61fca39694651c82ba43a5b2c"
   },
   "source": [
    "\n",
    "### Sorting\n",
    "\n",
    "A DataFrame can be sorted by the value of one of the variables (i.e columns). For example, we can sort by *Total day charge* (use `ascending=False` to sort in descending order):\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "_uuid": "419200266ac1e7d0e96ba1a20d756c9d87eb2541"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>account length</th>\n",
       "      <th>area code</th>\n",
       "      <th>phone number</th>\n",
       "      <th>international plan</th>\n",
       "      <th>voice mail plan</th>\n",
       "      <th>number vmail messages</th>\n",
       "      <th>total day minutes</th>\n",
       "      <th>total day calls</th>\n",
       "      <th>total day charge</th>\n",
       "      <th>total eve minutes</th>\n",
       "      <th>total eve calls</th>\n",
       "      <th>total eve charge</th>\n",
       "      <th>total night minutes</th>\n",
       "      <th>total night calls</th>\n",
       "      <th>total night charge</th>\n",
       "      <th>total intl minutes</th>\n",
       "      <th>total intl calls</th>\n",
       "      <th>total intl charge</th>\n",
       "      <th>customer service calls</th>\n",
       "      <th>churn</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>365</th>\n",
       "      <td>CO</td>\n",
       "      <td>154</td>\n",
       "      <td>415</td>\n",
       "      <td>343-5709</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>350.8</td>\n",
       "      <td>75</td>\n",
       "      <td>59.64</td>\n",
       "      <td>216.5</td>\n",
       "      <td>94</td>\n",
       "      <td>18.40</td>\n",
       "      <td>253.9</td>\n",
       "      <td>100</td>\n",
       "      <td>11.43</td>\n",
       "      <td>10.1</td>\n",
       "      <td>9</td>\n",
       "      <td>2.73</td>\n",
       "      <td>1</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>985</th>\n",
       "      <td>NY</td>\n",
       "      <td>64</td>\n",
       "      <td>415</td>\n",
       "      <td>345-9140</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>346.8</td>\n",
       "      <td>55</td>\n",
       "      <td>58.96</td>\n",
       "      <td>249.5</td>\n",
       "      <td>79</td>\n",
       "      <td>21.21</td>\n",
       "      <td>275.4</td>\n",
       "      <td>102</td>\n",
       "      <td>12.39</td>\n",
       "      <td>13.3</td>\n",
       "      <td>9</td>\n",
       "      <td>3.59</td>\n",
       "      <td>1</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2594</th>\n",
       "      <td>OH</td>\n",
       "      <td>115</td>\n",
       "      <td>510</td>\n",
       "      <td>348-1163</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>345.3</td>\n",
       "      <td>81</td>\n",
       "      <td>58.70</td>\n",
       "      <td>203.4</td>\n",
       "      <td>106</td>\n",
       "      <td>17.29</td>\n",
       "      <td>217.5</td>\n",
       "      <td>107</td>\n",
       "      <td>9.79</td>\n",
       "      <td>11.8</td>\n",
       "      <td>8</td>\n",
       "      <td>3.19</td>\n",
       "      <td>1</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>156</th>\n",
       "      <td>OH</td>\n",
       "      <td>83</td>\n",
       "      <td>415</td>\n",
       "      <td>370-9116</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>337.4</td>\n",
       "      <td>120</td>\n",
       "      <td>57.36</td>\n",
       "      <td>227.4</td>\n",
       "      <td>116</td>\n",
       "      <td>19.33</td>\n",
       "      <td>153.9</td>\n",
       "      <td>114</td>\n",
       "      <td>6.93</td>\n",
       "      <td>15.8</td>\n",
       "      <td>7</td>\n",
       "      <td>4.27</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>605</th>\n",
       "      <td>MO</td>\n",
       "      <td>112</td>\n",
       "      <td>415</td>\n",
       "      <td>373-2053</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>335.5</td>\n",
       "      <td>77</td>\n",
       "      <td>57.04</td>\n",
       "      <td>212.5</td>\n",
       "      <td>109</td>\n",
       "      <td>18.06</td>\n",
       "      <td>265.0</td>\n",
       "      <td>132</td>\n",
       "      <td>11.93</td>\n",
       "      <td>12.7</td>\n",
       "      <td>8</td>\n",
       "      <td>3.43</td>\n",
       "      <td>2</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>975</th>\n",
       "      <td>DE</td>\n",
       "      <td>129</td>\n",
       "      <td>510</td>\n",
       "      <td>332-6181</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>334.3</td>\n",
       "      <td>118</td>\n",
       "      <td>56.83</td>\n",
       "      <td>192.1</td>\n",
       "      <td>104</td>\n",
       "      <td>16.33</td>\n",
       "      <td>191.0</td>\n",
       "      <td>83</td>\n",
       "      <td>8.59</td>\n",
       "      <td>10.4</td>\n",
       "      <td>6</td>\n",
       "      <td>2.81</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>NY</td>\n",
       "      <td>161</td>\n",
       "      <td>415</td>\n",
       "      <td>351-7269</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>332.9</td>\n",
       "      <td>67</td>\n",
       "      <td>56.59</td>\n",
       "      <td>317.8</td>\n",
       "      <td>97</td>\n",
       "      <td>27.01</td>\n",
       "      <td>160.6</td>\n",
       "      <td>128</td>\n",
       "      <td>7.23</td>\n",
       "      <td>5.4</td>\n",
       "      <td>9</td>\n",
       "      <td>1.46</td>\n",
       "      <td>4</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1679</th>\n",
       "      <td>OK</td>\n",
       "      <td>82</td>\n",
       "      <td>510</td>\n",
       "      <td>393-4823</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>329.8</td>\n",
       "      <td>73</td>\n",
       "      <td>56.07</td>\n",
       "      <td>208.3</td>\n",
       "      <td>120</td>\n",
       "      <td>17.71</td>\n",
       "      <td>267.1</td>\n",
       "      <td>102</td>\n",
       "      <td>12.02</td>\n",
       "      <td>10.6</td>\n",
       "      <td>6</td>\n",
       "      <td>2.86</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2028</th>\n",
       "      <td>SD</td>\n",
       "      <td>93</td>\n",
       "      <td>510</td>\n",
       "      <td>408-4836</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>328.1</td>\n",
       "      <td>106</td>\n",
       "      <td>55.78</td>\n",
       "      <td>151.7</td>\n",
       "      <td>89</td>\n",
       "      <td>12.89</td>\n",
       "      <td>303.5</td>\n",
       "      <td>114</td>\n",
       "      <td>13.66</td>\n",
       "      <td>8.7</td>\n",
       "      <td>3</td>\n",
       "      <td>2.35</td>\n",
       "      <td>1</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>197</th>\n",
       "      <td>TX</td>\n",
       "      <td>208</td>\n",
       "      <td>510</td>\n",
       "      <td>378-3625</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>326.5</td>\n",
       "      <td>67</td>\n",
       "      <td>55.51</td>\n",
       "      <td>176.3</td>\n",
       "      <td>113</td>\n",
       "      <td>14.99</td>\n",
       "      <td>181.7</td>\n",
       "      <td>102</td>\n",
       "      <td>8.18</td>\n",
       "      <td>10.7</td>\n",
       "      <td>6</td>\n",
       "      <td>2.89</td>\n",
       "      <td>2</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     state  account length  area code phone number international plan  \\\n",
       "365     CO             154        415     343-5709                 no   \n",
       "985     NY              64        415     345-9140                yes   \n",
       "2594    OH             115        510     348-1163                yes   \n",
       "156     OH              83        415     370-9116                 no   \n",
       "605     MO             112        415     373-2053                 no   \n",
       "975     DE             129        510     332-6181                 no   \n",
       "15      NY             161        415     351-7269                 no   \n",
       "1679    OK              82        510     393-4823                 no   \n",
       "2028    SD              93        510     408-4836                 no   \n",
       "197     TX             208        510     378-3625                 no   \n",
       "\n",
       "     voice mail plan  number vmail messages  total day minutes  \\\n",
       "365               no                      0              350.8   \n",
       "985               no                      0              346.8   \n",
       "2594              no                      0              345.3   \n",
       "156               no                      0              337.4   \n",
       "605               no                      0              335.5   \n",
       "975               no                      0              334.3   \n",
       "15                no                      0              332.9   \n",
       "1679              no                      0              329.8   \n",
       "2028              no                      0              328.1   \n",
       "197               no                      0              326.5   \n",
       "\n",
       "      total day calls  total day charge  total eve minutes  total eve calls  \\\n",
       "365                75             59.64              216.5               94   \n",
       "985                55             58.96              249.5               79   \n",
       "2594               81             58.70              203.4              106   \n",
       "156               120             57.36              227.4              116   \n",
       "605                77             57.04              212.5              109   \n",
       "975               118             56.83              192.1              104   \n",
       "15                 67             56.59              317.8               97   \n",
       "1679               73             56.07              208.3              120   \n",
       "2028              106             55.78              151.7               89   \n",
       "197                67             55.51              176.3              113   \n",
       "\n",
       "      total eve charge  total night minutes  total night calls  \\\n",
       "365              18.40                253.9                100   \n",
       "985              21.21                275.4                102   \n",
       "2594             17.29                217.5                107   \n",
       "156              19.33                153.9                114   \n",
       "605              18.06                265.0                132   \n",
       "975              16.33                191.0                 83   \n",
       "15               27.01                160.6                128   \n",
       "1679             17.71                267.1                102   \n",
       "2028             12.89                303.5                114   \n",
       "197              14.99                181.7                102   \n",
       "\n",
       "      total night charge  total intl minutes  total intl calls  \\\n",
       "365                11.43                10.1                 9   \n",
       "985                12.39                13.3                 9   \n",
       "2594                9.79                11.8                 8   \n",
       "156                 6.93                15.8                 7   \n",
       "605                11.93                12.7                 8   \n",
       "975                 8.59                10.4                 6   \n",
       "15                  7.23                 5.4                 9   \n",
       "1679               12.02                10.6                 6   \n",
       "2028               13.66                 8.7                 3   \n",
       "197                 8.18                10.7                 6   \n",
       "\n",
       "      total intl charge  customer service calls  churn  \n",
       "365                2.73                       1   True  \n",
       "985                3.59                       1   True  \n",
       "2594               3.19                       1   True  \n",
       "156                4.27                       0   True  \n",
       "605                3.43                       2   True  \n",
       "975                2.81                       0   True  \n",
       "15                 1.46                       4   True  \n",
       "1679               2.86                       0   True  \n",
       "2028               2.35                       1   True  \n",
       "197                2.89                       2   True  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sort_values(by='total day charge', ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sorting Total day charge in ascending order"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>account length</th>\n",
       "      <th>area code</th>\n",
       "      <th>phone number</th>\n",
       "      <th>international plan</th>\n",
       "      <th>voice mail plan</th>\n",
       "      <th>number vmail messages</th>\n",
       "      <th>total day minutes</th>\n",
       "      <th>total day calls</th>\n",
       "      <th>total day charge</th>\n",
       "      <th>total eve minutes</th>\n",
       "      <th>total eve calls</th>\n",
       "      <th>total eve charge</th>\n",
       "      <th>total night minutes</th>\n",
       "      <th>total night calls</th>\n",
       "      <th>total night charge</th>\n",
       "      <th>total intl minutes</th>\n",
       "      <th>total intl calls</th>\n",
       "      <th>total intl charge</th>\n",
       "      <th>customer service calls</th>\n",
       "      <th>churn</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1345</th>\n",
       "      <td>SD</td>\n",
       "      <td>98</td>\n",
       "      <td>415</td>\n",
       "      <td>392-2555</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>159.6</td>\n",
       "      <td>130</td>\n",
       "      <td>13.57</td>\n",
       "      <td>167.1</td>\n",
       "      <td>88</td>\n",
       "      <td>7.52</td>\n",
       "      <td>6.8</td>\n",
       "      <td>1</td>\n",
       "      <td>1.84</td>\n",
       "      <td>4</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1397</th>\n",
       "      <td>VT</td>\n",
       "      <td>101</td>\n",
       "      <td>510</td>\n",
       "      <td>413-7655</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>192.1</td>\n",
       "      <td>119</td>\n",
       "      <td>16.33</td>\n",
       "      <td>168.8</td>\n",
       "      <td>95</td>\n",
       "      <td>7.60</td>\n",
       "      <td>7.2</td>\n",
       "      <td>4</td>\n",
       "      <td>1.94</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2736</th>\n",
       "      <td>OK</td>\n",
       "      <td>127</td>\n",
       "      <td>510</td>\n",
       "      <td>403-1128</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>27</td>\n",
       "      <td>2.6</td>\n",
       "      <td>113</td>\n",
       "      <td>0.44</td>\n",
       "      <td>254.0</td>\n",
       "      <td>102</td>\n",
       "      <td>21.59</td>\n",
       "      <td>242.7</td>\n",
       "      <td>156</td>\n",
       "      <td>10.92</td>\n",
       "      <td>9.2</td>\n",
       "      <td>5</td>\n",
       "      <td>2.48</td>\n",
       "      <td>3</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2753</th>\n",
       "      <td>OH</td>\n",
       "      <td>134</td>\n",
       "      <td>415</td>\n",
       "      <td>406-4158</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>7.8</td>\n",
       "      <td>86</td>\n",
       "      <td>1.33</td>\n",
       "      <td>171.4</td>\n",
       "      <td>100</td>\n",
       "      <td>14.57</td>\n",
       "      <td>186.5</td>\n",
       "      <td>80</td>\n",
       "      <td>8.39</td>\n",
       "      <td>12.9</td>\n",
       "      <td>2</td>\n",
       "      <td>3.48</td>\n",
       "      <td>2</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1986</th>\n",
       "      <td>WI</td>\n",
       "      <td>70</td>\n",
       "      <td>415</td>\n",
       "      <td>405-9233</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9</td>\n",
       "      <td>100</td>\n",
       "      <td>1.34</td>\n",
       "      <td>136.4</td>\n",
       "      <td>83</td>\n",
       "      <td>11.59</td>\n",
       "      <td>156.6</td>\n",
       "      <td>89</td>\n",
       "      <td>7.05</td>\n",
       "      <td>12.1</td>\n",
       "      <td>1</td>\n",
       "      <td>3.27</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1052</th>\n",
       "      <td>OR</td>\n",
       "      <td>98</td>\n",
       "      <td>415</td>\n",
       "      <td>378-6772</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>12.5</td>\n",
       "      <td>67</td>\n",
       "      <td>2.13</td>\n",
       "      <td>256.6</td>\n",
       "      <td>90</td>\n",
       "      <td>21.81</td>\n",
       "      <td>169.4</td>\n",
       "      <td>88</td>\n",
       "      <td>7.62</td>\n",
       "      <td>7.7</td>\n",
       "      <td>9</td>\n",
       "      <td>2.08</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2252</th>\n",
       "      <td>NH</td>\n",
       "      <td>148</td>\n",
       "      <td>408</td>\n",
       "      <td>333-7449</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>17.6</td>\n",
       "      <td>121</td>\n",
       "      <td>2.99</td>\n",
       "      <td>161.7</td>\n",
       "      <td>125</td>\n",
       "      <td>13.74</td>\n",
       "      <td>203.1</td>\n",
       "      <td>82</td>\n",
       "      <td>9.14</td>\n",
       "      <td>10.6</td>\n",
       "      <td>6</td>\n",
       "      <td>2.86</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3046</th>\n",
       "      <td>MI</td>\n",
       "      <td>110</td>\n",
       "      <td>510</td>\n",
       "      <td>357-5784</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>18.9</td>\n",
       "      <td>92</td>\n",
       "      <td>3.21</td>\n",
       "      <td>258.4</td>\n",
       "      <td>81</td>\n",
       "      <td>21.96</td>\n",
       "      <td>109.6</td>\n",
       "      <td>74</td>\n",
       "      <td>4.93</td>\n",
       "      <td>14.8</td>\n",
       "      <td>4</td>\n",
       "      <td>4.00</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1621</th>\n",
       "      <td>SC</td>\n",
       "      <td>138</td>\n",
       "      <td>510</td>\n",
       "      <td>370-9533</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>21</td>\n",
       "      <td>19.5</td>\n",
       "      <td>149</td>\n",
       "      <td>3.32</td>\n",
       "      <td>140.9</td>\n",
       "      <td>109</td>\n",
       "      <td>11.98</td>\n",
       "      <td>179.7</td>\n",
       "      <td>111</td>\n",
       "      <td>8.09</td>\n",
       "      <td>7.9</td>\n",
       "      <td>1</td>\n",
       "      <td>2.13</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1076</th>\n",
       "      <td>WY</td>\n",
       "      <td>53</td>\n",
       "      <td>415</td>\n",
       "      <td>337-4339</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>27</td>\n",
       "      <td>25.9</td>\n",
       "      <td>119</td>\n",
       "      <td>4.40</td>\n",
       "      <td>206.5</td>\n",
       "      <td>96</td>\n",
       "      <td>17.55</td>\n",
       "      <td>228.1</td>\n",
       "      <td>64</td>\n",
       "      <td>10.26</td>\n",
       "      <td>6.5</td>\n",
       "      <td>7</td>\n",
       "      <td>1.76</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     state  account length  area code phone number international plan  \\\n",
       "1345    SD              98        415     392-2555                 no   \n",
       "1397    VT             101        510     413-7655                 no   \n",
       "2736    OK             127        510     403-1128                 no   \n",
       "2753    OH             134        415     406-4158                 no   \n",
       "1986    WI              70        415     405-9233                 no   \n",
       "1052    OR              98        415     378-6772                yes   \n",
       "2252    NH             148        408     333-7449                 no   \n",
       "3046    MI             110        510     357-5784                 no   \n",
       "1621    SC             138        510     370-9533                 no   \n",
       "1076    WY              53        415     337-4339                 no   \n",
       "\n",
       "     voice mail plan  number vmail messages  total day minutes  \\\n",
       "1345              no                      0                0.0   \n",
       "1397              no                      0                0.0   \n",
       "2736             yes                     27                2.6   \n",
       "2753              no                      0                7.8   \n",
       "1986              no                      0                7.9   \n",
       "1052              no                      0               12.5   \n",
       "2252              no                      0               17.6   \n",
       "3046              no                      0               18.9   \n",
       "1621             yes                     21               19.5   \n",
       "1076             yes                     27               25.9   \n",
       "\n",
       "      total day calls  total day charge  total eve minutes  total eve calls  \\\n",
       "1345                0              0.00              159.6              130   \n",
       "1397                0              0.00              192.1              119   \n",
       "2736              113              0.44              254.0              102   \n",
       "2753               86              1.33              171.4              100   \n",
       "1986              100              1.34              136.4               83   \n",
       "1052               67              2.13              256.6               90   \n",
       "2252              121              2.99              161.7              125   \n",
       "3046               92              3.21              258.4               81   \n",
       "1621              149              3.32              140.9              109   \n",
       "1076              119              4.40              206.5               96   \n",
       "\n",
       "      total eve charge  total night minutes  total night calls  \\\n",
       "1345             13.57                167.1                 88   \n",
       "1397             16.33                168.8                 95   \n",
       "2736             21.59                242.7                156   \n",
       "2753             14.57                186.5                 80   \n",
       "1986             11.59                156.6                 89   \n",
       "1052             21.81                169.4                 88   \n",
       "2252             13.74                203.1                 82   \n",
       "3046             21.96                109.6                 74   \n",
       "1621             11.98                179.7                111   \n",
       "1076             17.55                228.1                 64   \n",
       "\n",
       "      total night charge  total intl minutes  total intl calls  \\\n",
       "1345                7.52                 6.8                 1   \n",
       "1397                7.60                 7.2                 4   \n",
       "2736               10.92                 9.2                 5   \n",
       "2753                8.39                12.9                 2   \n",
       "1986                7.05                12.1                 1   \n",
       "1052                7.62                 7.7                 9   \n",
       "2252                9.14                10.6                 6   \n",
       "3046                4.93                14.8                 4   \n",
       "1621                8.09                 7.9                 1   \n",
       "1076               10.26                 6.5                 7   \n",
       "\n",
       "      total intl charge  customer service calls  churn  \n",
       "1345               1.84                       4   True  \n",
       "1397               1.94                       1  False  \n",
       "2736               2.48                       3  False  \n",
       "2753               3.48                       2  False  \n",
       "1986               3.27                       0  False  \n",
       "1052               2.08                       1  False  \n",
       "2252               2.86                       1  False  \n",
       "3046               4.00                       1  False  \n",
       "1621               2.13                       0  False  \n",
       "1076               1.76                       1  False  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sort_values(by='total day charge').head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ff41ebcef7b883413530260388df7e23996862ba"
   },
   "source": [
    "### Indexing and retrieving data\n",
    "\n",
    "A DataFrame can be indexed in a few different ways. \n",
    "\n",
    "To get a single column/feature, you can use a `DataFrame['Name']` construction. Let's use this to answer a question about that column alone: **what is the proportion of churned users in our dataframe?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "_uuid": "b020796a83bc9e10b1bc8e9c6439fc006302d999"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.14491449144914492"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['churn'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "8abe3b0982e4599e775f08d8f2feab2b8a702a7b"
   },
   "source": [
    "14.5% is actually quite bad for a company; such a churn rate can make the company go bankrupt.\n",
    "\n",
    "**Boolean indexing** with one column is also very convenient. The syntax is `df[df['Name'] == P]`, where `P` is some logical condition that is checked for each element of the `Name` column. The result of such indexing is the DataFrame consisting only of rows that satisfy the `P` condition on the `Name` column. \n",
    "\n",
    "Lets use it to see all customers who have churned out of the company **('churn' == 1)**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>account length</th>\n",
       "      <th>area code</th>\n",
       "      <th>phone number</th>\n",
       "      <th>international plan</th>\n",
       "      <th>voice mail plan</th>\n",
       "      <th>number vmail messages</th>\n",
       "      <th>total day minutes</th>\n",
       "      <th>total day calls</th>\n",
       "      <th>total day charge</th>\n",
       "      <th>total eve minutes</th>\n",
       "      <th>total eve calls</th>\n",
       "      <th>total eve charge</th>\n",
       "      <th>total night minutes</th>\n",
       "      <th>total night calls</th>\n",
       "      <th>total night charge</th>\n",
       "      <th>total intl minutes</th>\n",
       "      <th>total intl calls</th>\n",
       "      <th>total intl charge</th>\n",
       "      <th>customer service calls</th>\n",
       "      <th>churn</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>IN</td>\n",
       "      <td>65</td>\n",
       "      <td>415</td>\n",
       "      <td>329-6603</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>129.1</td>\n",
       "      <td>137</td>\n",
       "      <td>21.95</td>\n",
       "      <td>228.5</td>\n",
       "      <td>83</td>\n",
       "      <td>19.42</td>\n",
       "      <td>208.8</td>\n",
       "      <td>111</td>\n",
       "      <td>9.40</td>\n",
       "      <td>12.7</td>\n",
       "      <td>6</td>\n",
       "      <td>3.43</td>\n",
       "      <td>4</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>NY</td>\n",
       "      <td>161</td>\n",
       "      <td>415</td>\n",
       "      <td>351-7269</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>332.9</td>\n",
       "      <td>67</td>\n",
       "      <td>56.59</td>\n",
       "      <td>317.8</td>\n",
       "      <td>97</td>\n",
       "      <td>27.01</td>\n",
       "      <td>160.6</td>\n",
       "      <td>128</td>\n",
       "      <td>7.23</td>\n",
       "      <td>5.4</td>\n",
       "      <td>9</td>\n",
       "      <td>1.46</td>\n",
       "      <td>4</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>CO</td>\n",
       "      <td>77</td>\n",
       "      <td>408</td>\n",
       "      <td>393-7984</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>62.4</td>\n",
       "      <td>89</td>\n",
       "      <td>10.61</td>\n",
       "      <td>169.9</td>\n",
       "      <td>121</td>\n",
       "      <td>14.44</td>\n",
       "      <td>209.6</td>\n",
       "      <td>64</td>\n",
       "      <td>9.43</td>\n",
       "      <td>5.7</td>\n",
       "      <td>6</td>\n",
       "      <td>1.54</td>\n",
       "      <td>5</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>AZ</td>\n",
       "      <td>12</td>\n",
       "      <td>408</td>\n",
       "      <td>360-1596</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>249.6</td>\n",
       "      <td>118</td>\n",
       "      <td>42.43</td>\n",
       "      <td>252.4</td>\n",
       "      <td>119</td>\n",
       "      <td>21.45</td>\n",
       "      <td>280.2</td>\n",
       "      <td>90</td>\n",
       "      <td>12.61</td>\n",
       "      <td>11.8</td>\n",
       "      <td>3</td>\n",
       "      <td>3.19</td>\n",
       "      <td>1</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>MD</td>\n",
       "      <td>135</td>\n",
       "      <td>408</td>\n",
       "      <td>383-6029</td>\n",
       "      <td>yes</td>\n",
       "      <td>yes</td>\n",
       "      <td>41</td>\n",
       "      <td>173.1</td>\n",
       "      <td>85</td>\n",
       "      <td>29.43</td>\n",
       "      <td>203.9</td>\n",
       "      <td>107</td>\n",
       "      <td>17.33</td>\n",
       "      <td>122.2</td>\n",
       "      <td>78</td>\n",
       "      <td>5.50</td>\n",
       "      <td>14.6</td>\n",
       "      <td>15</td>\n",
       "      <td>3.94</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3301</th>\n",
       "      <td>CA</td>\n",
       "      <td>84</td>\n",
       "      <td>415</td>\n",
       "      <td>417-1488</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>113</td>\n",
       "      <td>47.60</td>\n",
       "      <td>202.2</td>\n",
       "      <td>90</td>\n",
       "      <td>17.19</td>\n",
       "      <td>156.8</td>\n",
       "      <td>103</td>\n",
       "      <td>7.06</td>\n",
       "      <td>10.4</td>\n",
       "      <td>4</td>\n",
       "      <td>2.81</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3304</th>\n",
       "      <td>IL</td>\n",
       "      <td>71</td>\n",
       "      <td>510</td>\n",
       "      <td>330-7137</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>186.1</td>\n",
       "      <td>114</td>\n",
       "      <td>31.64</td>\n",
       "      <td>198.6</td>\n",
       "      <td>140</td>\n",
       "      <td>16.88</td>\n",
       "      <td>206.5</td>\n",
       "      <td>80</td>\n",
       "      <td>9.29</td>\n",
       "      <td>13.8</td>\n",
       "      <td>5</td>\n",
       "      <td>3.73</td>\n",
       "      <td>4</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3320</th>\n",
       "      <td>GA</td>\n",
       "      <td>122</td>\n",
       "      <td>510</td>\n",
       "      <td>411-5677</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>140.0</td>\n",
       "      <td>101</td>\n",
       "      <td>23.80</td>\n",
       "      <td>196.4</td>\n",
       "      <td>77</td>\n",
       "      <td>16.69</td>\n",
       "      <td>120.1</td>\n",
       "      <td>133</td>\n",
       "      <td>5.40</td>\n",
       "      <td>9.7</td>\n",
       "      <td>4</td>\n",
       "      <td>2.62</td>\n",
       "      <td>4</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3322</th>\n",
       "      <td>MD</td>\n",
       "      <td>62</td>\n",
       "      <td>408</td>\n",
       "      <td>409-1856</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>321.1</td>\n",
       "      <td>105</td>\n",
       "      <td>54.59</td>\n",
       "      <td>265.5</td>\n",
       "      <td>122</td>\n",
       "      <td>22.57</td>\n",
       "      <td>180.5</td>\n",
       "      <td>72</td>\n",
       "      <td>8.12</td>\n",
       "      <td>11.5</td>\n",
       "      <td>2</td>\n",
       "      <td>3.11</td>\n",
       "      <td>4</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3323</th>\n",
       "      <td>IN</td>\n",
       "      <td>117</td>\n",
       "      <td>415</td>\n",
       "      <td>362-5899</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>118.4</td>\n",
       "      <td>126</td>\n",
       "      <td>20.13</td>\n",
       "      <td>249.3</td>\n",
       "      <td>97</td>\n",
       "      <td>21.19</td>\n",
       "      <td>227.0</td>\n",
       "      <td>56</td>\n",
       "      <td>10.22</td>\n",
       "      <td>13.6</td>\n",
       "      <td>3</td>\n",
       "      <td>3.67</td>\n",
       "      <td>5</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>483 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     state  account length  area code phone number international plan  \\\n",
       "10      IN              65        415     329-6603                 no   \n",
       "15      NY             161        415     351-7269                 no   \n",
       "21      CO              77        408     393-7984                 no   \n",
       "33      AZ              12        408     360-1596                 no   \n",
       "41      MD             135        408     383-6029                yes   \n",
       "...    ...             ...        ...          ...                ...   \n",
       "3301    CA              84        415     417-1488                 no   \n",
       "3304    IL              71        510     330-7137                yes   \n",
       "3320    GA             122        510     411-5677                yes   \n",
       "3322    MD              62        408     409-1856                 no   \n",
       "3323    IN             117        415     362-5899                 no   \n",
       "\n",
       "     voice mail plan  number vmail messages  total day minutes  \\\n",
       "10                no                      0              129.1   \n",
       "15                no                      0              332.9   \n",
       "21                no                      0               62.4   \n",
       "33                no                      0              249.6   \n",
       "41               yes                     41              173.1   \n",
       "...              ...                    ...                ...   \n",
       "3301              no                      0              280.0   \n",
       "3304              no                      0              186.1   \n",
       "3320              no                      0              140.0   \n",
       "3322              no                      0              321.1   \n",
       "3323              no                      0              118.4   \n",
       "\n",
       "      total day calls  total day charge  total eve minutes  total eve calls  \\\n",
       "10                137             21.95              228.5               83   \n",
       "15                 67             56.59              317.8               97   \n",
       "21                 89             10.61              169.9              121   \n",
       "33                118             42.43              252.4              119   \n",
       "41                 85             29.43              203.9              107   \n",
       "...               ...               ...                ...              ...   \n",
       "3301              113             47.60              202.2               90   \n",
       "3304              114             31.64              198.6              140   \n",
       "3320              101             23.80              196.4               77   \n",
       "3322              105             54.59              265.5              122   \n",
       "3323              126             20.13              249.3               97   \n",
       "\n",
       "      total eve charge  total night minutes  total night calls  \\\n",
       "10               19.42                208.8                111   \n",
       "15               27.01                160.6                128   \n",
       "21               14.44                209.6                 64   \n",
       "33               21.45                280.2                 90   \n",
       "41               17.33                122.2                 78   \n",
       "...                ...                  ...                ...   \n",
       "3301             17.19                156.8                103   \n",
       "3304             16.88                206.5                 80   \n",
       "3320             16.69                120.1                133   \n",
       "3322             22.57                180.5                 72   \n",
       "3323             21.19                227.0                 56   \n",
       "\n",
       "      total night charge  total intl minutes  total intl calls  \\\n",
       "10                  9.40                12.7                 6   \n",
       "15                  7.23                 5.4                 9   \n",
       "21                  9.43                 5.7                 6   \n",
       "33                 12.61                11.8                 3   \n",
       "41                  5.50                14.6                15   \n",
       "...                  ...                 ...               ...   \n",
       "3301                7.06                10.4                 4   \n",
       "3304                9.29                13.8                 5   \n",
       "3320                5.40                 9.7                 4   \n",
       "3322                8.12                11.5                 2   \n",
       "3323               10.22                13.6                 3   \n",
       "\n",
       "      total intl charge  customer service calls  churn  \n",
       "10                 3.43                       4   True  \n",
       "15                 1.46                       4   True  \n",
       "21                 1.54                       5   True  \n",
       "33                 3.19                       1   True  \n",
       "41                 3.94                       0   True  \n",
       "...                 ...                     ...    ...  \n",
       "3301               2.81                       0   True  \n",
       "3304               3.73                       4   True  \n",
       "3320               2.62                       4   True  \n",
       "3322               3.11                       4   True  \n",
       "3323               3.67                       5   True  \n",
       "\n",
       "[483 rows x 21 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df['churn'] == True]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's answer the question:\n",
    "\n",
    "**What are average values of numerical features for churned users?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "_uuid": "3712d185e50d44c65128468c603b120b75786c84"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "account length            102.66\n",
       "area code                 437.82\n",
       "number vmail messages       5.12\n",
       "total day minutes         206.91\n",
       "total day calls           101.34\n",
       "total day charge           35.18\n",
       "total eve minutes         212.41\n",
       "total eve calls           100.56\n",
       "total eve charge           18.05\n",
       "total night minutes       205.23\n",
       "total night calls         100.40\n",
       "total night charge          9.24\n",
       "total intl minutes         10.70\n",
       "total intl calls            4.16\n",
       "total intl charge           2.89\n",
       "customer service calls      2.23\n",
       "churn                       1.00\n",
       "dtype: float64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df['churn'] == True].mean(numeric_only=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What are average values of numerical features for non-churned users?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "account length            100.79\n",
       "area code                 437.07\n",
       "number vmail messages       8.60\n",
       "total day minutes         175.18\n",
       "total day calls           100.28\n",
       "total day charge           29.78\n",
       "total eve minutes         199.04\n",
       "total eve calls           100.04\n",
       "total eve charge           16.92\n",
       "total night minutes       200.13\n",
       "total night calls         100.06\n",
       "total night charge          9.01\n",
       "total intl minutes         10.16\n",
       "total intl calls            4.53\n",
       "total intl charge           2.74\n",
       "customer service calls      1.45\n",
       "churn                       0.00\n",
       "dtype: float64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df['churn'] == False].mean(numeric_only=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "126e527e04c244c71edd9eac3abcc4a0a8bcd2cb"
   },
   "source": [
    "**How much time (on average) do churned users spend on the phone during daytime?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "_uuid": "314dfa8e0aeaa0a1d333e32234bd9b3e0a001348"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "206.91407867494814"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[df['churn'] == True]['total day minutes'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ef0358493f1731814d1bb34759ec6a0fad822487"
   },
   "source": [
    "\n",
    "**What is the maximum length of international calls among loyal users (`churn == 0`) who do not have an international plan?**\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "_uuid": "41874ec9bde4b2474a0bb9faf93795b2ae6cdf24"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "18.9"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[(df['churn'] == False) & (df['international plan'] == 'no')]['total intl minutes'].max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "22f7916fb05500116fdd6e52b86bf7949912bd9c"
   },
   "source": [
    "\n",
    "### Grouping\n",
    "\n",
    "In general, grouping data in Pandas works as follows:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "850ace3defaca6b72ef976c86a5020c4ed8f65b3"
   },
   "source": [
    "\n",
    "```python\n",
    "df.groupby(by=grouping_columns)[columns_to_show].function()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "1ac45d322f049449259ff21197e9977e220e82d1"
   },
   "source": [
    "\n",
    "1. First, the `groupby` method divides the `grouping_columns` by their values. They become a new index in the resulting dataframe.\n",
    "2. Then, columns of interest are selected (`columns_to_show`). If `columns_to_show` is not included, all non groupby columns will be included.\n",
    "3. Finally, one or several functions are applied to the obtained groups per selected columns.\n",
    "\n",
    "Here is an example where we group the data according to the values of the `Churn` variable and display statistics of three columns in each group:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "_uuid": "180e6b3d26c8adc52e46d8ab82dad34f9a49e234"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"6\" halign=\"left\">total day minutes</th>\n",
       "      <th colspan=\"6\" halign=\"left\">total eve minutes</th>\n",
       "      <th colspan=\"6\" halign=\"left\">total night minutes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>50%</th>\n",
       "      <th>max</th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>50%</th>\n",
       "      <th>max</th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>50%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>churn</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>False</th>\n",
       "      <td>2850.0</td>\n",
       "      <td>175.18</td>\n",
       "      <td>50.18</td>\n",
       "      <td>0.0</td>\n",
       "      <td>177.2</td>\n",
       "      <td>315.6</td>\n",
       "      <td>2850.0</td>\n",
       "      <td>199.04</td>\n",
       "      <td>50.29</td>\n",
       "      <td>0.0</td>\n",
       "      <td>199.6</td>\n",
       "      <td>361.8</td>\n",
       "      <td>2850.0</td>\n",
       "      <td>200.13</td>\n",
       "      <td>51.11</td>\n",
       "      <td>23.2</td>\n",
       "      <td>200.25</td>\n",
       "      <td>395.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>True</th>\n",
       "      <td>483.0</td>\n",
       "      <td>206.91</td>\n",
       "      <td>69.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>217.6</td>\n",
       "      <td>350.8</td>\n",
       "      <td>483.0</td>\n",
       "      <td>212.41</td>\n",
       "      <td>51.73</td>\n",
       "      <td>70.9</td>\n",
       "      <td>211.3</td>\n",
       "      <td>363.7</td>\n",
       "      <td>483.0</td>\n",
       "      <td>205.23</td>\n",
       "      <td>47.13</td>\n",
       "      <td>47.4</td>\n",
       "      <td>204.80</td>\n",
       "      <td>354.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      total day minutes                                   total eve minutes  \\\n",
       "                  count    mean    std  min    50%    max             count   \n",
       "churn                                                                         \n",
       "False            2850.0  175.18  50.18  0.0  177.2  315.6            2850.0   \n",
       "True              483.0  206.91  69.00  0.0  217.6  350.8             483.0   \n",
       "\n",
       "                                         total night minutes                 \\\n",
       "         mean    std   min    50%    max               count    mean    std   \n",
       "churn                                                                         \n",
       "False  199.04  50.29   0.0  199.6  361.8              2850.0  200.13  51.11   \n",
       "True   212.41  51.73  70.9  211.3  363.7               483.0  205.23  47.13   \n",
       "\n",
       "                            \n",
       "        min     50%    max  \n",
       "churn                       \n",
       "False  23.2  200.25  395.0  \n",
       "True   47.4  204.80  354.9  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "columns_to_show = ['total day minutes', 'total eve minutes', 'total night minutes']\n",
    "\n",
    "df.groupby(['churn'])[columns_to_show].describe(percentiles=[])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "221a34086734d3b52deccfda2db2edfb6f1b92a5"
   },
   "source": [
    "\n",
    "### Summary tables\n",
    "\n",
    "Suppose we want to see how the observations in our sample are distributed in the context of two variables - `churn` and `international plan`. To do so, we can build a **contingency table** using the `crosstab` method:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "_uuid": "ebcff1fdb2e6805fa186a0f5c27b46655222e9c2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "churn               False  True\n",
      "international plan             \n",
      "no                   2664   346\n",
      "yes                   186   137\n",
      "precision= 0.8850498338870432\n",
      "recall= 0.9347368421052632\n",
      "accuracy= 0.8403840384038403\n"
     ]
    }
   ],
   "source": [
    "crss_arr = pd.crosstab(df['international plan'], df['churn'])\n",
    "print(crss_arr)\n",
    "crss_arr = crss_arr.values\n",
    "precision = crss_arr[0,0]/np.sum(crss_arr[0,:])\n",
    "recall = crss_arr[0,0]/np.sum(crss_arr[:,0])\n",
    "accuracy = np.sum(crss_arr[0,0]+crss_arr[1,1])/np.sum(crss_arr)\n",
    "\n",
    "print(\"precision=\",precision)\n",
    "print(\"recall=\",recall)\n",
    "print(\"accuracy=\",accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "_uuid": "ebcff1fdb2e6805fa186a0f5c27b46655222e9c2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.4241486068111455\n",
      "0.5758513931888545\n",
      "0.11495016611295682\n",
      "0.8850498338870432\n",
      "--------\n",
      "0.2836438923395445 , 0.7163561076604554\n",
      "0.06526315789473684 , 0.9347368421052632\n"
     ]
    }
   ],
   "source": [
    "print(crss_arr[1,1]/np.sum(crss_arr[1,:]))\n",
    "print(crss_arr[1,0]/np.sum(crss_arr[1,:]))\n",
    "print(crss_arr[0,1]/np.sum(crss_arr[0,:]))\n",
    "print(crss_arr[0,0]/np.sum(crss_arr[0,:]))\n",
    "print(\"--------\")\n",
    "print(crss_arr[1,1]/np.sum(crss_arr[:,1]),\",\",crss_arr[0,1]/np.sum(crss_arr[:,1]))\n",
    "print(crss_arr[1,0]/np.sum(crss_arr[:,0]),\",\",crss_arr[0,0]/np.sum(crss_arr[:,0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "_uuid": "b2228aa42dd23fb1c70525f4ab105c5e2ee6196d",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>voice mail plan</th>\n",
       "      <th>no</th>\n",
       "      <th>yes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>churn</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>False</th>\n",
       "      <td>2008</td>\n",
       "      <td>842</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>True</th>\n",
       "      <td>403</td>\n",
       "      <td>80</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "voice mail plan    no  yes\n",
       "churn                     \n",
       "False            2008  842\n",
       "True              403   80"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.crosstab(df['churn'], df['voice mail plan'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "ea35f7ad664ff32785231dcae2292e9a66b28b90"
   },
   "source": [
    "\n",
    "### DataFrame transformations\n",
    "\n",
    "Like many other things in Pandas, adding columns to a DataFrame can be performed in many ways.\n",
    "\n",
    "For example, if we want to calculate the total charge of calls for all users, let's create the `total charge` Series and paste it into the DataFrame:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "244a645e9676b8bd2546bc84926cb9e43124e4c2"
   },
   "source": [
    "Notice the last column \"total charge,\" which we have added in our data frame "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "_uuid": "c5ec80ecb3b2e917834ccfb6cb0a76aced20315f"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>account length</th>\n",
       "      <th>area code</th>\n",
       "      <th>phone number</th>\n",
       "      <th>international plan</th>\n",
       "      <th>voice mail plan</th>\n",
       "      <th>number vmail messages</th>\n",
       "      <th>total day minutes</th>\n",
       "      <th>total day calls</th>\n",
       "      <th>total day charge</th>\n",
       "      <th>total eve minutes</th>\n",
       "      <th>total eve calls</th>\n",
       "      <th>total eve charge</th>\n",
       "      <th>total night minutes</th>\n",
       "      <th>total night calls</th>\n",
       "      <th>total night charge</th>\n",
       "      <th>total intl minutes</th>\n",
       "      <th>total intl calls</th>\n",
       "      <th>total intl charge</th>\n",
       "      <th>customer service calls</th>\n",
       "      <th>churn</th>\n",
       "      <th>total charge</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>KS</td>\n",
       "      <td>128</td>\n",
       "      <td>415</td>\n",
       "      <td>382-4657</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>25</td>\n",
       "      <td>265.1</td>\n",
       "      <td>110</td>\n",
       "      <td>45.07</td>\n",
       "      <td>197.4</td>\n",
       "      <td>99</td>\n",
       "      <td>16.78</td>\n",
       "      <td>244.7</td>\n",
       "      <td>91</td>\n",
       "      <td>11.01</td>\n",
       "      <td>10.0</td>\n",
       "      <td>3</td>\n",
       "      <td>2.70</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "      <td>75.56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>OH</td>\n",
       "      <td>107</td>\n",
       "      <td>415</td>\n",
       "      <td>371-7191</td>\n",
       "      <td>no</td>\n",
       "      <td>yes</td>\n",
       "      <td>26</td>\n",
       "      <td>161.6</td>\n",
       "      <td>123</td>\n",
       "      <td>27.47</td>\n",
       "      <td>195.5</td>\n",
       "      <td>103</td>\n",
       "      <td>16.62</td>\n",
       "      <td>254.4</td>\n",
       "      <td>103</td>\n",
       "      <td>11.45</td>\n",
       "      <td>13.7</td>\n",
       "      <td>3</td>\n",
       "      <td>3.70</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "      <td>59.24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NJ</td>\n",
       "      <td>137</td>\n",
       "      <td>415</td>\n",
       "      <td>358-1921</td>\n",
       "      <td>no</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>243.4</td>\n",
       "      <td>114</td>\n",
       "      <td>41.38</td>\n",
       "      <td>121.2</td>\n",
       "      <td>110</td>\n",
       "      <td>10.30</td>\n",
       "      <td>162.6</td>\n",
       "      <td>104</td>\n",
       "      <td>7.32</td>\n",
       "      <td>12.2</td>\n",
       "      <td>5</td>\n",
       "      <td>3.29</td>\n",
       "      <td>0</td>\n",
       "      <td>False</td>\n",
       "      <td>62.29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>OH</td>\n",
       "      <td>84</td>\n",
       "      <td>408</td>\n",
       "      <td>375-9999</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>299.4</td>\n",
       "      <td>71</td>\n",
       "      <td>50.90</td>\n",
       "      <td>61.9</td>\n",
       "      <td>88</td>\n",
       "      <td>5.26</td>\n",
       "      <td>196.9</td>\n",
       "      <td>89</td>\n",
       "      <td>8.86</td>\n",
       "      <td>6.6</td>\n",
       "      <td>7</td>\n",
       "      <td>1.78</td>\n",
       "      <td>2</td>\n",
       "      <td>False</td>\n",
       "      <td>66.80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>OK</td>\n",
       "      <td>75</td>\n",
       "      <td>415</td>\n",
       "      <td>330-6626</td>\n",
       "      <td>yes</td>\n",
       "      <td>no</td>\n",
       "      <td>0</td>\n",
       "      <td>166.7</td>\n",
       "      <td>113</td>\n",
       "      <td>28.34</td>\n",
       "      <td>148.3</td>\n",
       "      <td>122</td>\n",
       "      <td>12.61</td>\n",
       "      <td>186.9</td>\n",
       "      <td>121</td>\n",
       "      <td>8.41</td>\n",
       "      <td>10.1</td>\n",
       "      <td>3</td>\n",
       "      <td>2.73</td>\n",
       "      <td>3</td>\n",
       "      <td>False</td>\n",
       "      <td>52.09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  state  account length  area code phone number international plan  \\\n",
       "0    KS             128        415     382-4657                 no   \n",
       "1    OH             107        415     371-7191                 no   \n",
       "2    NJ             137        415     358-1921                 no   \n",
       "3    OH              84        408     375-9999                yes   \n",
       "4    OK              75        415     330-6626                yes   \n",
       "\n",
       "  voice mail plan  number vmail messages  total day minutes  total day calls  \\\n",
       "0             yes                     25              265.1              110   \n",
       "1             yes                     26              161.6              123   \n",
       "2              no                      0              243.4              114   \n",
       "3              no                      0              299.4               71   \n",
       "4              no                      0              166.7              113   \n",
       "\n",
       "   total day charge  total eve minutes  total eve calls  total eve charge  \\\n",
       "0             45.07              197.4               99             16.78   \n",
       "1             27.47              195.5              103             16.62   \n",
       "2             41.38              121.2              110             10.30   \n",
       "3             50.90               61.9               88              5.26   \n",
       "4             28.34              148.3              122             12.61   \n",
       "\n",
       "   total night minutes  total night calls  total night charge  \\\n",
       "0                244.7                 91               11.01   \n",
       "1                254.4                103               11.45   \n",
       "2                162.6                104                7.32   \n",
       "3                196.9                 89                8.86   \n",
       "4                186.9                121                8.41   \n",
       "\n",
       "   total intl minutes  total intl calls  total intl charge  \\\n",
       "0                10.0                 3               2.70   \n",
       "1                13.7                 3               3.70   \n",
       "2                12.2                 5               3.29   \n",
       "3                 6.6                 7               1.78   \n",
       "4                10.1                 3               2.73   \n",
       "\n",
       "   customer service calls  churn  total charge  \n",
       "0                       1  False         75.56  \n",
       "1                       1  False         59.24  \n",
       "2                       0  False         62.29  \n",
       "3                       2  False         66.80  \n",
       "4                       3  False         52.09  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['total charge'] = df['total day charge'] + df['total eve charge'] + \\\n",
    "                     df['total night charge'] + df['total intl charge']\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "83e0b3c996775535de55a8773bdd7dee215e2a3b"
   },
   "source": [
    "To delete columns or rows, use the `drop` method, passing the required indexes and the `axis` parameter (`1` if you delete columns, and nothing or `0` if you delete rows). The `inplace` argument tells whether to change the original DataFrame. With `inplace=False`, the `drop` method doesn't change the existing DataFrame and returns a new one with dropped rows or columns. With `inplace=True`, it alters the DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "_uuid": "48fbc2e42a623ee78f9e2d9222186387561bbb0a"
   },
   "outputs": [],
   "source": [
    "# get rid of just created column and phone number                \n",
    "df.drop(['total charge', 'phone number'], axis=1, inplace=True) \n",
    "# and here’s how you can delete rows\n",
    "#df.drop([1, 2]).head() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "482e5e694de9c2173dd607a2821c54d704bc88ea"
   },
   "source": [
    "## 2. First attempt at predicting telecom churn\n",
    "\n",
    "\n",
    "Let's see how churn rate is related to the *International plan* feature. We'll do this using a `crosstab` contingency table and also through visual analysis with `Seaborn`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "_uuid": "8bc65b91a9041032168dfd50d5fab097c003d9c0",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>international plan</th>\n",
       "      <th>no</th>\n",
       "      <th>yes</th>\n",
       "      <th>All</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>churn</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>False</th>\n",
       "      <td>2664</td>\n",
       "      <td>186</td>\n",
       "      <td>2850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>True</th>\n",
       "      <td>346</td>\n",
       "      <td>137</td>\n",
       "      <td>483</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>All</th>\n",
       "      <td>3010</td>\n",
       "      <td>323</td>\n",
       "      <td>3333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "international plan    no  yes   All\n",
       "churn                              \n",
       "False               2664  186  2850\n",
       "True                 346  137   483\n",
       "All                 3010  323  3333"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.crosstab(df['churn'], df['international plan'], margins=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "_uuid": "c809a62f4d1077993a0f0f2d36d1066c108ca23d"
   },
   "outputs": [],
   "source": [
    "# some imports to set up plotting \n",
    "import matplotlib.pyplot as plt\n",
    "# pip install seaborn \n",
    "import seaborn as sns\n",
    "# Graphics in retina format are sharper and more legible\n",
    "%config InlineBackend.figure_format = 'retina'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "_uuid": "ae3bb6cdde2a6d72112e6076d92b35f2f2caaa3e"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxYAAAILCAYAAAB1gQpWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAABYlAAAWJQFJUiTwAAA7xElEQVR4nO3de5xVZd3//9cHEFTkIJFhJKHmKTNJTFNLkfKYpaWmX++7xA7elcc0zUoFsYOKt5aa1W2lVnep2e8WNc1KwbMpYJpJeRzPhxRBEUWRz++PtUY3wx6YYe1hM8Pr+XjsxzX7Wte1rmvN4Ljfs9a1VmQmkiRJklRFr2ZPQJIkSVL3Z7CQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZX1afYEtHQR8TAwEGhp8lQkSZLUs40EXszMdTvb0WDRPQxcbbXVhmyyySZDmj0RSZIk9VwzZ87klVdeWaa+BovuoWWTTTYZMn369GbPQ5IkST3Y6NGjmTFjRsuy9HWNhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsznWEiSJK3kFi5cyKxZs3jppZeYP38+mdnsKakBIoJ+/foxYMAAhgwZQq9eXXtOwWAhSZK0Elu4cCGPPfYY8+bNa/ZU1GCZyauvvsqrr77Kyy+/zDrrrNOl4cJgIUmStBKbNWsW8+bNo0+fPgwbNoz+/ft3+V+2tXwsXLiQl19+maeffpp58+Yxa9Yshg4d2mXj+a9GkiRpJfbSSy8BMGzYMAYMGGCo6EF69erFgAEDGDZsGPDWz7rLxuvSvUuSJGmFNn/+fAD69+/f5Jmoq7T+bFt/1l3FYCFJkrQSa12o7ZmKnisiALp8Ub7/giRJkqQerDVYdDWDhSRJkqTKDBaSJEmSKjNYSJIkqdtoaWkhIhg3blyzp6I2DBaSJEmSKvMBeeqU0cf8stlT0Apo+qTPNXsKkiSpyTxjIUmSJKkyg4UkSZJWGLfffjv77bcfw4cPp1+/fqy99trsvPPOXHLJJYu1bWlpYf/992fo0KGsuuqqbLnlllx55ZWLtZswYQIRwdSpU+vuo96ajXHjxhERPPTQQ5x99tm8//3vZ7XVVmPMmDGLbG9paeGnP/0pm222GauuuirveMc7OPjgg5kzZ04jvh3dipdCSZIkaYVw3nnn8ZWvfIXevXvzyU9+kg022IBnn32WadOmce655/KZz3zmzbaPPPIIW221Feuttx6f/exnmTVrFhdffDF77rknf/nLX9hxxx0bMqcjjjiCG2+8kY9//OPsvvvu9O7de5Htxx57LNdccw2f+MQn2HnnnZkyZQrnnXceDzzwANddd11D5tBdGCwkSZLUdPfeey9f/epXGThwIDfeeCObbrrpItsff/zxRd5PnTqVCRMmMH78+DfrDjjgAHbddVcmTZrUsGAxY8YM7rzzTtZdd92622+77Tb+/ve/M2LECAAWLFjA2LFjmTJlCrfffjtbbbVVQ+bRHXgplCRJkpruxz/+MQsWLOCEE05YLFQAvOtd71rk/bvf/W6OP/74Rep22WUXRowYwe23396weR177LHthgqAE0888c1QAdCnTx8OOugggIbOozswWEiSJKnpbrvtNgB22223DrUfNWrUYpclAayzzjq88MILDZvX0s44bLnllnXnADR0Ht2BwUKSJElNN3v2bACGDx/eofaDBw+uW9+nTx8WLlzYoFnBsGHDOj2PPn2K1QZvvPFGw+bRHRgsJEmS1HStH9CfeOKJhu+7V6/iI++CBQsW29YaaNoTEQ2fT09lsJAkSVLTfehDHwLg6quvbvi+11xzTQAee+yxxbZNmzat4eOtrAwWkiRJarqvfOUr9OnTh5NPPpl77713se1t7wrVGa3rJM4///xFzlo89thjTJw4cZn3q0V5u1lJkiQ13Xvf+17OPfdcvvzlL/OBD3yAPffckw022IDnn3+eadOmMWDAAKZMmbJM+956663ZfvvtueGGG9hqq60YO3YszzzzDFdccQW77LJL3TMZ6jzPWEiSJGmF8KUvfYmbbrqJPfbYg6lTpzJp0iQuv/xyhg4dyiGHHFJp35MnT+aLX/wijz/+OGeffTZ33nknp512GqeeemqDZq/IzGbPQUsREdO32GKLLaZPn97sqTD6mF82ewpaAU2f9LlmT0GStIxmzpwJwCabbNLkmagrdfTnPHr0aGbMmDEjM0d3dgzPWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpsm4VLCLibRHxxYj4v4h4ICJeiYg5EXFTRHwhInq1aT8yInIJr4uWMNaBEXF7RMwtx5gaEXssof1qEXFSRPwrIl6NiGcj4pKI8N5tkiRJ6vG625O39wV+DDwFTAEeBd4BfBr4GbBbROybiz+c4y7gsjr7u6feIBFxOnA08DhwHtAX2B+4IiIOy8xz2rTvB/wZ2A6YBvwQWKec78cjYmxm/rXTRytJkiR1E90tWNwHfBL4Q2YubK2MiG8BtwN7U4SM37fp97fMnNCRASJiW4pQ8SDwwcx8oayfBEwHTo+IKzOzpabbURSh4lJgv9a5RcTFFIHmFxGxWe2cJUmSpJ6kW10KlZnXZeYVbT+gZ+bTwE/Kt2MqDvPlsvxua6gox2gBfgT0Aw5qrY+IqOlzbO3cMnMycCPwXmCHivOSJEmSVljdKlgsxetluaDOtndGxH9FxLfK8v1L2M/YsvxjnW1Xt2kDsD4wArgvMx/uYB9JkiSpR+lul0LVFRF9gM+Vb+sFgp3KV22fqcCBmfloTV1/YDgwNzOfqrOf+8tyw5q6jcryvnamV69PXRExvZ1NGy+tryRJktRMPeWMxSnA+4CrMvOamvp5wMnAaGDN8rUDxcLvMcC1ZZhoNags57QzTmv94Ip9JEmSpB6l2weLiDicYrH1P4HP1m7LzGcz88TMnJGZs8vXDcDOwF+B9wBfXIZh2951aolT7GifzBxd70VxbJIkSepmWlpaiAjGjRvX7Kl0uW59KVREHEJxa9d7gY9m5qyO9MvMBRHxM2BrYPtyH/DW2YVBdTvWPzuxtD4D6/SRJEnqNkYf88tmT2GJpk/63NIbdUBxT572nX/++StFQFhW3TZYRMSRwJkUz6L4aGY+28ld/Lss37wUKjNfjogngOERsXaddRYblGXteop/lWV7ayjq9ZEkSdIKavz48XXrR40atXwn0s10y2AREd+gWFfxN2CnzHxuGXbzobJ8qE39dRSXVO0KnN9m2241bVo9SPGgvg0jYt06d4aq10eSJEkrqAkTJjR7Ct1St1tjEREnUISK6RRnKtoNFRGxdUT0rVM/Fvha+fbXbTa3Pg/j2xGxZk2fkcAhwHxqAkf5lO/WPqdFRK+aPnsCH6G4VOv6jhyfJEmSVlxPPvkkEydOZLvttmPYsGH07duXd77znRxwwAHMnDmzw/t55pln+PrXv85GG21E//79GTx4MBtttBHjxo3joYfa/t0brrnmGnbffXeGDh1Kv379WH/99TnmmGOYPXt2A4+umm51xiIiDgQmAm9QPHju8DrXwrVk5gXl16cCm5a3ln28rHs/bz1T4oTMvKW2c2beEhFnUDxN++6IuBToC+wHDAEOa/PUbYAzgD2AfYC/RsS1FM+22JfizlSf96nbkiRJ3d8NN9zAKaecwo477sjee+/NGmuswf3338+ll17K5Zdfzs0338zmm2++xH3MmzeP7bbbjgcffJCddtqJT3ziE2QmjzzyCJMnT2afffZhvfXWe7P9xIkTGT9+PEOGDGGPPfZgrbXW4u677+b000/nqquu4tZbb2XgwIFLGHH56FbBAli3LHsDR7bT5nrggvLrXwGfAj5IcUnSKsAzwCXAOZl5Y70dZObREXE3cChwMLAQmAFMyswr67SfHxEfA44DDqA4G/IicBkwPjPv7cxBSpIkqXnqXQo1cuRIxo0bx9ixY3nmmWcYMGDAItvvuusutttuO4477jiuvvrqxfrXuvbaa3nwwQc58sgjOfPMMxfZ9tprrzF//vw330+ZMoXx48ezzTbbcNVVVzF48OA3t11wwQUcdNBBjB8/frH9NEO3ChaZOQGY0In2Pwd+voxjXQhc2In2rwDjy5ckSZK6qZNOOmmxuh122IFx48ax1lpr1e2z+eabM3bsWP70pz/x+uuvs8oqqyx1nNVWW22xur59+9K371tX8p911lkAnHfeeYuECoBx48bxwx/+kP/93/81WEiSJEkrmmIJbfv+8Ic/8JOf/IRp06bx3HPPsWDBgkW2P/fcc6y99trt9t9hhx0YPnw4p5xyCjNmzGD33Xdnu+22Y9SoUfTu3XuRtrfeeiurrLIKv/vd7/jd73632L5ee+01/v3vf/P888/ztre9rRNH2XgGC0mSJKmDzjrrLI444gjWXHNNdtppJ0aMGMHqq69ORHDZZZdx1113LXIpUz0DBw7ktttuY/z48Vx++eVcc801AAwdOpSvfvWrHH/88W+e8Xj++edZsGBB3bMotebOnWuwkCRJkrqDBQsWMH78eIYNG8aMGTMWOytx6623dnhf73rXu/j5z39OZnLvvfdy3XXX8aMf/YiJEyeycOFCTj75ZAAGDRrEwoULmTWrQ8+Bbqpud7tZSZIkqRmee+45Zs+ezbbbbrtYqJg7dy4zZszo9D4jgk033ZTDDjuMP//5zwBcdtllb27/0Ic+xAsvvMA//vGPSnNfHgwWkiRJUgestdZarL766kyfPp25c+e+Wf/6669zxBFH8NxzHXtm8z333ENLS8ti9c888wwAq6+++pt1X/ta8ei1L33pSzz55JOL9Xn55Ze57bbbOnMYXcZLoSRJkqQO6NWrF4cffjinnHIKm222GXvuuSevvfYaU6ZMYdasWey4445MmTJlqfv5y1/+wlFHHcW2227LxhtvzFprrcXjjz/O5MmT6dWrF8ccc8ybbT/60Y9yyimn8M1vfpMNNtiA3XffnXXXXZe5c+fyyCOPcP311/PhD3+YP/7xj1156B1isJAkSZI66OSTT+btb387P/vZz/jpT3/KoEGD2GmnnfjOd77D+PEde+rALrvswpFHHskNN9zA5MmTefHFF1l77bXZaaed3gwctb7xjW+w3XbbcdZZZ3HTTTcxefJkBg0axPDhwzn44IM54IADuuJQOy2WdjstNV9ETN9iiy22mD59erOnwuhjftnsKWgFNH3S55o9BUnSMpo5cyYAm2yySZNnoq7U0Z/z6NGjmTFjxozMHN3ZMVxjIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkyvo0ewKSJElasT06cbNmT2GJRpz490r9I6JT7c8//3zGjRtXacyeyGAhSZKkldr48eMXq/vBD37AnDlzOOKIIxg8ePAi20aNGrV8JtbNGCwkSZK0UpswYcJidRdccAFz5szhyCOPZOTIkct9Tt2RaywkSZKkDhozZgwRwWuvvcbEiRPZaKON6Nev35uXRk2YMIGIYOrUqYv1bWlpISLqXkY1b948vv/97zNq1Cj69+/PGmuswTbbbMNvf/vbrj2gBvKMhSRJktRJe++9N3fccQe77bYbe+21F2uttdYy72v27NmMHTuWO++8ky222ILPf/7zLFy4kGuuuYYDDjiAf/zjH3znO99p4Oy7hsFCkiRJ6qRHHnmEe+65h6FDh1be15FHHsmdd97JqaeeyrHHHvtm/auvvspee+3F9773PfbZZ58Vfm2Hl0JJkiRJnXTyySc3JFQ8//zz/PrXv2bLLbdcJFQArLrqqpx66qlkJr/5zW8qj9XVPGMhSZIkddJWW23VkP3ccccdvPHGG0RE3UXkr7/+OgAzZ85syHhdyWAhSZIkddKwYcMasp/nn38eKALGHXfc0W67uXPnNmS8ruSlUJIkSVIntfdQvV69io/XCxYsWGzb7NmzF6sbNGgQAF/72tfIzHZfU6ZMadzku4jBQpIkSWqQNddcE4DHHntssW3Tpk1brG6rrbaiV69e3HjjjV0+t65msJAkSZIapHXtxfnnn7/IWYvHHnuMiRMnLtZ+rbXW4j/+4z+YNm0aJ598ct0zHQ8++CAPP/xw1026QVxjIUmSJDXI1ltvzfbbb88NN9zAVlttxdixY3nmmWe44oor2GWXXeqeyTjnnHO4//77OfHEE/nVr37Fhz/8Yd7xjnfw5JNPMnPmTO644w5++9vfsu666zbhiDrOMxaSJElSA02ePJkvfvGLPP7445x99tnceeednHbaaZx66ql12w8cOJDrr7+es88+m6FDh/L73/+eM844gylTpjBgwADOPPNMdtppp+V8FJ3nGQtJkiQt0YgT/97sKSx3LS0tdeunTp261L6DBw/mvPPO47zzzltsW2bW7dO3b18OPfRQDj300M5Mc4XiGQtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJknqw9m5x22gGC0mSpJVYRACwcOHCJs9EXaU1WLT+rLuKwUKSJGkl1q9fPwBefvnlJs9EXaX1Z9v6s+4qBgtJkqSV2IABAwB4+umneemll1i4cOFyu3RGXSczWbhwIS+99BJPP/008NbPuqv06dK9S5IkaYU2ZMgQXn75ZebNm8fjjz/e7Omoi6y++uoMGTKkS8cwWEiSJK3EevXqxTrrrMOsWbN46aWXmD9/vmcseoiIoF+/fgwYMIAhQ4bQq1fXXqxksJAkSVrJ9erVi6FDhzJ06NBmT0XdmGssJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFXWrYJFRLwtIr4YEf8XEQ9ExCsRMSciboqIL0RE3eOJiG0j4qqImBUR8yLi7og4MiJ6L2GsAyPi9oiYW44xNSL2WEL71SLipIj4V0S8GhHPRsQlEbFJI45dkiRJWpF1q2AB7AucB2wN/BX4AfB74H3Az4BLIiJqO0TEnsANwPbA/wE/AvoCZwIX1RskIk4HLgDWLsf7NbAZcEVEHFqnfT/gz8CJwIvAD4G/AJ8CpkXE1st+yJIkSdKKr0+zJ9BJ9wGfBP6QmQtbKyPiW8DtwN7ApynCBhExkCIYvAGMycxpZf0JwHXAPhGxf2ZeVLOvbYGjgQeBD2bmC2X9JGA6cHpEXJmZLTXzOgrYDrgU2K91bhFxMXAZ8IuI2Kx2zpIkSVJP0q3OWGTmdZl5RdsP6Jn5NPCT8u2Ymk37AG8HLmoNFWX7V4Hjy7dfaTPMl8vyu62houzTQnG2ox9wUGt9eYaktc+xtXPLzMnAjcB7gR06fKCSJElSN9OtgsVSvF6WC2rqxpblH+u0vwGYB2xbXsrUkT5Xt2kDsD4wArgvMx/uYB9JkiSpR+lul0LVFRF9gM+Vb2sDwUZleV/bPpm5ICIeBjYF1gNmRkR/YDgwNzOfqjPU/WW5YUfGWEKfuiJiejubNl5aX0mSJKmZesoZi1MoFnBflZnX1NQPKss57fRrrR+8jO2XtY8kSZLUo3T7MxYRcTjFYut/Ap/tbPeyzE7260z7Do+RmaPr7qA4k7FFJ8aUJEmSlqtufcYiIg6huLXrvcCOmTmrTZPWswWDqG9gm3ZLa1/v7ERnx5AkSZJ6nG4bLCLiSOAc4B6KUPF0nWb/KsvF1jeU6zLWpVjs/RBAZr4MPAGsERFr19nfBmVZu56i3TGW0EeSJEnqUbplsIiIb1A84O5vFKHi2XaaXleWu9bZtj2wOnBLZs7vYJ/d2rSB4nkXjwIbRsS6HewjSZIk9SjdLliUD7c7heJhdR/NzOeW0PxS4Dlg/4jYsmYfqwLfKd/+uE2f1udhfDsi1qzpMxI4BJgPnN9an5lZ0+e0iOhV02dP4CMUl2pd38FDlCRJkrqdbrV4OyIOBCZSPEn7RuDw4vl0i2jJzAsAMvPFiPgSRcCYGhEXAbMont69UVl/cW3nzLwlIs6geJr23RFxKdAX2A8YAhzW5qnbAGcAe1A8kO+vEXEtxbMt9qV4Vsbnfeq2JEmSerJuFSwo1kQA9AaObKfN9cAFrW8y87KI2AH4NrA3sCrwAEVwOKs847CIzDw6Iu4GDgUOBhYCM4BJmXllnfbzI+JjwHHAAcDXgBeBy4DxmXlvZw9UkiRJ6k66VbDIzAnAhGXodzOweyf7XAhc2In2rwDjy5ckSZK0Uul2aywkSZIkrXgMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSaqsocEiIkZExMCltBkQESMaOa4kSZKk5mr0GYuHgSOW0ubwsp0kSZKkHqLRwSLKlyRJkqSVSDPWWLwDeLkJ40qSJEnqIn2q7iAiPtemalSdOoDewAjgs8DfK4y3D7ADMArYHBgA/G9m/medtiNZ8mVXF2fm/u2McyBwCPBe4A3gTuD0zLyynfarAccB+wPvBl4EpgLjM3NmBw5NkiRJ6rYqBwvgAiDLrxPYs3y11XqJ1DzgpArjHU8RKOYCjwMbd6DPXcBldervqdc4Ik4Hji73fx7QlyIwXBERh2XmOW3a9wP+DGwHTAN+CKwD7At8PCLGZuZfOzBPSZIkqVtqRLA4qCwD+AXFB/jJddq9ATwP3JqZsyuM9zWKD/wPUJy5mNKBPn/LzAkd2XlEbEsRKh4EPpiZL5T1k4DpwOkRcWVmttR0O4oiVFwK7JeZC8s+F1N8P34REZu11kuSJEk9TeVgkZkXtn5dXj50WWb+sup+lzDem0EiokvWiX+5LL/bGirKcVsi4kfACRRhanw5h6jpc2xteMjMyRFxI/AROh6CJEmSpG6nEWcs3pSZOzZyfw30zoj4L+BtvHXW5O522o4tyz/W2XY1RbAYSxksgPUp1o7cl5n11nNcTREsxrKUYBER09vZ1JHLvSRJkqSmaWiwWIHtVL7eFBFTgQMz89Gauv7AcGBuZj5VZz/3l+WGNXUbleV97Yxdr48kSZLUozQ8WETEDsAxwFbAmtS/pW1m5vIINfOAkynWOTxU1r0fmADsCFwbEaMys/X2t4PKck47+2utH1xTtyx96srM0fXqyzMZWyytvyRJktQsDf1wHxEfp/gQ3xt4FPgXsKCRY3RGZj4LnNim+oaI2Bm4Cdga+CLFXZw6tetOtG1dCNKZPpIkSVK30uizBhOA14GPZ+afGrzvhsnMBRHxM4pgsT1vBYvWswuD6nasf3ZiaX0G1ukjSZIk9SiNfvL2+ygeOrfChooa/y7L/q0V5SVRTwBrRMTadfpsUJa16yn+VZbtraGo10eSJEnqURodLOYCsxq8z67yobJ8qE39dWW5a50+u7VpA8XzLh4FNoyIdTvYR5IkSepRGh0srgW2afA+l1lEbB0RfevUj6V40B7Ar9ts/klZfjsi1qzpMxI4BJgPnN9an5lZ0+e0iOhV02dPilvN3gtcX+lgJEmSpBVYo9dYfAO4PSKOp3jAXMMXLEfEXsBe5dthZblNRFxQfv1cZn69/PpUYNPy1rKPl3Xv561nVZyQmbfU7j8zb4mIMyiepn13RFwK9AX2A4YAh7V56jbAGcAewD7AXyPiWopnW+xLcWeqz/vUbUmSJPVkjQ4W44F/ACcBn4+IvwGz67TLzPzCMo4xCjiwTd165QvgEaA1WPwK+BTwQYpLklYBngEuAc7JzBvrDZCZR0fE3cChwMHAQmAGMCkzr6zTfn5EfAw4DjiA4mzIixR3yBqfmfcuy4FKkiRJ3UWjg8W4mq9Hlq96ElimYJGZEyjuPtWRtj8Hfr6M41wIXNiJ9q9QBKvxS2srSZIk9TSNDhb1Fi9LkiRJ6uEaGiwy85FG7k+SJElS99Dou0JJkiRJWgk19IxFRIzoaNvMfLSRY0uSJElqnkavsWihWJi9NNkFY0uSJElqkkZ/uP8l9YPFYIrbxL4bmEpxS1hJkiRJPUSjF2+Pa29b+UTqE4Avs/hzKCRJkiR1Y8tt8XZmLszMkygulzpleY0rSZIkqes1465QtwA7N2FcSZIkSV2kGcFiCNC/CeNKkiRJ6iLLNVhExMeA/YB7lue4kiRJkrpWo59jcd0SxlkHaH3OxcRGjitJkiSpuRp9u9kx7dQn8AJwDXB6ZrYXQCRJkiR1Q42+3Wwz1mxIkiRJajKDgCRJkqTKGn0p1CIiYiAwCJiTmS925ViSJEmSmqfhZywiondEHBcRD1Csq2gBXoiIB8r6Lg0zkiRJkpa/Rt8Vqi/wR2AHigXbjwFPAWsDI4HvArtGxM6Z+Vojx5YkSZLUPI0+Y3EUxZ2h/gBskpkjM3ObzBwJbARcAXykbCdJkiSph2h0sDiA4uF3e2Xm/bUbMvNB4NPAP4D/aPC4kiRJkpqo0cHiPcDVmbmw3say/mpg/QaPK0mSJKmJGh0sXgPWWEqb/sDrDR5XkiRJUhM1OljcDewTEW+vtzEihgL7AHc1eFxJkiRJTdToYHEO8Hbg9oj4QkSsFxGrRcS6EXEQ8Ndy+zkNHleSJElSEzX0drOZeUlEjAKOA/6nTpMATsvMSxo5riRJkqTmavjD6jLzWxFxOfAF4AOUT94G7gR+kZm3NnpMSZIkSc3VJU/BzszbgNu6Yt+SJEmSVjwNXWMREftGxHUR8c52tg+PiGsj4tONHFeSJElSczV68fYXgcGZ+WS9jZn5BDCwbCdJkiSph2h0sNgMmLaUNtOA9zd4XEmSJElN1OhgMQR4diltngeGNnhcSZIkSU3U6GDxHLDBUtpsAMxu8LiSJEmSmqjRweJm4JMRsXG9jRGxCbAncGODx5UkSZLURI0OFqdT3ML2pog4PCI2jIj+ZXkERaDoXbaTJEmS1EM0+snbd0TEV4EfAWeWr1pvAF/JzL82clxJkiRJzdUVT94+LyJuAr4KbA0MplhTcRvw48yc2egxJUmSJDVXVz15eyZwWFfsW5IkSdKKp9FrLCRJkiSthAwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkyrpdsIiIfSLi7Ii4MSJejIiMiF8vpc+2EXFVRMyKiHkRcXdEHBkRvZfQ58CIuD0i5kbEnIiYGhF7LKH9ahFxUkT8KyJejYhnI+KSiNikyvFKkiRJ3UG3CxbA8cChwCjgiaU1jog9gRuA7YH/A34E9AXOBC5qp8/pwAXA2sB5wK+BzYArIuLQOu37AX8GTgReBH4I/AX4FDAtIrbuxPFJkiRJ3U6fZk9gGXwNeBx4ANgBmNJew4gYSBEM3gDGZOa0sv4E4Dpgn4jYPzMvqumzLXA08CDwwcx8oayfBEwHTo+IKzOzpWaoo4DtgEuB/TJzYdnnYuAy4BcRsVlrvSRJktTTdLszFpk5JTPvz8zsQPN9gLcDF7WGinIfr1Kc+QD4Sps+Xy7L77aGirJPC8XZjn7AQa31ERE1fY6tDQ+ZORm4EXgvRQiSJEmSeqRuFyw6aWxZ/rHOthuAecC25aVMHelzdZs2AOsDI4D7MvPhDvaRJEmSepTueClUZ2xUlve13ZCZCyLiYWBTYD1gZkT0B4YDczPzqTr7u78sN+zIGEvoU1dETG9n08ZL6ytJkiQ1U08/YzGoLOe0s721fvAytl/WPpIkSVKP0tPPWCxNlGVH1mvU6kz7Do+RmaPr7qA4k7FFJ8aUJEmSlquefsai9WzBoHa2D2zTbmnt652d6OwYkiRJUo/T04PFv8pysfUNEdEHWBdYADwEkJkvUzwbY42IWLvO/jYoy9r1FO2OsYQ+kiRJUo/S04PFdWW5a51t2wOrA7dk5vwO9tmtTRsonnfxKLBhRKzbwT6SJElSj9LTg8WlwHPA/hGxZWtlRKwKfKd8++M2fX5Slt+OiDVr+owEDgHmA+e31pfP02jtc1pE9KrpsyfwEeBe4PoGHI8kSZK0Qup2i7cjYi9gr/LtsLLcJiIuKL9+LjO/DpCZL0bElygCxtSIuAiYBXyS4jaxlwIX1+4/M2+JiDMonqZ9d0RcCvQF9gOGAIe1eeo2wBnAHhQP5PtrRFxL8WyLfSmelfF5n7otSZKknqzbBQtgFHBgm7r1yhfAI8DXWzdk5mURsQPwbWBvYFXgAYrgcFa9J3hn5tERcTdwKHAwsBCYAUzKzCvrtJ8fER8DjgMOAL4GvAhcBozPzHuX9WAlSZKk7qDbBYvMnABM6GSfm4HdO9nnQuDCTrR/BRhfviRJkqSVSk9fYyFJkiRpOTBYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpspUiWERES0RkO6+n2+mzbURcFRGzImJeRNwdEUdGRO8ljHNgRNweEXMjYk5ETI2IPbruyCRJkqQVQ59mT2A5mgP8oE793LYVEbEn8HvgVeBiYBbwCeBMYDtg3zp9TgeOBh4HzgP6AvsDV0TEYZl5TkOOQpIkSVoBrUzBYnZmTlhao4gYSBEM3gDGZOa0sv4E4Dpgn4jYPzMvqumzLUWoeBD4YGa+UNZPAqYDp0fElZnZ0thDkiRJklYMK8WlUJ20D/B24KLWUAGQma8Cx5dvv9Kmz5fL8rutoaLs0wL8COgHHNRVE5YkSZKabWUKFv0i4j8j4lsRcURE7NjOeomxZfnHOttuAOYB20ZEvw72ubpNG0mSJKnHWZkuhRoG/KpN3cMRcVBmXl9Tt1FZ3td2B5m5ICIeBjYF1gNmRkR/YDgwNzOfqjPu/WW54dImGBHT29m08dL6SpIkSc20spyxOB/4KEW46A9sBvwUGAlcHRGb17QdVJZz2tlXa/3gZWwvSZIk9TgrxRmLzDypTdU9wJcjYi7FousJwKc6uLto3W1np7HUBpmj6w5YnMnYopPjSZIkScvNynLGoj0/Kcvta+pazzAMor6Bbdotrf3SzmhIkiRJ3d7KHiyeLcv+NXX/KsvF1kRERB9gXWAB8BBAZr4MPAGsERFr1xljg7JcbM2GJEmS1FOs7MFim7J8qKbuurLctU777YHVgVsyc34H++zWpo0kSZLU4/T4YBERm0bEkDr17wZan4b965pNlwLPAftHxJY17VcFvlO+/XGb3bVeUvXtiFizps9I4BBgPsUCckmSJKlHWhkWb+8LHBcRU4CHgZeA9YGPA6sCVwGntzbOzBcj4ksUAWNqRFwEzAI+SXEr2kuBi2sHyMxbIuIM4Cjg7oi4FOgL7AcMAQ7zqduSJEnqyVaGYDGFIhB8gOLSp/7AbOAmiuda/CozF7ljU2ZeFhE7AN8G9qYIIA9QBIez2rYv+xwdEXcDhwIHAwuBGcCkzLyyaw5NkiRJWjH0+GBRPvzu+qU2XLzfzcDunexzIXBhZ8eSJEmSursev8ZCkiRJUtczWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqbI+zZ6ApO7v0YmbNXsKWgGNOPHvzZ6CJGk58oyFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpsj7NnoAkST3d6GN+2ewpaAU0fdLnmj0FqaE8YyFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkyrzdrCRJUhM8OnGzZk9BK6ARJ/692VNYZp6xkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLBooIt4VEb+IiCcjYn5EtETEDyJizWbPTZIkSepKfZo9gZ4iItYHbgHWAiYD/wS2Ao4Ado2I7TLz+SZOUZIkSeoynrFonHMpQsXhmblXZh6XmWOBM4GNgO82dXaSJElSFzJYNEBErAfsDLQAP2qzeTzwMvDZiOi/nKcmSZIkLRcGi8YYW5Z/ysyFtRsy8yXgZmB14EPLe2KSJEnS8uAai8bYqCzva2f7/RRnNDYErm1vJxExvZ1Nm8+cOZPRo0cv+wwbZOYTLhPR4j7e65lmT0EroL6Tm/87a0Xh707V4+9O1dPs350zZ84EGLksfQ0WjTGoLOe0s721fvAy7v+NV155Zc6MGTNalrG/1JU2vqco/9ncaWiF89SMZs9AWpH5u1P1Nf9350jgxWXpaLBYPqIsc0mNMtM/76nbaT3T5r9fSeo4f3eqJ3KNRWO0npEY1M72gW3aSZIkST2KwaIx/lWWG7azfYOybG8NhiRJktStGSwaY0pZ7hwRi3xPI2IAsB3wCnDb8p6YJEmStDwYLBogMx8E/kSx2OWQNptPAvoDv8zMl5fz1CRJkqTlwsXbjfNV4BbgrIj4KDAT2BrYkeISqG83cW6SJElSl4rMJd6oSJ0QEesAE4FdgbcBTwGXASdl5qwmTk2SJEnqUgYLSZIkSZW5xkKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0l1RcTIiMiIuKD8+qKIeC4iXo2IaRGxR50+/SLiuIi4OyLmRcSLEXFjRHymGccgSV0lIjYuf0det4Q2f4+I1yNiWE3dLhFxVfn7dH5EPBgRkyJicJ3+74+I30ZES9n23xExIyJ+EBGrdNGhScssMrPZc5C0AoqIkcDDwFRgU+Ah4FZgCLAfsArwscycUrbvC/wJ2AH4J3AlsDqwD7AW8P3M/NZyPQhJ6kJlqNgR2Cgz72uzbVvgZuD3mblPWXcicBIwi+J35LPA+4GdgXuBbTLzxbLt+4G/AglcTvH7eCDwnnLMIZk5t6uPUeqMPs2egKQV3hhgQmae1FoREb8B/ggcA0wpq4+mCBVXA5/MzAVl25OA24FvRsSVmXnLcpy7JHWlcyk+5B8MfL3NtoPL8qcAEbEjRai4Fdg9M2e3NoyIccD55favldUHAqsCe2Xm5NodR8SawLwGHofUEJ6xkFRXzRmLR4D1M/ONNtsfAfpn5tDy/f3A+sB7M/Ofbdp+AfgZcH5mfn45TF+SulxE9KH4HdkPGJ6Z88v6wcCT5WuDzMyI+D9gL+B9mfmPOvu6s9zHWuX7/waOAnbJzD8th8ORKvOMhaSl+VvbUFF6DNgGICIGUJyef6JtqCi1XoP8ga6ZoiQtf5m5ICJ+BpwI7A38ptz0WWA14H/yrb/gbgO8DuwbEfvW2V1f4O0R8bbMfB64GDgCuCwiLgX+AtycmQ923RFJ1XjGQlJdNWcsLszMcXW2TwV2KP4QF++iCBrTMvODddquCrwCtGTmul05b0laniJiONAC3JKZO5R1fwc2BN6Vmf8u616nY3/QHZmZj5R9tgG+DYylCCoA/wJOyszfNvI4pEbwjIWkRphTlsPa2b52m3aS1CNk5hMRcQXwqYjYBFgTeB9wcWuoKM0BemXmkE7s+1Zgj4joB4wGdgUOA34TEf/OzL807ECkBvB2s5Iqy8yXgAeB4RGxQZ0mO5bljOU3K0labs4ty4Nps2i7xm3AmhGxaWd3npnzM/OWzDwROLys3nOZZip1IYOFpEb5BRDApIjo3VoZEUOBE2raSFJPcy1wH8WdnD4D3Nd6K+4aZ5bleRHxzrY7iIj+EfGhmvcfiYhBdcZ6R1l6VyitcLwUSlKjnA7sRvFXtLsi4iqK51jsS/Eci9My86Ymzk+SukR516efAGeUVW3PVpCZ10bEccD3gfvL35EPA2sA76a4XfdNFJc7QXEL753L9WwPAXMpnim0G/AC8D9ddkDSMnLxtqS6OrN4u6ZuVYrbIx5AcevZBcBdwI9caCipJyufLfEcxZ2fhpd3dqrX7sMUlzN9GBhKsfbiCYq75/0mM6eV7XYG/h+wNTCc4o/BjwPXAP/dusBbWpEYLCRJkiqKiDEUDwz9dWZ+trmzkZrDNRaSJEnVHVuW5zR1FlITucZCkiRpGUTEZsAeFLeC3Q24MjP/2txZSc1jsJAkSVo2o4HvAS8CvwO+2tzpSM3lGgtJkiRJlbnGQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkKQmioiREZERcUGz59IdRcQF5fdvZLPnsjQRMTUiuuzhUf5bktRsBgtJ6iEioiUiWpo9j0aKiAnlh+UxzZ6LJGnJ+jR7ApK0knsC2ASY0+yJdFPfBE6h+D5KkprIYCFJTZSZrwP/bPY8uqvMfAp4qtnzkCR5KZQkNVV718XXrh2IiP+KiL9HxKsR8UxE/E9EDKppO6a8dv/dwLvLftnOfjcu9/1YRMwv9/ebiNioztxa57BeRBwWEXdHxCsRMbWzc6zZ547ltnsj4sVyf/dExPiIWLVN2xZgfPl2Su1x1fs+1RnrMxFxQ0TMKcf5e0R8MyL61WnbUr5Wj4hJEfFo+f15ICK+ERFRp8+4iPh9RDxU7v/FiLg5Iv6zbdtlUTOnQRFxTkQ8UX5/742Iw+vNqZ39bBgRp0TEtIj4d3lcj5Q/h3fVaT+m/J5OiIhREfGHiJgdEfMi4vqI2LYRxyep5/GMhSSt2E4DdgGuAP4E7Ah8CXgPMLZs0wKcBBxZvv9BTf+/tX4REbsC/x+wSrm/B4B3AZ8GPh4RO2bmjDpz+CHwEeAPwFXAG8swx1bfADYGbin3tyqwHTABGBMRH8vM1v3/ANgL2AG4sDzODomI71FcJvUc8BtgLrAb8D1gl4jYqTxbVGuVcv7vBK4GFpTjn1LO86Q27X8M3AvcQHHW5G3A7sCvImKjzDyho/Ndgr7AX4DBwEXl+70pfiYbAYd0YB+fBr4MTKH4vr8GbAp8EfhERGyZmfUuJdsSOBa4FfgZMKIc+9qIGJWZ/1r2w5LUI2WmL1++fPlq0gsYCSRwQZv6C8r6R4ERNfV9KD7IJrBVmz4tQEs746wJvEDxQfu9bbZtSvHBe0Y7c3gCWLfOPpdljusBUWdfJ5ft92tTP6GsH9POcbXOYWRN3TY18xrWZl5XlNu+Ved7lxTBabWa+rWA2eVrlTZ91q8zn77AtcDrwPA226YW/9vt8L+N1jndBPSrqR8CPFhu274D/5aG1/avqd+ZIiT+uE39mHI/CYxrs+2/yvpzm/3fji9fvla8l5dCSdKKbWJmPtr6JjMXAOeXb7fqxH4+R/FX7/GZeW/thsz8B3Ae8IGIeG+dvqdl5sONmGNmPpSZ9W65+oOy3GVJB9FBny/L72Tm023mdTSwkOKv9fUcnpmv1PR5FpgMDKI4Q0DNtgfbds7M14AfUYSYj1Y4hlrfzMz5NWPMoghiAActrXNmPlHbv6b+T8A/aP97fnNmXtCm7hcUZ3I6829P0krCS6EkacU2rU7dY2W5Zif2s01Zbh4RE+ps37AsN6G4vKfW7UvZd4fnGBH9gSOAT5VjDgBq1woMX8pYHbFFWV7XdkNm3hcRjwPrRsTgzJxds3lOZj5QZ3/tHcsIiku7PkpxmdBqbfo14lgWUFy+1NbUsvzA0nZQrsX4D2AcsDnFcfSuafJaO10X+7lm5usR8Qyd+7cnaSVhsJCkFdvsOnULyrJ3nW3teVtZfmkp7daoU/d0nbpas+vULTbHiFiF4sP+VsA9wMXAvykuG4JiofZiC6uXQeui8fbuFvUURRAYxKJzn12vMfWPZT2KwLUmcCPF2ow5FJcWjQQOpDHH8ly+teakVuvPZLEF8nWcQbH+5ingGopL21rPyoyjWPRfz+x26hfQuX97klYSBgtJWjm0Pidj88y8u5N9G/W06D0pQsWFmTmudkNErM1bd4CqqvVYh1GsRWhr7TbtlsVRFGHtoLaXC0XE/6MIFo0wNCJ61wkXw8pyiccQEWsBh1MEuW0z86U6c5WkhnCNhST1HG/Q/l+SbyvLjyynudTznrL8fZ1tO7TTp/UDdWf+Qn5nWY5puyEi3kNxJ6yH21wG1VnLcizLog9Q7/auY8ryzjrbaq1H8f/6P9UJFe8qt0tSQxgsJKnneB54e0S0vdYfisXUs4HxEbHYwtuI6BURY7p0dm/dLnaRccrLik5tp8/zZTmiE+P8oiyPj4i314zTGzid4v99P+/E/uppKcsxtZURsQvtLwxfVt+vffZGRAwBji/fnl+/y5tayvLD5fG37mMNigX7XrkgqWH8hSJJPce1wAeBP0bEDcB84K7MvCIzn4+IfYD/A26LiGsp7gi0kOJD+zYUl/asWn/XDdH67IyjImIzir+2jwD2oHimRb3wMKWc4/cj4n0Ut8wlM7/T3iCZeUtEnEbxDIZ7IuJS4GWK51i8j+L2rZMqHsu5FHdk+l1E/J5i3cL7gF2BS4D9Ku6/1VMUazXuiYjLKZ61sQ/F5VznZuYNS+qcmU9HxEXA/sDfIuJPFOsydgJepXjOyagGzVXSSs5gIUk9x3cobin7CYqHzvWmeLDcFQCZeW1EvB/4OsUtRj9CcUegJykWVde7rKdhMvPliBhL8cC5MeX4D1HcOvUM6nwYz8yZEXFgOeev8lbwaTdYlP2+ERF3AodS3Gp3FYr1FscD/13eFrbKsdwdETuW89id4v+nd1E8jG52vWNZRq8BH6N4sN/+wFCK79kpwNkd3McXyj77UTxQ79/A5cCJdPHPXNLKJerfTlySJDVTRLQAZObI5s5EkjrGNRaSJEmSKjNYSJIkSarMYCFJkiSpMtdYSJIkSarMMxaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKvv/AX7rZ3kLSNFjAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 261,
       "width": 395
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(x='international plan', hue='churn', data=df);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "c1134a7f1fe7a5c8dae399a84f58611043f6e757"
   },
   "source": [
    "\n",
    "We see that, with *International Plan*, the churn rate is much higher, which is an interesting observation! Perhaps large and poorly controlled expenses with international calls are very conflict-prone and lead to dissatisfaction among the telecom operator's customers.\n",
    "\n",
    "Next, let's look at another important feature – *Customer service calls*. Let's also make a summary table and a picture."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "_uuid": "75cf7fe86ae75e01d49a07fb52fe5c8ce1c6a334"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>customer service calls</th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>All</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>churn</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>False</th>\n",
       "      <td>605</td>\n",
       "      <td>1059</td>\n",
       "      <td>672</td>\n",
       "      <td>385</td>\n",
       "      <td>90</td>\n",
       "      <td>26</td>\n",
       "      <td>8</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2850</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>True</th>\n",
       "      <td>92</td>\n",
       "      <td>122</td>\n",
       "      <td>87</td>\n",
       "      <td>44</td>\n",
       "      <td>76</td>\n",
       "      <td>40</td>\n",
       "      <td>14</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>483</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>All</th>\n",
       "      <td>697</td>\n",
       "      <td>1181</td>\n",
       "      <td>759</td>\n",
       "      <td>429</td>\n",
       "      <td>166</td>\n",
       "      <td>66</td>\n",
       "      <td>22</td>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "customer service calls    0     1    2    3    4   5   6  7  8  9   All\n",
       "churn                                                                  \n",
       "False                   605  1059  672  385   90  26   8  4  1  0  2850\n",
       "True                     92   122   87   44   76  40  14  5  1  2   483\n",
       "All                     697  1181  759  429  166  66  22  9  2  2  3333"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.crosstab(df['churn'], df['customer service calls'], margins=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "_uuid": "bc0a6cb1fb72cc66bc1d7c33301b3ab0d2c7d81b"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxYAAAILCAYAAAB1gQpWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAABYlAAAWJQFJUiTwAABEHUlEQVR4nO3deZgcVb3/8fc3xAQIWQgBgwiyXAREFAkGAWWJsoqAihcuCgYvcNkEBMENSAh6BUFQUNGLyuIGij9W2QQSwyokQRGIyhZkRwiELBAI+f7+qBrSTHoymVRPdzLzfj3PPJWuc6rOqZrJTH/61KmKzESSJEmSqujT6g5IkiRJWvYZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZX1b3QF1LiIeBQYB01rcFUmSJPVsawMvZ+Y6Xd3QYLFsGLTCCisM3WijjYa2uiOSJEnquaZOncorr7yyRNsaLJYN0zbaaKOhkydPbnU/JEmS1IONGDGCKVOmTFuSbZ1jIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKvM5FpLUS8yfP5/p06czc+ZM5s6dS2a2uktqgIigf//+DBw4kKFDh9Knj58ZSmoNg4Uk9QLz58/n8ccfZ86cOa3uihosM3n11Vd59dVXmT17NmuuuabhQlJLGCwkqReYPn06c+bMoW/fvgwfPpwBAwb45rOHmD9/PrNnz+aZZ55hzpw5TJ8+nWHDhrW6W5J6If+qSFIvMHPmTACGDx/OwIEDDRU9SJ8+fRg4cCDDhw8HFnyvJanZ/MsiSb3A3LlzARgwYECLe6Lu0va9bfteS1KzGSwkqRdom6jtSEXPFREATsqX1DL+hZEkqQdoCxaS1CoGC0mSJEmVGSwkSZIkVWawkCQtlaZNm0ZEMHr06FZ3RZK0GAwWkiRJkirzAXnqUUYcd1HT2pp8+v5Na0uSJGlp54iFJEmSpMoMFpKklrjrrrvYe++9WWONNejfvz+rr746O+64I7/97W8Xqjtt2jT22Wcfhg0bxvLLL8/mm2/O1VdfvVC9sWPHEhFMmDCh7j7qzdkYPXo0EcEjjzzCOeecw/ve9z5WWGEFtttuu7eUT5s2jZ/85CdssskmLL/88rz97W/n4IMPZsaMGY04HZK0zPNSKElS05133nkceuihLLfccuy+++6sv/76PPfcc0yaNIkf/ehH/Od//uebdR977DFGjhzJuuuuy3777cf06dO55JJL2GOPPbjxxhvZfvvtG9Kno446iltuuYWPf/zj7Lrrriy33HJvKT/++OO5/vrr+cQnPsGOO+7I+PHjOe+883jooYe4+eabG9IHSVqWGSwkSU31wAMPcNhhhzFo0CBuueUWNt5447eUP/HEE295PWHCBMaOHcuYMWPeXLfvvvuy8847c/rppzcsWEyZMoV77rmHddZZp275nXfeyd/+9jfWWmstAObNm8eoUaMYP348d911FyNHjmxIPyRpWeWlUJKkpjr33HOZN28eJ5544kKhAuCd73znW16/613v4oQTTnjLup122om11lqLu+66q2H9Ov744zsMFQAnnXTSm6ECoG/fvhxwwAEADe2HJC2rDBaSpKa68847Adhll10Wq/6mm2660GVJAGuuuSYvvvhiw/rV2YjD5ptvXrcPQEP7IUnLKoOFJKmpXnrpJQDWWGONxao/ZMiQuuv79u3L/PnzG9QrGD58eJf70bdvcUXxG2+80bB+SNKyymAhSWqqtjfoTz75ZMP33adP8Wdt3rx5C5W1BZqORETD+yNJvYnBQpLUVB/60IcAuPbaaxu+75VXXhmAxx9/fKGySZMmNbw9SdICBgtJUlMdeuih9O3bl1NOOYUHHnhgofL2d4XqirZ5Eueff/5bRi0ef/xxxo0bt8T7lSR1ztvNSpKa6j3veQ8/+tGPOOSQQ/jABz7AHnvswfrrr88LL7zApEmTGDhwIOPHj1+ifW+xxRZss802TJw4kZEjRzJq1CieffZZrrrqKnbaaae6IxmSpMZwxEKS1HQHHXQQt956K7vtthsTJkzg9NNP58orr2TYsGEcfvjhlfZ9xRVXcOCBB/LEE09wzjnncM899/Cd73yH0047rUG9lyTVE5nZ6j6oExExebPNNtts8uTJre7KUm/EcRc1ra3Jp+/ftLakqqZOnQrARhtt1OKeqDv5fZZU1YgRI5gyZcqUzBzR1W0dsZAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVbbMBYuI2CsizomIWyLi5YjIiPhlJ9tsFRHXRMT0iJgTEfdGxNERsdwitvl8RNwVEbMiYkZETIiI3RZRf4WIODki/hERr0bEcxHx24hwBp0kSZJ6vGUuWAAnAEcAmwJPdlY5IvYAJgLbAJcBPwT6AWcBF3ewzRnABcDqwHnAL4FNgKsi4og69fsDfwROAl4Gvg/cCHwSmBQRW3Th+CRJkqRlzrL4gLwvAU8ADwHbAh0+RSkiBlEEgzeA7TJzUrn+ROBmYK+I2CczL67ZZivgWOBh4IOZ+WK5/nRgMnBGRFydmdNqmjoG2Bq4FNg7M+eX21wCXA78PCI2aVsvSZIk9TTL3IhFZo7PzAdz8R7AsRewKnBxW6go9/EqxcgHwKHttjmkXH6rLVSU20yjGO3oDxzQtj4iomab42vDQ2ZeAdwCvIciBEmSJEk90jIXLLpoVLm8rk7ZRGAOsFV5KdPibHNtuzoA6wFrAf/MzEcXcxtJkiSpR1kWL4Xqig3K5T/bF2TmvIh4FNgYWBeYGhEDgDWAWZn5dJ39PVgu3704bSxim7oioqNHa2/Y2baSJElSK/X0EYvB5XJGB+Vt64csYf0l3UaSJEnqUXr6iEVnolwuznyNWl2pv9htZOaIujsoRjI260KbkiRJUlP19BGLttGCwR2UD2pXr7P69UYnutqGJEmS1OP09GDxj3K50PyGiOgLrAPMAx4ByMzZFM/GWCkiVq+zv/XLZe18ig7bWMQ2kqQeZNq0aUQEo0ePbnVXJKllevqlUDcDnwV2Bn7TrmwbYEVgYmbObbfNfuU257fbZpeaOm0eBv4FvDsi1qlzZ6h620jSUmfEcRe1uguLNPn0/Ruyn+Iu4R07//zzDQiStAR6erC4FDgN2Ccizql5QN7ywDfLOue22+bHFMHiGxFxec0D8tYGDgfmUhM4MjMj4sfA/wLfiYjaB+TtAXwEeAD4U/ccoiRpSYwZM6bu+k033bS5HZGkHmKZCxYRsSewZ/lyeLncMiIuKP/9fGZ+GSAzX46IgygCxoSIuBiYDuxOcZvYS4FLavefmbdHxJkUT9O+NyIuBfoBewNDgS+2e+o2wJnAbhQP5PtzRNxE8WyLz1A8K+MLPnVbkpYuY8eObXUXJKlHWRbnWGwKfL782qlct27Nur1qK2fm5RRPvZ4IfBr4IvA6RXDYp94TvDPzWGA08AxwMLA/cD/wicz8QZ36c4GPAeMobiv7JWAH4HLgg5n55yU9WElS8z311FOMGzeOrbfemuHDh9OvXz/e8Y53sO+++zJ16tTF3s+zzz7Ll7/8ZTbYYAMGDBjAkCFD2GCDDRg9ejSPPPLIQvWvv/56dt11V4YNG0b//v1Zb731OO6443jppZcaeHSS1D2WuRGLzBwLjO3iNrcBu3ZxmwuBC7tQ/xVgTPklSVqGTZw4kVNPPZXtt9+eT3/606y00ko8+OCDXHrppVx55ZXcdtttvP/971/kPubMmcPWW2/Nww8/zA477MAnPvEJMpPHHnuMK664gr322ot11133zfrjxo1jzJgxDB06lN12243VVluNe++9lzPOOINrrrmGO+64g0GDBi2iRUlqrWUuWEiS1Aj1LoVae+21GT16NKNGjeLZZ59l4MCBbyn/61//ytZbb81Xv/pVrr322kXu/6abbuLhhx/m6KOP5qyzznpL2WuvvcbcuQvuGzJ+/HjGjBnDlltuyTXXXMOQIUPeLLvgggs44IADGDNmzEL7kaSlicFCktQrnXzyyQut23bbbRk9ejSrrbZa3W3e//73M2rUKG644QZef/113va2t3XazgorrLDQun79+tGvX783X5999tkAnHfeeW8JFQCjR4/m+9//Pr/61a8MFpKWagYLSVKvVGeK3Vv84Q9/4Mc//jGTJk3i+eefZ968eW8pf/7551l99XqPPCpsu+22rLHGGpx66qlMmTKFXXfdla233ppNN92U5ZZb7i1177jjDt72trfxu9/9jt/97ncL7eu1117j3//+Ny+88AKrrLJKF45SkprHYCFJUjtnn302Rx11FCuvvDI77LADa621FiuuuCIRweWXX85f//rXt1zKVM+gQYO48847GTNmDFdeeSXXX389AMOGDeOwww7jhBNOeHPE44UXXmDevHl1R1FqzZo1y2AhaallsJAkqca8efMYM2YMw4cPZ8qUKQuNStxxxx2Lva93vvOd/OxnPyMzeeCBB7j55pv54Q9/yLhx45g/fz6nnHIKAIMHD2b+/PlMnz69occiSc20LN5uVpKkbvP888/z0ksvsdVWWy0UKmbNmsWUKVO6vM+IYOONN+aLX/wif/zjHwG4/PLL3yz/0Ic+xIsvvsj9999fqe+S1EoGC0mSaqy22mqsuOKKTJ48mVmzZr25/vXXX+eoo47i+eefX6z93HfffUybNm2h9c8++ywAK6644pvrvvSlLwFw0EEH8dRTTy20zezZs7nzzju7chiS1HReCiVJUo0+ffpw5JFHcuqpp7LJJpuwxx578NprrzF+/HimT5/O9ttvz/jx4zvdz4033sgxxxzDVlttxYYbbshqq63GE088wRVXXEGfPn047rjj3qz70Y9+lFNPPZWvfe1rrL/++uy6666ss846zJo1i8cee4w//elPfPjDH+a6667rzkOXpEoMFpIktXPKKaew6qqr8tOf/pSf/OQnDB48mB122IFvfvObjBmzeM9B3WmnnTj66KOZOHEiV1xxBS+//DKrr746O+yww5uBo9ZXvvIVtt56a84++2xuvfVWrrjiCgYPHswaa6zBwQcfzL777tsdhypJDROd3W5PrRcRkzfbbLPNJk+e3OquLPVGHHdR09qafPr+TWtLqmrq1KkAbLTRRi3uibqT32dJVY0YMYIpU6ZMycwRXd3WORaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSaqsb6s7IElaOvxr3Cat7sIirXXS3yptHxFdqn/++eczevToSm1KUm9isJAk9QpjxoxZaN33vvc9ZsyYwVFHHcWQIUPeUrbppps2p2OS1EMYLCRJvcLYsWMXWnfBBRcwY8YMjj76aNZee+2m90mSehLnWEiS1M52221HRPDaa68xbtw4NthgA/r37//mpVFjx44lIpgwYcJC206bNo2IqHsZ1Zw5c/j2t7/NpptuyoABA1hppZXYcsst+c1vftO9ByRJTeCIhSRJHfj0pz/N3XffzS677MKee+7JaquttsT7eumllxg1ahT33HMPm222GV/4wheYP38+119/Pfvuuy/3338/3/zmNxvYe0lqLoOFJEkdeOyxx7jvvvsYNmxY5X0dffTR3HPPPZx22mkcf/zxb65/9dVX2XPPPfnf//1f9tprL+d2SFpmeSmUJEkdOOWUUxoSKl544QV++ctfsvnmm78lVAAsv/zynHbaaWQmv/71ryu3JUmt4oiFJEkdGDlyZEP2c/fdd/PGG28QEXUnkb/++usATJ06tSHtSVIrGCwkSerA8OHDG7KfF154ASgCxt13391hvVmzZjWkPUlqBS+FkiSpAx09VK9Pn+LP57x58xYqe+mllxZaN3jwYAC+9KUvkZkdfo0fP75xnZekJjNYSJLURSuvvDIAjz/++EJlkyZNWmjdyJEj6dOnD7fccku3902SWsVgIUlSF7XNvTj//PPfMmrx+OOPM27cuIXqr7baanz2s59l0qRJnHLKKXVHOh5++GEeffTR7uu0JHUz51hIktRFW2yxBdtssw0TJ05k5MiRjBo1imeffZarrrqKnXbaqe5Ixg9+8AMefPBBTjrpJH7xi1/w4Q9/mLe//e089dRTTJ06lbvvvpvf/OY3rLPOOi04IkmqzhELSZKWwBVXXMGBBx7IE088wTnnnMM999zDd77zHU477bS69QcNGsSf/vQnzjnnHIYNG8bvf/97zjzzTMaPH8/AgQM566yz2GGHHZp8FJLUOI5YSJIAWOukv7W6C003bdq0uusnTJjQ6bZDhgzhvPPO47zzzluoLDPrbtOvXz+OOOIIjjjiiK50U5KWCY5YSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkST1AR7e4laRmMVhIUi8QEQDMnz+/xT1Rd2kLFm3fa0lqNoOFJPUC/fv3B2D27Nkt7om6S9v3tu17LUnNZrCQpF5g4MCBADzzzDPMnDmT+fPne+lMD5CZzJ8/n5kzZ/LMM88AC77XktRsfVvdAUlS9xs6dCizZ89mzpw5PPHEE63ujrrJiiuuyNChQ1vdDUm9lMFCknqBPn36sOaaazJ9+nRmzpzJ3LlzHbHoISKC/v37M3DgQIYOHUqfPl6MIKk1DBaS1Ev06dOHYcOGMWzYsFZ3RZLUA/mxhiRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqrNcEi4j4eETcEBFPRMQrEfFIRPwuIrbsoP5WEXFNREyPiDkRcW9EHB0Ryy2ijc9HxF0RMSsiZkTEhIjYrfuOSpIkSVo69IpgERGnAVcDmwHXAd8HpgB7ALdFxOfa1d8DmAhsA1wG/BDoB5wFXNxBG2cAFwCrA+cBvwQ2Aa6KiCMaflCSJEnSUqRvqzvQ3SJiOPBl4FngfZn5XE3Z9sDNwDiKIEBEDKIIBm8A22XmpHL9iWXdvSJin8y8uGY/WwHHAg8DH8zMF8v1pwOTgTMi4urMnNbNhytJkiS1RG8YsXgXxXH+uTZUAGTmeGAmsGrN6r3K1xe3hYqy7qvACeXLQ9u1cUi5/FZbqCi3mUYx2tEfOKDykUiSJElLqd4QLB4EXgNGRsSw2oKI2AYYCNxYs3pUubyuzr4mAnOArSKi/2Juc227OpIkSVKP0+MvhcrM6RHxFeBM4IGIuBx4AVgP2B34I/A/NZtsUC7/WWdf8yLiUWBjYF1gakQMANYAZmXm03W68GC5fHdnfY2IyR0UbdjZtpIkSVIr9fhgAZCZ34uIacDPgYNqih4CLmh3idTgcjmjg921rR+yhPUlSZKkHqc3XApFRBwPXEpx16b1gAHACOAR4FcR8Z2u7K5cZhe70Wn9zBxR7wv4exfbkiRJkpqqxweLiNgOOA24MjOPycxHMnNOZk4BPgk8CRwbEeuWm7SNMAxeaGeFQe3qdVa/sxENSZIkaZnX44MF0PaAuvHtCzJzDnAXxXn4QLn6H+VyoTkREdEXWAeYRzHaQWbOpggnK0XE6nXaX79cLjRnQ5IkSeopekOwaLt706odlLetf61c3lwud65TdxtgReD2zJxbs35R2+zSro4kSZLU4/SGYHFLuTw4ItaoLYiIXYCtgVeB28vVlwLPA/tExOY1dZcHvlm+PLddGz8ul9+IiJVrtlkbOByYC5xf+UgkSZKkpVRvuCvUpRTPqfgYxe1hLwOeATaiuEwqgK9m5gsAmflyRBxUbjchIi4GplPcmnaDcv0ltQ1k5u0RcSZwDHBvRFwK9AP2BoYCX/Sp25IkSerJenywyMz5EbErxcjBPhQTtlekCAvXAGdn5g3ttrk8IrYFvgF8Glie4ta0x5T1F7rDU2YeGxH3AkcABwPzgSnA6Zl5dXcdnyRJkrQ06PHBAiAzXwe+V34t7ja3Abt2sZ0LgQu7so0kSZLUE/SGORaSJEmSupnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUWd9Wd0BS9xhx3EVNa2vy6fs3rS1JkrR0csRCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRV1tBgERFrRcSgTuoMjIi1GtmuJEmSpNZq9IjFo8BRndQ5sqwnSZIkqYdodLCI8mupFBEfiYjfR8TTETG3XN4QEbvWqbtVRFwTEdMjYk5E3BsRR0fEcovY/+cj4q6ImBURMyJiQkTs1r1HJUmSJLVeK+ZYvB2Y3exGI+IEYCKwDXAd8F3gKmBlYLt2dfeoqXsZ8EOgH3AWcHEH+z8DuABYHTgP+CWwCXBVRBzR6OORJEmSliZ9q+4gIvZvt2rTOusAlgPWAvYD/la13a6IiM8ApwA3Ap/KzJntyt9W8+9BFMHgDWC7zJxUrj8RuBnYKyL2ycyLa7bZCjgWeBj4YGa+WK4/HZgMnBERV2fmtO47SkmSJKl1KgcLik/ps/x3AnuUX+21XSI1Bzi5Ae0ulojoA5xWtrtv+1ABkJmv17zcC1gVuKgtVJR1Xi1HPW4CDuWtIxeHlMtvtYWKcptpEfFD4ETgAGBMY45KkiRJWro0IlgcUC4D+DlwOXBFnXpvAC8Ad2TmSw1od3FtBawDXAq8GBEfB94LvArclZl3tKs/qlxeV2dfEykCylYR0T8z5y7GNtdSBItRdFOwGHHcRd2x27omn15vMEqSJEm9XeVgkZkXtv07Ij4PXJ6ZzXun27kPlstngSkU8x7eFBETgb0y89/lqg3K5T/b7ygz50XEo8DGwLrA1IgYAKwBzMrMp+u0/2C5fHdnHY2IyR0UbdjZtpIkSVIrNXTydmZuv5SFCoDVyuUhwArAx4CBFKMW11NM0P5dTf3B5XJGB/trWz9kCetLkiRJPU4jLoVa2rXdHjYoRib+Wr6+PyI+STEysW1EbFnnsqh62uaK5CJrLazT+pk5om6DxUjGZl1sT5IkSWqaht9uNiK2jYirI+K5iHg9It6o8zWv0e0uQttk6kdqQgUAmfkKxagFwMhy2TbCMJj6BrWr11n9zkY0JEmSpGVeQ0csyonRl1OMEvwL+AfQzBBRzz/K5UsdlLcFjxVq6m9OMSfiLXMeIqIvxUTwecAjAJk5OyKeBNaIiNXrzLNYv1wuNGdDkiRJ6ikafSnUWOB14OOZeUOD972kJlIEgfUjol9mvtau/L3lclq5vBn4LLAz8Jt2dbcBVgQm1twRqm2b/cptzm+3zS41dSRJkqQeqdGXQr0XuGQpChVk5vPAJRSXJJ1UWxYROwA7UVym1Har2EuB54F9ImLzmrrLA98sX57brpkfl8tvRMTKNdusDRwOzGXhwCFJkiT1GI0esZgFTG/wPhvhGGALijf+2wB3Ae8CPknxfI2D2p6tkZkvR8RBFAFjQkRcTHFMu1PcivZSiqDypsy8PSLOLNu5NyIuBfoBewNDgS/61G1JkiT1ZI0OFjcBWzZ4n5Vl5nMRsQVwAkWY+BAwE/gD8O3MvLNd/csjYlvgG8CngeWBhyiCw9mZudAdnjLz2Ii4FzgCOBiYT/HcjNMz8+puOzhJkiRpKdDoYPEV4K6IOAH4Vr034K2SmdMpgsExi1n/NmDXLrZxIXBhpxUlSZKkHqbRwWIMcD9wMvCFiPgL9e/GlJn53w1uW5IkSVKLNDpYjK7599rlVz0JGCwkSZKkHqLRwWKdBu9PkiRJ0jKgocEiMx9r5P4kSZIkLRsa/RwLSZIkSb1QQ0csImKtxa2bmf9qZNuSJEmSWqfRcyymUUzM7kx2Q9uSJEmSWqTRb+4von6wGAJsSvG06wmAczEkSZKkHqTRk7dHd1QWEX2AE4FDgM83sl1JkiRJrdW0yduZOT8zT6a4XOrUZrUrSZIkqfu14q5QtwM7tqBdSZIkSd2kFcFiKDCgBe1KkiRJ6iZNDRYR8TFgb+C+ZrYrSZIkqXs1+jkWNy+inTWBtudcjGtku5IkSZJaq9G3m92ug/UJvAhcD5yRmR0FEEmSJEnLoEbfbrYVczYkSZIktZhBQJIkSVJljb4U6i0iYhAwGJiRmS93Z1uSJEmSWqfhIxYRsVxEfDUiHqKYVzENeDEiHirXd2uYkSRJktR8jb4rVD/gOmBbignbjwNPA6sDawPfAnaOiB0z87VGti1JkiSpdRo9YnEMxZ2h/gBslJlrZ+aWmbk2sAFwFfCRsp4kSZKkHqLRwWJfioff7ZmZD9YWZObDwKeA+4HPNrhdSZIkSS3U6GDxH8C1mTm/XmG5/lpgvQa3K0mSJKmFGh0sXgNW6qTOAOD1BrcrSZIkqYUaHSzuBfaKiFXrFUbEMGAv4K8NbleSJElSCzU6WPwAWBW4KyL+OyLWjYgVImKdiDgA+HNZ/oMGtytJkiSphRp6u9nM/G1EbAp8Ffi/OlUC+E5m/raR7UqSJElqrYY/rC4zvx4RVwL/DXyA8snbwD3AzzPzjka3KUmSJKm1uuUp2Jl5J3Bnd+xbkiRJ0tKnoXMsIuIzEXFzRLyjg/I1IuKmiPhUI9uVJEmS1FqNnrx9IDAkM5+qV5iZTwKDynqSJEmSeohGB4tNgEmd1JkEvK/B7UqSJElqoUYHi6HAc53UeQEY1uB2JUmSJLVQo4PF88D6ndRZH3ipwe1KkiRJaqFGB4vbgN0jYsN6hRGxEbAHcEuD25UkSZLUQo0OFmdQ3ML21og4MiLeHREDyuVRFIFiubKeJEmSpB6i0U/evjsiDgN+CJxVftV6Azg0M//cyHYlSZIktVZ3PHn7vIi4FTgM2AIYQjGn4k7g3Myc2ug2JUmSJLVWdz15eyrwxe7YtyRJkqSlT6PnWEiSJEnqhQwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKmyXhksImK/iMjy68AO6mwVEddExPSImBMR90bE0RGx3CL2+/mIuCsiZkXEjIiYEBG7dd+RSJIkSUuHXhcsImJN4Bxg1iLq7AFMBLYBLgN+CPQDzgIu7mCbM4ALgNWB84BfApsAV0XEEY07AkmSJGnp06uCRUQEcD7wAvDjDuoMoggGbwDbZeZ/Z+ZxwKbAHcBeEbFPu222Ao4FHgbel5lfyszDgRHAdOCMiFi7Ww5KkiRJWgr0qmABHAmMAg4AZndQZy9gVeDizJzUtjIzXwVOKF8e2m6bQ8rltzLzxZptplGMdvQv25QkSZJ6pF4TLCJiI+BU4PuZOXERVUeVy+vqlE0E5gBbRUT/xdzm2nZ1JEmSpB6nb6s70AwR0Rf4BfAv4OudVN+gXP6zfUFmzouIR4GNgXWBqRExAFgDmJWZT9fZ34Pl8t2L0c/JHRRt2Nm2kiRJUiv1imABnAR8APhwZr7SSd3B5XJGB+Vt64csYX1JkiSpx+nxwSIiRlKMUnw3M+9oxC7LZXZxu07rZ+aIug0WIxmbdbE9SZIkqWl69ByLmkug/gmcuJibtY0wDO6gfFC7ep3V72xEQ5IkSVrm9ehgAaxEMbdhI+DVmofiJTCmrHNeue575et/lMuF5kSUQWUdYB7wCEBmzgaeBFaKiNXr9GH9crnQnA1JkiSpp+jpl0LNBX7WQdlmFPMubqUIE22XSd0MfBbYGfhNu222AVYEJmbm3Jr1NwP7lduc326bXWrqSJIkST1Sjw4W5UTtA+uVRcRYimBxYWb+tKboUuA0YJ+IOKftWRYRsTzwzbLOue1292OKYPGNiLi87VkW5UPxDqcIOO0DhyRJktRj9OhgsSQy8+WIOIgiYEyIiIspnp69O8WtaC8FLmm3ze0RcSZwDHBvRFwK9AP2BoYCXywfliepBUYcd1HT2pp8+v5Na0uSpKWJwaKOzLw8IrYFvgF8GlgeeIgiOJydmQvd4Skzj42Ie4EjgIOB+cAU4PTMvLppnZckSZJaoNcGi8wcC4xdRPltwK5d3OeFwIWVOiZJkiQtg3r6XaEkSZIkNYHBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFXW44NFRKwSEQdGxGUR8VBEvBIRMyLi1oj474ioew4iYquIuCYipkfEnIi4NyKOjojlFtHW5yPiroiYVbYxISJ2676jkyRJkpYOPT5YAJ8BzgO2AP4MfA/4PfBe4KfAbyMiajeIiD2AicA2wGXAD4F+wFnAxfUaiYgzgAuA1cv2fglsAlwVEUc0+JgkSZKkpUrfVnegCf4J7A78ITPnt62MiK8DdwGfBj5FETaIiEEUweANYLvMnFSuPxG4GdgrIvbJzItr9rUVcCzwMPDBzHyxXH86MBk4IyKuzsxp3XyskiRJUkv0+BGLzLw5M6+qDRXl+meAH5cvt6sp2gtYFbi4LVSU9V8FTihfHtqumUPK5bfaQkW5zTSK0Y7+wAHVjkSSJElaevWGEYtFeb1czqtZN6pcXlen/kRgDrBVRPTPzLmLsc21wIllnTGL6kxETO6gaMNFbSdJkiS1Wo8fsehIRPQF9i9f1gaCDcrlP9tvk5nzgEcpAtm65X4GAGsAszLz6TpNPVgu392AbkuSJElLpd48YnEqxQTuazLz+pr1g8vljA62a1s/ZAnrdygzR9RbX45kbNbZ9pIkSVKr9MoRi4g4kmKy9d+B/bq6ebnMLm7X1fqSJEnSMqPXBYuIOBz4PvAAsH1mTm9XpW2EYTD1DWpXr7P6nY1oSJIkScu8XhUsIuJo4AfAfRSh4pk61f5RLheaE1HOy1iHYrL3IwCZORt4ElgpIlavs7/1y+VCczYkSZKknqLXBIuI+ArFA+7+QhEqnuug6s3lcuc6ZdsAKwK319wRqrNtdmlXR5IkSepxekWwKB9udyrFw+o+mpnPL6L6pcDzwD4RsXnNPpYHvlm+PLfdNm3Pw/hGRKxcs83awOHAXOD8KscgSZIkLc16/F2hIuLzwDiKJ2nfAhwZEe2rTcvMCwAy8+WIOIgiYEyIiIuB6RRP796gXH9J7caZeXtEnAkcA9wbEZcC/YC9gaHAF33qtiRJknqyHh8sKOZEACwHHN1BnT8BF7S9yMzLI2Jb4BvAp4HlgYcogsPZmbnQHZ4y89iIuBc4AjgYmA9MAU7PzKsbciSSJEnSUqrHB4vMHAuMXYLtbgN27eI2FwIXdrUtSZIkaVnXK+ZYSJIkSepeBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZX1b3QFpWfWvcZs0ra21Tvpb09qSJElaEo5YSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTK+ra6A1q2/GvcJk1ra62T/ta0tqTeYsRxFzWtrcmn79+0tiRJreeIhSRJkqTKHLGQVJkjWZIkyRELSZIkSZUZLCRJkiRVZrCQJEmSVJlzLCRJ3cK5N5LUuzhiIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIq69vqDkhST/KvcZs0ra21Tvpb09qSJKkzjlhIkiRJqswRC0mSulmzRrIcxZLUSo5YSJIkSarMYCFJkiSpMoOFJEmSpMoMFpIkSZIqc/K2JKlXGnHcRU1r67KBTWtKklrGEQtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmVO3m6giHgnMA7YGVgFeBq4HDg5M19sYdckSVoq+BRyqecyWDRIRKwH3A6sBlwB/B0YCRwF7BwRW2fmCy3soiRJktRtDBaN8yOKUHFkZp7TtjIizgS+BHwLOKRFfZMkqUPeeldSIzjHogEiYl1gR2Aa8MN2xWOA2cB+ETGgyV2TJEmSmsIRi8YYVS5vyMz5tQWZOTMibqMIHh8Cbmp25yRJ0uJp7ujN6U1px/kmapbIzFb3YZkXEacDXwa+nJnfrVP+A+Bw4LDMPHcR+5ncQdH7V1hhheU22mijuoVTn2ze1I31+jzbtLb6rf6eLm/juVjAc7GA52IBz8UCPfFcLMl5AM9FLc/FAp6L1njt6Qea0k5H52Lq1Km88sor0zNzla7u02DRABHxf8BBwEGZ+dM65d8Cvg58PTO/vYj9dBQs3gvMorjUqpU2LJd/b2kvlg6eiwU8Fwt4LhbwXCzguVjAc7GA52IBz8UCS8O5WBt4OTPX6eqGXgrVHFEuF5niMnNEE/qyxNqCz9Lez2bwXCzguVjAc7GA52IBz8UCnosFPBcLeC4WWNbPhZO3G2NGuRzcQfmgdvUkSZKkHsVg0Rj/KJfv7qB8/XL5zyb0RZIkSWo6g0VjjC+XO0bEW85pRAwEtgZeAe5sdsckSZKkZjBYNEBmPgzcQDHZ5fB2xScDA4CLMnN2k7smSZIkNYWTtxvnMOB24OyI+CgwFdgC2J7iEqhvtLBvkiRJUrfydrMNFBFrAuOAnYFVgKeBy4GTM3N6C7smSZIkdSuDhSRJkqTKnGMhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEmSpMoMFupURLwzIn4eEU9FxNyImBYR34uIlVvdt2aKiL0i4pyIuCUiXo6IjIhftrpfzRYRq0TEgRFxWUQ8FBGvRMSMiLg1Iv47InrV75WIOC0iboqIx8tzMT0i7omIMRGxSqv710oRsV/5/yQj4sBW96eZyt+T2cHXM63uXytExEci4vcR8XT5t+TpiLghInZtdd+aISJGL+Jnou3rjVb3s5ki4uPlz8AT5e/PRyLidxGxZav71ixR+EJE3BkRMyNiTvk35MiIWK7V/esqn2OhRYqI9SieKL4acAXwd2AkxRPF/wFsnZkvtK6HzRMRfwHeD8wCngA2BH6VmZ9rZb+aLSIOAc6leADkeOBfwNuBTwGDgd8Dn8le8sslIl4DpgAPAM8BA4APAZsDTwEfyszHW9fD1igfGPo3YDlgJeCgzPxpa3vVPBExDRgCfK9O8azMPKOZ/Wm1iDgBOAV4Hria4vfHMOADwPjMPL6F3WuKiNgU2LOD4o8Ao4A/ZOZuzepTK0XEacDxwAsUDxN+HvgPYHegL7B/Zvb4D+8i4iJgP4q/H1cBs4GPAe9hGfx7arDQIkXE9cCOwJGZeU7N+jOBLwE/ycxDWtW/ZoqI7SkCxUPAthRvqntjsBhF8eb5D5k5v2b9cOAuYE1gr8z8fYu62FQRsXxmvlpn/beArwPnZuZhze9Z60REAH8E1gH+H/BlemewIDPXbm1PWi8iPgP8FrgR+FRmzmxX/rbMfL0lnVtKRMQdFB9I7JGZV7a6P92t/HvxJPBv4H2Z+VxN2fbAzcCjmblui7rYFBGxJ3AZ8CgwMjOfL9e/jeL/zJ7AAZl5QYu62GW96pIFdU1ErEsRKqYBP2xXPIYiVe8XEQOa3LWWyMzxmfngsvTJQXfIzJsz86raUFGufwb4cflyu6Z3rEXqhYrSb8vl+s3qy1LkSIpPXw+g+D2hXqq8NPI0YA6wb/tQAWCoiPdShIongT+0uDvN8i6K96B/rg0VUPytBWYCq7aiY032qXL53bZQAW/+nzixfPnFpveqgr6t7oCWaqPK5Q113kTOjIjbKILHh4Cbmt05LZXa3iDMa2kvlg6fKJf3trQXTRYRGwGnAt/PzInlCFdv1T8iPgesRRGw7gUmZmZvuo5+K4qRq0uBFyPi48B7gVeBuzLzjlZ2binxP+XyZ73oZ+NB4DVgZEQMq31THRHbAAMpLo/q6YaXy0fqlLWt2ywihmTmS83pUjUGCy3KBuXynx2UP0gRLN6NwaLXi4i+wP7ly+ta2ZdWiIgvU8wlGEwxv+LDFG8kT21lv5qp/Bn4BcW8m6+3uDtLg+EU56PWoxFxQGb+qRUdaoEPlstnKeYibVJbGBETKS6d/HezO7Y0iIgVgM8B84Fec6lgZk6PiK8AZwIPRMTlFHMt1qOYY/FHFgSunqwtUK1Tp6z2MrANgTu7vzvVeSmUFmVwuZzRQXnb+iHd3xUtA06l+CTymsy8vtWdaYEvU1wieDRFqLgO2LGXvWE6iWIy7ujMfKXVnWmx84GPUoSLARRvqH8CrA1cGxHvb13Xmmq1cnkIsALFpNSBFL8rrge2AX7Xmq4tFf6T4m/otb3tJg+Z+T2KS4H6AgcBXwU+AzwOXND+Eqke6upyeUxEDG1bWX5Ic3JNvWXmLpwGC1UR5bJXzzkQRMSRwLEUdw3br8XdaYnMHJ6ZQfFG8lMUnzbdExGbtbZnzRERIylGKb7r5S2QmSeX85Gezcw5mXlfeaOLMyneYI9tbQ+bpu12mUExMnFTZs7KzPuBT1LcEGPb3nR70XYOLpc/aWkvWiAijqe4RO4CipGKAcAIikuAfhUR32ld75rmYuBaiuN/ICL+LyK+B/wF2JXiyhCAZeYSOYOFFqVtRGJwB+WD2tVTLxQRhwPfp7jd6vaZOb3FXWqp8o3kZRSXCa4CXNTiLnW7mkug/smCCYeqr+0GB9u0tBfN82K5fCQz/1pbUI5qtY1ujmxqr5YCEfEeijkoTwDXtLg7TRUR21FM6r8yM4/JzEfKAD6FInA+CRxb3kSmxyrnr+5OMeL9DMUHc1+g+Jn4MMXlYVDcinaZYLDQovyjXL67g/K2u910NAdDPVxEHA38ALiPIlT0ygd/1ZOZj1GErY0jYlir+9PNVqL4PbER8GrtA78oLg8DOK9c971WdXIp0fYGoVfcTY8Ff0de6qC8LXis0P1dWer0xknbbdqe1TG+fUFmzqG4dXkfiksre7TMnJeZ383MTTNzhcwclJk7U/z92BR4Bbi/pZ3sAidva1Ha/sPvGBF92j2zYCCwNcUP/DIxoUiNVU68O5ViyHaH2rt66E3vKJc9/U3DXOBnHZRtRvHm4FaKN5m9/TKptkt+6t0FpieaSHGXuPUjol9mvtau/L3lclpTe9ViEbE8xafT8+n4/05P1r9cdnRL2bb17X9eepP9gOWBC5elWzI7YqEOZebDwA0Ukw0Pb1d8MsUnbhdlpvep72Ui4kSKUDEZ+GhvDRURsWH5oKf26/uUD8hbDbg9M19ceOueIzNfycwD630BbQ/7urBcd0kr+9oMEbFx7UTMmvXvohjhA+jxTxQGKH83XEJxSe1JtWURsQOwE8XltL3tTnKfoZiQe01vm7RduqVcHhwRa9QWRMQuFB9cvgrc3uyONVtEDKqz7oMUf2NnAeOa3qkKHLFQZw6j+I99dkR8FJgKbAFsT3EJ1Dda2LemKp+QuWf5su3N5JYRcUH57+cz88tN7lbTRcTnKX7RvUHxx+HI4kHLbzFtWXpSaAU7A6eXt8x8mOJ62LdTPJl9XYprZg9qXffUIp8BvhoR4ymeqDuTYnLmxyk+gbwGOKN13Wu6Yyj+bnyjfEbBXRQPSPskxe+Rg5aVe/Q3UNuk7f9raS9a51KKJ7F/DJgaEZdR/L7ciOIyqQC+mpkvdLyLHuOPEfEKxSXFM4GNKSZuz6V4Uv0yNboZvfwhwloMEbEmxRvJnSkmoz5N8eCak3vTRN2IGMuC68XreSwz125Ob1pnMc4DwJ8yc7vu701rlU/MPZTi07V3Utw2cjZF6P4DcHZv+j9ST83Py0GZ2Svu0x8R21LcXvUDLLjd7EsUlw3+AvhF9rI/vuUIzgkUYWINijdQtwLfzsxedTlt+RDJBygm6K7dC+dXABARb6O4GmIf4D3AisB0iuB5dmbe0MLuNU1EHEdxDtajmGv0FMVNDU7NzGkt7NoSMVhIkiRJqsw5FpIkSZIqM1hIkiRJqsxgIUmSJKkyg4UkSZKkygwWkiRJkiozWEiSJEmqzGAhSZIkqTKDhSRJkqTKDBaSJEmSKjNYSJIkSarMYCFJkiSpMoOFJEndKCLWjoiMiAta3ZfuFhFjy2Pdrt36jIgJLemUpKYxWEjSMsQ3aJKkpVXfVndAkqQe7klgI2BGqzsiSd3JYCFJUjfKzNeBv7e6H5LU3bwUSpIWU0SMjIhLIuLJiJgbEU9HxA0R8Z81dbYrL1ca28E+pkXEtHbr+kXEkRExJSJejIg5Zb0rIuJjZZ3REZHlJtuWbWS9tiLiPyNiYkTMiIhXIuJvEfG1iOjfUX8iYqWIOCsiHi+3+UtE7FnW6RsRX4+IByPi1Yh4OCKOWMR52ikiromI58vz9HBEnB4RQxbR/qCIOLP89+sdnb922+4eETeV34e5EfFURPwpIg6rU3doRHw7IqaWxzej3HbHOnVHl+d1dETsHBETyvoZEWtExBsRMWUR/bqurPve8nWHcywiYsWI+EpETIqImRExq+zj2RHx9jp1v1Z+b2aXde+IiP/q7FzVafedZRtt39PpEXFXRJzYrt72EfF/EfFARLxcnrv7ImJMRCzf1Xbb7XtgRJxY7u/l8vgfjuL/2Igq+5bUGo5YSNJiiIiDgHOBN4ArgQeB1YDNgcOA31bY/QXAfwH3ARcBrwDvAD4M7AzcCPwFOBkYAzxWbtNmQk0//xf4GvA88GtgFrAL8L/AThGxQ/kJeq23AX8EhgJXAP3K/vy+fON9GLAFcC0wF/gMcE5E/DszL6ndUUScVPZzOnA18BzwPuDLwK4RsWVmvtyu/X7AzWX7NwAvA48u6oRFxMHAT4BngKvK412tbOsA4Ec1dd9VnqO1gVuA64ABwG7AdRHxP5l5Xp1m9qI4/9cCPwbWzswnI+JGYMeI2CQz/9auX6sDHwMmZ+Z9nRzDysB44P3AP4CfA68B6wFfAP4f8GxZd0h5jj4ATCnr9gF2An4dERtn5gmLaq+m3c2B6ynO98SynRWB9wBjgVNqqn8F2BC4HfgDsDywdVlvu4j4WGa+sTjttutDUHwftgLuAH4KzAPWBLaj+D5N7up+JbVYZvrll19++bWIL4o3XK9TvFneuE75O2v+vR2QwNgO9jUNmFbzejAwH5gELFen/irtXicwoYN9b1mW/wsYXrO+L8Wb7wS+Xqc/WZb3r1n/kXL9dOBuYEhN2boUb4Dvabev7cttbq+tX5aNLsvO6qD9G4EBXfieTKYIOavVKRvW7vWE8hzv0279EIrA9grw9jp9nQ/sXGf//1WWn1Gn7Liy7Is169Yu113Qru6vy/XnAn3alQ0EBte8vqCse3y7estTvEGfD2y6GOetH0VoS2DfOuVrtnu9LhB16p1S7mPvduvHluu3W9TPLbBJue6yOvvuA6y8uD8Lfvnl19Lz5aVQktS5QynenJ+Smfe3L8zMJyrsO4GgeJM8v86+X+jCvr5QLr+Zmc/U7GMecGy5/wM72PbozJxbs80tFG9AVwa+kpkv1ZQ9AtwGbBIRy9Xs48hyeVBt/XKbCyjexH+2g/aPzczZizi2euZRBL63yMzn2/4dEe8HtgV+n5kXt6v3EsUI0PLAp+vs/4rMvK7O+sspJmJ/tt3xA3y+7NNvFtXxiFgN2Bt4GvhyZr7le5+ZMzNzRll3FeBzwKTM/E67eq9SjCoEsO+i2ix9giLoXJmZv25fmJmPt3v9SGZm+3rA98rlTovR5qK8UqcP8zPzxYr7ldQCXgolSZ37ULm8ttE7zsyXI+Iqijd8f4mI31NcBvLnzJzTxd1tVi5vrtPOPyPiCWCdiBjS7o3/S5n5cJ39PQWsQ/1LUp4ElgOGl/+GYsTkdeAzEfGZOtv0A1aNiFXaBaZXgXs7Pqy6fgV8F7g/Ii4B/gTclpn/bldvy3I5uIN5G6uWy43qlN1Vr+HMfCUifgscRPHG+hqAcl7AxhSfwj9fb9saH6T4ZH7iYgSqD1Kc647m7rxtEcfQXpd+liNiAHAU8Eng3RQjKVFTZY3F2U8dD1AEzf8qL1W7AriVIjy9toT7lNRiBgtJ6tyQcvnkoipVsDfFp877UsxPAHg1Ii6l+DT72cXcz+By+XQH5U8Da5X1XqpZ39FtUOcBtH1yXq+MBW9qAVah+LsyppN+rgTUBovnOvhUvEOZeWZEPE8x/+NI4GiKN95/Ao7LzEk1fQLYofxaVJ/ae6bOujYXUASLz1MGi/LfABd21n+69jPVdgwfLL86Uu8YlrjdiHgbRUgdSTH/5xLg3ywYJRoDLHRDgMWRmW9ExCjgJIq5LKeVRTMj4kLga5k5a0n2Lal1vBRKkjr3UrlcnE9n2y5p6eiDm8HtV2TmK5k5NjPfTfHG/3MUn95+Dri0C/1sCwDDOyhfvV29RpsBvJiZ0cnXY+2261KoeHOjzIsy80MUb7w/DvwM2Aa4vrzUqK1PAEd10qcD6jWxiLZvp5jAv0dEDCnfhP8XxSTyazrarsZL5XJxfqbajuGsTo5h+wa3uwdFqLgwMzfJzIMz8xuZOZZi4nwlmfliZn4pM9cE1qe4TO/vwBEU804kLWMMFpLUuTvL5S6LUbft2vA12xdExH+w4BPjujLz8cz8FcUlNg8CHy6vsW8zn+KymHruKZfbddD2O4FH289/aKA7gZUjYuNu2n9dmflSZl6TmQdRjCQMpZh83tYnal430oUUn9jvTRFshgG/zoXvulXPXRTfy23Ky40Wp24jjqErP8v/US5/X6ds2wb05U2Z+VBm/qzc7yyKUCNpGWOwkKTOnUtx6c+JEfGe9oUR8c6al3+nuF3qHjWfmhMRKwBn19l21YjYok6bAyiuZ59HcQemNi9QJ7SUfl4uT4iItrkDlBOMz6D4nf+zDrZthLPK5XkR8Y72hRExICI+1H79kiifL1FvVKjtnM8BKC+JugX4VER8oU59ImKT2u9VF1xE8YZ///IL3nob4A6Vc0EuphhFOiMi3vL3OIrnigwu6z5HMadk8/K5Dwsdd0SsFxHrLEbTV1HciWv3es+/iIjakYxp5XK7dnXWZcGlS0skItbpIICuTBHWFprULWnp5xwLSepEZj4QxUPXfgzcExFXUIwmrELxHIuZFLdaJTNfj4jvAyeWdS+j+F27A8Vk6Kfa7X4N4M6ImErxfILHgUEUz1gYDpydmTNr6t8E7FNO+J5METwmZubEzLw9Ir4DHA/cV87RmE3x6fR7KS6vOr2Bp+YtMvOmiPgq8G3gwYi4huLOUisB76L4NPpWimdDVHUxxTyUWyneAAfFJ/ofpDgvN9bU3ZdirsDPIuJI4M8UlwS9k+K5F++lmOT9XFc6kJmPR8R44KMU34e/ZeY9nWxW64iy7UMonglxPUWIXIdixGp3Fjyj5AiKy4XGAfuVx/0sxfNONiqP+7/o5PkfmflaObH+BornX/wPxSjG8uV+PsqC9wZXAQ8Bx0TEJhQjYmtR/Gz+ofz3kno/cFlETKaYv/EUxUT6PSjm7VQKLpJaw2AhSYshM8+LiPsoHvS2HbAnxfX091I83KvWGIpPzA8CDqaYBHwxxT3+H2hXd1pZfzuKcDKM4tkR/wC+Wm5X6yiKa/8/CuxKMQpxMsWDzsjMr0TEPRRvRPeneJP2MHAC8N3uvuNOZp4WEbdRTKj+MMUbxRkUk4X/j+LZDY3wVYo335tRnIdXKR4c+BXg3NrLkTLzifKOTV+kuK3sZykuJ3uG4vtxDvCWB911wQUseDO+OJO235SZL0bEVhQTz/em+Fl5gyJc/pyan5Xy7mHblnX2LY9jeYpw8SDwJYqHHC5Ou5MiYlOKc7gLxUPqZlKEiDE19WaXE6xPpfj5/AjwCMUzLM4s+7ykJlEE0G0pgubKFBPDJ1OE6YbfgU1S94su3ohDkiRJkhbiHAtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmVGSwkSZIkVWawkCRJklSZwUKSJElSZQYLSZIkSZUZLCRJkiRVZrCQJEmSVJnBQpIkSVJlBgtJkiRJlRksJEmSJFVmsJAkSZJUmcFCkiRJUmUGC0mSJEmV/X9DvZadxcNsXAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 261,
       "width": 395
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(x='customer service calls', hue='churn', data=df);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "f07e73f5b67964229579f87c626b7a22d3b4e622"
   },
   "source": [
    "Although it's not so obvious from the summary table, it's easy to see from the above plot that the churn rate increases sharply from 4 customer service calls and above.\n",
    "\n",
    "Now let's add a binary feature to our DataFrame – `Customer service calls > 3`. And once again, let's see how it relates to churn. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "_uuid": "37b91ac20b481d221d97515aa46c9cda713a6991"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>churn</th>\n",
       "      <th>False</th>\n",
       "      <th>True</th>\n",
       "      <th>All</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>many_service_calls</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2721</td>\n",
       "      <td>345</td>\n",
       "      <td>3066</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>129</td>\n",
       "      <td>138</td>\n",
       "      <td>267</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>All</th>\n",
       "      <td>2850</td>\n",
       "      <td>483</td>\n",
       "      <td>3333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "churn               False  True   All\n",
       "many_service_calls                   \n",
       "0                    2721   345  3066\n",
       "1                     129   138   267\n",
       "All                  2850   483  3333"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['many_service_calls'] = (df['customer service calls'] > 3).astype('int')\n",
    "\n",
    "pd.crosstab(df['many_service_calls'], df['churn'], margins=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "_uuid": "58463ad14986d044a406debf885fd25f43fee525"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxYAAAIMCAYAAABohDruAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAABYlAAAWJQFJUiTwAAA7JUlEQVR4nO3de5xXVb3/8dcHEVTiIpJiJIIeb3lIEu+UF8q7paUePfYrscw075rmOamDWOd4O1pqFw+VWFZqdo54TUtBMEMFzBsc76PiBQMEBBTFWb8/9h798uU7zGV/h+/M8Ho+Ht/HZtZea6+1Z3zgvFl77RUpJSRJkiSpiG61HoAkSZKkzs9gIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIK617rAah5EfEi0Aeor/FQJEmS1LUNARallIa2tqHBonPos+666/bfZptt+td6IJIkSeq6Zs2axTvvvNOmtgaLzqF+m2226T99+vRaj0OSJEld2IgRI5gxY0Z9W9q6xkKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBXmPhaSJElruIaGBubPn8/bb7/NsmXLSCnVekiqgoigZ8+e9O7dm/79+9OtW/vOKRgsJEmS1mANDQ288sorLF26tNZDUZWllHj33Xd59913WbJkCZtsskm7hguDhSRJ0hps/vz5LF26lO7duzNw4EB69erV7v+yrdWjoaGBJUuW8MYbb7B06VLmz5/PgAED2q0//6uRJElag7399tsADBw4kN69exsqupBu3brRu3dvBg4cCHz0s263/tr16pIkSerQli1bBkCvXr1qPBK1l8afbePPur0YLCRJktZgjQu1nanouiICoN0X5ftfkCRJktSFNQaL9mawkCRJklSYwUKSJElSYQYLSZIkdRr19fVEBKNHj671UFTGYCFJkiSpMDfIU6uMOOvXtR6COqDpl3691kOQJEk15oyFJEmSpMIMFpIkSeowHn74YY444ggGDRpEz5492Xjjjdlnn3246aabVqpbX1/PkUceyYABA1hnnXXYYYcduP3221eqN2bMGCKCSZMmVbxGpTUbo0ePJiJ44YUXuOqqq/j0pz/Nuuuuy5577rnC+fr6eq655hqGDRvGOuusw0YbbcRxxx3HwoULq/Ht6FR8FEqSJEkdwrhx4zjhhBNYa621+NKXvsQWW2zBm2++ybRp0/jpT3/Kv/zLv3xY96WXXmKnnXZis80242tf+xrz58/nxhtv5OCDD+Yvf/kLe+21V1XGdOqppzJlyhQOPPBADjjgANZaa60Vzp999tncfffdfPGLX2SfffZh4sSJjBs3jueee4777ruvKmPoLAwWkiRJqrmZM2fyne98hz59+jBlyhS23XbbFc7Pnj17ha8nTZrEmDFjqKur+7DsqKOOYr/99uPSSy+tWrCYMWMGjz76KEOHDq14furUqTzxxBMMHjwYgOXLlzNq1CgmTpzIww8/zE477VSVcXQGPgolSZKkmvvZz37G8uXLOe+881YKFQCf/OQnV/h600035dxzz12hbN9992Xw4ME8/PDDVRvX2Wef3WSoADj//PM/DBUA3bt355hjjgGo6jg6A4OFJEmSam7q1KkA7L///i2qP3z48JUeSwLYZJNNeOutt6o2ruZmHHbYYYeKYwCqOo7OwGAhSZKkmluwYAEAgwYNalH9fv36VSzv3r07DQ0NVRoVDBw4sNXj6N49W23wwQcfVG0cnYHBQpIkSTXX+Av6q6++WvVrd+uW/cq7fPnylc41BpqmRETVx9NVGSwkSZJUc7vssgsAd911V9Wvvf766wPwyiuvrHRu2rRpVe9vTWWwkCRJUs2dcMIJdO/enQsvvJCZM2eudL78rVCt0bhO4tprr11h1uKVV15h7Nixbb6uVuTrZiVJklRzn/rUp/jpT3/K8ccfz2c+8xkOPvhgtthiC+bNm8e0adPo3bs3EydObNO1d955Z3bffXcmT57MTjvtxKhRo5gzZw633XYb++67b8WZDLWeMxaSJEnqEL71rW/xwAMPcNBBBzFp0iQuvfRSbr31VgYMGMCJJ55Y6NoTJkzg2GOPZfbs2Vx11VU8+uijXHLJJVx88cVVGr0ipVTrMagZETF9++2333769Om1Hgojzvp1rYegDmj6pV+v9RAkSW00a9YsALbZZpsaj0TtqaU/5xEjRjBjxowZKaURre3DGQtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSpHZSX19PRDB69OhaD6Xdda/1AFojIjYAvgwcCAwDBgHvAU8A1wLXppQaSuoPAV5cxSVvTCkd2URfRwMnAp8CPgAeBS5LKd3eRP11gXOAI4FNgUXAJKAupTSrxTcpSZLUwYw469e1HsIqTb/061W5TkSs8vy11167RgSEtupUwQI4HPgZ8DowEXgZ2Aj4CvALYP+IODyllMraPQbcUuF6T1bqJCIuA84EZgPjgB5kgeG2iDg5pXR1Wf2ewJ+BkcA04MfAJvl4D4yIUSmlh1p9t5IkSVrt6urqKpYPHz589Q6kk+lsweIZ4EvAHWUzE/8OPAwcShYy/ljW7u8ppTEt6SAidiMLFc8DO6aU3srLLwWmA5dFxO0ppfqSZmeQhYqbgSMaxxYRN5IFml9FxLDSMUuSJKljGjNmTK2H0Cl1qjUWKaX7Ukq3lf+CnlJ6A/h5/uWeBbs5Pj/+sDFU5H3UAz8BegLHNJZHNmfW2Obs0rGllCYAU8gep9qj4LgkSZJUY6+99hpjx45l5MiRDBw4kB49evCJT3yCo446ilmzWv70+5w5c/jud7/LVlttRa9evejXrx9bbbUVo0eP5oUXXlip/t13380BBxzAgAED6NmzJ5tvvjlnnXUWCxYsqOLdFdOpgkUz3s+Pyyuc+0REfDsi/j0/fnoV1xmVH/9U4dxdZXUANgcGA8+klCqt56jURpIkSZ3Q5MmTueiii+jXrx+HHnoop59+Orvssgs333wzO+64I4899liz11i6dCkjR47kv/7rv9h000054YQT+OY3v8mwYcOYMGECM2fOXKH+2LFj2W+//XjooYc48MADOeWUU/inf/onLrvsMkaOHMmiRYva63ZbpbM9ClVRRHQHGlftVAoEe+ef0jaTgKNTSi+XlPUiWxC+OKX0eoXrPJsftywp2yo/PtPE8Cq1qSgipjdxauvm2kqSJKk6Kj0KNWTIEEaPHs2oUaOYM2cOvXv3XuH8Y489xsiRIznnnHO46667Vmpf6t577+X555/ntNNO44orrljh3HvvvceyZcs+/HrixInU1dWx6667cuedd9KvX78Pz40fP55jjjmGurq6la5TC10iWAAXAf8M3JlSurukfClwIdk6h8Y5pU8DY4C9gHsjYnhKaUl+rm9+XNhEP43l/UrK2tJGkiRJHdQFF1ywUtkee+zB6NGj2XDDDSu22W677Rg1ahT33HMP77//PmuvvXaz/ay77rorlfXo0YMePXp8+PWVV14JwLhx41YIFQCjR4/mxz/+Mb/97W8NFtUQEaeQLbb+P+BrpedSSm8C55c1mRwR+wAPADsDx5K9xak1yt86tcohtrRNSmlExQtkMxnbt6JPSZIktdHKLxhd0R133MHPf/5zpk2bxty5c1m+fMUn8efOncvGG2/cZPs99tiDQYMGcdFFFzFjxgwOOOAARo4cyfDhw1lrrbVWqPu3v/2Ntddemz/84Q/84Q9/WOla7733Hv/4xz+YN28eG2ywQSvusvo6dbCIiBPJQsFM4PMppfktaZdSWh4RvyALFrvzUbBonF3oW7Fh5dmJ5tr0qdBGkiRJndCVV17Jqaeeyvrrr8/ee+/N4MGDWW+99YgIbrnlFh577LEVHmWqpE+fPkydOpW6ujpuvfVW7r47e+BmwIABfOc73+Hcc8/9cMZj3rx5LF++vOIsSqnFixcbLNoqIk4DriDbi+Lz+exEa/wjP/ZqLEgpLYmIV4FBEbFxhXUWW+TH0vUUT+fHptZQVGojSZKkTmb58uXU1dUxcOBAZsyYsdKsxN/+9rcWX+uTn/wkv/zlL0kpMXPmTO677z5+8pOfMHbsWBoaGrjwwgsB6Nu3Lw0NDcyf36J/P6+pTvlWqIj4Hlmo+DuwVxtCBcAu+bH8fV735cf9KrTZv6wOZPtdvAxsGRFDW9hGkiRJnczcuXNZsGABu+2220qhYvHixcyYMaPV14wItt12W04++WT+/Oc/A3DLLbd8eH6XXXbhrbfe4qmnnio09tWh0wWLiDiPbLH2dLKZirmrqLtzRPSoUD4KOD3/8vqy0437YXw/ItYvaTMEOBFYBlzbWJ7v8t3Y5pKI6FbS5mDgc2SPat3fkvuTJElSx7Thhhuy3nrrMX36dBYvXvxh+fvvv8+pp57K3LlN/lq6gieffJL6+vqVyufMmQPAeuut92HZ6adnv7J+61vf4rXXXlupzZIlS5g6dWprbqPddKpHoSLiaGAs8AHZxnOnZPvTraA+pTQ+//PFwLb5q2Vn52Wf5qM9Jc5LKT1Y2jil9GBEXE62m/bjEXEz0AM4AugPnFy26zbA5cBBwGHAQxFxL9neFoeTvZnqG+66LUmS1Ll169aNU045hYsuuohhw4Zx8MEH89577zFx4kTmz5/PXnvtxcSJE5u9zl/+8hfOOOMMdtttN7beems23HBDZs+ezYQJE+jWrRtnnXXWh3U///nPc9FFF/Fv//ZvbLHFFhxwwAEMHTqUxYsX89JLL3H//ffz2c9+lj/9qdKOC6tXpwoWQOOjRmsBpzVR535gfP7n3wBfBnYkeyRpbWAOcBNwdUppSqULpJTOjIjHgZOA44AGYAZwaUrp9gr1l0XEF4BzgKPIZkMWkb3mti6lNLO8jSRJkjqfCy+8kI9//OP84he/4JprrqFv377svffe/OAHP6Curq5F19h333057bTTmDx5MhMmTGDRokVsvPHG7L333h8GjlLf+973GDlyJFdeeSUPPPAAEyZMoG/fvgwaNIjjjjuOo446qj1utdWiuddpqfYiYvr222+//fTpTe2ft/qMOOvXtR6COqDpl369+UqSpA5p1qxZAGyzzTY1HonaU0t/ziNGjGDGjBkzmtoGYVU63RoLSZIkSR2PwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYd1rPQBJkiR1bC+PHVbrIazS4POfKNQ+IlpV/9prr2X06NGF+uyKDBaSJElao9XV1a1U9qMf/YiFCxdy6qmn0q9fvxXODR8+fPUMrJMxWEiSJGmNNmbMmJXKxo8fz8KFCznttNMYMmTIah9TZ+QaC0mSJKmF9txzTyKC9957j7Fjx7LVVlvRs2fPDx+NGjNmDBHBpEmTVmpbX19PRFR8jGrp0qX853/+J8OHD6dXr1587GMfY9ddd+X3v/99+95QFTljIUmSJLXSoYceyiOPPML+++/PIYccwoYbbtjmay1YsIBRo0bx6KOPsv322/ONb3yDhoYG7r77bo466iieeuopfvCDH1Rx9O3DYCFJkiS10ksvvcSTTz7JgAEDCl/rtNNO49FHH+Xiiy/m7LPP/rD83Xff5ZBDDuE//uM/OOywwzr82g4fhZIkSZJa6cILL6xKqJg3bx7XX389O+ywwwqhAmCdddbh4osvJqXE7373u8J9tTdnLCRJkqRW2mmnnapynUceeYQPPviAiKi4iPz9998HYNasWVXprz0ZLCRJkqRWGjhwYFWuM2/ePCALGI888kiT9RYvXlyV/tqTj0JJkiRJrdTUpnrdumW/Xi9fvnylcwsWLFiprG/fvgCcfvrppJSa/EycOLF6g28nBgtJkiSpStZff30AXnnllZXOTZs2baWynXbaiW7dujFlypR2H1t7M1hIkiRJVdK49uLaa69dYdbilVdeYezYsSvV33DDDfnqV7/KtGnTuPDCCyvOdDz//PO8+OKL7TfoKnGNhSRJklQlO++8M7vvvjuTJ09mp512YtSoUcyZM4fbbruNfffdt+JMxtVXX82zzz7L+eefz29+8xs++9nPstFGG/Haa68xa9YsHnnkEX7/+98zdOjQGtxRyzljIUmSJFXRhAkTOPbYY5k9ezZXXXUVjz76KJdccgkXX3xxxfp9+vTh/vvv56qrrmLAgAH88Y9/5PLLL2fixIn07t2bK664gr333ns130XrOWMhSZKkVRp8/hO1HsJqV19fX7F80qRJzbbt168f48aNY9y4cSudSylVbNOjRw9OOukkTjrppNYMs0NxxkKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSpC6sqVfcVpvBQpIkaQ0WEQA0NDTUeCRqL43BovFn3V4MFpIkSWuwnj17ArBkyZIaj0TtpfFn2/izbi8GC0mSpDVY7969AXjjjTd4++23aWhoWG2Pzqj9pJRoaGjg7bff5o033gA++lm3l+7tenVJkiR1aP3792fJkiUsXbqU2bNn13o4aifrrbce/fv3b9c+DBaSJElrsG7durHJJpswf/583n77bZYtW+aMRRcREfTs2ZPevXvTv39/unVr34eVDBaSJElruG7dujFgwAAGDBhQ66GoE3ONhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKqxTBYuI2CAijo2I/42I5yLinYhYGBEPRMQ3I6Li/UTEbhFxZ0TMj4ilEfF4RJwWEWutoq+jI+LhiFic9zEpIg5aRf11I+KCiHg6It6NiDcj4qaI2KYa9y5JkiR1ZJ0qWACHA+OAnYGHgB8BfwT+GfgFcFNERGmDiDgYmAzsDvwv8BOgB3AFcEOlTiLiMmA8sHHe3/XAMOC2iDipQv2ewJ+B84FFwI+BvwBfBqZFxM5tv2VJkiSp4+te6wG00jPAl4A7UkoNjYUR8e/Aw8ChwFfIwgYR0YcsGHwA7JlSmpaXnwfcBxwWEUemlG4oudZuwJnA88COKaW38vJLgenAZRFxe0qpvmRcZwAjgZuBIxrHFhE3ArcAv4qIYaVjliRJkrqSTjVjkVK6L6V0W/kv6CmlN4Cf51/uWXLqMODjwA2NoSKv/y5wbv7lCWXdHJ8ff9gYKvI29WSzHT2BYxrL8xmSxjZnl44tpTQBmAJ8CtijxTcqSZIkdTKdKlg04/38uLykbFR+/FOF+pOBpcBu+aNMLWlzV1kdgM2BwcAzKaUXW9hGkiRJ6lI626NQFUVEd+Dr+ZelgWCr/PhMeZuU0vKIeBHYFtgMmBURvYBBwOKU0usVuno2P27Zkj5W0aaiiJjexKmtm2srSZIk1VJXmbG4iGwB950ppbtLyvvmx4VNtGss79fG+m1tI0mSJHUpnX7GIiJOIVts/X/A11rbPD+mVrZrTf0W95FSGlHxAtlMxvat6FOSJElarTr1jEVEnEj2ateZwF4ppfllVRpnC/pSWZ+yes3VrzQ70do+JEmSpC6n0waLiDgNuBp4kixUvFGh2tP5caX1Dfm6jKFki71fAEgpLQFeBT4WERtXuN4W+bF0PUWTfayijSRJktSldMpgERHfI9vg7u9koeLNJqrelx/3q3Bud2A94MGU0rIWttm/rA5k+128DGwZEUNb2EaSJEnqUjpdsMg3t7uIbLO6z6eU5q6i+s3AXODIiNih5BrrAD/Iv/xZWZvG/TC+HxHrl7QZApwILAOubSxPKaWSNpdERLeSNgcDnyN7VOv+Ft6iJEmS1Ol0qsXbEXE0MJZsJ+0pwCnZ/nQrqE8pjQdIKS2KiG+RBYxJEXEDMJ9s9+6t8vIbSxunlB6MiMvJdtN+PCJuBnoARwD9gZPLdt0GuBw4iGxDvoci4l6yvS0OJ9sr4xvuui1JkqSurFMFC7I1EQBrAac1Ued+YHzjFymlWyJiD+D7wKHAOsBzZMHhynzGYQUppTMj4nHgJOA4oAGYAVyaUrq9Qv1lEfEF4BzgKOB0YBFwC1CXUprZ2huVJEmSOpNOFSxSSmOAMW1o91fggFa2uQ64rhX13wHq8o8kSZK0Rul0aywkSZIkdTwGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYVUNFhExOCL6NFOnd0QMrma/kiRJkmqr2jMWLwKnNlPnlLyeJEmSpC6i2sEi8o8kSZKkNUgt1lhsBCypQb+SJEmS2kn3oheIiK+XFQ2vUAawFjAY+BrwRNF+JUmSJHUchYMFMB5I+Z8TcHD+Kdf4iNRS4IIq9CtJkiSpg6hGsDgmPwbwK+AWYEKFeh8A84C/pZQWVKFfSZIkSR1E4WCRUrqu8c8RcTRwS0rp10WvK0mSJKnzqOri7ZTSXu0dKiLisIi4KiKmRMSiiEgRcX0TdYfk55v63LCKfo6OiIcjYnFELIyISRFx0CrqrxsRF0TE0xHxbkS8GRE3RcQ21bhvSZIkqSOrxqNQq9u5wHbAYmA2sHUL2jxG9ohWuScrVY6Iy4Az8+uPA3oARwK3RcTJKaWry+r3BP4MjASmAT8GNgEOBw6MiFEppYdaME5JkiSpU6p6sIiIPYCzgJ2A9ak8K5JSSm3t+3SyX/ifA/YAJragzd9TSmNacvGI2I0sVDwP7JhSeisvvxSYDlwWEbenlOpLmp1BFipuBo5IKTXkbW4kCzS/iohhjeWSJElSV1PVYBERB5L9Ir0W8DLwNLC8mn2klD4MEhHtshff8fnxh42hIu+3PiJ+ApxHtmC9Lh9DlLQ5uzQ8pJQmRMQU4HO0PARJkiRJnU61ZyzGAO8DB6aU7qnytYv4RER8G9iAj95M9XgTdUflxz9VOHcXWbAYRR4sgM3J9ud4JqX0YhNtPpe3MVhIkiSpS6p2sPhn4IYOFioA9s4/H4qIScDRKaWXS8p6AYOAxSml1ytc59n8uGVJ2Vb58Zkm+q7UpqKImN7EqZasI5EkSZJqpqpvhSJbUD2/ytcsYilwITCCbL3H+nz0SNKewL15mGjUNz8ubOJ6jeX9CraRJEmSupRqz1jcC+xa5Wu2WUrpTeD8suLJEbEP8ACwM3As2VucWnXpVtRtXAjSbJuU0oiKF8hmMrZvRZ+SJEnSalXtGYvvAZtHxLnRTiurqyGltBz4Rf7l7iWnGmcX+lJZpdmJ5tr0qdBGkiRJ6lKqPWNRBzwFXAB8IyL+DiyoUC+llL5Z5b5b6x/58cNHoVJKSyLiVWBQRGxcYZ3FFvmxdD3F0/mxqTUUldpIkiRJXUq1g8Xokj8PyT+VJKDWwWKX/PhCWfl9wNeA/YBry87tX1Kn0fNkr9bdMiKGVngzVKU2kiRJUpdS7Uehhrbws1mV+60oInaOiB4VykeRbbQHcH3Z6Z/nx+9HxPolbYYAJwLLKAkcKaVU0uaSiOhW0uZgslfNzgTuL3QzkiRJUgdW1RmLlNJL1bxeJRFxCHBI/uXA/LhrRIzP/zw3pfTd/M8XA9vmr5adnZd9mo/2qjgvpfRg6fVTSg9GxOVku2k/HhE3Az2AI4D+wMllu24DXA4cBBwGPBQR95LtbXE42ZupvuGu25IkSerKqv0o1OowHDi6rGwzPpoFeQloDBa/Ab4M7Ej2SNLawBzgJuDqlNKUSh2klM6MiMeBk4DjgAZgBnBpSun2CvWXRcQXgHOAo8hmQxaR7UJel1Ka2ZYblSRJkjqLqgaLiBjc0rqlG9O1RkppDNkO3y2p+0vgl23s5zrgulbUf4ds8Xpdc3UlSZKkrqbaMxb1tGyPh9QOfUuSJEmqkWr/cv9rKgeLfmSPMG0KTCJ7XEmSJElSF1HtxdujmzqXvy3pPOB4Vl4jIUmSJKkTq/brZpuUUmpIKV1A9rjURaurX0mSJEntb7UFixIPAvvUoF9JkiRJ7aQWwaI/0KsG/UqSJElqJ6s1WOR7PRwBPLk6+5UkSZLUvqq9j8V9q+hnE7LdqAHGVrNfSZIkSbVV7dfN7tlEeQLeAu4GLkspNRVAJEmSJHVC1X7dbC3WbEiSJEmqMYOAJEmSpMKq/SjUCiKiD9AXWJhSWtSefUmSJEmqnarPWETEWhFxTkQ8R7auoh54KyKey8vbNcxIkiRJWv2q/VaoHsCfgD3IFmy/ArwObAwMAX4I7BcR+6SU3qtm35IkSZJqp9ozFmeQvRnqDmCblNKQlNKuKaUhwFbAbcDn8nqSJEmSuohqB4ujyDa/OySl9GzpiZTS88BXgKeAr1a5X0mSJEk1VO1g8U/AXSmlhkon8/K7gM2r3K8kSZKkGqp2sHgP+FgzdXoB71e5X0mSJEk1VO1g8ThwWER8vNLJiBgAHAY8VuV+JUmSJNVQtYPF1cDHgYcj4psRsVlErBsRQyPiGOCh/PzVVe5XkiRJUg1V9XWzKaWbImI4cA7w3xWqBHBJSummavYrSZIkqbaqvlldSunfI+JW4JvAZ8h33gYeBX6VUvpbtfuUJEmSVFvtsgt2SmkqMLU9ri1JkiSp46nqGouIODwi7ouITzRxflBE3BsRX6lmv5IkSZJqq9qLt48F+qWUXqt0MqX0KtAnrydJkiSpi6h2sBgGTGumzjTg01XuV5IkSVINVTtY9AfebKbOPGBAlfuVJEmSVEPVDhZzgS2aqbMFsKDK/UqSJEmqoWoHi78CX4qIrSudjIhtgIOBKVXuV5IkSVINVTtYXEb2CtsHIuKUiNgyInrlx1PJAsVaeT1JkiRJXUS1d95+JCK+A/wEuCL/lPoAOCGl9FA1+5UkSZJUW+2x8/a4iHgA+A6wM9CPbE3FVOBnKaVZ1e5TkiRJUm21187bs4CT2+PakiRJkjqeaq+xkCRJkrQGMlhIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgrrdMEiIg6LiKsiYkpELIqIFBHXN9Nmt4i4MyLmR8TSiHg8Ik6LiLVW0eboiHg4IhZHxMKImBQRB62i/roRcUFEPB0R70bEmxFxU0RsU+R+JUmSpM6g0wUL4FzgJGA48GpzlSPiYGAysDvwv8BPgB7AFcANTbS5DBgPbAyMA64HhgG3RcRJFer3BP4MnA8sAn4M/AX4MjAtInZuxf1JkiRJnU73Wg+gDU4HZgPPAXsAE5uqGBF9yILBB8CeKaVpefl5wH3AYRFxZErphpI2uwFnAs8DO6aU3srLLwWmA5dFxO0ppfqSrs4ARgI3A0eklBryNjcCtwC/iohhjeWSJElSV9PpZixSShNTSs+mlFILqh8GfBy4oTFU5Nd4l2zmA+CEsjbH58cfNoaKvE092WxHT+CYxvKIiJI2Z5eGh5TSBGAK8CmyECRJkiR1SZ0uWLTSqPz4pwrnJgNLgd3yR5la0uausjoAmwODgWdSSi+2sE1FETG90gfYurm2kiRJUi119WCxVX58pvxESmk58CLZ42CbAUREL2AQsDil9HqF6z2bH7dsSR+raCNJkiR1KZ1xjUVr9M2PC5s431jer43129qmopTSiErl+azF9s21lyRJkmqlq89YNCfyY0vWa5RqTf229iFJkiR1Gl09WDTOFvRt4nyfsnrN1a80O9HaPiRJkqQup6sHi6fz40rrGyKiOzAUWA68AJBSWkK2N8bHImLjCtfbIj+Wrqdoso9VtJEkSZK6lK4eLO7Lj/tVOLc7sB7wYEppWQvb7F9WB7L9Ll4GtoyIoS1sI0mSJHUpXT1Y3AzMBY6MiB0aCyNiHeAH+Zc/K2vz8/z4/YhYv6TNEOBEYBlwbWN5vp9GY5tLIqJbSZuDgc8BM4H7q3A/kiRJUofU6d4KFRGHAIfkXw7Mj7tGxPj8z3NTSt8FSCktiohvkQWMSRFxAzAf+BLZa2JvBm4svX5K6cGIuJxsN+3HI+JmoAdwBNAfOLls122Ay4GDyDbkeygi7iXb2+Jwsr0yvuGu25IkSerKOl2wAIYDR5eVbZZ/AF4Cvtt4IqV0S0TsAXwfOBRYB3iOLDhcWWkH75TSmRHxOHAScBzQAMwALk0p3V6h/rKI+AJwDnAUcDqwCLgFqEspzWzrzUqSJEmdQacLFimlMcCYVrb5K3BAK9tcB1zXivrvAHX5R5IkSVqjdPU1FpIkSZJWA4OFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpsDUiWEREfUSkJj5vNNFmt4i4MyLmR8TSiHg8Ik6LiLVW0c/REfFwRCyOiIURMSkiDmq/O5MkSZI6hu61HsBqtBD4UYXyxeUFEXEw8EfgXeBGYD7wReAKYCRweIU2lwFnArOBcUAP4Ejgtog4OaV0dVXuQpIkSeqA1qRgsSClNKa5ShHRhywYfADsmVKalpefB9wHHBYRR6aUbihpsxtZqHge2DGl9FZefikwHbgsIm5PKdVX95YkSZKkjmGNeBSqlQ4DPg7c0BgqAFJK7wLn5l+eUNbm+Pz4w8ZQkbepB34C9ASOaa8BS5IkSbW2Js1Y9IyI/wcMBpYAjwOTU0oflNUblR//VOEak4GlwG4R0TOltKwFbe4Czsvr1K1qgBExvYlTW6+qnSRJklRra1KwGAj8pqzsxYg4JqV0f0nZVvnxmfILpJSWR8SLwLbAZsCsiOgFDAIWp5Rer9Dvs/lxy0KjlyRJkjqwNSVYXAtMAZ4C3iYLBScBxwF3RcSuKaXH8rp98+PCJq7VWN6vjfWblFIaUak8n8nYvrn2kiRJUq2sEcEipXRBWdGTwPERsZhs0fUY4MstvFw0Xra1w2hlfUmSJKnTWNMXb/88P+5eUtY4w9CXyvqU1WuufnMzGpIkSVKnt6YHizfzY6+Ssqfz40prIiKiOzAUWA68AJBSWgK8CnwsIjau0McW+XGlNRuSJElSV7GmB4td8+MLJWX35cf9KtTfHVgPeLDkjVDNtdm/rI4kSZLU5XT5YBER20ZE/wrlmwKNu2FfX3LqZmAucGRE7FBSfx3gB/mXPyu7XOMjVd+PiPVL2gwBTgSWkS0glyRJkrqkNWHx9uHAORExEXiR7K1QmwMHAusAdwKXNVZOKS2KiG+RBYxJEXEDMB/4EtmraG8GbiztIKX0YERcDpwBPB4RNwM9gCOA/sDJ7rotSZKkrmxNCBYTyQLBZ8gefeoFLAAeINvX4jcppRXe2JRSuiUi9gC+DxxKFkCeIwsOV5bXz9ucGRGP89FrbBuAGcClKaXb2+fWJEmSpI6hyweLfPO7+5utuHK7vwIHtLLNdcB1re1LkiRJ6uy6/BoLSZIkSe3PYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpsO61HoCkzu/lscNqPQR1QIPPf6LWQ5AkrUbOWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMK613oAkiRJa6KXxw6r9RDUAQ0+/4laD6HNDBaSJLWzEWf9utZDUAf0v71rPQKpunwUSpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBosqiohPRsSvIuK1iFgWEfUR8aOIWL/WY5MkSZLaU/daD6CriIjNgQeBDYEJwP8BOwGnAvtFxMiU0rwaDlGSJElqN85YVM9PyULFKSmlQ1JK56SURgFXAFsBP6zp6CRJkqR2ZLCogojYDNgHqAd+Una6DlgCfC0ieq3moUmSJEmrhcGiOkblx3tSSg2lJ1JKbwN/BdYDdlndA5MkSZJWB9dYVMdW+fGZJs4/SzajsSVwb1MXiYjpTZzabtasWYwYMaLtI6ySWa+6TEQrO7DbnFoPQR1Qjwm1/zuro/DvTlXi352qpNZ/d86aNQtgSFvaGiyqo29+XNjE+cbyfm28/gfvvPPOwhkzZtS3sb3UnrZ+Mjv+X22HoQ7n9Rm1HoHUkfl3pyqr/d+dQ4BFbWlosFg9Ij+mVVVKKfnPe+p0Gmfa/O9XklrOvzvVFbnGojoaZyT6NnG+T1k9SZIkqUsxWFTH0/lxyybOb5Efm1qDIUmSJHVqBovqmJgf94mIFb6nEdEbGAm8A0xd3QOTJEmSVgeDRRWklJ4H7iFb7HJi2ekLgF7Ar1NKS1bz0CRJkqTVwsXb1fMd4EHgyoj4PDAL2BnYi+wRqO/XcGySJElSu4qUVvmiIrVCRGwCjAX2AzYAXgduAS5IKc2v4dAkSZKkdmWwkCRJklSYaywkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQlKbRMQnI+JXEfFaRCyLiPqI+FFErF/rsUlSRxMRh0XEVRExJSIWRUSKiOtrPS6pmtx5W1KrRcTmZDvNbwhMAP4P2Ak4FdgvIkamlObVcIiS1NGcC2wHLAZmA1vXdjhS9TljIaktfkoWKk5JKR2SUjonpTQKuALYCvhhTUcnSR3P6cCWQB/ghBqPRWoX7rwtqVUiYjPgeaAe2Dyl1FByrjfwOhDAhimlJTUZpCR1YBGxJzAR+G1K6f/VdjRS9ThjIam1RuXHe0pDBUBK6W3gr8B6wC6re2CSJKl2DBaSWmur/PhME+efzY9broaxSJKkDsJgIam1+ubHhU2cbyzv1/5DkSRJHYXBQlK1RX50AZckSWsQg4Wk1mqckejbxPk+ZfUkSdIawGAhqbWezo9NraHYIj82tQZDkiR1QQYLSa01MT/uExEr/B2Sv252JPAOMHV1D0ySJNWOwUJSq6SUngfuAYYAJ5advgDoBfzaPSwkSVqzuEGepFaLiM2BB8l2354AzAJ2BvYiewRqt5TSvNqNUJI6log4BDgk/3IgsC/wAjAlL5ubUvru6h+ZVD0GC0ltEhGbAGOB/YANyHbcvgW4IKU0v4ZDk6QOJyLGAHWrqPJSSmnI6hmN1D4MFpIkSZIKc42FJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJUpmIGB0RKSJG13osq1t+35PKysbk5XvWZFCSOgWDhSRJkqTCutd6AJIkdUD/C0wFXq/1QCSpszBYSJJUJqW0EFhY63FIUmfio1CS1M4iYkj+fPr4iNg8Im6OiHkR8XZE3BMR/5zX+3hE/HdEvB4R70bEIxGxV9m1PhER50fEXyPijYh4LyJei4jfRcQ2zfQ9JCJuiIi5+fWnRcRBZfWPz+uf38S9DIyI9yPiiTZ8H3pExCkRMSMi3oqIpRFRHxETIuILFepvnY/7lYhYFhFz8vvcqkLd8fm4N4uIkyPi8Yh4JyImRcS/5ucub2JcPfPxvBER3fOyJtdYRMQnI+LKiHg2/z7Oj4iHI+K8JupeHREv5PcwLyJujYgdW/v9q3DtIyLi3rz/d/Pv5e8jYoeSOn0j4qyIuC8iZuf/vfwjH8MuVRjD5yLitvzay/Lv4dSIqCt6bUmdj8FCklafIcBDwEbAeOAe4AvApIjYguzRmx2BG4GbgO2AuyJicMk1dgfOARYAfwSuyNsdBjwSEds10femwMP5GH6T9/HPwISy8HI9sAg4NiLWqnCdb5DNdl/T4rv+yHjgx8DawK+BK4HJwDBgv9KKEbEfMAP4KvBI3u5e4CvAwxGxfRN9/Bi4EHgi//NfyR5rWgh8tTE4lDkY6Adcn1JavqobyH9pfww4GXgt7+O3wNvAmLK62wN/B74DPA1cBdxG9jN8ICIOWFVfqxhDRMR44Abg08D/kP13MAX4HFAaFrcBfgg0AHcAlwN/BkYBU/Lvc5vkbScBnyX72fwXcAuwjOyeJa1pUkp+/Pjx46cdP2S/zKf88/2yc+fl5fOBnwPdSs59LT93RUnZhkDvCn1sBywG7lpF33Vl5/bNy+8sK786Lz+orDyAF4AlQN9Wfg/6kv1yOw1Yq8L5DUr+vD7wFjAX+FRZvW3z+5xRVj4+H/OrwNAK17+m0j3l5+7Izw0rKRudl40uKesBvJiXH1XhOpuU/Lk78BzwLrBHWb1P5ON8HejZhv+ejsvH8HD5zwFYC9i47Ps+oMI1PkkWjGZVOJeASWVlY/LyPUvK/piXbVfhGiv16cePn67/ccZCklafeuCisrLr8mNP4KyUUkPJud8By4HhjQUppTdTSm+XXzil9BhwH7BXRKxdoe+XgB+UtbkbeBnYqazuz/Ljt8vK9wGGAjembA1CaySyYLKMLGCUj39eyZdfJ5tBqEspzSyr9xQwDvhMRHyqQj+XpJRerFDe+H0+urQwIgaSBaxHU0rNPd71RbKgdmtK6XcV7uGVki8PBDYHrkop3V9W7zXgEmAg8Plm+qzk5Pz47fKfQ0rpg5TS6yVfL0wpza0w1tnAzcDWZTNibfFOheuv1Kekrs/F25K0+vw9pfRBWdlr+fGZ8sCQUvogIuaQ/evyhyLiQOB4YAdgACv/XT6Ald9mVKlvgFeAXcv6fSoiJgP7R8QmJb8wH5cff17x7lYhpbQoIm4j++X87xHxR7JHdx5KKS0tq944nu0iYkyFy22ZH7cBZpade7iJ/h+MiGeAL0bE+imlt/JTXyX7V/7xLbiNxjUJd7WgbuM9bNrEPWyRH7cB7mzB9QCIiF5kj7DNSSk92sI2I4FT8zFtSDbzUmoQWcBsrd+SPZr2UETcCEwE/pqHFklrIIOFJK0+K/0rf0ppeURUPJdbTrYmAYCIOIXsuf63yJ6VfxlYSjYjcAjZI1E9K1xnwSquX2n2+qdkawGOBeryf9n/EllAqfjLewscAXwPOAq4IC97NyJuBr6bUpqTl22QH7/VzPU+VqHsjVXUv45svcGRfDQrczTwPvD7ZvqCbBYFsseYmtN4D4c3U6/SPVRrDETEl8lmJt4l++/lebJH2RqAPYE9qPzfS7NSSv+TL/4/k2ztzbfzPqcD/5ZS+nNbriup8zJYSFInkS88voDsl+ftSx95yc/vWrFh2/wPMAf4ZkSMpdiibQBSSu+QPas/JiI2IQsuo4H/R/aI0efyqo0ha7uU0uOt7WYV535DtrD7aOBnEfEZsoXjE1JK/2jBtRfkx0EtqNt4DwenlG5tQf2Was0YILvf94AdUkqzSk9ExDVkwaLNUkp3AHfkMyk7ky0cPwG4PSI+U/4om6SuzTUWktR5DCD7F+sHK4SKjwFNvSmp1VJK7wO/IPsF9otkMxeLyR5/qcb1X0kp/ZZsfcOzwGcjovFf+afmx89VbFygT7J1KDvnr6xtXG9xXdOtVtA4rv1bUbfa97AEeBLYKA9GzfknYGaFUNGN7G1OVRtXSum+lNIZwH+QPW7Vku+TpC7EYCFJncebZI89jciDBAD5Yu0fkwWPavpv4AOyt0QNBX5XaeF4S0S2R8fOFU71AnqTPZL1Xl52Ldm/zNdFRPnCciKiW0Ts2ZZx8NFaim8C/wrMA25vYdvbyBbgfyki/rXCuEpnESaQPXZ0YlOvlY2IXSNivRb2XerK/HhNRPQtu2a3iNi4pKge2CIiPlFSJ4A6oNLi9xaLiM9HxLoVTm2UH8vXzkjq4nwUSpI6iZRSQ0RcSbaPxRMRMYHsX4b3AvqTLZ7daxWXaG1/L0fEHWRrK6DAY1BkMx9TI2IW2f4UrwB9yB6dGQhc2RhaUkrzIuIwsv0npkbEvcBTZOsCBpMtQt4AWKcN4/gfsn06TiNbu3JVPjvTrJTSexFxONn+I7+LiG+TzUysQ7YI+/Pk/19NKb0fEV8B7iZ7VOhBsj0tlgKbkO1XshmwMa3/BfwXZLMNXweezf87+AfZa2xHAb/ioz01riBbbP9ovmD+fWAkWahoXEzfVv8FDImISWQB5j1gRD6Gl8j22ZC0BjFYSFLnch7ZL5HHki2WXUi2KPdcPloQXU2/IgsW01JKMwpcp57sX8n3JAs/A8j27niaLCit8EtoSuneiPg08F2yx6U+R/aL62tkjzP9sS2DSCm9ExF/IJuxgJY/BtXYflpEDM/HvD+wG9nmeM+R3V9p3cfzDQvPIAtQx5CFo9eBR/P6rX4ta0opAUdHxD1kb+r6F7IF2K+TvWnr1pK610TEMrIgdTTZq2Gn5GM5lGLB4j+AL5O9newLZPf2cl7+o5I3b0laQ0T295MkSSvLX5VaBxybUvpljYcjSerADBaSpIoiojfZwuq1yXaV9pl5SVKTfBRKkrSCfAO+7ckek9mIbI8JQ4UkaZUMFpKkcoeTPY8/B/hPsgXAK8nfzLRnC663IKX0o+oMrevx+yipq/BRKElSm5Ssv2jOSymlIe07ms7L76OkrsJgIUmSJKkwN8iTJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVNj/B6GR/ifZgsTLAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 262,
       "width": 395
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(x='many_service_calls', hue='churn', data=df);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "_uuid": "303b7f45cfd75c98ddfb31c686a8e023c123ad45"
   },
   "source": [
    "\n",
    "Let's construct another contingency table that relates *Churn* with both *International plan* and freshly created *Many_service_calls*.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "_uuid": "4679d8985f7b32943cd990a1dc4e88f41edca039",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "churn  False  True   All\n",
      "row_0                   \n",
      "False   2841   464  3305\n",
      "True       9    19    28\n",
      "All     2850   483  3333\n",
      "precision= 0.3392857142857143\n",
      "recall= 0.019668737060041408\n",
      "accuracy= 0.2145214521452145\n"
     ]
    }
   ],
   "source": [
    "crss_arr=pd.crosstab(df['many_service_calls'] & (df['international plan'] == 'yes') , df['churn'], margins=True)\n",
    "\n",
    "print(crss_arr); crss_arr = crss_arr.values\n",
    "precision = crss_arr[1,1]/np.sum(crss_arr[1,:])\n",
    "recall = crss_arr[1,1]/np.sum(crss_arr[:,1])\n",
    "accuracy = np.sum(crss_arr[0,0]+crss_arr[1,1])/np.sum(crss_arr)\n",
    "\n",
    "print(\"precision=\",precision); print(\"recall=\",recall); print(\"accuracy=\",accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Therefore, predicting that a customer is not loyal (*Churn*=1) in the case when the number of calls to the service center is greater than 3 and the *International Plan* is added (and predicting *Churn*=0 otherwise), we might expect an accuracy of 85.8% (we are mistaken only 464 + 9 times). This number, 85.8%, that we got through this very simple reasoning serves as a good starting point (*baseline*) for the further machine learning models that we will build. \n",
    "\n",
    "Recall that, before the advent of machine learning, the data analysis process looked something like this. Let's recap what we've covered:\n",
    "    \n",
    "- The share of loyal clients in the sample is 85.5%. The most naive model that always predicts a \"loyal customer\" on such data will guess right in about 85.5% of all cases. That is, the proportion of correct answers (*accuracy*) of subsequent models should be no less than this number, and will hopefully be significantly higher;\n",
    "- With the help of a simple forecast that can be expressed by the following formula: \"International plan = True & Customer Service calls > 3 => Churn = 1, else Churn = 0\", we can expect a guessing rate of 85.8%, which is just above 85.5%. \n",
    "- We got these two baselines without applying machine learning, and they'll serve as the starting point for our subsequent models. \n",
    "\n",
    "This notebook is adaption from the following kaggle notebook https://www.kaggle.com/kashnitsky/topic-1-exploratory-data-analysis-with-pandas. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "#The below are my code executions to answer some of the questions "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "51\n",
      "['KS' 'OH' 'NJ' 'OK' 'AL' 'MA' 'MO' 'LA' 'WV' 'IN' 'RI' 'IA' 'MT' 'NY'\n",
      " 'ID' 'VT' 'VA' 'TX' 'FL' 'CO' 'AZ' 'SC' 'NE' 'WY' 'HI' 'IL' 'NH' 'GA'\n",
      " 'AK' 'MD' 'AR' 'WI' 'OR' 'MI' 'DE' 'UT' 'CA' 'MN' 'SD' 'NC' 'WA' 'NM'\n",
      " 'NV' 'DC' 'KY' 'ME' 'MS' 'TN' 'PA' 'CT' 'ND']\n"
     ]
    }
   ],
   "source": [
    "lis=pd.unique(df['state'])\n",
    "n=len(lis)\n",
    "print(n)\n",
    "print(lis)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"6\" halign=\"left\">customer service calls</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>50%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>churn</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>False</th>\n",
       "      <td>2850.0</td>\n",
       "      <td>1.45</td>\n",
       "      <td>1.16</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>True</th>\n",
       "      <td>483.0</td>\n",
       "      <td>2.23</td>\n",
       "      <td>1.85</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      customer service calls                           \n",
       "                       count  mean   std  min  50%  max\n",
       "churn                                                  \n",
       "False                 2850.0  1.45  1.16  0.0  1.0  8.0\n",
       "True                   483.0  2.23  1.85  0.0  2.0  9.0"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "columns_to_show = ['customer service calls']\n",
    "\n",
    "df.groupby(['churn'])[columns_to_show].describe(percentiles=[])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    3333.00\n",
       "mean       30.56\n",
       "std         9.26\n",
       "min         0.00\n",
       "25%        24.43\n",
       "50%        30.50\n",
       "75%        36.79\n",
       "max        59.64\n",
       "Name: total day charge, dtype: float64"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['total day charge'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "account length             1.87\n",
      "area code                  0.74\n",
      "number vmail messages     -3.49\n",
      "total day minutes         31.74\n",
      "total day calls            1.05\n",
      "total day charge           5.40\n",
      "total eve minutes         13.37\n",
      "total eve calls            0.52\n",
      "total eve charge           1.14\n",
      "total night minutes        5.10\n",
      "total night calls          0.34\n",
      "total night charge         0.23\n",
      "total intl minutes         0.54\n",
      "total intl calls          -0.37\n",
      "total intl charge          0.15\n",
      "customer service calls     0.78\n",
      "churn                      1.00\n",
      "many_service_calls         0.24\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "a=df[df['churn'] == True].mean(numeric_only=True)\n",
    "b=df[df['churn'] == False].mean(numeric_only=True)\n",
    "print(a-b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "churn               False  True   All\n",
      "international plan                   \n",
      "False                 186   137   323\n",
      "True                 2664   346  3010\n",
      "All                  2850   483  3333\n",
      "precision= 0.4673684210526316\n",
      "recall= 0.4425249169435216\n",
      "accuracy= 0.2100960096009601\n"
     ]
    }
   ],
   "source": [
    "crss_arr=pd.crosstab(df['international plan'] == 'no' , df['churn'], margins=True)\n",
    "\n",
    "print(crss_arr); crss_arr = crss_arr.values\n",
    "precision = crss_arr[1,0]/np.sum(crss_arr[:,0])\n",
    "recall = crss_arr[1,0]/np.sum(crss_arr[1,:])\n",
    "accuracy = np.sum(crss_arr[1,0]+crss_arr[0,1])/np.sum(crss_arr)\n",
    "\n",
    "print(\"precision=\",precision); print(\"recall=\",recall); print(\"accuracy=\",accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "churn               False  True   All\n",
      "international plan                   \n",
      "False                 186   137   323\n",
      "True                 2664   346  3010\n",
      "All                  2850   483  3333\n",
      "precision= 0.05747508305647841\n",
      "recall= 0.3581780538302277\n",
      "accuracy= 0.039903990399039906\n",
      "------------------------------------\n",
      "churn               False  True   All\n",
      "international plan                   \n",
      "False                2664   346  3010\n",
      "True                  186   137   323\n",
      "All                  2850   483  3333\n",
      "precision= 0.4425249169435216\n",
      "recall= 0.4673684210526316\n",
      "accuracy= 0.2100960096009601\n"
     ]
    }
   ],
   "source": [
    "crss_arr=pd.crosstab(df['international plan'] == 'no' , df['churn'], margins=True)\n",
    "\n",
    "print(crss_arr); crss_arr = crss_arr.values\n",
    "precision = crss_arr[1,1]/np.sum(crss_arr[1,:])\n",
    "recall = crss_arr[1,1]/np.sum(crss_arr[:,1])\n",
    "accuracy = np.sum(crss_arr[0,0]+crss_arr[1,1])/np.sum(crss_arr)\n",
    "\n",
    "print(\"precision=\",precision); print(\"recall=\",recall); print(\"accuracy=\",accuracy)\n",
    "print(\"------------------------------------\")\n",
    "crss_arr=pd.crosstab(df['international plan'] == 'yes' , df['churn'], margins=True)\n",
    "\n",
    "print(crss_arr); crss_arr = crss_arr.values\n",
    "precision = crss_arr[0,0]/np.sum(crss_arr[0,:])\n",
    "recall = crss_arr[0,0]/np.sum(crss_arr[:,0])\n",
    "accuracy = np.sum(crss_arr[0,0]+crss_arr[1,1])/np.sum(crss_arr)\n",
    "\n",
    "print(\"precision=\",precision); print(\"recall=\",recall); print(\"accuracy=\",accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "churn                   False  True   All\n",
      "customer service calls                   \n",
      "False                    2245   391  2636\n",
      "True                      605    92   697\n",
      "All                      2850   483  3333\n",
      "precision= 0.06599713055954089\n",
      "recall= 0.09523809523809523\n",
      "accuracy= 0.1752925292529253\n"
     ]
    }
   ],
   "source": [
    "crss_arr=pd.crosstab(df['customer service calls'] == 0 , df['churn'], margins=True)\n",
    "\n",
    "print(crss_arr); crss_arr = crss_arr.values\n",
    "precision = crss_arr[1,1]/np.sum(crss_arr[1,:])\n",
    "recall = crss_arr[1,1]/np.sum(crss_arr[:,1])\n",
    "accuracy = np.sum(crss_arr[0,0]+crss_arr[1,1])/np.sum(crss_arr)\n",
    "\n",
    "print(\"precision=\",precision); print(\"recall=\",recall); print(\"accuracy=\",accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxYAAAILCAYAAAB1gQpWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAABYlAAAWJQFJUiTwAAA25UlEQVR4nO3de5hWZb3/8fcXEVTkIBJipoH+PGUmieKBUqE8ZmJbTX/2K7HU7VnUNHepINreKl6ap7RNJZa71OzaoqaZCYhnBDyksj2PiscQQQFFce7fH2sN+2F4hplhPTPPPMP7dV1zLeZe9+kJr2k+3Oted6SUkCRJkqQiulR7ApIkSZJqn8FCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBXWtdoTUPMi4hWgF1BX5alIkiSpcxsIfJBSGtTahgaL2tBr7bXX7rv11lv3rfZEJEmS1HnNnj2bjz76aJXaGixqQ93WW2/dd+bMmdWehyRJkjqxIUOGMGvWrLpVaeseC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjnWEiSJK3m6uvrmTdvHh9++CFLliwhpVTtKakCIoLu3bvTs2dP+vbtS5cubbumYLCQJElajdXX1/P666+zePHiak9FFZZS4uOPP+bjjz9m0aJFbLzxxm0aLgwWkiRJq7F58+axePFiunbtyoABA+jRo0eb/8u22kd9fT2LFi3i7bffZvHixcybN49+/fq12Xj+VyNJkrQa+/DDDwEYMGAAPXv2NFR0Il26dKFnz54MGDAA+N+/6zYbr017lyRJUoe2ZMkSAHr06FHlmaitNPzdNvxdtxWDhSRJ0mqsYaO2KxWdV0QAtPmmfP8LkiRJkjqxhmDR1gwWkiRJkgozWEiSJEkqzGAhSZKkmlFXV0dEMGrUqGpPRY0YLCRJkiQV5gF5apUhZ/yu2lNQBzRz/A+qPQVJklRlrlhIkiRJKsxgIUmSpA5j+vTpHHrooWy00UZ0796dDTfckL322oubb755hbp1dXUcdthh9OvXj7XWWosddtiBO+64Y4V6Y8eOJSKYOnVq2T7K7dkYNWoUEcHLL7/MlVdeyVe+8hXWXntt9thjj+Xu19XV8atf/Yptt92WtdZaiw022IBjjjmGBQsWVOJ/jprio1CSJEnqECZMmMBxxx3HGmuswQEHHMDmm2/Ou+++y4wZM/jlL3/Jd7/73WV1X331VYYOHcqmm27K97//febNm8dNN93EyJEj+fvf/87w4cMrMqdTTjmF+++/n29961vst99+rLHGGsvdP/PMM7n77rv59re/zV577cWUKVOYMGECL774IpMnT67IHGqFwUKSJElV9+yzz3L88cfTq1cv7r//frbZZpvl7s+ZM2e576dOncrYsWMZM2bMsrLDDz+cffbZh/Hjx1csWMyaNYvHH3+cQYMGlb3/yCOP8I9//INNNtkEgKVLlzJixAimTJnC9OnTGTp0aEXmUQt8FEqSJElVd80117B06VLOOeecFUIFwBe+8IXlvv/iF7/I2WefvVzZ3nvvzSabbML06dMrNq8zzzyzyVABcO655y4LFQBdu3blyCOPBKjoPGqBwUKSJElV98gjjwCw7777tqj+4MGDV3gsCWDjjTfm/fffr9i8mltx2GGHHcrOAajoPGqBwUKSJElVN3/+fAA22mijFtXv06dP2fKuXbtSX19foVnBgAEDWj2Prl2z3QafffZZxeZRCwwWkiRJqrqGX9DfeOONivfdpUv2K+/SpUtXuNcQaJoSERWfT2dlsJAkSVLV7bzzzgDcddddFe97vfXWA+D1119f4d6MGTMqPt7qymAhSZKkqjvuuOPo2rUr559/Ps8+++wK9xu/Fao1GvZJXHfddcutWrz++uuMGzdulfvV8nzdrCRJkqruS1/6Er/85S859thj+epXv8rIkSPZfPPNee+995gxYwY9e/ZkypQpq9T3TjvtxG677ca0adMYOnQoI0aM4J133uH2229n7733LruSodZzxUKSJEkdwtFHH80DDzzA/vvvz9SpUxk/fjy33XYb/fr144QTTijU96RJkzjqqKOYM2cOV155JY8//jgXX3wxF110UYVmr0gpVXsOakZEzNx+++23nzlzZrWnwpAzflftKagDmjn+B9WegiRpFc2ePRuArbfeusozUVtq6d/zkCFDmDVr1qyU0pDWjuGKhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJElSG6mrqyMiGDVqVLWn0ua6VnsCkiRJ6tiGnPG7ak9hpWaO/0FF+omIld6/7rrrVouAsKoMFpIkSVKJMWPGlC0fPHhw+06kxhgsJEmSpBJjx46t9hRqknssJEmSpBZ68803GTduHMOGDWPAgAF069aNz3/+8xx++OHMnj27xf288847/PjHP2bLLbekR48e9OnThy233JJRo0bx8ssvr1D/7rvvZr/99qNfv350796dzTbbjDPOOIP58+dX8NMV44qFJEmS1ELTpk3jwgsvZPjw4Rx00EGsu+66vPDCC9xyyy3cdtttPPjgg2y33XYr7WPx4sUMGzaMl156iT333JNvf/vbpJR49dVXmTRpEgcffDCbbrrpsvrjxo1jzJgx9O3bl/3335/+/fvz1FNPcckll3DnnXfy8MMP06tXr7b+6M0yWEiSJEklyj0KNXDgQEaNGsWIESN455136Nmz53L3n3zySYYNG8ZZZ53FXXfdtdL+7733Xl566SVGjx7NZZddtty9Tz75hCVLliz7fsqUKYwZM4ZddtmFO++8kz59+iy7N3HiRI488kjGjBmzQj/VYLCQJEmSSpx33nkrlO2+++6MGjWK/v37l22z3XbbMWLECP72t7/x6aefsuaaazY7ztprr71CWbdu3ejWrduy76+44goAJkyYsFyoABg1ahSXX345//Vf/2WwkCRJkjqalNJK7//lL3/h2muvZcaMGcydO5elS5cud3/u3LlsuOGGTbbffffd2WijjbjwwguZNWsW++23H8OGDWPw4MGsscYay9V9+OGHWXPNNfnTn/7En/70pxX6+uSTT/jnP//Je++9x/rrr9+KT1l5BgtJkiSpha644gpOOeUU1ltvPfbcc0822WQT1llnHSKCW2+9lSeffHK5R5nK6dWrF4888ghjxozhtttu4+677wagX79+HH/88Zx99tnLVjzee+89li5dWnYVpdTChQsNFpIkSVItWLp0KWPGjGHAgAHMmjVrhVWJhx9+uMV9feELX+A3v/kNKSWeffZZJk+ezNVXX824ceOor6/n/PPPB6B3797U19czb968in6WtuDrZiVJkqQWmDt3LvPnz2fXXXddIVQsXLiQWbNmtbrPiGCbbbbhpJNO4p577gHg1ltvXXZ/55135v333+eZZ54pNPf2YLCQJEmSWqB///6ss846zJw5k4ULFy4r//TTTznllFOYO3dui/p5+umnqaurW6H8nXfeAWCdddZZVnbqqacCcPTRR/Pmm2+u0GbRokU88sgjrfkYbcZHoSRJkqQW6NKlCyeffDIXXngh2267LSNHjuSTTz5hypQpzJs3j+HDhzNlypRm+/n73//Oaaedxq677spWW21F//79mTNnDpMmTaJLly6cccYZy+p+4xvf4MILL+Tf/u3f2Hzzzdlvv/0YNGgQCxcu5NVXX+W+++7ja1/7Gn/961/b8qO3iMFCkiRJaqHzzz+fz33uc/z617/mV7/6Fb1792bPPffkggsuYMyYMS3qY++992b06NFMmzaNSZMm8cEHH7Dhhhuy5557LgscpX7yk58wbNgwrrjiCh544AEmTZpE79692WijjTjmmGM4/PDD2+Kjtlo09zotVV9EzNx+++23nzlzZrWnwpAzflftKagDmjn+B9WegiRpFc2ePRuArbfeusozUVtq6d/zkCFDmDVr1qyU0pDWjuEeC0mSJEmFGSwkSZIkFVZTwSIi1o+IoyLivyPixYj4KCIWRMQDEfGjiOjSqP7AiEgr+bpxJWMdERHTI2JhPsbUiNh/JfXXjojzIuK5iPg4It6NiJsjwnVFSZIkdXq1tnn7EOAa4C1gCvAasAHwL8CvgX0j4pC04saRJ4Fby/T3dLlBIuIS4HRgDjAB6AYcBtweESellK5qVL87cA8wDJgBXA5snM/3WxExIqX0aKs/rSRJklQjai1YPA8cAPwlpVTfUBgRPwWmAweRhYw/N2r3REppbEsGiIhdyULFS8COKaX38/LxwEzgkoi4I6VUV9LsNLJQcQtwaMPcIuImskDz24jYtnTOkiRJUmdSU49CpZQmp5Rub/wLekrpbeDa/Ns9Cg5zbH79eUOoyMeoA64GugNHNpRHRJS0ObN0bimlScD9wJeA3QvOS5IkSeqwaipYNOPT/Lq0zL3PR8S/RsRP8+tXVtLPiPxa7pSRuxrVAdgM2AR4PqX0SgvbSJIkSZ1KrT0KVVZEdAUaXqRfLhDsmX+VtpkKHJFSeq2krAewEbAwpfRWmX5eyK9blJRtmV+fb2J65dqUFRFNHVSxVXNtJUmSpGrqLCsWFwJfBu5MKd1dUr4YOB8YAqyXf+1OtvF7D+DePEw06J1fFzQxTkN5n4JtJEmSpE6l5lcsIuJkss3W/wN8v/ReSuld4NxGTaZFxF7AA8BOwFFkb3FqjdYcVx4tbdPUCYf5Ssb2rRhTkiRJalc1vWIRESeQhYJngeEppXktaZdSWkr2elqA3UpuNawu9Ka8cqsTzbXpVaaNJEmS1KnUbLCIiNHAVWRnUQzP3wzVGv/Mr8sehUopLQLeANaNiA3LtNk8v5bup3guvza1h6JcG0mSJKlTqclgERE/AS4DniALFe+uQjc759eXG5VPzq/7lGmzb6M6kJ138RqwRUQMamEbSZIkqVOpuWAREeeQbdaeCXwjpTR3JXV3iohuZcpHAKfm397Q6HbDeRg/i4j1StoMBE4AlgDXNZTnp3w3tLk4IrqUtBkJfJ3sUa37WvL5JEmSpFpUU5u3I+IIYBzwGdnBcydn59Mtpy6lNDH/80XANvmrZefkZV/hf8+UOCel9FBp45TSQxFxKdlp2k9FxC1AN+BQoC9wUqNTtwEuBfYHDgYejYh7yc62OITszVQ/9NRtSZJUq14bt221p7BSm5z7j0Lty/w+uVLXXXcdo0aNKjRmZ1RTwQJoeNRoDWB0E3XuAybmf/498B1gR7JHktYE3gFuBq5KKd1froOU0ukR8RRwInAMUA/MAsanlO4oU39JRHwTOAs4nGw15APgVmBMSunZ1nxISZIktZ8xY8asUPaLX/yCBQsWcMopp9CnT5/l7g0ePLh9JlZjaipYpJTGAmNbUf83wG9WcazrgetbUf8jYEz+JUmSpBoxduzYFcomTpzIggULGD16NAMHDmz3OdWimttjIUmSJFXLHnvsQUTwySefMG7cOLbccku6d+++7NGosWPHEhFMnTp1hbZ1dXVERNnHqBYvXsx//Md/MHjwYHr06MG6667LLrvswh//+Me2/UAVVFMrFpIkSVJHcNBBB/HYY4+x7777cuCBB9K/f/9V7mv+/PmMGDGCxx9/nO23354f/vCH1NfXc/fdd3P44YfzzDPPcMEFF1Rw9m3DYCFJkiS10quvvsrTTz9Nv379Cvc1evRoHn/8cS666CLOPPPMZeUff/wxBx54IP/+7//OwQcf3OH3dvgolCRJktRK559/fkVCxXvvvccNN9zADjvssFyoAFhrrbW46KKLSCnxhz/8ofBYbc0VC0mSJKmVhg4dWpF+HnvsMT777DMiouwm8k8//RSA2bNnV2S8tmSwkCRJklppwIABFennvffeA7KA8dhjjzVZb+HChRUZry35KJQkSZLUSk0dqtelS/br9dKlS1e4N3/+/BXKevfuDcCpp55KSqnJrylTplRu8m3EYCFJkiRVyHrrrQfA66+/vsK9GTNmrFA2dOhQunTpwv33lz23uaYYLCRJkqQKadh7cd111y23avH6668zbty4Fer379+f733ve8yYMYPzzz+/7ErHSy+9xCuvvNJ2k64Q91hIkiRJFbLTTjux2267MW3aNIYOHcqIESN45513uP3229l7773LrmRcddVVvPDCC5x77rn8/ve/52tf+xobbLABb775JrNnz+axxx7jj3/8I4MGDarCJ2o5VywkSZKkCpo0aRJHHXUUc+bM4corr+Txxx/n4osv5qKLLipbv1evXtx3331ceeWV9OvXjz//+c9ceumlTJkyhZ49e3LZZZex5557tvOnaD1XLCRJkrRSm5z7j2pPod3V1dWVLZ86dWqzbfv06cOECROYMGHCCvdSSmXbdOvWjRNPPJETTzyxNdPsUFyxkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiSpE2vqFbeVZrCQJElajUUEAPX19VWeidpKQ7Bo+LtuKwYLSZKk1Vj37t0BWLRoUZVnorbS8Hfb8HfdVgwWkiRJq7GePXsC8Pbbb/Phhx9SX1/fbo/OqO2klKivr+fDDz/k7bffBv7377qtdG3T3iVJktSh9e3bl0WLFrF48WLmzJlT7emojayzzjr07du3TccwWEiSJK3GunTpwsYbb8y8efP48MMPWbJkiSsWnURE0L17d3r27Enfvn3p0qVtH1YyWEiSJK3munTpQr9+/ejXr1+1p6Ia5h4LSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmF1VSwiIj1I+KoiPjviHgxIj6KiAUR8UBE/Cgiyn6eiNg1Iu6MiHkRsTginoqI0RGxxkrGOiIipkfEwnyMqRGx/0rqrx0R50XEcxHxcUS8GxE3R8TWlfjskiRJUkdWU8ECOASYAOwEPAr8Avgz8GXg18DNERGlDSJiJDAN2A34b+BqoBtwGXBjuUEi4hJgIrBhPt4NwLbA7RFxYpn63YF7gHOBD4DLgb8D3wFmRMROq/6RJUmSpI6va7Un0ErPAwcAf0kp1TcURsRPgenAQcC/kIUNIqIXWTD4DNgjpTQjLz8HmAwcHBGHpZRuLOlrV+B04CVgx5TS+3n5eGAmcElE3JFSqiuZ12nAMOAW4NCGuUXETcCtwG8jYtvSOUuSJEmdSU2tWKSUJqeUbm/8C3pK6W3g2vzbPUpuHQx8DrixIVTk9T8Gzs6/Pa7RMMfm1583hIq8TR3Zakd34MiG8nyFpKHNmaVzSylNAu4HvgTs3uIPKkmSJNWYmgoWzfg0vy4tKRuRX/9apv40YDGwa/4oU0va3NWoDsBmwCbA8ymlV1rYRpIkSepUau1RqLIioivwg/zb0kCwZX59vnGblNLSiHgF2AbYFJgdET2AjYCFKaW3ygz1Qn7doiVjrKRNWRExs4lbWzXXVpIkSaqmzrJicSHZBu47U0p3l5T3zq8LmmjXUN5nFeuvahtJkiSpU6n5FYuIOJlss/X/AN9vbfP8mlrZrjX1WzxGSmlI2Q6ylYztWzGmJEmS1K5qesUiIk4ge7Xrs8DwlNK8RlUaVgt6U16vRvWaq19udaK1Y0iSJEmdTs0Gi4gYDVwFPE0WKt4uU+25/LrC/oZ8X8Ygss3eLwOklBYBbwDrRsSGZfrbPL+W7qdocoyVtJEkSZI6lZoMFhHxE7ID7p4gCxXvNlF1cn7dp8y93YB1gIdSSkta2GbfRnUgO+/iNWCLiBjUwjaSJElSp1JzwSI/3O5CssPqvpFSmruS6rcAc4HDImKHkj7WAi7Iv72mUZuG8zB+FhHrlbQZCJwALAGuayhPKaWSNhdHRJeSNiOBr5M9qnVfCz+iJEmSVHNqavN2RBwBjCM7Sft+4OTsfLrl1KWUJgKklD6IiKPJAsbUiLgRmEd2eveWeflNpY1TSg9FxKVkp2k/FRG3AN2AQ4G+wEmNTt0GuBTYn+xAvkcj4l6ysy0OITsr44eeui1JkqTOrKaCBdmeCIA1gNFN1LkPmNjwTUrp1ojYHfgZcBCwFvAiWXC4Il9xWE5K6fSIeAo4ETgGqAdmAeNTSneUqb8kIr4JnAUcDpwKfADcCoxJKT3b2g8qSZIk1ZKaChYppbHA2FVo9yCwXyvbXA9c34r6HwFj8i9JkiRptVJzeywkSZIkdTwGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFVbRYBERm0REr2bq9IyITSo5riRJkqTqqvSKxSvAKc3UOTmvJ0mSJKmTqHSwiPxLkiRJ0mqkGnssNgAWVWFcSZIkSW2ka9EOIuIHjYoGlykDWAPYBPg+8I+i40qSJEnqOAoHC2AikPI/J2Bk/tVYwyNSi4HzKjCuJEmSpA6iEsHiyPwawG+BW4FJZep9BrwHPJxSml+BcSVJkiR1EIWDRUrp+oY/R8QRwK0ppd8V7VeSJElS7ajEisUyKaXhlexPkiRJUm3w5G1JkiRJhVU8WETE7hFxR0S8GxGfRsRnZb6WVnpcSZIkSdVT0UehIuJbZJu31wBeA54DDBGSJElSJ1fRYAGMBT4FvpVS+luF+5YkSZLUQVX6UagvAzcZKiRJkqTVS6WDxUJgXoX7lCRJktTBVTpY3AvsUuE+JUmSJHVwlQ4WPwE2i4izIyIq3LckSZKkDqrSm7fHAM8A5wE/jIgngPll6qWU0o8qPLYkSZKkKql0sBhV8ueB+Vc5CTBYSJIkSZ1EpR+FGtTCr01XdYCIODgiroyI+yPig4hIEXFDE3UH5veb+rpxJeMcERHTI2JhRCyIiKkRsf9K6q8dEedFxHMR8XF+QODNEbH1qn5WSZIkqVZUdMUipfRqJftrwtnAdmRvoJoDbNWCNk+SHdzX2NPlKkfEJcDpef8TgG7AYcDtEXFSSumqRvW7A/cAw4AZwOXAxsAhwLciYkRK6dEWzFOSJEmqSZV+FKo9nEr2C/+LwO7AlBa0eSKlNLYlnUfErmSh4iVgx5TS+3n5eGAmcElE3JFSqitpdhpZqLgFODSlVJ+3uYks0Pw2IrZtKJckSZI6m4oGi4jYpKV1U0qvrcoYKaVlQaKNXjx1bH79eUOoyMeti4irgXOAI8k2qpO//aqhzZml4SGlNCki7ge+TstDkCRJklRzKr1iUUe2Mbs5qQ3GXpnPR8S/AusD7wEPp5SeaqLuiPz61zL37iILFiPIgwWwGbAJ8HxK6ZUm2nw9b2OwkCRJUqdU6V/uf0f5YNEHGAx8EZgKtMdejFJ75l/LRMRU4IjSlZOI6AFsBCxMKb1Vpp8X8usWJWVb5tfnmxi7XJuyImJmE7daso9EkiRJqppKb94e1dS9iOhC9q/9xwJHVHLclVgMnE+2z+HlvOwrwFhgOHBvRAxOKS3K7/XOrwua6K+hvE9J2aq0kSRJkjqVdnscKd97cF5E7ANcCHyvHcZ8Fzi3UfG0iNgLeADYCTiK7C1Oreq6FXUbNoI02yalNKRsB9lKxvatGFOSJElqV5U+x6IlHgL2qsK4y6SUlgK/zr/dreRWw+pCb8ortzrRXJteZdpIkiRJnUo1gkVfoEcVxm3sn/l12VzyR6LeANaNiA3LtNk8v5bup3guvza1h6JcG0mSJKlTaddgERHfBA6liYPp2tnO+fXlRuWT8+s+Zdrs26gOZOddvAZsERGDWthGkiRJ6lQqfY5FU788dyU7ibrhnItxlRx3JfPZCXg8pfRJo/IRZAftAdzQqNm1wPeBn0XErSUH5A0ETgCWANc1VE4ppYi4Fvh34OKIKD0gbyTZq2afBe6r8MeTJEmSOoxKb97eo4nyBLwP3A1cklJa5X+9j4gDgQPzbwfk110iYmL+57kppR/nf74I2CZ/teycvOwr/O9ZFeeklB5abqIpPRQRl5Kdpv1URNwCdCNbaekLnNTo1G2AS4H9gYOBRyPiXrIQdQjZm6l+6KnbkiRJ6swq/brZ9ni0ajArvq520/wLsjMyGoLF74HvADuSPZK0JvAOcDNwVUrp/nIDpJROj4ingBOBY4B6YBYwPqV0R5n6S/LHvM4CDidbDfmA7DW3Y1JKz67KB5UkSZJqRXuefl0RKaWxZOdQtKTub4DfrOI41wPXt6L+R2SncY9prq4kSZLU2bRpsIiIXmSvYV2QUvqgLceSJEmSVD0Vf3QpItaIiLMi4kWyfRV1wPsR8WJeXnOrJJIkSZJWrtJvheoG/BXYnWzD9uvAW8CGwEDg58A+EbFX4zc1SZIkSapdlV6xOI3szVB/AbZOKQ1MKe2SUhoIbAncTvb61dMqPK4kSZKkKqp0sDic7PC7A1NKL5TeSCm9BPwL8AzwvQqPK0mSJKmKKh0s/g9wV1NnNuTldwGbVXhcSZIkSVVU6WDxCbBuM3V6AJ9WeFxJkiRJVVTpYPEUcHBEfK7czYjoR3Y69ZMVHleSJElSFVU6WFwFfA6YHhE/iohNI2LtiBgUEUcCj+b3r6rwuJIkSZKqqKKvm00p3RwRg4GzgP8sUyWAi1NKN1dyXEmSJEnVVfHD6lJKP42I24AfAV8lP3kbeBz4bUrp4UqPKUmSJKm62uQU7JTSI8AjbdG3JEmSpI6nonssIuKQiJgcEZ9v4v5GEXFvRPxLJceVJEmSVF2V3rx9FNAnpfRmuZsppTeAXnk9SZIkSZ1EpYPFtsCMZurMAL5S4XElSZIkVVGlg0Vf4N1m6rwH9KvwuJIkSZKqqNLBYi6weTN1NgfmV3hcSZIkSVVU6WDxIHBARGxV7mZEbA2MBO6v8LiSJEmSqqjSweISslfYPhARJ0fEFhHRI7+eQhYo1sjrSZIkSeokKn3y9mMRcTxwNXBZ/lXqM+C4lNKjlRxXkiRJUnW1xcnbEyLiAeB4YCegD9meikeAa1JKsys9piRJkqTqaquTt2cDJ7VF35IkSZI6nkrvsZAkSZK0GjJYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqbCaCxYRcXBEXBkR90fEBxGRIuKGZtrsGhF3RsS8iFgcEU9FxOiIWGMlbY6IiOkRsTAiFkTE1IjYfyX1146I8yLiuYj4OCLejYibI2LrIp9XkiRJqgU1FyyAs4ETgcHAG81VjoiRwDRgN+C/gauBbsBlwI1NtLkEmAhsCEwAbgC2BW6PiBPL1O8O3AOcC3wAXA78HfgOMCMidmrF55MkSZJqTtdqT2AVnArMAV4EdgemNFUxInqRBYPPgD1SSjPy8nOAycDBEXFYSunGkja7AqcDLwE7ppTez8vHAzOBSyLijpRSXclQpwHDgFuAQ1NK9Xmbm4Bbgd9GxLYN5ZIkSVJnU3MrFimlKSmlF1JKqQXVDwY+B9zYECryPj4mW/kAOK5Rm2Pz688bQkXepo5staM7cGRDeURESZszS8NDSmkScD/wJbIQJEmSJHVKNRcsWmlEfv1rmXvTgMXArvmjTC1pc1ejOgCbAZsAz6eUXmlhG0mSJKlTqcVHoVpjy/z6fOMbKaWlEfEKsA2wKTA7InoAGwELU0pvlenvhfy6RUvGWEmbsiJiZhO3tmqurSRJklRNnX3Fond+XdDE/YbyPqtYf1XbSJIkSZ1KZ1+xaE7k15bs1yjVmvotHiOlNKRsB9lKxvatGFOSJElqV519xaJhtaB3E/d7NarXXP1yqxOtHUOSJEnqdDp7sHguv66wvyEiugKDgKXAywAppUVkZ2OsGxEblulv8/xaup+iyTFW0kaSJEnqVDp7sJicX/cpc283YB3goZTSkha22bdRHcjOu3gN2CIiBrWwjSRJktSpdPZgcQswFzgsInZoKIyItYAL8m+vadTm2vz6s4hYr6TNQOAEYAlwXUN5fp5GQ5uLI6JLSZuRwNeBZ4H7KvB5JEmSpA6p5jZvR8SBwIH5twPy6y4RMTH/89yU0o8BUkofRMTRZAFjakTcCMwDDiB7TewtwE2l/aeUHoqIS8lO034qIm4BugGHAn2Bkxqdug1wKbA/2YF8j0bEvWRnWxxCdlbGDz11W5IkSZ1ZzQULYDBwRKOyTfMvgFeBHzfcSCndGhG7Az8DDgLWAl4kCw5XlDvBO6V0ekQ8BZwIHAPUA7OA8SmlO8rUXxIR3wTOAg4HTgU+AG4FxqSUnl3VDytJkiTVgpoLFimlscDYVrZ5ENivlW2uB65vRf2PgDH5lyRJkrRa6ex7LCRJkiS1A4OFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCVotgERF1EZGa+Hq7iTa7RsSdETEvIhZHxFMRMToi1ljJOEdExPSIWBgRCyJiakTs33afTJIkSeoYulZ7Au1oAfCLMuULGxdExEjgz8DHwE3APODbwGXAMOCQMm0uAU4H5gATgG7AYcDtEXFSSumqinwKSZIkqQNanYLF/JTS2OYqRUQvsmDwGbBHSmlGXn4OMBk4OCIOSyndWNJmV7JQ8RKwY0rp/bx8PDATuCQi7kgp1VX2I0mSJEkdw2rxKFQrHQx8DrixIVQApJQ+Bs7Ovz2uUZtj8+vPG0JF3qYOuBroDhzZVhOWJEmSqm11ChbdI+L/RcRPI+KUiBjexH6JEfn1r2XuTQMWA7tGRPcWtrmrUR1JkiSp01mdHoUaAPy+UdkrEXFkSum+krIt8+vzjTtIKS2NiFeAbYBNgdkR0QPYCFiYUnqrzLgv5NctmptgRMxs4tZWzbWVJEmSqml1WbG4DvgGWbjoAWwL/AoYCNwVEduV1O2dXxc00VdDeZ9VrC9JkiR1OqvFikVK6bxGRU8Dx0bEQrJN12OB77Swu2jotrXTaLZCSkPKDpitZGzfyvEkSZKkdrNaBIuVuJYsWOxWUtawwtB7xeoA9GpUr7n6za1oSJKk1dBr47at9hTUAW1y7j+qPYVVtro8CtWUd/Nrj5Ky5/LrCnsiIqIrMAhYCrwMkFJaBLwBrBsRG5YZY/P8usKeDUmSJKmzWN2DxS759eWSssn5dZ8y9XcD1gEeSiktaWGbfRvVkSRJkjqdTh8sImKbiOhbpvyLQMNp2DeU3LoFmAscFhE7lNRfC7gg//aaRt1dm19/FhHrlbQZCJwALCHbQC5JkiR1SqvDHotDgLMiYgrwCvAhsBnwLWAt4E7gkobKKaUPIuJosoAxNSJuBOYBB5C9ivYW4KbSAVJKD0XEpcBpwFMRcQvQDTgU6Auc5KnbkiRJ6sxWh2AxhSwQfJXs0acewHzgAbJzLX6fUlrujU0ppVsjYnfgZ8BBZAHkRbLgcEXj+nmb0yPiKeBE4BigHpgFjE8p3dE2H02SJEnqGDp9sMgPv7uv2YortnsQ2K+Vba4Hrm/tWJIkSVKt6/R7LCRJkiS1PYOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSJEmSCjNYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBYVFBFfiIjfRsSbEbEkIuoi4hcRsV615yZJkiS1pa7VnkBnERGbAQ8B/YFJwP8AQ4FTgH0iYlhK6b0qTlGSJElqM65YVM4vyULFySmlA1NKZ6WURgCXAVsCP6/q7CRJkqQ2ZLCogIjYFNgLqAOubnR7DLAI+H5E9GjnqUmSJEntwmBRGSPy699SSvWlN1JKHwIPAusAO7f3xCRJkqT24B6Lytgyvz7fxP0XyFY0tgDubaqTiJjZxK3tZs+ezZAhQ1Z9hhUy+w23iWhFQyZfXu0pSFLN+eStF6s9BXVA3SZV9/e92bNnAwxclbYGi8ronV8XNHG/obzPKvb/2UcffbRg1qxZdavYXmpLW81651XIXlggSWqZrfKrPzu1vLdmVXsGA4EPVqWhwaJ9RH5NK6uUUqr+koTUSg0rbf73K0kt589OdUbusaiMhhWJ3k3c79WoniRJktSpGCwq47n8ukUT9zfPr03twZAkSZJqmsGiMqbk170iYrn/TSOiJzAM+Ah4pL0nJkmSJLUHg0UFpJReAv5GttnlhEa3zwN6AL9LKS1q56lJkiRJ7cLN25VzPPAQcEVEfAOYDewEDCd7BOpnVZybJEmS1KYipZW+qEitEBEbA+OAfYD1gbeAW4HzUkrzqjg1SZIkqU0ZLCRJkiQV5h4LSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJK2yiBgYESkiJlZ7LpIkqboMFtJqIg8AK/saVe05SlJH1IKfn/48lfDkbWl1dF4T5U+05yQkqYaU+7k5GugNXA7Mb3TvibadjtQxeUCetJqIiASQUooK9jkQeAW4PqU0qlL9SlJHFxF1wBeBQSmluurORuoYfBRK0jIR8fmIODciHoyItyPik4h4MyL+EBFbt6KfDSLikoh4LiIWRcT8/M8TI2LTMvX3jog7I2JuRCyJiJciYnxE9KnoB5SkNhYRU/PHobrlP0+fy3+uTczvj83v71GmbZP71iJinYj4t4h4Iv+5ujAiHo6I/9vWn0lqKR+FklRqN+AsYArwZ2AhsDlwMHBARAxLKT25sg4iYh3gQWAz4B7gdiDI/mVvJHAL8HJJ/XPJHjOYB9wBvAt8BfgxsF9E7JJS+qCCn1GS2sOfgR2Bu4BbyX62rZL8H1kmA18FZgG/JfvH4b2BP0TENimlswvOVyrMYCGtZiJibJniupTSRLL/49ogpfRhozbbkYWFC4F9mxniG2Sh4hcppVMb9dMN6F7y/XCyUPEwsF9KaX7JvVHAdfn95fqRpBrwReDLKaW5FejrF2Sh4icppYsbCiNiLbLQ8tOIuCWl9EQFxpJWmcFCWv2MKVN2HzAxpVT2X9RSSk9GxGRgr4hYM6X0aQvG+ahMP58An5QUnZxfjy4NFXndiRFxCvA9DBaSas85lQgVEbE+8P+AGaWhAiCl9HFE/IRs5eJw3DSuKjNYSKuZ5jZvR8S3gGOBHYB+rPhzoh/w1kq6uA94AzgrIrYH7iRb7XgipfRZo7q7AJ8Ch0TEIWX66gZ8LiLWTym9t7J5S1IHM71C/ewIrAGkJlac18yvLd4HJ7UVg4WkZSLiZLJXJ75Ptj/iNWAxkIADge0oeZSpnJTSBxGxM9kjTAeQ/UsawNyI+CVwQcmKx/pkP4fKraKUWhcwWEiqJW9XqJ/18+uO+VdT1q3QeNIqM1hIAiAiupKFgbeB7VNKbzW6v0tL+0opzQF+FBEBfAkYAZwAnEu24fCcvOoCoEtKqW/xTyBJHUdq+n3+9fm13O9gfcqULcivl6WUTis6L6kt+bpZSQ36kf2f2kNlQsW6wPat7TBlnkkpXQnsmRcfWFLlEWC9iNhmlWYsSbXn/fy6cZl7O5Qpm04WRr7eZjOSKsRgIanBu2SPPQ3JgwQAEbEm2eNR/VrSSUR8OT84r7EN8uvikrLL8uuEiPh8mb565I9VSVJn0bD34sh8pRiAiNiYbFV3OflLNf4L2CEiziltU9J2s4gY1FYTllrKR6EkAZBSqo+IK8jOsfhHREwi2zw9HOhLdrbF8BZ09U3g0oh4CPgfssDyBbIzLOqB8SVj3hsRZwH/AbwQEXeSneS9LtmrGncHHgD2qciHlKQqSyk9GhHTyM4Nmp6/cW8D4NvA3ZRfyTiR7EyhccD3I+IB4B3g82SbtncE/i/Zz0+pagwWkkqdA/wTOAr4V7Jne+8Bzibbf9ESd5O9c303sjDRi+wtUvcAl6aUHiqtnFK6KCIeJHv17NfyNgvI3iz1n8AfCn0iSep4RpL9I8tI4CTgBeBM4G/AdxtXzl+KsTtwDNlrZQ8C1iILFy+QvZL7nnaZubQS0fTeIkmSJElqGfdYSJIkSSrMYCFJkiSpMIOFJEmSpMIMFpIkSZIKM1hIkiRJKsxgIUmSJKkwg4UkSZKkwgwWkiRJkgozWEiSJEkqzGAhSZIkqTCDhSRJkqTCDBaSpA4pIgZGRIqIidWeiySpeQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpJUFRExNCJuiog3ImJJRLwVEX+LiO+WqTswIm6MiLkR8XFEzIiI/cvUG5vvy9ijiT5W2LMRERPz8k0j4qSIeCoiPoqIqY3uD4yIf42If+RzeCci/jMielfqfxNJqmVdqz0BSdLqJyKOBq4BPgNuA14A+gM7AMcDN5dU/yIwHXgZ+D3QFzgUmBQR30wpTanQtC4Hvg78Bbgzn1upi4G9gduBvwHDgaOB/wOMqNAcJKlmGSwkSe0qIr4E/BL4APh6SumZRve/0KjJHsDYlNJ5JXX+APwVOAOoVLDYHvhqSumVJu7vDGybUnotn0NXYDIwPCKGppSmV2geklSTfBRKktTejiP7h63zG4cKgJTSnEZFrwIXNKpzN/AaMLSC87p4JaECYFxDqMjnsBS4Lv+2kvOQpJpksJAktbed8+tdLaz/REqp8WNJAK8D61VmSkD2uNXKzGhiDlR4HpJUkwwWkqT21ie/vtHC+vObKF9KZf9/7O1VmMfS/LpGBechSTXJYCFJam/z8+tGbdB3fX4tt4ewTzNtU2WnIkmrF4OFJKm9PZJf922Dvt/PrxuXubdDG4wnScoZLCRJ7e0askeIzsnfELWcMm+Fao2GfRJH5m9tauhzY+DcAv1Kkprh62YlSe0qpfRsRBwPXAs8HhGTyM6xWJ9sVeFDsjMiVqXvRyNiGrAbMD0iJgMbAN8G7qb8SoYkqQJcsZAktbuU0gTga8AdZOdUnAEcAMwFri7Y/Ujg18AXgJOArwJnAj8p2K8kaSUiJfeqSZIkSSrGFQtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmFGSwkSZIkFWawkCRJklSYwUKSJElSYQYLSZIkSYUZLCRJkiQVZrCQJEmSVJjBQpIkSVJhBgtJkiRJhRksJEmSJBVmsJAkSZJUmMFCkiRJUmEGC0mSJEmF/X/7aGIMJUP+5wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 261,
       "width": 395
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(x='churn', hue='churn', data=df);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}