03-SklearnIntroduction.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! pip install graphviz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv('train.csv')\n",
    "df = df.fillna(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Revisiting the results from our quiz in the beginning:\n",
    "\n",
    "What kind of learning problem do we have here:\n",
    "- [x] Supervised Learning\n",
    "- [ ] Unsupervised Learning\n",
    "- [ ] Reinforcement Learning\n",
    "\n",
    "What is our target here:\n",
    "- [ ] Regression\n",
    "- [x] Classification\n",
    "- [ ] Clustering\n",
    "\n",
    "We can now choose a machine learning algorithm from the [sklearn webpage](https://scikit-learn.org).\n",
    "Let us start with a `DecisionTreeClassifier` as it is easy to understand and can provide us with a lot of valuable insights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "features = [\"Pclass\", \"Age\", \"SibSp\", \"Parch\", \"Fare\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us score our algorithm using `.score`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nice! You trained your first machine learning algorithm and it can really well predict who survives and who doesn't! But how does it do that and is it actually that awesome or did we do something wrong?\n",
    "Let us have a look in how the algorithm decides who dies and who doesn't."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import List\n",
    "\n",
    "import graphviz\n",
    "import numpy as np\n",
    "\n",
    "from matplotlib import pyplot as plt\n",
    "from sklearn import tree\n",
    "\n",
    "def train_classifier_and_plot_decision_boundaries(\n",
    "    df: pd.DataFrame,\n",
    "    x_column: str,\n",
    "    y_column: str,\n",
    "    max_depth: int\n",
    ") -> DecisionTreeClassifier:\n",
    "\n",
    "    target_column = \"Survived\"\n",
    "    # Parameters\n",
    "    n_classes = 2\n",
    "    plot_colors = \"rb\"\n",
    "    plot_step = 0.5\n",
    "\n",
    "    decision_tree = DecisionTreeClassifier(max_depth=max_depth, random_state=42)\n",
    "    decision_tree.fit(df[[x_column, y_column]], df[target_column])\n",
    "\n",
    "    target_names = [\"Deceased\", \"Survived\"]\n",
    "\n",
    "    x_min = df[x_column].min() - 1\n",
    "    x_max = df[x_column].max() + 1\n",
    "    y_min = df[y_column].min() - 1\n",
    "    y_max = df[y_column].max() + 1\n",
    "    y_max = y_max if y_max < 200 else 200\n",
    "    xx, yy = np.meshgrid(np.arange(x_min, x_max, plot_step),\n",
    "                         np.arange(y_min, y_max, plot_step))\n",
    "    \n",
    "    plt.figure(figsize=(16,12))\n",
    "    plt.tight_layout(h_pad=0.5, w_pad=0.5, pad=2.5)\n",
    "\n",
    "    Z = decision_tree.predict(np.c_[xx.ravel(), yy.ravel()])\n",
    "    Z = Z.reshape(xx.shape)\n",
    "    cs = plt.contourf(xx, yy, Z, cmap=plt.cm.RdYlBu)\n",
    "\n",
    "    plt.xlabel(x_column)\n",
    "    plt.ylabel(y_column)\n",
    "    plt.ylim([y_min, y_max])\n",
    "\n",
    "    # Plot the training points\n",
    "    for i, color in zip(range(n_classes), plot_colors):\n",
    "        idx = np.where(df[target_column] == i)\n",
    "        plt.scatter(df.iloc[idx][x_column], df.iloc[idx][y_column], c=color, label=target_names[i],\n",
    "                    cmap=plt.cm.RdYlBu, edgecolor='black', s=30)\n",
    "        \n",
    "    return decision_tree\n",
    "\n",
    "def show_decisions(decision_tree: DecisionTreeClassifier, features: List[str]):\n",
    "    dot_data = tree.export_graphviz(\n",
    "        decision_tree,\n",
    "        out_file=None, \n",
    "        feature_names=features,  \n",
    "        class_names=[\"Deceased\", \"Survived\"],  \n",
    "        filled=True,\n",
    "        rounded=True,  \n",
    "        special_characters=True)  \n",
    "    graph = graphviz.Source(dot_data)   \n",
    "    return graph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "decision_tree = train_classifier_and_plot_decision_boundaries(df, \"Age\", \"Fare\", 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "show_decisions(decision_tree, [\"Age\", \"Fare\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "decision_tree = train_classifier_and_plot_decision_boundaries(df, \"Age\", \"Fare\", 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "show_decisions(decision_tree, [\"Age\", \"Fare\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "decision_tree = train_classifier_and_plot_decision_boundaries(df, \"Age\", \"Fare\", None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wow, this is a lot of decisions and some of them seem to be very interesting. They only occure because there is just someone who by chance survived or deceased eventhough one should not generate a good rule from that. The problem is that we cannot always plot the decisions an algorithm took, so how can we decide if the algorithm learned something useful or not?\n",
    "\n",
    "The solution for this is to have a [training and test set](./04-TrainTestSplit.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}