Lily_Data_144_Random_Forest_(Excerpt).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/PoojaDoctor/Data144-FinalProject/blob/master/Lily_Data_144_Random_Forest_(Excerpt).ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PzFRBnjMAbdE"
   },
   "source": [
    "### **Data 144 Final Project - Lily Li - Only Random Forest**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "o-5Z-48e_os-"
   },
   "source": [
    "Recommend Methods:\n",
    "1. Random forest\n",
    "2. Decision tree\n",
    "3. Neural network\n",
    "4. Linear Regression\n",
    "\n",
    "Try different models and throw the data into them!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZqY1KkxcHqRK"
   },
   "source": [
    "Use Mean square error;\n",
    "Mean absolute error;\n",
    "Train the data of the log of views\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "11wWfeYJMJac"
   },
   "source": [
    "# **RANDOM FOREST:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "g4PU09JDMQQ4"
   },
   "outputs": [],
   "source": [
    "# Import necessary library\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import numpy as np\n",
    "from scipy.stats import norm\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from scipy import stats\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 400
    },
    "colab_type": "code",
    "id": "9iUwk7SMzuwB",
    "outputId": "e2b2ac9d-7e99-4654-9e6a-3ad841fdb615"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>comments</th>\n",
       "      <th>description</th>\n",
       "      <th>duration</th>\n",
       "      <th>languages</th>\n",
       "      <th>main_speaker</th>\n",
       "      <th>name</th>\n",
       "      <th>num_speaker</th>\n",
       "      <th>related_talks</th>\n",
       "      <th>speaker_occupation</th>\n",
       "      <th>tags</th>\n",
       "      <th>title</th>\n",
       "      <th>url</th>\n",
       "      <th>views</th>\n",
       "      <th>transcript</th>\n",
       "      <th>Funny</th>\n",
       "      <th>Beautiful</th>\n",
       "      <th>Ingenious</th>\n",
       "      <th>Courageous</th>\n",
       "      <th>Longwinded</th>\n",
       "      <th>Confusing</th>\n",
       "      <th>Informative</th>\n",
       "      <th>Fascinating</th>\n",
       "      <th>Unconvincing</th>\n",
       "      <th>Persuasive</th>\n",
       "      <th>Jaw-dropping</th>\n",
       "      <th>OK</th>\n",
       "      <th>Obnoxious</th>\n",
       "      <th>Inspiring</th>\n",
       "      <th>max_rating</th>\n",
       "      <th>yr</th>\n",
       "      <th>month</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>4553</td>\n",
       "      <td>Sir Ken Robinson makes an entertaining and pro...</td>\n",
       "      <td>1164</td>\n",
       "      <td>60</td>\n",
       "      <td>Ken Robinson</td>\n",
       "      <td>Ken Robinson: Do schools kill creativity?</td>\n",
       "      <td>1</td>\n",
       "      <td>[{'id': 865, 'hero': 'https://pe.tedcdn.com/im...</td>\n",
       "      <td>Author/educator</td>\n",
       "      <td>['children', 'creativity', 'culture', 'dance',...</td>\n",
       "      <td>Do schools kill creativity?</td>\n",
       "      <td>https://www.ted.com/talks/ken_robinson_says_sc...</td>\n",
       "      <td>47227110</td>\n",
       "      <td>Good morning. How are you?(Laughter)It's been ...</td>\n",
       "      <td>19645</td>\n",
       "      <td>4573</td>\n",
       "      <td>6073</td>\n",
       "      <td>3253</td>\n",
       "      <td>387</td>\n",
       "      <td>242</td>\n",
       "      <td>7346</td>\n",
       "      <td>10581</td>\n",
       "      <td>300</td>\n",
       "      <td>10704</td>\n",
       "      <td>4439</td>\n",
       "      <td>1174</td>\n",
       "      <td>209</td>\n",
       "      <td>24924</td>\n",
       "      <td>Inspiring</td>\n",
       "      <td>2006</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>265</td>\n",
       "      <td>With the same humor and humanity he exuded in ...</td>\n",
       "      <td>977</td>\n",
       "      <td>43</td>\n",
       "      <td>Al Gore</td>\n",
       "      <td>Al Gore: Averting the climate crisis</td>\n",
       "      <td>1</td>\n",
       "      <td>[{'id': 243, 'hero': 'https://pe.tedcdn.com/im...</td>\n",
       "      <td>Climate advocate</td>\n",
       "      <td>['alternative energy', 'cars', 'climate change...</td>\n",
       "      <td>Averting the climate crisis</td>\n",
       "      <td>https://www.ted.com/talks/al_gore_on_averting_...</td>\n",
       "      <td>3200520</td>\n",
       "      <td>Thank you so much, Chris. And it's truly a gre...</td>\n",
       "      <td>544</td>\n",
       "      <td>58</td>\n",
       "      <td>56</td>\n",
       "      <td>139</td>\n",
       "      <td>113</td>\n",
       "      <td>62</td>\n",
       "      <td>443</td>\n",
       "      <td>132</td>\n",
       "      <td>258</td>\n",
       "      <td>268</td>\n",
       "      <td>116</td>\n",
       "      <td>203</td>\n",
       "      <td>131</td>\n",
       "      <td>413</td>\n",
       "      <td>Funny</td>\n",
       "      <td>2006</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>124</td>\n",
       "      <td>New York Times columnist David Pogue takes aim...</td>\n",
       "      <td>1286</td>\n",
       "      <td>26</td>\n",
       "      <td>David Pogue</td>\n",
       "      <td>David Pogue: Simplicity sells</td>\n",
       "      <td>1</td>\n",
       "      <td>[{'id': 1725, 'hero': 'https://pe.tedcdn.com/i...</td>\n",
       "      <td>Technology columnist</td>\n",
       "      <td>['computers', 'entertainment', 'interface desi...</td>\n",
       "      <td>Simplicity sells</td>\n",
       "      <td>https://www.ted.com/talks/david_pogue_says_sim...</td>\n",
       "      <td>1636292</td>\n",
       "      <td>(Music: \"The Sound of Silence,\" Simon &amp; Garfun...</td>\n",
       "      <td>964</td>\n",
       "      <td>60</td>\n",
       "      <td>183</td>\n",
       "      <td>45</td>\n",
       "      <td>78</td>\n",
       "      <td>27</td>\n",
       "      <td>395</td>\n",
       "      <td>166</td>\n",
       "      <td>104</td>\n",
       "      <td>230</td>\n",
       "      <td>54</td>\n",
       "      <td>146</td>\n",
       "      <td>142</td>\n",
       "      <td>230</td>\n",
       "      <td>Funny</td>\n",
       "      <td>2006</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  comments  ...    yr  month\n",
       "0           0      4553  ...  2006      6\n",
       "1           1       265  ...  2006      6\n",
       "2           2       124  ...  2006      6\n",
       "\n",
       "[3 rows x 32 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "url = 'https://raw.githubusercontent.com/PoojaDoctor/Data144-FinalProject/master/datasets/cleaned.csv'\n",
    "df_cleanned = pd.read_csv(url)\n",
    "df_cleanned.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Fg2iAudoMLHt"
   },
   "outputs": [],
   "source": [
    "#Import scikit-learn dataset library\n",
    "from sklearn import datasets\n",
    "data = df_cleanned\n",
    "\n",
    "# Import train_test_split function\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X=data[['comments','duration','languages','num_speaker','month']]  # Features\n",
    "# 'Funny','Beautiful','Ingenious','Courageous','Longwinded',\n",
    "#        'Confusing','Informative','Fascinating','Unconvincing','Persuasive','Jaw-dropping','OK','Obnoxious','Inspiring',\n",
    "#        'max_rating','yr',\n",
    "y=data['views']  # Labels\n",
    "\n",
    "# Split dataset into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3) # 70% training and 30% test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "0PdS_Q_9NfO0"
   },
   "outputs": [],
   "source": [
    "# Feature Scaling\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "sc = StandardScaler()\n",
    "X_train = sc.fit_transform(X_train)\n",
    "X_test = sc.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "HUfqptXjOUil"
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestRegressor\n",
    "\n",
    "regressor = RandomForestRegressor(n_estimators=20, random_state=0)\n",
    "regressor.fit(X_train, y_train)\n",
    "y_pred = regressor.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 69
    },
    "colab_type": "code",
    "id": "tDAmEo4kOXgu",
    "outputId": "4bcd5494-137c-465c-dbd9-10e98d69419c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean Absolute Error: 892053.7568151146\n",
      "Mean Squared Error: 3133925128657.0005\n",
      "Root Mean Squared Error: 1770289.5606812465\n"
     ]
    }
   ],
   "source": [
    "from sklearn import metrics\n",
    "\n",
    "print('Mean Absolute Error:', metrics.mean_absolute_error(y_test, y_pred))\n",
    "print('Mean Squared Error:', metrics.mean_squared_error(y_test, y_pred))\n",
    "print('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(y_test, y_pred)))"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "1_SXrZu3FxyN"
   ],
   "include_colab_link": true,
   "name": "Lily_Data 144_Random Forest (Excerpt).ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}