neural_network.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 뉴럴 네트워크 (Neural Network)\n",
    "\n",
    "Date: 2019-08-29  \n",
    "Author: skettee  \n",
    "Categories: Deep Learning, Neural Network  \n",
    "Tags: Neuron, One-hot encoding, Softmax\n",
    "<!--eofm-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "딥러닝의 세계로 들어가기 위해 알아야 하는 세번째 모델인 뉴럴 네트워크(Neural Network)에 대해 알아보고 keras를 이용해서 모델링을 해보자!\n",
    "<!--more-->\n",
    "\n",
    "실제로 돌려 보고 싶으면 구글 코랩으로 ~  \n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/skettee/notebooks/blob/master/neural_network.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 문제 (Problem)\n",
    "\n",
    "💰 고객\n",
    "\n",
    "> 덕분에 친추가 되었어요. 고마워요~\n",
    ">\n",
    "> 그 친구가 숙제를 부탁했는데... 해결해 주세요.\n",
    ">\n",
    "> 아래와 같이 입력과 출력을 하는 프로그램이 필요해요.  \n",
    ">\n",
    "> 입력: 1 0 0 0 0 0 0 0 0 0, 출력: 0  \n",
    "> 입력: 0 1 0 0 0 0 0 0 0 0, 출력: 1  \n",
    "> 입력: 0 0 1 0 0 0 0 0 0 0, 출력: 2  \n",
    "> 입력: 0 0 0 1 0 0 0 0 0 0, 출력: 3  \n",
    "> 입력: 0 0 0 0 1 0 0 0 0 0, 출력: 4  \n",
    "> 입력: 0 0 0 0 0 1 0 0 0 0, 출력: 5  \n",
    "> 입력: 0 0 0 0 0 0 1 0 0 0, 출력: 6  \n",
    "> 입력: 0 0 0 0 0 0 0 1 0 0, 출력: 7   \n",
    "> 입력: 0 0 0 0 0 0 0 0 1 0, 출력: 8   \n",
    "> 입력: 0 0 0 0 0 0 0 0 0 1, 출력: 9     \n",
    ">  \n",
    "> 데이터는 아래에 있어요.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_data = [\n",
    "    [1, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
    "    [0, 1, 0, 0, 0, 0, 0, 0, 0, 0],\n",
    "    [0, 0, 1, 0, 0, 0, 0, 0, 0, 0],\n",
    "    [0, 0, 0, 1, 0, 0, 0, 0, 0, 0],\n",
    "    [0, 0, 0, 0, 1, 0, 0, 0, 0, 0],\n",
    "    [0, 0, 0, 0, 0, 1, 0, 0, 0, 0],\n",
    "    [0, 0, 0, 0, 0, 0, 1, 0, 0, 0],\n",
    "    [0, 0, 0, 0, 0, 0, 0, 1, 0, 0],\n",
    "    [0, 0, 0, 0, 0, 0, 0, 0, 1, 0],\n",
    "    [0, 0, 0, 0, 0, 0, 0, 0, 0, 1],\n",
    "]\n",
    "\n",
    "output_data = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "⚙️ 엔지니어\n",
    "\n",
    "> 이건 제가 코딩해서 바로 만들어 드릴 수 있습니다!  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input data is [0, 0, 0, 0, 0, 1, 0, 0, 0, 0] and ouput is 5\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "def human_coding(x):\n",
    "    for i in range(len(x)):\n",
    "        if x[i] == 1:\n",
    "            return i\n",
    "    return None\n",
    "\n",
    "test_data =  [0, 0, 0, 0, 0, 1, 0, 0, 0, 0]\n",
    "result = human_coding(test_data)\n",
    "print('input data is {} and ouput is {}'.format(test_data, result))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "💰 고객\n",
    "\n",
    "> 아... 조건이 있어요.  \n",
    ">\n",
    "> 2-레이어 뉴럴 네트워크(NN)로 만들어 주세요.  \n",
    "> 히든 레이어(hidden layer)는 3개의 유닛들(units)로 구성되게 해 주세요.  \n",
    "\n",
    "⚙️ 엔지니어\n",
    "\n",
    "> ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 뉴럴 네트워크(Neural Network) 구성\n",
    "\n",
    "### 뉴런 모델 (Neuron model)\n",
    "\n",
    "뉴런(Neuron) 모델 은 앞에서 학습한 로지스틱 모델과 같다.  \n",
    "예를 들어서 3개의 변수($x_1$, $x_2$, $x_3$)를 입력 받을 경우    \n",
    "$z=w_{1}x_{1} + w_{2}x_{2} + w_{3}x_{3} + b$가 되고  \n",
    "$a=\\sigma(z)$ 가 된다.  \n",
    "아래 그림과 같이 표현 된다.  \n",
    "\n",
    "![뉴런 모델](https://skettee.github.io/post/neural_network/neuron_model.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 뉴럴 네트워트 (Neural Network)\n",
    "\n",
    "뉴럴 네트워트(Neural Network)는  \n",
    "하나의 뉴런, 즉 로지스틱 모델을 **유닛(Unit)** 이라고 하고   \n",
    "유닛들을 서로 연결한 구조이다.  \n",
    "유닛들을 모아서 **레이어(Layer)** 를 형성하고  \n",
    "레이어를 계속해서 쌓아가는 모양을 가지고 있다.  \n",
    "\n",
    "아래 그림은  \n",
    "2-레이어 (2-layer) 뉴럴 네트워크이다.  \n",
    "4개의 유닛(4-units)들로 구성된 히든 레이어(hidden layer) 1개와   \n",
    "1개의 유닛(1-unit)으로 구성된 출력 레이어(output layer) 1개로 되어 있다.   \n",
    "\n",
    "![뉴럴 네트워크](https://skettee.github.io/post/neural_network/neural_network.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 데이터 분석 (Data Analysis)\n",
    "\n",
    "이 문제는 데이터의 규칙에 따라 출력값을 결정하는 것이다.  \n",
    "\n",
    "10개의 입력값 중에  \n",
    "- 첫번째 입력값이 1이고 나머지가 0이면 출력값은 0  \n",
    "- 두번째 입력값이 1이고 나머지가 0이면 출력값은 1  \n",
    "-   ...    \n",
    "\n",
    "이 규칙을 학습 시켜서 원하는 출력값으로 분류하는 것이 목표다.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 데이터 변환 (Data Transformation)\n",
    "\n",
    "Keras가 원하는 입력과 출력의 모양(shape)는 다음과 같다.  \n",
    "\n",
    "- (batch_size, input_dimension)    \n",
    "- batch_size는 학습할 데이터의 개수이다.  \n",
    "- input_demension은 학습을 할 때에 사용할 입력 변수의 개수이다.  \n",
    "\n",
    "**입력 데이터 변환**  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "batch_size=  10\n"
     ]
    }
   ],
   "source": [
    "print('batch_size= ', len(input_data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input_dimension=  10\n"
     ]
    }
   ],
   "source": [
    "print('input_dimension= ', len(input_data[0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x.shape is  (10, 10)\n",
      "\n",
      "x is \n",
      " [[1 0 0 0 0 0 0 0 0 0]\n",
      " [0 1 0 0 0 0 0 0 0 0]\n",
      " [0 0 1 0 0 0 0 0 0 0]\n",
      " [0 0 0 1 0 0 0 0 0 0]\n",
      " [0 0 0 0 1 0 0 0 0 0]\n",
      " [0 0 0 0 0 1 0 0 0 0]\n",
      " [0 0 0 0 0 0 1 0 0 0]\n",
      " [0 0 0 0 0 0 0 1 0 0]\n",
      " [0 0 0 0 0 0 0 0 1 0]\n",
      " [0 0 0 0 0 0 0 0 0 1]]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "x = np.array(input_data)\n",
    "print('x.shape is ', x.shape)\n",
    "print('\\nx is \\n', x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**출력 데이터 변환**\n",
    "\n",
    "⚙️ 엔지니어\n",
    "\n",
    "> yes or no 문제라면 0과 1로 변환 하면 된다.     \n",
    "> 이번 문제는 0에서 9까지 10개다.  \n",
    "> 출력 데이터를 어떻게 변환해야 할까?  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 원-핫 인코딩 (One-hot encoding)\n",
    "\n",
    "'강아지', '고양이', '토끼'를 분류하는 문제가 있다고 하자  \n",
    "손실 함수(loss function)를 계산하기 위해서는 출력 데이터를 숫자로 표시 해야 한다.  \n",
    "\n",
    "Label | 숫자\n",
    "--------|---\n",
    "강아지 | 0\n",
    "고양이 | 1\n",
    "토끼    | 2\n",
    "\n",
    "시그모이드(Sigmoid)나 소프트맥스(softmax)와 같은 활성 함수(activation function)을 사용하기 위해서는 각 레이블(label)이 0과 1로 표현 되어야 한다. 원-핫-인코딩은 '강아지', '고양이', '토끼'를 0과 1로 표현하는 방법이다.\n",
    "\n",
    "Label | 숫자 | 원-핫 벡터\n",
    "--------|-------|---\n",
    "강아지 | 0    | [1, 0, 0]\n",
    "고양이 | 1    | [0, 1, 0]\n",
    "토끼    | 2    | [0, 0, 1]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "⚙️ 엔지니어\n",
    "\n",
    "> 아하~    \n",
    "> 0에서 9까지의 값을   \n",
    "> 원-핫 벡터 (one-hot vector)로 표시하면 되겠군  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "y.shape is  (10, 10)\n",
      "\n",
      "one-hot encoded y is \n",
      " [[1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 1. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 1. 0. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 1. 0. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 1. 0. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 1. 0. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 1. 0. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 0. 0. 0. 0. 0. 0. 1.]]\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.keras.utils import to_categorical\n",
    "\n",
    "y = to_categorical(output_data)\n",
    "print('y.shape is ', y.shape)\n",
    "print('\\none-hot encoded y is \\n', y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "⚙️ 엔지니어\n",
    "\n",
    "> 엇... 어어... 입력 데이터와 출력 데이터가 ???  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 뉴럴 네트워크 모델링 (NN Modeling)\n",
    "\n",
    "입력 레이어(input layer)는 $x_1$에서 $x_{10}$까지 10개로 되어 있다.   \n",
    "히든 레이어(hidden layer)는 3개의 units로 구성 한다.  \n",
    "출력 레이어(output layer)는 10개의 units로 구성 한다.  \n",
    "그림으로 나타내면 다음과 같다.  \n",
    "\n",
    "⚙️ 엔지니어\n",
    "\n",
    "> 선들이 빠짐 없이 조밀하게 연결 되어 있다.  \n",
    "> 음... **덴스(Dense)** 하게 연결 되어있군...   \n",
    "\n",
    "![오토 코딩 모델](https://skettee.github.io/post/neural_network/auto_coding_model.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "히든 레이어에 있는 유닛들의 활성함수(activation function)는 sigmoid로 한다.   \n",
    "출력 레이어에 있는 유닛들의 활성함수(activation function)를 sigmoid로 하면 ~~ㅈ된다~~안된다.  \n",
    "왜냐하면  \n",
    "각 확률의 총합은 1이 나와야 하는데   \n",
    "sigmoid는   \n",
    "$\\hat y_{1}+\\hat y_{2}+\\hat y_{3}+...+\\hat y_{9}+\\hat y_{10}=1$ 을 만족하지 않는다.  \n",
    "\n",
    "⚙️ 엔지니어\n",
    "\n",
    ">  걱정마라.  엔지니어를 갈아서 만든 것이 있다.  \n",
    ">  그것은 바로...\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 소프트맥스 회귀 (Softmax Regression)\n",
    "\n",
    "0과 1, 두가지의 경우에 대한 분류는 로지스틱 회귀(logistic regression)로 모델링을 한다.  \n",
    "세가지 이상에 대한 분류는 소프트맥스 회귀(softmax regression)로 모델링을 한다.  \n",
    "소프트맥스의 활성 함수(activation function)는 아래와 같이 정의한다.  \n",
    "\n",
    "$\\begin{align}\n",
    "z & = w^{T}x+b, \\\\  \n",
    "a & =\\sigma(z) = {\\dfrac{e^z}{\\sum_{j=1}^{m} e^z}},\n",
    "\\end{align}$ \n",
    "\n",
    "$m$ = number of classes (분류 개수)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**소프트맥스 회귀를 이용한 분류**\n",
    "\n",
    "3개로 분류된 아래 데이터를 가지고 10X10 공간을 3개의 클래스로 분류해 보자.  \n",
    "여기서 **클래스(Class)** 란 '공통적인 성질을 가진 종류'라는 뜻이다.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "x_data = np.array([\n",
    "    (1,4), (1,5), (1,6), (1,7), (1,8), (2,5), (2,6), (2,7), (2,8), (2,9), \n",
    "    (3,6), (3,7), (3,8), (3,9), (4,7), (4,8), (4,9), (5,8), (5,9), (6,9),\n",
    "    (3,1), (4,1), (4,2), (5,1), (5,2), (5,3), (6,1), (6,2), (7,1), (8,1),\n",
    "    (7,5), (8,4), (8,5), (8,6), (9,3), (9,4), (9,5), (9,6), (9,7), (9,8)\n",
    "])\n",
    "\n",
    "y_data = np.array([\n",
    "    [1,0,0], [1,0,0], [1,0,0], [1,0,0], [1,0,0], [1,0,0], [1,0,0], [1,0,0], [1,0,0], [1,0,0], \n",
    "    [1,0,0], [1,0,0], [1,0,0], [1,0,0], [1,0,0], [1,0,0], [1,0,0], [1,0,0], [1,0,0], [1,0,0],\n",
    "    [0,1,0], [0,1,0], [0,1,0], [0,1,0], [0,1,0], [0,1,0], [0,1,0], [0,1,0], [0,1,0], [0,1,0],\n",
    "    [0,0,1], [0,0,1], [0,0,1], [0,0,1], [0,0,1], [0,0,1], [0,0,1], [0,0,1], [0,0,1], [0,0,1]\n",
    "])\n",
    "\n",
    "label_data = ['red', 'green', 'blue']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "x_data를 그래프로 표시하면 아래와 같다.  \n",
    "y_data에서 [1,0,0]은 빨강, [0,1,0]은 초록, [0,0,1]은 파랑으로 클래스를 구분한다.    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xt4VeWd9vHvL9k7JDshyCEKKihgi4JYlShFKp6tx6IWq7Y6XqWWqr0ctdN2OtNOx2nnnRnrvG3fGa0tVauO1o4nKvUESj2gIDYcPCCIFTygAlFBCCHn3/vHkxCSbMjeuHdWVrg/15ULXKysdZvsfe9nP2slj7k7IiISHwVRBxARkeyouEVEYkbFLSISMypuEZGYUXGLiMSMiltEJGZU3CIiMaPiFhGJGRW3iEjMJPJx0CFDhviBBx6Yj0OLiPRJixcv/tDdKzLZNy/FfeCBB1JVVZWPQ4uI9Elm9nam+2qqREQkZlTcIiIxo+IWEYkZFbeISMyouEVEYiaj4jazq83sVTNbbmbX5DtUr1FXBz/8IQwbBoMHw/TpsGFD1Kmgtha+9z0YOhSGDIHLL4ePPoo6FWzZAtdeC3vvHT6uugo++STqVLBpE3z721BREXJ95ztQUxN1KvjwQ/jWt8L3cOhQ+Pu/D99bkW5YdyvgmNmhwB+Ao4EG4HHgCnd/Y2efU1lZ6X3idsCTToIFC0KBAyQSocRXroRUKppM7nDMMbBsWXuuZBJGjIDly6Ffv2hytbTAkUeGr019fdhWVAQHHQQvvRS+dlFobITDDoPVq6GhIWzr1w/GjYO//AUKInrTWVcHY8fC2rUhI0BxMUyYAPPng1k0uSQyZrbY3Ssz2TeTR+0hwAvuXuvuTcAzwLmfJmAsVFXBokXt5QjQ1AQffwy//310uebPh1df7ZirsRHWr4cHH4wu19y58Oab7aUNoSjfeQceeSS6XLNnh3JsK20IGVetgnnzost1331QXd1e2hC+p8uWhcGCyC5kUtyvAlPMbLCZpYAzgOH5jdULLF0aRredbd0KL7zQ83naLF3a8cnepqYmjCCjsnQpbNvWdXtNTSijqCxdmn5apK0ko/Lii+lzNTdHm0tiodvidvcVwPXAE4RpkpeAps77mdkMM6sys6rq6uqcB+1xo0ZBYWHX7SUlcPDBPZ+nzahRYQqis9JS+Oxnez5Pm1Gj0k8flZXByJE9n6fN6NEhQ2clJdHmGjMm/dcrmYw2l8RCt3PcXT7B7N+Ate7+q53t0yfmuFtawhzkm2+GKZI2AwbAX/8aLihFoakpzBuvXRtGZxDmQwcOhDVroLw8mlz19aFw1q8PXzsI88dDhoRcUV0T2Lo15Proo/ZchYXhYuDq1elfBHvCpk3hxW7TpvZ3doWF4VrFG2+kHzRIn5brOW7MbO/WP0cA5wH37H68mCgogGeegVNPDaOgZDJcfJs/P7rShnCR7/nn4YQT2nMdfXTYFlVpQ7jgt2ABHHtse65jjgnboiptCO9Enn8eJk1qzzVlSsgVVWkD7LUXPPccHHVUe64TTwzbVNrSjYxG3GY2HxgMNALfcfddXtXpEyPuHdXWhpFulMWYTm1tGHX37x91ko62bg2jyHRTFFGqqQnvTkpLo07S0ZYtoayjfIGTyGUz4s7oHi13P/bTRYq53vqE6q25elsxtultLyRtetsLr/R6+slJEZGYUXGLiMSMiltEJGZU3CIiMaPiFhGJGRW3iEjMqLhFRGJGxS0iEjMqbhGRmFFxi4jEjIpbRCRmVNwiIjGj4hYRiRkVt4hIzKi4RURiJqPfx90jWlrCqupbt4bVSnrL73Rubg6LA9fVhRVdSkqiThQ0NcHCheHPY44JK9D0Bo2NIVdLS8gV5SozO2poCKveFBS0r4YjkiNbtoSaGDAgLGpklt/zZVTcZnYtcBngwCvA1929LmcpXnkFTj8dNm8O/8dNTfDrX8Mll+TsFLtl8WI488yw0oxZKKPf/Q6mTYs218KF8KUvhXUe2x4hd98NZ50Vba6nn4YvfzmUt1koyXvvhVNOiTbXnDlwwQXtazsmEjBrVljCTORT+vWv4TvfCWOBlhaoqIDHH8/v2t3dLl1mZvsBzwFj3X2bmd0LPOrut+/sc7JauqypCfbbDzZs6Li9pARefBEOPTSz4+RaXR3suy9s3Ng118svhwV7o1BTE75emzd33J5Kweuvw/77R5Nr40YYPjy8Y+qca80a2HvvaHKtXx8W5a2t7bi9rAzefTes/SiymxYtCkuF7vjwMgtPhTVrwtglUzlfLJgwMi8xswSQAt7PPE435s2Dbdu6bm9ogN/+Nmenydqjj3Zc3b1NU1MYdUflj39sX618R01NcNddPZ+nzX33tY9od+QO//u/PZ+nzT33pP96ucMDD/R8HulTbrqpa325h3HMCy/k77zdFre7vwf8J/AO8AHwibvP7byfmc0wsyozq6qurs48QecRbZvm5q6j8J60cWP6J3xjI2Tz/5drGzemf0FpaICPPur5PG02bgwZOqur2/n3uCd8/HGYUuqsoSHaXNInVFenH6+YhYdevnRb3GY2EJgKjAT2BUrN7OLO+7n7THevdPfKioqKzBNMmZL+CV9aClOnZn6cXDvxxPDi0VlZWbRzySedlP7KR2kpfPGLPZ+nzcknp78QmUqFf4vKySenX1Q5mYw2l/QJ55yT/uHV0ACTJ+fvvJlMlZwMrHH3andvBB4EjslZgn33he9/v+NdJKkUjB8fLnRFZeRIuPLKjrlKS8Ml4zPPjC7X2LHhom3nXMcdF0o9KhMmwLnnds11+unhLo6oHHssnHpq11zTpsHhh0eXS/qESy+Fz3ymY3mnUvDTn8LAgfk7byYXJycCtwFHAduA24Eqd//vnX1OVhcn2zzxBNx8c7iv5oILQjlFfYubOzz2GMycGa4+fPWr8LWvRX8rmTvMng233BJe2v/mb8LXLBHx3Z0tLfDgg+EagHt4VJ9/fnZXaPKhuTnMwd9xBxQWwvTp4UUm3/dsyR5h27bwkH/gARg8OIz3jj8+++Nkc3Gy2+JuPeC/ABcATcBS4DJ3TzNxGOxWcYuI7MGyKe6Mhmju/s/AP3+qVCIikhP6kXcRkZhRcYuIxIyKW0QkZlTcIiIxo+IWEYkZFbeISMyouEVEYkbFLSISMypuEZGYUXGLiMSMiltEJGZU3CIiMaPiFhGJGRW3iEjMqLhFRGImkzUnx5jZsh0+NpvZNTlP8txzYSWXc86B3/8+LMrbGzz1FFx8MZx3Htx7b/p1KHuaO8ydCxddFJZ3e/DB9AsbR5Hr0UfDajznnw9/+lP6lVRF+pD6erj99rBE7vTpsGhRD5zU3TP+AAqBdcABu9pvwoQJnpV//3f3VMrdzB3cS0vdTzjBvbExu+Pk2o9+FHKF+gm5zjjDvbk52lzXXhuy7Jhr2jT3lpZoc33zm11zXXJJtJlE8qiuzv3oo9sf9gUFoTJuuin7YxGWhMysizPdMRyXU4Hnu9svq+Jet869uLj9yb7jk/6++7L/v8+Vt99On6uszP2RR6LLtXKle0lJ+q/XU09Fl2vp0o4vcm0fqZT7okXR5RLJo1tv7ThWafsoKXHftCm7Y2VT3NnOcV8I3JPDAX+Yiki3+O7WrWEKICpPPhkWlu2spgYeeqjn87R54on00w+1tfDIIz2fp83cuemnt+rqYM6cns8j0gPuvz9UVWfJZJj9zZeMi9vMioAvAfft5N9nmFmVmVVVV1dnnqC8PP1q24WFMGhQ5sfJtfLy9KuTJxIwcGDP52lTXp5+NfdkEvbaq+fztCkvT/8CXFQU/k2kDxo0KH19uef3YZ/NiPt0YIm7r0/3j+4+090r3b2yoqIi86OefPLOn/CXXZZFvBw744z0xZ1Mwte/3vN52pxzTvoRd2FhuIgalWnT0m83CxcrRfqgK66AkpKu2/v3h8mT83febIr7InI9TQKhoOfMgYqK8BJVXh6+Er/4BRx+eM5Pl7FUCh57LLyk7pjrN7+BMWOiy1VeDg8/DAMGtOdKpeCOO+CAA6LLNWRImNrq3789V1lZuBNn6NDoconk0eTJ8NOfQnFxeMj37w/DhoWZw3Tjvlwxz+B2LTNLAe8Co9z9k+72r6ys9KqqquySNDXBM8+ECaPjjgvF1Bs0NsLTT4e52uOPD9+Z3qC+PuRqagq5SkujThTU1YXrFu5wwgnphyMifczHH8P8+aG2jj02/eWx7pjZYnevzGjfTIo7W7tV3CIie7Bsils/OSkiEjMqbhGRmFFxi4jEjIpbRCRmVNwiIjGj4hYRiRkVt4hIzKi4RURiRsUtIhIzKm4RkZhRcYuIxIyKW0QkZlTcIiIxo+IWEYkZFbeISMz0ruJuboaGhqhTdNXUlH4h3Kj11lwiedTQEKqit+nJXBkVt5ntZWb3m9lKM1thZpNymmLTJvja18ISXCUlMGkSvPJKTk+xWz76CM4/P+QqLoYpU2DlyqhTwfr1Ye3JkpLwceKJ8OabUacSyauXX4bPfz485FOpsMTqJ92ux5V/ixfDhAkhV2kpTJ8ONTX5PWemS5fdAcx391taV3tPufumne2f1Qo47uG7sWxZx9F2eTmsWgX77JPZcXLNHQ47DF5/vX1UaxZWUn/zzehWem9uhoMPhrfeCiNuCIvbDR4Mq1eHdR5F+pgPPggP+82b27cVFcGRR8LChdHlevttOPTQjkXdr19Yi3LevOyOldMVcMysHJgC3Arg7g27Ku2sVVXB8uVdp0gaGuC3v83ZabL27LOhHHecinAPaz3ecUdksZgzJ4y420oboKUFamvhntyv5SzSG/zmN+Gpt6OGhvDGfMmSaDIB3Hhj11z19eHFZMWK/J03k6mSUUA18DszW2pmt5hZl5VpzWyGmVWZWVV1dXXmCVatSr8ccl1deG8UlVWrQiF2Vlsb7TTOG2+kvw6wdWt+HykiEXr55a4FCaE6Vq3q+TxtXnop/WWmoqLwVM2XTIo7ARwJ3OzuRwBbgR903sndZ7p7pbtXVlRUZJ5g/Pj0M/qpFEycmPlxcm38+DA10llpKRx1VM/naTN+PCSTXbeXlcERR/R8HpEeMHFimEPurKkpPCWiMnFimBrprL4+TKHkSybFvRZY6+6LWv/7fkKR58Zhh4UJoeLiHVIVhOKePj1np8naxIlw+OEdvyuFhdC/f7gqEpUTToDPfKZjrkQCBg0KF1JF+qDLLgtjph3fnBcXw3HHwbhx0eX69rdDVe04xispgTPPhFGj8nfebovb3dcB75rZmNZNJwGv5TTF7Nlw1VXhAltpabhjoqoquguAEL4Tc+bA5ZeHHKWl8OUvh1xRXgA0g6efDi9qAwaELBdeCC++2PHFT6QPGTw4PMSnTg1FOXgwXH01zJoVba6hQ2HRolDUqRRUVMDf/V3+LzdlelfJ4cAtQBGwGvi6u2/c2f5Z3VUiIiJZ3VWSyGQnd18GZHRAERHJr971k5MiItItFbeISMyouEVEYkbFLSISMypuEZGYUXGLiMSMiltEJGZU3CIiMaPiFhGJGRW3iEjMqLhFRGJGxS0iEjMqbhGRmFFxi4jEjIpbRCRmMvp93Gb2FrAFaAaaMv1l31nZsgX+9KewGO+pp8KIETk/xW755JOQq74eTjsN9tsv6kQiebd2bVgAqrgYzj4bysujThS8/TbMnRsWfjr77GgXo9rR6tUwb174Op19dlgNJ6/cvdsP4C1gSCb7ujsTJkzwrMyb515W5t6/v3sq5V5c7H7dddkdIx8efTTkKStrz3XDDVGnEsmr//iP8FAvLQ1PydJS98cfjzqV+49/HHKlUiFXWZn7U09Fm6mlxf273+2Yq7zcfcGC7I8FVHmGHZvp0mVvAZXu/mEmLwZZLV22bRvss08Yce8olYInn4RJkzI7Tq5t3gzDhoV3ADsqKYGFC+Fzn4sml0geLVkCxx7b9WFfWgrr1kU3wn3uOfjiF7vmKi+H9eujW2517lw47zzYurXj9kGDwtcrmcz8WNksXZbpHLcDc81ssZnNyDxKBubO7bhEcptt2+D223N6qqw88khY1b2z+nq4886ezyPSA+64A+rqum4vKIDHHuv5PG1uuy1UQjrz5vVslh3dckvX0gZoagovNvmS0Rw3MNnd3zezvYEnzGyluz+74w6thT4DYEQ289MNDZBu1O++8+9UT9hVrnSPbJE+oK4OWlq6bncPY5ao1NWlfzpC9Ll2Jp+5Mhpxu/v7rX9uAGYBR6fZZ6a7V7p7ZUVFReYJTj4ZGhu7bi8thQsuyPw4uXbaaeFls7NUCqZN6/k8Ij3g/PPDU6+zxsYwVRGVCy/cea6TTur5PG2++tX0uZqawpRTvnRb3GZWamb92/4OnAq8mrMEAwfCjTeGueNE6xuA0lI44ww4/fScnSZr++wDP/tZyFVYGKZzSkvDI/v446PLJZJHJ50E554bHupm4SlZUgI//zlkMx7LtbPOCi8cbSXZluumm2DAgOhynX8+HHdc+9x/Mhly3XZb+kLPlW4vTprZKMIoG8LUyu/d/f/s6nOyujjZ5vXXw9zxli0wdSqceGL6ue+e9uqrcNddYdrmvPNgypTekUskT9zhmWdg1qzwBvOSS2Ds2KhThVzz5sFDD4WyvuQSGDMm6lRhamnuXHj44XBR8tJLYfTo7I+TzcXJjO4qydZuFbeIyB4sH3eViIhIL6HiFhGJGRW3iEjMqLhFRGJGxS0iEjMqbhGRmFFxi4jEjIpbRCRmVNwiIjGj4hYRiRkVt4hIzKi4RURiRsUtIhIzKm4RkZhRcYuIxIyKW0QkZjIubjMrNLOlZvZwzlO4h1VmJk2C8ePhJz8JK+GIiLTavBmuuw4OPTRUxd1373wB4Z60cSP86Ech1+TJcN99+c+V6SrvAFcDK4DynKe48kr4n/9pX+f+r3+Fe+6BJUvCAm4iskerrYWjj4a3325fWf2VV2DBgrDuZFS2bIEJE+D999tXdX/pJfjLX8KStfmS0YjbzPYHzgRuyXmCNWvg9tvbSxvCd+bdd0N5i8ge7667YO3a9tKGUBm33QbvvBNdrltvhfXr20u7Ldd//3fYni+ZTpX8Evg+0JLzBAsXhqWRO9u6FebMyfnpRCR+5szpOLZrk0yGConKY4+FdwOdFRXBiy/m77zdFreZnQVscPfF3ew3w8yqzKyquro68wRDh6bfnkzCiBGZH0dE+qwRIyCxk4ndnVVITxgxAgoLu25vaclvrkxG3JOBL5nZW8AfgBPN7K7OO7n7THevdPfKioqKzBMcf3xY076gU5RkEr71rcyPIyJ91hVXhFHsjgoKYMgQOPbYaDIB/O3fQr9+HbcVFsLw4VCZ0Xrtu6fb4nb3f3D3/d39QOBC4M/ufnHuEhTAU0+Fu0lKSqCsDCoq4P774aCDcnYaEYmvz34W7r03FHVZWaiKww6DP/+565ivJ40fD3feCQMHQv/+IdeRR8ITT4BZ/s6bzV0l+TNyJCxbBqtXh4mssWPTv/8QkT3WmWfCunXw2muhvEeOjDpR8OUvw9SpIdeAAXDAAfk/Z1bF7e5PA0/nJQnAqFF5O7SIxF9hYRjl9jaJRHgH0FP0k5MiIjGj4hYRiRkVt4hIzKi4RURiRsUtIhIzKm4RkZhRcYuIxIyKW0QkZlTcIiIxo+IWEYkZFbeISMyouEVEYkbFLSISMypuEZGYUXGLiMSMilv2CO7Omo1reHvT21FHkT6ouRlWrYIPPuiZ82WyWHCxmb1oZi+Z2XIz+5eeCCaSK0s+WMJnb/ws4341jkNuOoRxN41j+YblUceSPuKRR2DffcOSZSNHwnHHwfr1+T2nufuudzAzoNTda8wsCTwHXO3uL+zscyorK72qqiq3SUV2w6a6TRzwywPYXL95+zbDGFQyiHeufYdUMhVhOom7116Do46C2tr2bYlEWH1x2bLs1p00s8XuntESw5ksFuzuXtP6n8nWj123vUgv8YdX/0BTS1OHbY5T31zPrBWzIkolfcV//RfU13fc1tQEb74JS5fm77wZzXGbWaGZLQM2AE+4+6I0+8wwsyozq6qurs51TpHd8t7m96htrO2yva6pjve3vB9BIulL1qwJ89udFRbCe+/l77wZFbe7N7v74cD+wNFmdmiafWa6e6W7V1ZUVOQ6p8humTR8EmVFZV229yvsx+f3/3wEiaQvOeUUKCnpur2+HiozmvTYPVndVeLumwirvJ+WlzQiOfbF0V9kXMU4ShLtz65UMsXn9/88XxjxhQiTSV/wzW/CkCFQVNS+rbQULr8chg3L33kzuaukwsz2av17CXAysDJ/kURyp7CgkKcufYp/mvJPjBk8hkOGHMJPjv8Jj37tUSybK0ciaQwYAIsXw1VXwejR4c6Sm2+GX/wiv+fN5K6Sw4A7gEJC0d/r7j/Z1eforhIRkexkc1dJorsd3P1l4IhPnUpERHJCPzkpIhIzKm4RkZhRcYuIxIyKW0QkZlTcIiIxo+IWEYkZFbeISMyouEVEYkbFLSISMypuEZGYUXGLiMSMiltEJGZU3CIiMaPiFhGJGRW3iEjMdPv7uM1sOHAnMBRoAWa6+//LdzCJp1c3vMq9y+/F3Zk2dhqfG/q5qCMBsGzdMh547QEKrICvjPsK4/YeF3UkAKrer2LWilkkC5NcdOhFjBkyJupIEgOZrIAzDBjm7kvMrD+wGDjH3V/b2edoBZw907/N/zf+9dl/paG5AYCiwiK+e8x3+ckJu1wwKe9+OO+H/HLRL6lrqqOAApKFSX583I/5wRd+EGmuax+/lplLZlLXWEdBQQHJgiTXn3I9Vx19VaS5JBrZrIDTbXGnOfhDwI3u/sTO9lFx73n++vFfGX/zeOqa6jpsL0mUUDWjirEVYyPJ9cr6V5h4y0S2NW3rsL04UczyK5czauCoSHItWruIE+88kdrG2o65Cot58+o32bf/vpHkkuhkU9xZzXGb2YGEZcwWZR9L+rLZr88m3SCgsaWRP678YwSJglkrZ21/B9DZ7Ndn93CadvevuJ9tjdu6bC8oKODhVQ9HkEjiJOPiNrMy4AHgGnffnObfZ5hZlZlVVVdX5zKjxECiIJF21XTDSBYkI0gUJAuSFFjXh7lySZxlVNxmliSU9t3u/mC6fdx9prtXuntlRUVFLjNKDJx3yHlptycKEkwbO62H07Q7f9z5JAq6XoN3nHMPOTeCRMFFh15EUWFRl+3N3szZY86OIJHESbfFbWEYdSuwwt1/nv9IEkf7l+/Pr874FcWJYlLJFCWJEooTxfz81J8zcuDIyHIdNOggbjjlBooTxZQkSkglUxQnivn1mb+OdB55/D7jue746zrkKkmUcPs5tzMkNSSyXBIPmdxV8gVgPvAK4XZAgH9090d39jm6OLnnWlezjodWPkSLtzD14Km95iLbe5vfY/brsymwAqYePJWhZUOjjgTAO5+8w8OrHiZZkOScg8+holTvVvdUeb2rJBMqbhGR7OTtrhIREYmeiltEJGZU3CIiMaPiFhGJGRW3iEjMqLhFRGJGxS0iEjMqbhGRmFFxi4jEjIpbRCRmVNwiIjGj4hYRiRkVt4hIzKi4RURiRsUtIhIzKm4RkZjJZOmy28xsg5m92hOBpHtbG7by3bnfZe8b9mbw9YOZ8acZfFj7YdSx2Fy/masfu5qKGyoY8rMhXPnIlWyq2xR1LDZu28gVD1/B4J8NpuKGCq55/Bq21G+JOhbVW6u5bPZlDL5+MPvcsA/fm/s9ahtro47FB1s+4NI/Xsqg6wcx9D+H8sN5P6SuqS7qWKzdvJavPvBV9vqPvdj3/+7LdU9fR0NzQ9SxIpHJ0mVTgBrgTnc/NJODagWc/HF3Jt06iWXrllHfXA+EFcOHDxjOa1e+Rr9Ev0hyNbc0c+TMI3n9w9e35yoqLGL0wNG8fMXLaRfs7QmNzY2Mv3k8azat2f4k71fYj7EVY6maUZV2pfWeUNdUx9ibxrJ281oaWxoBKE4Uc+TQI3lu+nOEpV57Xk1DDQffeDDrt66nqaVpe67Jwyfz5N88GUkmgE11mxhz4xg+qv2IZm8GoCRRwimjTuGhix6KLFcu5XQFHHd/Fvj4U6eSnHj27WdZXr18ezkCNLY0smHrBh5Y8UBkuea+OZfVG1d3yNXQ3MC7m9/l4VUPR5Zr9uuzeW/Lex1GZvXN9bzx8Rs8uTq6Irpv+X1U11ZvL20IZf7S+pd4/t3nI8t118t3salu0/bSbsu1cO1ClnywJLJcty65lS31W7aXNsC2pm08sfoJVn64MrJcUdEcd8wsW7eMxubGLttrGmqoej+6dznL1i1jW+O2LttrGmp4ad1LESQKlq5bSk1DTZftdU11keZ68f0X0+Zq9uZIc72w9gW2Nm7tst2wSHMtWLuAbU1dH1+JgkSkuaKSs+I2sxlmVmVmVdXV1bk6rHQyetBoigqLumxPJVOMGTwmgkTB6EGjSSVTXbaXFZUxauCoCBIFoweOpjRZ2mV7SaIk0lxjBo9J+/VKFiQjzXXIkEMoThR32W5mkeYaO2Qs/Qq7TgO2eEukuaKSs+J295nuXunulRUVFbk6rHRy2kGnMbhkMIVWuH2bYRQnirlo/EWR5Zo6ZiplRWUdchVYAalkimljp0WW6yvjvkIqmeowl11ohZT3K+fsMWdHluviwy6mX2E/jPa57IQlGJIawqmjT40s1/QjpncZGCQLkowYMIIpB0yJKBVcXnk5ycJkh21FBUUcPORgKvfNaFq4T9FUScwkChI8/43nOWnkSSQKEiQKEkzafxILpi+gvF95ZLn6Jfqx8BsLmXLAlO25vjD8Cyz8xkJKkiWR5SotKmXBNxYwefjk7bmOO/A4FnxjQdp3Lj1lr+K9WPCNBUzcb+L2XCePOpnnpz9PYUFh9wfIk4rSCuZ/fT4Thk0gUZAgWZDktINO4+lLn47sginAfuX78fSlT/O5fT5HoiBBUWERZ485m7mXzI00V1QyuavkHuB4YAiwHvhnd791V5+ju0p6xrbGbbR4C6VFXacCotR2S1u6qYAobW3Yipn1ylwFVhDpC1w6NQ01JAoSaadOorSlfgtFhUWR3UGVL9ncVdLtPVqjwL/uAAAEW0lEQVTuHt37b9ml3vZEb9PbirFNb3uBa9Nbc5UVlUUdIa3+/fpHHSFymioREYkZFbeISMyouEVEYkbFLSISMypuEZGY6fZ2wN06qFk18PZufvoQIPpfddeVcmVHubKjXNnpi7kOcPeMfnoxL8X9aZhZVab3MvYk5cqOcmVHubKzp+fSVImISMyouEVEYqY3FvfMqAPshHJlR7myo1zZ2aNz9bo5bhER2bXeOOIWEZFd6DXF3VsXJTaz4Wb2lJmtMLPlZnZ11JkAzKzYzF40s5dac/1L1JnamFmhmS01s+jWLEvDzN4ys1fMbJmZ9YpfX2lme5nZ/Wa2svUxNinqTABmNqb169T2sdnMrukFua5tfby/amb3mFmv+NWFZnZ1a6blPfF16jVTJbuzKHFPMLNhwDB3X2Jm/YHFwDnu/lrEuQwodfcaM0sCzwFXu/sLUeYCMLPvAJVAubufFXWeNmb2FlDp7r3m/l8zuwOY7+63mFkRkHL3TVHn2pGZFQLvARPdfXd/PiMXOfYjPM7Huvs2M7sXeNTdb48qU2uuQ4E/AEcDDcDjwBXu/ka+ztlrRty9dVFid//A3Ze0/n0LsALYL9pU4EHbooXJ1o/IX4XNbH/gTOCWqLP0dmZWDkwBbgVw94beVtqtTgLejLK0d5AASswsAaSA9yPOA3AI8IK717p7E/AMcG4+T9hrijsOzOxA4AhgUbRJgtYpiWXABuAJd+8NuX4JfB9oiTpIGg7MNbPFZjYj6jDAKKAa+F3r1NItZtYbfzn3hcA9UYdw9/eA/wTeAT4APnH3udGmAuBVYIqZDTazFHAGMDyfJ1RxZ8jMyoAHgGvcfXPUeQDcvdndDwf2B45ufcsWGTM7C9jg7oujzLELk939SOB04Nut03NRSgBHAje7+xHAVuAH0UbqqHX65kvAfb0gy0BgKjAS2BcoNbOLo00F7r4CuB54gjBN8hLQlM9zqrgz0DqH/ABwt7s/GHWezlrfXj8NnBZxlMnAl1rnkv8AnGhmd0UbqZ27v9/65wZgFmFOMkprgbU7vFO6n1DkvcnpwBJ3Xx91EOBkYI27V7t7I/AgcEzEmQBw91vd/Uh3n0KY8s3b/DaouLvVehHwVmCFu/886jxtzKzCzPZq/XsJ4UG9MspM7v4P7r6/ux9IeHv9Z3ePfEQEYGalrReXaZ2OOJXwFjcy7r4OeNfMxrRuOgmI9KJ3GhfRC6ZJWr0DfN7MUq3Py5MI15wiZ2Z7t/45AjiPPH/Nul1zsqfsuCixma0lg0WJe8hk4BLgldb5ZIB/dPdHI8wEMAy4o/WKfwFwr7v3qtvvepl9gFmtK4IngN+7++PRRgLgKuDu1imJ1cDXI86zXet87SnAt6LOAuDui8zsfmAJYSpiKb3nJygfMLPBQCPwbXffmM+T9ZrbAUVEJDOaKhERiRkVt4hIzKi4RURiRsUtIhIzKm4RkZhRcYuIxIyKW0QkZlTcIiIx8/8BKZAZ2WEUHs0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.scatter([x[0] for x in x_data ], [x[1] for x in x_data], color=[label_data[i] for i in [np.argmax(x) for x in y_data]])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "소프트맥스 회귀 모델은 Keras로 간단하게 모델링할 수 있다.\n",
    "\n",
    "![소프트맥스 회귀 모델](https://skettee.github.io/post/neural_network/softmax_regression.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras import Sequential\n",
    "from tensorflow.keras.layers import Dense, Activation\n",
    "\n",
    "# 모델을 준비한다.\n",
    "model = Sequential()\n",
    "# 입력 변수 x가 2개, unit 개수가 3개인 output layer를 생성한다.\n",
    "model.add(Dense(3, input_shape=(2,), activation='softmax'))\n",
    "# Loss funtion과 Optimizer를 선택한다.\n",
    "model.compile(loss='categorical_crossentropy', optimizer='rmsprop')\n",
    "# epochs만큼 반복해서 손실값이 최저가 되도록 모델을 훈련한다.\n",
    "hist = model.fit(x_data, y_data, epochs=1000, verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "반복 학습할 수록 손실 값이 0에 가깝게 줄어 든다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEKCAYAAAARnO4WAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAHwNJREFUeJzt3WmQHOWd5/Hvv46+b/Wh1n2BDhAS0EiADAgMMsYYHOETYw8GNmAj7F3b6x2PGXvW4X01x+74iHDYYMAYm/FtjgHWXEKADBK0OHVfSKCzD6nV3eqrjmdfVHarJYTUkjo7qyp/n4iK6szKyvxnp/R7sp/KetKcc4iISP6LBF2AiIiMDQW+iEhIKPBFREJCgS8iEhIKfBGRkFDgi4iEhAJfRCQkFPgiIiGhwBcRCYlY0AUMV1tb66ZNmxZ0GSIiOWPNmjVtzrm6kSybVYE/bdo0mpubgy5DRCRnmNnOkS6rLh0RkZBQ4IuIhIQCX0QkJBT4IiIhocAXEQkJBb6ISEgo8EVEQiLnAz+ddvz0+a28sLk16FJERLJazgd+JGLc8+J2nl2/P+hSRESyWs4HPsCk6mJ2HewJugwRkayWR4HfG3QZIiJZLU8Cv4RdB3txzgVdiohI1sqLwJ9cXUxvIkX74YGgSxERyVp5Efgz68sA2Li3K+BKRESyV14E/sLJVUQMmnceCLoUEZGslReBX14UZ/b4CtbsPBh0KSIiWSsvAh/gwqlVvPFeB6m0PrgVETmevAn8pqk1dPcn2bRP/fgiIseTP4E/rRqAV99tD7gSEZHslDeBP6m6hKnjSli5VYEvInI8eRP4AEtm1bJqezuJVDroUkREsk5eBf5HZtXS3Z/k7V0dQZciIpJ18irwL505DjN4aUtb0KWIiGSdvAr8qpIC5k+sZKUCX0TkA/Iq8CHTj//G+x109SWCLkVEJKvkXeBfdlYtqbTjb1t1li8iMpyvgW9mO8zsHTN708ya/dzWoIum1VBRFOPZDS1jsTkRkZwRG4NtXOmcG7PT7Xg0wpVz6lm+sYVU2hGN2FhtWkQkq+Vdlw7ANfMaOHB4QIOpiYgM43fgO+BpM1tjZnf4vK0hV5xdRyxiLN+obh0RkUF+B/4S59wFwMeBr5rZ5ccuYGZ3mFmzmTW3traOykbLi+JcNK2GFZsU+CIig3wNfOfcHu+5BXgYWHScZe5xzjU555rq6upGbdtLZ9excV8Xew/p5uYiIuBj4JtZqZmVD/4MLAPW+rW9Y105px6AFzaNzl8NIiK5zs8z/AZgpZm9BbwKPOGc+6uP2zvKWfVlTKwq5nl164iIAD5elumc2w4s8Gv9J2NmXDG7jkff2M1AMk1BLC8vSBIRGbG8TsErZ9dzeCClm5uLiJDngX/pzHEURCOsUD++iEh+B35pYYxF03V5pogI5HngQ+byzM37u9l1sCfoUkREApX3gT94eebz6tYRkZDL+8CfUVvKlJoSntcwCyIScnkf+GbGVXPqeXlbG32JVNDliIgEJu8DHzLdOn2JNK9sbw+6FBGRwIQi8BdPr6E4HlW3joiEWigCvygeZcmsWpZvbME5F3Q5IiKBCEXgA1w5p45dB3vZ1toddCkiIoEIT+DPzlyeqZuiiEhYhSbwJ1QVM2d8uQJfREIrNIEPcNWcepp3HKSzLxF0KSIiYy5UgX/lnHqSacfKLW1BlyIiMuZCFfjnT66isjiubh0RCaVQBX4sGuGKs+tYsamFdFqXZ4pIuIQq8CHTj9/WPcA7uw8FXYqIyJgKXeBffnYdZuhetyISOqEL/JrSAs6fXKVhFkQkdEIX+JDp1nlr1yFau/qDLkVEZMyEMvAHb4rywmbdFEVEwiOUgT+vsYL68kL144tIqIQy8M2MK2fX8+LmVpKpdNDliIiMiVAGPmRGz+zqS7Jm58GgSxERGROhDfwls2qJR003NxeR0Aht4JcXxbloWg3LN+4PuhQRkTHhe+CbWdTM3jCzx/3e1qm6em4Dm/d3827b4aBLERHx3Vic4X8d2DAG2zll1547HoD/t3ZvwJWIiPjP18A3s0nAJ4B7/dzO6ZpQVcyCSZU8tXZf0KWIiPjO7zP8HwHfBrL22sdl54znrV2H2HeoL+hSRER85Vvgm9n1QItzbs1JlrvDzJrNrLm1deyvmPnYOQ0APLNBH96KSH7z8wx/CXCDme0AfgdcZWa/OXYh59w9zrkm51xTXV2dj+Uc38y6MmbUlvL0OnXriEh+8y3wnXN3OecmOeemAV8AljvnvuTX9k6XmXHNvAZWbW/XvW5FJK+F9jr84Zad00Ai5VihL2GJSB4bk8B3zq1wzl0/Fts6HQsnV1NTWsBy9eOLSB7TGT4QjRhLZ9exQoOpiUgeU+B7rp7bQEdPQoOpiUjeUuB7Lj+7joJohGfWq1tHRPKTAt9TVhhjyaxxPL1+P865oMsRERl1Cvxhlp0znvcO9LBpf1fQpYiIjDoF/jAfnVuPGTy9Tt06IpJ/FPjD1JcXccGUap5er2/dikj+UeAfY9m8Btbu7mR3R2/QpYiIjCoF/jGWnZMZI/8Zja0jInlGgX+M6bWlnFVfxtO6PFNE8owC/ziWndPA6ncP0NEzEHQpIiKjRoF/HMvmjSeVdjy3oSXoUkRERo0C/zjmT6xkfEWRrtYRkbyiwD+OSCQzRv4Lm1vpHUgFXY6IyKhQ4H+Ij50znr5EmpVb24IuRURkVCjwP8TiGTWUF8V060MRyRsK/A8Rj0b46Jx6nt2wX2Pki0heUOCfwLXnjudgT4JXtrcHXYqIyBlT4J/A0tn1lBfFePiN3UGXIiJyxhT4J1AUj3LduY08tXYffQldrSMiuU2BfxLXndfI4YEUL2/T1ToiktsU+CdxyYxxlBfGNEa+iOQ8Bf5JFMQiLPWu1kmldetDEcldCvwRWDavgbbuAd5472DQpYiInDYF/ggsnV1HPGoaMllEcpoCfwTKi+JcOrOWp9btwzl164hIblLgj9A18xrY2d7DlpbuoEsRETktvgW+mRWZ2atm9paZrTOzH/i1rbFwzbwGAI2tIyI5y88z/H7gKufcAmAhcK2ZXezj9nzVUFHEwslV6scXkZw1osA3s6+bWYVl3Gdmr5vZshO9x2UM9n/EvUdOd4AvO6eBt3cdYu+h3qBLERE5ZSM9w7/NOdcJLAPqgFuBfz7Zm8wsamZvAi3AM8651addaRZYNm88AM/qLF9EctBIA9+85+uAXzrn3ho270M551LOuYXAJGCRmZ37gRWb3WFmzWbW3NraOtK6AzGrvowZdaXq1hGRnDTSwF9jZk+TCfynzKwcGPEg8c65DmAFcO1xXrvHOdfknGuqq6sb6SoDs2zeeF7Z1s6h3kTQpYiInJKRBv7twHeAi5xzPWT642890RvMrM7Mqryfi4GrgY1nUGtWWHZOA8m0Y8WmlqBLERE5JSMN/EuATc65DjP7EvA94NBJ3tMIPG9mbwOvkenDf/z0S80OCydVUVdeqMHURCTnxEa43M+ABWa2APg2cB/wIHDFh73BOfc2cP4ZV5hlIhHjmnkNPPLGbnoHUhQXRIMuSURkREZ6hp90mTEFbgR+7Jz7MVDuX1nZ7fr5jfQMpFi+Ud06IpI7Rhr4XWZ2F/Bl4Akzi5Lpxw+lxTPGUVtWyONv7wm6FBGRERtp4H+ezDdnb3PO7QMmAv/mW1VZLhoxPjF/PMs3ttDdnwy6HBGRERlR4Hsh/xBQaWbXA33OuQd9rSzLXb9gAv3JtL6EJSI5Y6RDK3wOeBX4LPA5YLWZfcbPwrLdhVOqGV9RpG4dEckZI71K57tkrsFvgcw19sCzwJ/8KizbRSLGJ85r5MFXdnCoN0FlcWg/0hCRHDHSPvzIYNh72k/hvXnrkwsmkEg5DZksIjlhpKH9VzN7ysy+YmZfAZ4AnvSvrNywYFIlk2uKeewtdeuISPYb6Ye2fw/cA5wHLADucc79g5+F5QIz44YFE/jb1jZau/qDLkdE5IRG3C3jnPuzc+5/OOe+6Zx72M+icsmNCyeSdvCEPrwVkSx3wsA3sy4z6zzOo8vMOseqyGx2dkM5c8aX86i6dUQky50w8J1z5c65iuM8yp1zFWNVZLb71PkTeeO9Dt5r7wm6FBGRDxX6K21GwycXTADgsbd2B1yJiMiHU+CPgolVxSyaVsMjb+4hM8aciEj2UeCPkhsWTmBrSzcb9nYFXYqIyHEp8EfJdfMbiUWMR9WtIyJZSoE/SmpKC7j87Dr+8809pNPq1hGR7KPAH0U3LpzAnkN9NO88GHQpIiIfoMAfRVfPbaA4HuXRN9WtIyLZR4E/ikoLY1wzr4En3tnLQDIddDkiIkdR4I+yT50/gY6eBCu3tgZdiojIURT4o+yys+qoLonzyBsaakFEsosCf5TFoxGum9/IM+v3c1j3uxWRLKLA98GNCyfSm0jx7Abd71ZEsocC3wdNU6uZUFnEo2+qW0dEsocC3weRiPHJhRN4cXMrBw4PBF2OiAigwPfNjQsmkkw7nnxnb9CliIgACnzfzG0s56z6Mh5Tt46IZAnfAt/MJpvZ82a2wczWmdnX/dpWNjIzblw4gVd3HGB3R2/Q5YiI+HqGnwS+5ZybC1wMfNXM5vm4vaxz48KJABpqQUSygm+B75zb65x73fu5C9gATPRre9lock0Ji6bX8PvX3tcImiISuDHpwzezacD5wOrjvHaHmTWbWXNra/4NR3Dz4insbO/h5W3tQZciIiHne+CbWRnwZ+AbzrnOY193zt3jnGtyzjXV1dX5Xc6Yu/bc8VSXxHlo9c6gSxGRkPM18M0sTibsH3LO/cXPbWWrwliUzzZN5pn1+2np7Au6HBEJMT+v0jHgPmCDc+7f/dpOLrhp0RSSaccfmt8PuhQRCTE/z/CXAF8GrjKzN73HdT5uL2tNry3l0pnj+O2r75PSh7ciEhA/r9JZ6Zwz59x5zrmF3uNJv7aX7W5ePJXdHb28uCX/PpgWkdygb9qOkWvmNVBbVsBDq94LuhQRCSkF/hgpiEX4bNNklm/cz95D+uatiIw9Bf4Y+uKiKTjgP1brLF9Exp4CfwxNrinh6rkN/HrVTnoGdDcsERlbCvwxduflM+joSfCnNbuCLkVEQkaBP8aaptWwYHIVD7y8Q+PriMiYUuAH4CuXTmV762FWbm0LuhQRCREFfgCum99IbVkhD7y8I+hSRCREFPgBKIxFuXnxFJ7f1MKOtsNBlyMiIaHAD8jNi6cQNePBVzSKpoiMDQV+QOorivjEeY38sfl9uvt1iaaI+E+BH6DbPzKdrv4k9698N+hSRCQEFPgBOm9SFdfMa+Del7bT1ZcIuhwRyXMK/ID9t6tm0dmX5CENtyAiPlPgB+y8SVVcdlYt9770Ln2JVNDliEgeU+Bnga9eOYu27n7dEUtEfKXAzwKLp9dw4dRq7n5hO4lUOuhyRCRPKfCzgJnxtatmsbujl9+9prN8EfGHAj9LLD27jotn1PCjZzbrih0R8YUCP0uYGd+9bh7thwf4+Qvbgi5HRPKQAj+LzJ9UyacWTuDel95ld4dugygio0uBn2X+58dmEzHjnx5Zi3MaL19ERo8CP8tMqi7hW8vOZvnGFh57a0/Q5YhIHlHgZ6Fbl0zn/ClVfP+xdbR09QVdjojkCQV+FopGjH/7zHn0DKT4x7+oa0dERocCP0vNqi/n75fN5tkN+3XDcxEZFQr8LHbbR6azeHoN331kLau3twddjojkON8C38zuN7MWM1vr1zbyXTRi/PxLFzK5upj/8mAzm/Z1BV2SiOQwP8/wHwCu9XH9oVBdWsCvbltEcTzKLfe/ys523QNXRE6Pb4HvnHsROODX+sNkUnUJv7ptEf3JFJ/9+StsbdGZvoicOvXh54i5jRX87o5LSDv4/N2rWL+nM+iSRCTHBB74ZnaHmTWbWXNra2vQ5WS12ePL+cOdF1MQi3DTL1bx9q6OoEsSkRwSeOA75+5xzjU555rq6uqCLifrzagr4w93XkJFcYybf7Ga5Rv3B12SiOSIwANfTt3kmhL+cOclTKwu5rYHmvnHh9/R7RFF5KT8vCzzt8ArwGwz22Vmt/u1rTBqrCzm0a8t4c4rZvAfq9/jUz/9mz7MFZET8vMqnZucc43OubhzbpJz7j6/thVWhbEod318Lr+89SJauvr5xE9W8su/vUs6raEYROSD1KWTB66cXc9fv3EZS2bV8oP/XM8X713F+wd6gi5LRLKMAj9P1JcXcd8tTfzrp89j3e5OPvajF3nwlR062xeRIQr8PGJmfO6iyTz1zctpmlbD/3p0HV+8dxVb9qtvX0QU+HlpQlUxv7r1Iv7l0/NZv6eTa3/8Et975B3auvuDLk1EAhQLugDxh5nx+YumcM288fzkuS38etVOHnljD393yVRuXTKduvLCoEsUkTFm2XRzjaamJtfc3Bx0GXlpa0s3P3xmM0+u3Us8GuFzTZO447KZTBlXEnRpInIGzGyNc65pRMsq8MNle2s3v3hpO39es5tkOs31503gv14xk3kTKoIuTUROgwJfTmp/Zx/3r3yX36zayeGBFFecXccXF0/hytn1FMT00Y5IrlDgy4gd6knwm9U7eeDlHbR29VNdEueGBRP49IWTmD+xEjMLukQROQEFvpyyZCrNS1va+PPru3h6/X4GkmmmjSth6ex6ls6u4+IZ4yiKR4MuU0SOocCXM3KoN8ETb+/l6fX7eGVbO/3JNIWxCJfMHMcVZ9exdHY902tLgy5TRFDgyyjqS6RYtb2dFZtaeWFzK++2ZW6xOHVcCUu98L94xjiKC3T2LxIEBb74Zmf7YV7Y3MqKTa28vK2NvkSagliExdNrWDq7nsXTa5gzvpxYVB/8iowFBb6Mib5Eitd2HGDFplZWbGphW2vm7L84HmXB5EounFrNBVMyj+rSgoCrFclPCnwJxO6OXtbsPMjrOw/y+nsHWbenk5Q3eNvEqmLmNlYwt7GcuY0VzBlfztRxpUQjugpI5EycSuBraAUZNROriplYVcwNCyYA0DuQ4u1dHbz+Xgfr93ayYW8nyzfuZ3AAz+J4lNnjy4c1AhXMbiinsiQe4F6I5C+d4cuY6kuk2LK/mw37Mg1A5tHFod7E0DK1ZYXMrCtlZn0ZM+vKmFlXytRxpUyoKqIwpg+HRYbTGb5kraJ4lPmTKpk/qXJonnOOfZ19bNjbyZb93Wxr7WZb62GefGcvHT2Jo95fX17IxOrMXxKTqkuYWF3MpOpiGiuLaCgvoqokri+LiXwIBb4EzsxorCymsbKYq+Y0HPXagcMDbG3p5v0DPezu6GXXwczz2t2HeGrdPhKpo/9CLYhFaKgoZHxFEeMri6krK6S2vIDaskLqygoZV5b5eVxZgf5akNBR4EtWqyktYNH0GhZNr/nAa+m0o7W7n10He9l7qJf9nf3s7+xjf2cf+w718fauDlq7+ukZSB133RVFMWrLCjMPr1GoHdYoDG8kSgv1X0Vyn/4VS86KRIyGiiIaKoqA6g9drmcgSXv3AK3d/bR19dN+eIC2rn7auvtp8+Zv2tfF37rbj/osYbjiePSoRqG2rICK4jgVRXEqiuOUF8YoL4pRVhijvChOedGRaX0nQbKFAl/yXklBjJKaGJNrTj72/0AyTfvh/qMaiLbuAdq6+2n3Goj3D/Tw5vsdHOpNMJBMn3SdxfFoJvyLvMbAaxxKC2OUFEQpKYhRWhClpDDzXFwQpbQgRklhlOJ4Zro4nnkUeT/H1YjIaVDgiwxTEIsMfZ4wEn2JFJ19Cbr7knT1JenuT9LVl6DrmOnu/iSdfUlvuQT7O/voGUhxeCBJT3+KgdTJG47hYhGjOB6lMB6luCBypEE4TgNRFDuyzPFeH5o/NO/I+vTXSX5R4IucgSIvLOvLz2w9iVSanoEUPQNJDvcfee5LpOhNpOgdyDz3JYbPSw/NG/76wcMD7Bm2zODyg1+COxXxqA01BkVHNRKRob8+jmosvHmFsQiF8SiF0QgFMe8RjVAYzzwPziuMRYhHI8SiEeJRoyCamY5707rianQp8EWyQDwaobI4QmWxf186S6S8BsJrHD7YkKSPmu4dtlxfIv2Bee2HB+g9ePTrPQNJTqNd+VCxiA2F/2CjEY8daTTiQ89GLHKkoRhsQOKRCLGoHT3fW+fgMrGIEY9FhpaNRSMUeOuLeY1QLOqtZ2h9RjQSGapv8LWot75oJPOcbQ2WAl8kJAbPnCuK/GtUnHMkUo7eRIqBZJqBVJqBZJr+pDedTNM/7DmROvIYSDmSQ9OOgaNed0PrSgx7HlxXXyJNMpUcWkcy7Ybem0y5oXUk0+kPXMrrp4hBLBIZagAGG4ShedFM41BbVsgf7rzE93oU+CIyasyMgphl9W0ynXOk0pmGKZHONAjJVKZxSnqNwkDySOOQ9BqLwWVT6SONR2b5I41MKu2GnhOp9FHTg+898vqR6bIxuuzX162Y2bXAj4EocK9z7p/93J6IyMmYmdd1A8WE68t3vjXDZhYFfgp8HJgH3GRm8/zanoiInJiff3ctArY657Y75waA3wE3+rg9ERE5AT8DfyLw/rDpXd48EREJgJ+Bf7zrkT7w8biZ3WFmzWbW3Nra6mM5IiLh5mfg7wImD5ueBOw5diHn3D3OuSbnXFNdXZ2P5YiIhJufgf8acJaZTTezAuALwGM+bk9ERE7At8synXNJM/sa8BSZyzLvd86t82t7IiJyYr5eh++cexJ40s9tiIjIyGTVPW3NrBXYeZpvrwXaRrGcXKB9Dgftc/47k/2d6pwb0QegWRX4Z8LMmkd6I998oX0OB+1z/hur/c3eAS9ERGRUKfBFREIinwL/nqALCID2ORy0z/lvTPY3b/rwRUTkxPLpDF9ERE4g5wPfzK41s01mttXMvhN0PaPFzCab2fNmtsHM1pnZ1735NWb2jJlt8Z6rvflmZj/xfg9vm9kFwe7B6TOzqJm9YWaPe9PTzWy1t8+/9765jZkVetNbvdenBVn36TKzKjP7k5lt9I73Jfl+nM3sm96/67Vm9lszK8q342xm95tZi5mtHTbvlI+rmd3iLb/FzG45k5pyOvDzfMz9JPAt59xc4GLgq96+fQd4zjl3FvCcNw2Z38FZ3uMO4GdjX/Ko+TqwYdj0vwA/9Pb5IHC7N/924KBzbhbwQ2+5XPRj4K/OuTnAAjL7nrfH2cwmAv8daHLOnUvmm/hfIP+O8wPAtcfMO6XjamY1wPeBxWSGnP/+YCNxWpxzOfsALgGeGjZ9F3BX0HX5tK+PAtcAm4BGb14jsMn7+W7gpmHLDy2XSw8yg+w9B1wFPE5m1NU2IHbsMSczbMcl3s8xbzkLeh9OcX8rgHePrTufjzNHhk6v8Y7b48DH8vE4A9OAtad7XIGbgLuHzT9quVN95PQZPiEZc9/7E/Z8YDXQ4JzbC+A913uL5cvv4kfAt4G0Nz0O6HDOJb3p4fs1tM/e64e85XPJDKAV+KXXjXWvmZWSx8fZObcb+D/Ae8BeMsdtDfl9nAed6nEd1eOd64E/ojH3c5mZlQF/Br7hnOs80aLHmZdTvwszux5occ6tGT77OIu6EbyWK2LABcDPnHPnA4c58mf+8eT8PntdEjcC04EJQCmZLo1j5dNxPpkP28dR3fdcD/wRjbmfq8wsTibsH3LO/cWbvd/MGr3XG4EWb34+/C6WADeY2Q4yt8S8iswZf5WZDQ70N3y/hvbZe70SODCWBY+CXcAu59xqb/pPZBqAfD7OVwPvOudanXMJ4C/ApeT3cR50qsd1VI93rgd+3o65b2YG3AdscM79+7CXHgMGP6m/hUzf/uD8v/M+7b8YODT4p2OucM7d5Zyb5JybRuZYLnfO3Qw8D3zGW+zYfR78XXzGWz6nzvycc/uA981stjfro8B68vg4k+nKudjMSrx/54P7nLfHeZhTPa5PAcvMrNr7y2iZN+/0BP2hxih8KHIdsBnYBnw36HpGcb8+QuZPt7eBN73HdWT6Lp8DtnjPNd7yRuaKpW3AO2SugAh8P85g/5cCj3s/zwBeBbYCfwQKvflF3vRW7/UZQdd9mvu6EGj2jvUjQHW+H2fgB8BGYC3wa6Aw344z8Fsyn1EkyJyp3346xxW4zdv3rcCtZ1KTvmkrIhISud6lIyIiI6TAFxEJCQW+iEhIKPBFREJCgS8iEhIKfJFRYGZLB0f3FMlWCnwRkZBQ4EuomNmXzOxVM3vTzO72xt7vNrP/a2avm9lzZlbnLbvQzFZ545M/PGzs8llm9qyZveW9Z6a3+rJh49o/5H2LVCRrKPAlNMxsLvB5YIlzbiGQAm4mM3jX6865C4AXyIw/DvAg8A/OufPIfPtxcP5DwE+dcwvIjAEzOLTB+cA3yNybYQaZsYFEskbs5IuI5I2PAhcCr3kn38VkBq9KA7/3lvkN8BczqwSqnHMvePN/BfzRzMqBic65hwGcc30A3vpedc7t8qbfJDMW+kr/d0tkZBT4EiYG/Mo5d9dRM83+6ZjlTjTeyIm6afqH/ZxC/78ky6hLR8LkOeAzZlYPQ/cXnUrm/8HgKI1fBFY65w4BB83sMm/+l4EXXOaeBLvM7FPeOgrNrGRM90LkNOkMRELDObfezL4HPG1mETKjGH6VzE1HzjGzNWTupvR57y23AD/3An07cKs3/8vA3Wb2v711fHYMd0PktGm0TAk9M+t2zpUFXYeI39SlIyISEjrDFxEJCZ3hi4iEhAJfRCQkFPgiIiGhwBcRCQkFvohISCjwRURC4v8D/V1U9Z7z+fwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(hist.history['loss'])\n",
    "plt.xlabel('epoch')\n",
    "plt.ylabel('loss')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "첫번째 x 데이터 (1,4)에 대해서 소프트맥스가 예측한 데이터는 다음과 같다.  \n",
    "예측된 확률을 모두 더하면 1이 되는 것을 확인할 수 있다. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "y data is  [1 0 0]\n",
      "predicted y data is  [0.980123   0.0051288  0.01474817]\n",
      "sum of probability is  0.99999994\n"
     ]
    }
   ],
   "source": [
    "predict = model.predict(x_data[0].reshape(1, 2))\n",
    "print('y data is ', y_data[0])\n",
    "print('predicted y data is ', predict[0])\n",
    "print('sum of probability is ', np.sum(predict[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "학습이 완료된 소프트맥스 회귀 모델을 이용해서  \n",
    "10X10 공간을 3개의 클래스로 분류한 그래프를 그려 보자.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnXl4U2Xaxu+3TdIsbWmhBZStgCwKoiwCyiqKyuaGCyq4sDkqOjrfjIMoOn7jMt+IjM6wiSLiBiIu6FAQ2Qoqi6yyKshaFilbgS40aZ7vj5dS0pyUpDTPyZHnd1251GPIuXvyy9PkPeHcioggCIIgWIc4swMIgiAIkSGDWxAEwWLI4BYEQbAYMrgFQRAshgxuQRAEiyGDWxAEwWLI4BYEQbAYMrgFQRAshgxuQRAEi2GLxoOmpaVRRkZGNB5aEAThd8mqVasOEVF6OPeNyuDOyMjAypUro/HQgiAIv0uUUrvCva8slQiCIFgMGdyCIAgWQwa3IAiCxZDBLQiCYDFkcAuCIFiM2BjceXnAk08CVasCiYlAv37A3r38ObZvB269FfB4gLQ04JlngFOn+HMsXAi0aQO4XEDDhsCUKfwZiIDx44F69XSOq68Gli7lz5GXBzz1VKkbd98NZGfz59ixA7jttlI3hg83x41Fi4Crrip1Y/Jk/gxEwIQJgW788AN7jPx84E9/ih01EhNL1SgsjPJOiajSb61bt6aw8fuJOnQgSkgg0koQxccT1axJdPx4+I9zvuTkEFWrRhQXV5rD5SLq1YsvAxHR4sVEbndpBkD/95tv8uZ48UUijyc4x8qVfBn8fqKOHYmcTnPdOHQo2A2nk6hnT74MRERLlhi78cYbvDn+/nfT3fD7iTp1ClajRg2i3Fy2GCHV6NEj8scCsJLCnLHmD+6lS4MlAPS2ceMi/+kryssv60FdNofLRbRpE1+ODh2CMwBEKSlEPh9PhoIC4+dEKaLevXkyEBEtW2acw+0mGjuWL8crr4R2Y+NGvhwdO4Z2w+vlyVCeG4xvcpYvD63GmDFsMegf/witxoYNkT1WJIPb/KWSDRv0z1qWvDyA8y/xrFgBFBQEb7fZdEYuNm0y3l5YCBw+zJMh1DIVEbBmDU8GILQb+fkXphsbNxpv53Rj3z7j7UTA2rU8GRD6sOfnAz/+yBYDy5ebo4b5g7tRIyDOIIbbDTRvzpfj8suBhITg7cXFOiMXDRoYb7fbgdRUngw1awJ+v/H/a9qUJwMQ2g2XC2jWjC9HrLjRsKHxdptNL/RyUJ4bjRvzZIA+7EoFb3e5YmNs+P1RViPct+aR3CJe427enMhuD/zYlZpKdPhwZJ81zoe9e4mSkgI/7yQk6KULTjIzjdcx//Y33hxPPWWcIyuLL4PfT3T55cFupKSY74bDQXTNNXwZiIhmzw7+XG6GG3/6k7EbixaxRfD7iVq0MFbj0CG2GLRvn7EaV18d+WPBUmvcRPpI33mnfhbi4vRZh82bI//Jz5e1a4nattUZHA6iAQN4z3SU8MknRHXq6BwpKXr93e/nzeDzEY0cqa2MiyOqX5/o6695MxDpAX22Gx07mudGu3bmuzF9erAbxcW8GXw+ouefN92Nw4eJ7rqrVI0OHXhPR5Wwbl2gGv37V0yNSAa3IqM1xPOkTZs2VKGLTBUX688YdnulZ4qIoiIgPl7fzIJI53A4jD8Tcucw+jzIibhRirgRQHGxvjkcpsY4bzWUUquIqE04943K1QErjNkviBLMNgDQL0izh2Us5RA3SomV5yRGclyIaph/clIQBEGICBncgiAIFkMGtyAIgsWQwS0IgmAxZHALgiBYDBncgiAIFkMGtyAIgsWQwS0IgmAxZHALgiBYDBncgiAIFkMGtyAIgsWQwS0IgmAxYmtwe70MLZthUFCgs5gJEXDyZOiL1nNRXKzbiKJwFcmIEDdKETcCuBDVCGtwK6WeUkptVEptUEpNVUo5KzXFoUPA7bfrBu3ERN0aHarCK5qsWQO0bg0kJekc990H5Oby55g6FahdWzfepKYCf/87/4u0uBgYMQJISdG3+vWBmTN5MwDajb59Y8ONNm1K3bj3XnPcmDat1I2UFOB//9d8NzIygC+/5M0A3dZ2xx2BaoRqd4sma9cGqnHPPcCxY1He6bku2A2gFoAdAFyn/3s6gAfL+zMRN+A0a2Z+y0l2tnGVRfv2fBmIiP77X+N2keef583xxBOmt5yU6wZnzUmsuDFrlvFzMnIkb44//tE4x8KFbBFCFWdVqUKUk8MWI2Q5Urt2kT8WolAWbAPgUkrZALgBhGgMrQBZWcCuXYGfMYiAU6eAyZMrbTfnZPx4fSX0sykqAtav5y3Iff553Xh6Nvn5wOjRfJ/D8vKAiRONc7z4Ik8GAFi8ODbcmDAhtBurV/PlCOXGv/4VnC9axIgbS5YAO3cGq1FUFBtqbNgArFoVvf2ec3AT0V4AowDsBrAfQC4RzS17P6XUUKXUSqXUypycnPATbNtm/FGvoID3I/GGDXoglCUuTmfkYscO4+0+H3D0KE+GAwdCX5n+l194MgDiRlm2bzfezunGb78ZFzgDwNatPBmgD7vR0npBAe9yiVlqnHNwK6VSAdwCoD6AiwF4lFL9y96PiCYSURsiapOenh5+ghYtjOuXPB7gqqvCf5zzpV07XRFdFp9PVzlzEaqi2uUCqlXjyVCrlvFzohTQqhVPBiB23GjfPrQbLVrw5QjlYay40bIlTwbow240uD0eoG1bthjmqXGutRQAdwKYdNZ/3w9gXHl/JuI17s6diZzO0kUim43o4ouJTpyIfKGoohw6RJSWphs/S3K4XEQ338yXgYjou++Mm7zHjuXN8fLLxuuYq1fzZfD7ibp0CXbjootiw40+ffgyEIV2Y8wY3hyvvELk8QTnWLWKNUbXroFqxMdrNY4f58tw+DBRenqwGr16Rf5YqMyWdwDtAGyEXttWAKYAeLy8PxNxy3teHtGf/0xUrZpe6b/3Xr3qz82OHUR9+xIlJhJVr65P+pw6xZ8jK0u3zbvdRI0aEX34IX8Gv59o4kTd4O12E3XqRLRsGX+O/HztRlqa+W7ccUepG889Z54b7dqVuvHBB/wZ/H6it98matBA5+jQgWjpUvYYZdW45x5z1Ni5U6uRlKTVGDGCqLAw8seJZHCH1fKulHoRwN0AfADWABhMRAYrO5oKt7wLgiBcoFR6yzsRvQDghfNKJQiCIFQKsfU3JwVBEIRzIoNbEATBYsjgFgRBsBgyuAVBECyGDG5BEASLIYNbEATBYsjgFgRBsBgyuAVBECyGDG5BEASLIYNbEATBYsjgFgRBsBgyuAVBECxGbA3ufft0A0wYVyyMGn4/8OuvuunDTAoLddvM8ePm5jh2TDebcFVjhWL//thx48AB8zIAMePGsWM6hqih9719O58asTG4d+7UjSYNGgDNmulG8e+/588xbx5Qp46urqhXD+jUSf8y4eaf/wTS03XjfI0awODB/K+OggLdcl+zpm69SU8Hxo3jzQBoN9q21U6UuPHdd/w55s0D6tbVbmRkmOfGqFGBbgwaxO7G2Wq0bg2kpZmjxq5durjKbDXmz9dj4/LLtRodOwJ790Z5p+FeuDuSW0RFCl4vUZ06gRUSgG7Y4Lwq+tatwY0v8fFEl16qLxzPxUcfBedwuYgee4wvAxHRffcF1ouUtJx89RVfBp+PqG5d893Yts3YjaZNed34+GNjNx59lC8DEQ0YYKzGzJl8GcpTIzubL0coNZo0iVwNRKHlPXrMm6c/c5UthfX5eOuax40LblEvLgb27AGWLePL8dJLwQ3aBQXAu+8at5JGg9xcYMYM/ZH8bPLzgVde4ckAaDeOHjV2Y9Ikvhzjxxu7kZ0NLF3KlyOUG5MnBz9XUeL4cWD6dPPVmD8/NtSYMMFYjb17gR9+iN5+zR/ce/fqn7Qsp06FbjyPBjt2BD8DgC5BjfrnnrMItUjm9/OtaR4+DNhCdGxkZ/NkAPRxN2p553Zj+3Zx4zSiRiBmqWH+4G7XzvisQmIi0KULX45u3QC3O3i718vfKG5Eaipfk3edOsavzrg4vbbLRbt2xq9Ojwfo2pUvRyy5YdSwnpqqF5oZqF0bsNuDt3Or0bbtBa5GuGsqkdwiLgvu2zdwoSghQa8fVqRxs6IcP05Urx6RwxG4cDd4MF8GIqK1a/VC3dmLd2430bRpvDkmTgx8TuLjdRvqzz/z5rjjjmA3mjQhKijgy3DiBFFGRrAbAwfyZSAiWrdOlxWXdWPqVNYYb78d+JTExZmjxl13BavRuHFsqPHQQ5E/Fiqz5b0it4gHt9dL9J//EF12GVHDhromOTc38p/8fDl0iOh//kc3m7dooYdXcTF/jg0b9MCqW5eoa1ei+fP5MxARZWbqBu+6dXW7+i+/8Gfw+QLdGD6c6Ngx/hyHDulKcbPd2LgxJtzIzCTq2NF8NcaMIWrWzFw1Dh8OVOOttyqmRiSDO6yW90iRlndBEITIiKTl3fw1bkEQBCEiZHALgiBYDBncgiAIFkMGtyAIgsWQwS0IgmAxZHALgiBYDBncgiAIFkMGtyAIgsWQwS0IgmAxZHALgiBYDBncgiAIFkMGtyAIgsWQwS0IgmAxYmdw+/3AqlW6JNjM2uiCAmDJEmDtWnNro3NygEWLeOs8jNi2DcjKAo4cMS9DrLhRWBgTbsSSGosWmasGUakaXM1+RpSosWYNkxrhXPsVQAqAGQC2ANgM4Ory7h/x9bjXrtWFwYmJRMnJ+sbZPFrChx+WZvB49EV+ua8OX1xMNGyYbmOtUkX/88Yb9RXbOTlyhKhzZ11GW5LjmWd4y3GJdHlAWTe+/JI3A5EucS7rxpYtrBGKi4kef1wXBpQ8JTfcwK/G0aNEXbqUqpGQQPTXv/Kr8dNPwWp8/jlvBqJANRITiRo0qJgaqOwiBQBTAAw+/e8OACnl3T+iwV1YSFStWml9xNnt1b/+GvlPX1HWrdP7PDuDUkS1a/NeMH/MmODa6IQEfbV6Tnr1Cqz1KKnQ/uADvgynThGlpZnvxk8/BT8nShHVqqWv5s/E2LGxoUbv3sZqvP8+X4ZQarjdunmdi/XrK0+NSAb3OZdKlFLJADoDmHT6HXoRER2rtLf8c+YYf/zlbnmfMCE4B5FuPF+8mC/H6NHBTd6nTgGffaaXcTg4ckQ3rJc9Hnl5Oh8Xc+YYf/7lrvJ+663gHES6oJfRjfLUKLs9Whw9Cnz7rflqfPONsRpeb+yokZUVvf2Gs8bdAEAOgMlKqTVKqXeUUp6yd1JKDVVKrVRKrczJyQk/weHDxq2fXi/w22/hP875cuCAcds8oDNycSzE70Qi/ergIDcXiI83/n+HDvFkAGLHjf37Y8KNUGoAokYJF8rYCGdw2wC0AjCeiFoCyAMwvOydiGgiEbUhojbp6enhJ+jSxfgnT0wEbrop/Mc5X/r00RXRZfF6gQ4d+HJ066Yrs8tSqxZfy3vduvr4l8Vm431OOnc2dsPjAXr04MsRyo2iIlY3rrvOWI2LLmIreRc1ytC7d2g1OnaM4o7PtZYCoCaAnWf9dycAs8r7MxGfnBw2TC+Snb1Qdc01ukSYi4ICoiuuCFyw8niInnuOLwORXqBLSSldRIyP15nmzuXN8fnner8ljeIJCXpRcc8e3hyx4EZhIdGVVwa78eyzfBlIL+vHghpffhkbajz+eLAaV18dG2qMGBH5YyEKJyeXAGhy+t//BuC18u4f8eD2+/WguPFGok6diMaP10eEm7w8ojff1M3mvXrpKmszyM7WbfPt2xM98IA+OWYGK1fqM1/t22sTDxzgz+D3E33xRWy48e9/azd69iSaNYs/A2k1/vxn/ZTcf795aqxaVarG8OGxoca4ceaokZ8fqMZ//1uxx4lkcIfV8q6UuhLAO9DfKNkO4CEiOhrq/tLyLgiCEBmRtLzbwrkTEa0FENYDCoIgCNEldv7mpCAIghAWMrgFQRAshgxuQRAEiyGDWxAEwWLI4BYEQbAYMrgFQRAshgxuQRAEiyGDWxAEwWLI4BYEQbAYMrgFQRAshgxuQRAEiyGDWxAEwWKEdZEpFgoKdCdSYSFw/fVA1arm5Dh4EFiwAEhKArp3BxwOc3Js3gysXg3Urw9cfTWgFH8Gvx/47jsgOxu46iqgUSP+DIB2Yu5c/c/rruMrlCiDqFEKkW4137MHaNv2glcDBw8CCxfqkonrrwcSEqK8w3Cv/xrJLeLrcS9cWFrTnJSk66snTIj8grbnyz//qa8Kn5Sks6SmEi1fzpuhqIjollt0IW5Skq6NvvxyooMHeXNkZxM1aqQzlDwn993HWo5LRESLFgW7MX48bwYieu01vWuz1bj11kA1mjfnV2PvXq1GYmLpU3LvvfxqZGUFqzFuHG8GIqJRo4LHxrJlkT8OKrtIIdJbRIP75EltgFGT94YNkf/0FeWHH4LrmgGiqlX1K4aLl14Kbpu323WxAycdO+qKlbIV2mPG8GU4eVK/GozcWL+eLcbSpaHVOHWKLQa9/HKwGjabvng/J506Gavxn//wZcjLC60GZ7nEsmXGbqSmRu5GJIPb/DXuWbOMP+sVFQFTpvDleOcd4xZ1r1d/BuLirbeCc3i9+vPgyZM8GQ4eBH78MbjULz8fGDeOJwMAZGYab2d2I5QaPp/5avh8wLx5fGrk5AArVsS2Gu+9x5dj0iS9TFOW4mK9rBYtzB/c+fnGdc3Fxbrjnovjx/Uvy7IoxVehDRhPiJIcRUV8GYxaaQHeY5Gfb/ycMLsRSg2A/3CE4tQpngxWUOPECb4cx48bjy+i6B4P8wd39+6h65pvu40vx513hq5r7tqVL8fNN+vK7LI0asR3wrZuXSA9PXi7wwHccQdPBkC74fMFb2d2ozw1rr2WLQZuucVYjUsu4TspV6cOUL168HYz1PB6g7dzj4077jB2w+uNshvhrqlEcov45OQrrwTWRns8RLffrttAufB6ia6/vrQ2Oj5eL5iNHcuXgYho/36iiy8uXThLSNDnAFas4M2xaJE+FiWV4h4PUf36RIcP8+Yo60Zioj5DV1zMFsHrJerePVAN7uV+otBqcJ8kzcoyVuPQId4cr74aPDZuuYVVDfL5iG64odSNuLiKu4HKLguOlAqVBS9fDkyerD8D3XUX0LNn6M9k0aK4GJg5E/jsMyA1FRg0CGjZkjcDoD9/TZkCfP890KQJMHQoUKsWf45du/TC6vbt+lPHgAHGby+izYoVwLvvajfuvBPo1YvdjeJi4KuvgBkzRA0A2L1bq/Hrr+arMXmyXpYwSY0gNwYOBFq1ivxxIikLjp3BLQiCcAETyeA2f41bEARBiAgZ3IIgCBZDBrcgCILFkMEtCIJgMWRwC4IgWAwZ3IIgCBZDBrcgCILFkMEtCIJgMWRwC4IgWAwZ3IIgCBZDBrcgCILFkMEtCIJgMWRwC4IgWIywW96VUvEAVgLYS0S9Kz3JwYPA55/rHqBevcyrjd64EfjmG13l3bevOW3zfr/uPVqzBsjI0FfQN6NSvLAQ+PJL3fLeti3QqZMpleJnq9GzJ9C4MXsEAMCmTcCcOVqN2283p1Hc79d1aatXazVuvpmhUdyAEjVKWt47dzanbT7W3EhM1GMj6m6Ee+FuAH8C8DGA/57rvhEXKXz6qS4tcLv11eFdLqKRIyO/Evn54PcTDRum9+1w6CxuN9GcObw5Tp4kuuoqfYV8m003ol50EdGOHbw5tmwhSk8vzeHxEHXpQlRYyBpjxoxANZxOoueeY41Afj/RE08EqzF7Nm+OkyeJ2raNTTU6dyYqKODN8dlnpW44HNqNZ5/lzeD3E/3xj8FuZGZG/lio7JZ3ALUBzAfQrdIH99GjwdXVJbXRnK0vc+eW1licfUtMJMrP58vx9NPawLMzxMXpVwYnV1xBpFRwhfarr7JFKE8NztaXb7+NDTX++ldjNTp14stARHTllcZqvPwyX4Zjx4zb1d1u3bzOxbx5od3Iy4vssSIZ3OGucb8B4GkABrWY50lmJhAfH7y9sBD46KNK311I3n/fuN0zLi66dc1l+eCD4Npovx9YupSvIHffPmDLluA21oIC3UTDxOzZodX48EO2GOWqMX8+bw4jNZYvB3JzeTLs3w9s3mysxuTJPBkAceOcg1sp1RvAQSJadY77DVVKrVRKrczJyQk/gVFRMKDNMCqKjRbl7StUxmhQ3r6M6qSjgd8fesGS8ViUp0YsPCXcOUI9/SXv87gyxIAaKC42/pkvFDfCecfdAcDNSqmdAKYB6KaUCvqdRkQTiagNEbVJN2oID0WPHsZD0+UC+vUL/3HOl/79jUvzfD6gWze+HP36BZ+IVEqX2KWk8GSoXVuf+SqL06nLBZmIFTXuu89YjeJi4Lrr+HL06xd8IlIp3X3JpUatWkD9+sHbnU79EuLippuM3XC7LxA3wl1T0Usw6IponJycPLl0db+kJnnYMN6Wd7+faMAAvW+lSk+SzpjBl4FIL95deqleJCuprq5WTZ8R4mTtWqIqVUoX8BITiVq10mfIGHnvPb2ua7YaDzwQrMann/JlINJqXHaZ+WqsWxcTahi68dhj/G48+GCwG9OnR/5YiFbLu1KqK4A/0zm+DlihsuCdO4FPPtGLZTffXLGa5POFSC8YZmYCVaoAd9+t331y4/UCX3+tv/PVoIFuvU9M5M+RmwtMm6a/89W+vX4LbLSwGGV27gSmT9cl72aqsWIFMGsWkJys39WZoYbPp9VYtUq/8737bvPU+OQT3fberp3+Kp4JagS40acP0Lo1f4YSNzIz9VdFK+qGtLwLgiBYDGl5FwRB+B0jg1sQBMFiyOAWBEGwGDK4BUEQLIYMbkEQBIshg1sQBMFiyOAWBEGwGDK4BUEQLIYMbkEQBIshg1sQBMFiyOAWBEGwGDK4BUEQLIYMbkEQBIsRdst71Nm4UXcOFRToCm0zGsWJgHnz9HUzq1QB7r/fnLb5wkJ9rcply3Rt9f33m9M2f/AgMGWKvnZmx466vtqEtvlNm3SjW0EBcNtt5jSKE+kqqq++Ml+NTz/VTXaNG+teCzPa5kvU2LFDv1RNUgObNumxkZ9vvhtff60v+TtgAEPbfLgX7o7kFnGRwpgx+urjNpu+GrnHo69OznlF9OJiottuK706vN2uM73/Pl8GIqJDh4gaNCi9Wr7Lpa9a/9NPvDmWL9c14iXttImJuuAhN5c1xtixwWo88AC/GrffXqqGzaYzTZnCl4EotBrr1vHmWLHCWI1jx3hzGLkxYAC/G337Brvx3nuRPxYqu+U90ltEg/vAgeDq6pJ6jwULIv/pK8oXXxjXNbtcvMPqD3/QvzTK5mjZki+D30/UsGFwhoQEomeeYYtRnhrz57PFoC+/DK0G57B65BHd9lI2x5VX8mXw+4kuucRYjb/+lS/Hb7+FdmPePL4cM2dWnhuRDG7z17jnzAFsBis2+fnAjBl8OaZNM65rttuBhQv5cnz2mW7AKcuGDcDRozwZ9uzRTe9lOXVKHycmvvnGWI28vAtXjaKi4O2bNgFHjvBkyM7Wt7KcOqUbcbgoz41PP+XLEcoNmw1YsCB6+zV/cCckGC9KKRXcjGpGDoB38c5uD/3/jEyNBg5H6EpxxmMR6imJi+NVw+kUNUpwOEI3ynMei/LccDr5coRyQ6noHg/zB3fPnsY99k6nPgPExcCBuiK6LErxtrwPGqRrzM8mPl6fdUlK4slQsybQooV+FZyN2w0MGcKTAVoNo98fCQm8ajz0UPBTAmg1OFveBw0KHkrx8fq8cXIyT4YaNYArrghWw+ViVQM9ehi7wT02QrkBxFDLe7i3iE9Ozpqla5ITE/U/nU6i//u/yB6jMhgxQu/b7dZnXxITiRYt4s1QUEDUtateOHO5dI769Yn27uXNsX07Ua1aev8ulz4mvXsTFRWxxsjMDFbjH/9gjUBERM8+G6zGwoW8GQoKiK69NliN7GzeHNu3E9WuHahGr17satDs2cFuvPoqbwYioueeK3UjMbHibiBaLe/hUqGy4Nxc/X2awkLgppvMqdAG9Fff5s7V72779DGnQptO10avWQNkZADdu5tToe3z6XMQ2dlA27bm1KtD1DgbUSOQ35Mb0vIuCIJgMaTlXRAE4XeMDG5BEASLIYNbEATBYsjgFgRBsBgyuAVBECyGDG5BEASLIYNbEATBYsjgFgRBsBgyuAVBECyGDG5BEASLIYNbEATBYsjgFgRBsBgyuAVBECzGOXszlFJ1ALwPoCYAP4CJRPRmpSeZPx945x1dWdavH3DnnXy1HiUUFQFTp+ruo9RU4OGH9VXqmTmcfxjjV47Hd7u/Q9O0pni87eNoWLUhe46fD/2M/6z4D3498iu61e+GIa2HIMWZwp5jwY4FeHvV28j35aNfs364s9mdsMXxulFUXIRpG6Zh+sbpSHWm4uE2D6NjXXPcmLByApbsXhITbmw7sg3d6nfD0NZDzXXDm49+zc1xw1vsxdQNUzF943SkOFPwcOuH0alep6ju85yXdVVKXQTgIiJarZRKArAKwK1EtCnUn4n4sq7PPgu8+WZpeZvHowdmZmZw1Ua08HqBrl2Bdet0DqV0tcXf/gb85S88GQDsyd2D1hNb40TRCRT6CmGPs8MR78Cc/nNYB8XcX+fitk9uQ5GvCD7ywWVzIdWVitVDV6NGYg22HM8teA5vLHsDeV7thsfuQYe6HZB5bybi43guRO0t9qLrlK5Yd2Ad8rx5UFBw2V14ocsLeLrD0ywZgNBuzL5vdtQHxdnEihsjF4zEv5b9K8CNa+pcg9n3zWZ1o9uUblhzYE2AGyM7j8TwjsMjeqxKvawrEe0notWn//0EgM0AakWUqDx27QJGjw5s3MzLA77/Hpg9u9J2c06mTy8d2oC+Yn1+PvD888ChQ2wxRiwYgSMFR1DoKwQAeP1e5HnzMPirwWwZ/OTHQzMfQr43Hz7yAQAKfAXIycvBy0teZsuxO3c3Xl/6+pkXJgDkefPww+4fMHsbnxufbvr0zNAGAAIh35uPFxa9gEP5fG48u+BZQzcGfTUI0biuvhFEhIEzBxq68dLil1gyAPqX2Kilo4LcWJq9FJlH7kaOAAARu0lEQVRbM9lyzNg048zQBkrdeDHrReTk5URtvxG9nVVKZQBoCWB5pSVYsMC4wuPkSeCrryptN+fkiy9CV3kvXswWY/bW2Sim4A7OHcd24HD+YZYMu3N341jhsaDtXr8XM3+eyZIB0B+D41WwGye9J/HVz3xufLH5i4ABUYI9zo6snVlsOUK5sSt3Fw4X8LlxpCC4Uj5m3ChidmNLOW7sip4bYQ9upVQigM8APElExw3+/1Cl1Eql1MqcnAh+0yQnGy+H2Gx6nZmLatVCL8twNbECSHKELgR22UO0klYyiY5EFPsNCpwBVEmowpIBAJITkhGngp8TW5wNqU4+N6q5qxnmAHRGLhITQvdhuWyMbhj88gCAKs7YcCPFxbfWXs1ljhthDW6llB16aH9ERJ8b3YeIJhJRGyJqk56eHn6Cnj2NB6bdriuUuRgyJLhCG9Dr3F27ssV4rO1jcNsC2+Yd8Q70btQbbrtBC30USHOnoWPdjrDH2QO2u+1uPNHuCZYMANDjkh6G76rscXY81JLPjSGthiAhPiFou8vmwrX1r2XLMaztsCAHHHEO9GrUCx6HhyVDNXc1dKrbKcgNj92DJ9oyutEotBsDrxzIlmNI6yFwxgfPDafNiW71u0Vtv+cc3EopBWASgM1ENLrSE7hcunW0WjX9zjY5GXC7gYkTgSZNKn13IWnTBhg1SudJTtatnzVrAt9+y/rtlqfaP4XbL70dTpsTyQnJcNvdaHVRK0y6ZRJbBgD4uO/HaJbeDB67B8kJyXDGO3F/i/sxqOUgtgwuuwtz+s9BNVc1JDuSzxyPt/q8haZpTdlytL64NUbfOBoumwvJCclIciShhqcG5g6Yy/oNhifbPYm+l/aFM76MGzfzujG171Q0r948wI3+LfpjUCs+N5w2J74Z8A3S3GlITjjths2NCb0n4NL0S9lytLqoFUbfxO9GON8q6QhgCYD10F8HBIARRBTyDECFyoK9XmDJEl3X3LmzORXagK6NXrJED+8OHcyp0Aaw69gurPttHTJSMtCiRgtTMhAR1hxYg+zj2Wh9UWvUSq68c9KR4PP7sGTXEhT4CtC5XmckOsxxI7cwF0t2L0GSIwkd63Zk++ZCWWLFjbUH1mLP8T1odVEr1E42p149ltz4bvd3SHQkVtgNaXkXBEGwGNLyLgiC8DtGBrcgCILFkMEtCIJgMWRwC4IgWAwZ3IIgCBZDBrcgCILFkMEtCIJgMWRwC4IgWAwZ3IIgCBZDBrcgCILFkMEtCIJgMWRwC4IgWAwZ3IIgCBYjNga33w+8/z5wzTVAy5bAa6/pvkdmjp86jr9n/R1XjL8CnSZ3wvSN09m6/M5m+9HtGPL1EDQf1xy3fXIbVuxdwZ4BAJbsWoI+H/dB83HNMSxzGPbk7mHP4Cc/Plj3Aa6ZdA1avtUS//z+n8j38rtx4tQJvLT4JVwxQbvxyYZPTHPj4a8fPuPG8uzKaxGMhO92f3fGjcdmPWaaGx/+9KF2Y4L5blw54Uo2N2Ljsq4PPgjMmFHa+ehyAU2bAsuX6yYcBvK9+Wj1Vivsyt11pozVY/dgaOuhGH1j5fdHhGLLoS1o+3ZbFHgL4CPfmdboT+74BL0b92bL8fH6jzHk6yFnXgi2OBuSHElY/fBqZKRksOUYOHMgpm+cfqbXz2VzoWlaUywbvAyOeAdLhlBuDGk1BP+66V8sGQDtRrt32iG/KD/AjWl9p6FPkz5sOaZumIrBXw0+44Y9zg6Pw4PVQ1ejfmp9thxGbjRJa4Llg5ezuVHgLUCria2w89jOADcGtxqMN256I6LHstZlXbds0Q3rZxf1FhQAW7cCnxu2pEWFD9Z9gOzj2WcOPqBbo8f/OB57j+9lyzF83nCcLDp5pkG7pDX60VmPsr3D8/l9eGL2EwHvXnx+H46fOo4XF73IkgEAfj70M6ZtmBZQxlrgK8DWw1vx+WY+Nz786UNjN1aOR/bxbLYcI+aPwIlTJ4LdyORzo9hfjMczHw9ww+v34sSpE3gxi8+NXw7/YujGtiPb8Nmmz9hyfLT+I+zJ3RPkxoSVE6L6KcT8wf3998adkydP6towJuZsm2Pc1hxvx9LspWw5Fu9aDELwi/Bg3kEcyj/EkmF37m4U+AqCthdTMebtmMeSAQC+3/O9YRHrSe9JfPur+W444h1YuofPjaxdWYZu5OTlICc/goLu86BcN7YzurE7hBtFJ/Htdj43Zm+bHdqNKM4N8wd3jRrGg9vhAGrz1SHVqVIHNmXcEVfDU4MtR7rHuGhZKYWkhNAN8JVJVVfVkC3vNRNrsmQA9HE3enE64h2sVVl1kstxI5HRDXc5bjh43Eh1pcaGG4kx5EaIbslozg3zB/eNN+pyYKUCt9tswEC+tuZH2jwChy1wXSxOxSHNnYYOdTuw5fjLNX8JavJ22py49/J74bQZtNBHgRRnCno37h3UbO6xe/D0NU+zZACAGxreALfdDYVAN2xxNtZi2keuMnajmrsaOtbtyJYjlBv3NL8HLruLJUOKMwV9GvcxdqMDrxseh8fYDcZC6z+0+UNQ432JG53qdYrafs0f3HY7sGgR0LixHuCJiUB6OvDll0DdumwxLk2/FB/f/jFSnalIciTBZXOhefXmWPDAAsPf7NFiUMtBeLLdk2da3p3xTvRu1BtjeoxhywAAk2+ZjOsbXH8mh8vmwrOdnsWdze5ky2CPtyPrwSw0rtYYbrsbiY5EpLvT8cXdX6BuFT43mqY1NXbjfl43BrYciKfaPxXkxtieY9kyAMDkWyeje8PuAW6M6DQCdzW7iy2DLc6GrAez0CStSZAb9VLqseVomtYU0+6YFuBGs/RmUXcjNr5VAgBEwC+/6Jb35s1Na1f3Fnux4eAGJCUk4ZKql5iSAdBfTdx6eCtqJ9dm/Theln0n9mH/if1oktbEtAZtIsIvh39Boa8Qzas3N61d3ef3Yf1v62PGjVrJtViXJ8oibpRSGW5Iy7sgCILFsNbXAQVBEISIkMEtCIJgMWRwC4IgWAwZ3IIgCBZDBrcgCILFkMEtCIJgMWRwC4IgWAwZ3IIgCBZDBrcgCILFkMEtCIJgMWRwC4IgWAwZ3IIgCBZDBrcgCILFCGtwK6VuUkr9rJTappQaXtkhioqL8PLil5HxRgYuev0iDMschsP5hyt7N+fkwMkDGPzVYNQcVRMN/90Qry99PWTbRzRZtW8VbvjgBqS/lo7WE1vj65+/Zs8AADM2zcCVE65E9deqo8/UPlj/23r2DEXFRXhlyStn3Hhs1mOmuTHk6yFn3Bj1wyj4/D72HKv2rcKNH9wYM26kv5aO3h/3Ns2NV5e8GuAGV73f2Rw4eQBDvx6KmqNqosGbDVjcOOdlXZVS8QB+AdAdQDaAHwHcQ0SbQv2ZSC/r2uOjHsjamXWmy84R50Ct5FrY+OhGtmaP3MJcXDr2UuTk55w56G67G7c2uRUf9f2IJQMArNy3El3e6xJQxuq2uzGu5zg8cOUDbDneXP4mRswfcSaHgoLH7sGywcvQrHozthy9PuqFhTsXnnHDHmc/40bZNphoEcqNmxvfjKl3TGXJAIR2Y2yPsXiw5YNsOf69/N94Zv4zQW4sHbwUzas3Z8vR++PeWLBjQaAbSbWw8TFeNy4bdxkO5h0McKNP4z6Ydse0iB6rsi/r2hbANiLaTkRFAKYBuCWiROWwZv8aLN61OKCAtMhfhIN5BzFtQ2Q/+Pkwac0kHCs8FvCbMt+bj8+3fI5fj/zKlmP4vOEBL8ySHH/+9s/wk58lQ1FxEUYuGBmQg0DI9+XjhUUvsGQAgLUH1mLRrkUBbnj9XuTk5bC68e6ad5F7KjfIjS9//hLbjmxjy3H2sDw7x1/m/YXtk2FRcRFGLgx2I8+bhxcW8rmx7sC6gF/owGk38nMwdT3fL9P31r5nODdm/jwTWw9vjdp+wxnctQCc3TOffXpbpbBq/yrD7XnePHy3+7vK2s05KfvLowR7nB2r969myxFqXydOnWD7GLgnd4/hLwk/+bEsexlLBkAvCxhhhhtlBybA70ao43Gy6CSbG9nHsw1/SRAIy/cuZ8kA6LlRtm8S0G4s2b2ELUfWrixT3AhncAcfHSBofUUpNVQptVIptTInJyfsAPWq1EO8Cq4bctqcaFStUdiPc740qdYEjnhH0HY/+Vk77EI1VMfHxSPFmcKSobqnOorJ+B1c/ZT6LBkAoF5KOW5UZXQjrRw3qvC5UadKHcPtcSqOzY10d3pINzJSMlgyAHpuGHU6OuOdaFytMVuOUHODQFGdG+EM7mwAZxtTG8C+sncioolE1IaI2qSnp4cdoFv9bkj3pAe9QO1xdgxsydjyftUjQW3N9jg7GlVthKsuvootxwtdXghan3Pb3Xj0qkcNBYkGSQlJuO/y++CyBZ5fcNvdGNllJEsGQLtR3VPdfDfaGLtxSdVL0LZWW7Ycodx4pM0jSLAlhPhTlUtSQhL6t+hv7EZnPjeurX8taiTWgE3ZArbb43ndMGp5t8fZ0SClAdrVahe1/YYzuH8E0EgpVV8p5QDQD8BXlRUgPi4eSx5ags71OsMR54Aj3oFm6c2w8IGFqO6pXlm7OScZKRmY038OLql6CRLiE+CId6B7g+749v5voZTRh47o0Peyvni9++tIdabCZXPBbXPjD23+gFeve5UtAwCM6zUOD1zxAFw2F5w2J9LcaZjQawJuaHgDW4Y4FYfFDy1Gl4wucMRrNy5LvwwLH1jIWqBcL6Uevun/DRpVbXTGjesbXI9vB/C6cfult2P0jaMD3Hi49cP4x/X/YMsAAGN7jsWDVz4Il80Fl82FNHcaxvccjxsvuZEtQ5yKQ9aDWUFuLHhgAWuBcr2Uepg7YG6AG9fVvw7z7p8XVTfCKgtWSvUE8AaAeADvEtHL5d2/omXBuYW5KCouQron/HfslQ0R4WDeQbjsLiQnJJuWw+f3IScvB6muVDhtTtNyFHgLcKzwmH7na1KDNiBunI24EUisuJGTnwOnzVlhN6TlXRAEwWJIy7sgCMLvGBncgiAIFkMGtyAIgsWQwS0IgmAxZHALgiBYjKh8q0QplQNgVwX/eBoA/kt8xS5yPEqRYxGIHI9Sfg/Hoh4RhfWdxqgM7vNBKbUy3K/EXAjI8ShFjkUgcjxKudCOhSyVCIIgWAwZ3IIgCBYjFgf3RLMDxBhyPEqRYxGIHI9SLqhjEXNr3IIgCEL5xOI7bkEQBKEcYmpwR7uU2CoopeoopRYqpTYrpTYqpf5odiazUUrFK6XWKKX+a3YWs1FKpSilZiiltpx25GqzM5mJUuqp06+TDUqpqUop8y6ZyETMDO7TpcRjAfQAcBmAe5RSl5mbyjR8AP6HiC4F0B7AYxfwsSjhjwA2mx0iRngTwBwiagrgClzAx0UpVQvAEwDaEFFz6EtP9zM3VfSJmcGNKJcSWwki2k9Eq0//+wnoF2al9XxaDaVUbQC9ALxjdhazUUolA+gMYBIAEFERER0zN5Xp2AC4lFI2AG4YNHT93oilwR3VUmKropTKANASAF8Ta+zxBoCnAfDU3Mc2DQDkAJh8eunoHaWUx+xQZkFEewGMArAbwH4AuUQ019xU0SeWBndYpcQXEkqpRACfAXiSiI6bnccMlFK9ARwkIuOK8wsPG4BWAMYTUUsAeQAu5PNBqdCfzOsDuBiARynV39xU0SeWBndYpcQXCkopO/TQ/oiIPjc7j4l0AHCzUmon9PJZN6XUh+ZGMpVsANlEVPIJbAb0IL9QuR7ADiLKISIvgM8BXGNypqgTS4M7qqXEVkLpltFJADYT0Wiz85gJET1DRLWJKAPaiQVE9Lt/RxUKIjoAYI9SqsnpTdcB2GRiJLPZDaC9Usp9+nVzHS6Ak7W2c9+FByLyKaWGAfgGpaXEG02OZRYdAAwAsF4ptfb0thFElGliJiF2eBzAR6ff4GwH8JDJeUyDiJYrpWYAWA39baw1uAD+FqX8zUlBEASLEUtLJYIgCEIYyOAWBEGwGDK4BUEQLIYMbkEQBIshg1sQBMFiyOAWBEGwGDK4BUEQLIYMbkEQBIvx//ydCTYRuwSKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_data = []\n",
    "y_data = []\n",
    "c_data = []\n",
    "for i in range(10):\n",
    "    for j in range(10):\n",
    "        x_data.append(i)\n",
    "        y_data.append(j)\n",
    "        c_data.append(label_data[np.argmax(model.predict(np.array([i, j]).reshape(1, 2))[0])])\n",
    "\n",
    "plt.scatter(x_data, y_data, color=c_data)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "⚙️ 엔지니어\n",
    "\n",
    "> 스스로 학습해서  \n",
    "> 근처에 있는 땅을 자기것으로 만들면서  \n",
    "> 천하삼분지계 하고 있다!!!  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 정리\n",
    "\n",
    "뉴럴 네트워크 모델(Neural Network model)을 만드는 방법을 정리해 보자.  \n",
    "\n",
    "1. 데이터를 변환한다.    \n",
    "    1. 입력 데이터의 형식과 모양을 변환한다.  \n",
    "    2. 출력 데이터의 형식과 모양을 변환한다. **원-핫 인코딩(one-hot encoding)** 을 사용한다.      \n",
    "2. **히든 레이어(hidden layer)**  정의  \n",
    "     1. 히든 레이어 개수를 정의한다. 여기서는 1-hidden layer를 사용한다.  \n",
    "     2. 히든 레이어 유닛 개수를 정의한다. 여기서는 3개이 유닛을 사용한다.  \n",
    "     3. 활성 함수(activation function)를 정의한다. 여기서는 **시그모이드(sigmoid)** 를 사용한다.  \n",
    "3. **출력 레이어(output layer)**  정의   \n",
    "     1. 출력 레이어 유닛 개수를 정의한다. 여기서는 10개의 클래스로 분류해야 되니까 10개의 유닛을 사용한다.  \n",
    "     2. 활성 함수(activation function)를 정의한다. 여기서는 **소프트맥스(softmax)** 를 사용한다.\n",
    "4. 손실 함수 (Loss function)를 정의한다. 여기서는 **크로스-엔트로피 손실(cross-entropy loss)** 를 사용한다.  \n",
    "5. 옵티마이저(Optimizer)를 정의한다. 여기서는 **RMSProp** 을 사용한다.  \n",
    "6. 반복할 회수(epoch)를 결정한다.  \n",
    "7. 주어진 조건으로 모델을 최적화(fit) 시킨다.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 케라스(Keras)로 모델링(Modeling)\n",
    "\n",
    "### 정규화 (Normalization)\n",
    "\n",
    "⚙️ 엔지니어  \n",
    "\n",
    "> 정규화 할거 없음!  \n",
    "> 0과 1로 되어 있는데 뭔... 필요???  \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = x\n",
    "Y_train = y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 모델링 (Modeling)\n",
    "\n",
    "⚙️ 엔지니어\n",
    "\n",
    "> 5줄로 모델링이 가능하다!   \n",
    "> \n",
    "> 프랑소와 숄레 만세!\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from tensorflow.keras import Sequential\n",
    "from tensorflow.keras.layers import Dense\n",
    "\n",
    "# 모델을 준비한다.\n",
    "model = Sequential()\n",
    "# unit 개수가 3이고 입력 변수 x가 10개인  hidden layer를 생성하고 dense하게 연결한다.\n",
    "model.add(Dense(3, input_shape=(10,), activation='sigmoid'))\n",
    "# unit 개수가 10인 output layer를 생성하고 dense하게 연결한다.\n",
    "model.add(Dense(10, activation='softmax'))\n",
    "# Loss funtion과 Optimizer를 선택한다.\n",
    "model.compile(loss='categorical_crossentropy', optimizer='rmsprop')\n",
    "# epochs만큼 반복해서 손실값이 최저가 되도록 모델을 훈련한다.\n",
    "hist = model.fit(X_train, Y_train, epochs=10000, verbose=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 손실값의 변화를 그래프로 확인"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8VPW9//HXJzPZV8gCYQ2bAi4IRAQU1Grd6tbWBeteFa1ttfX3a6/+2t57671d7u1yr1arUrXWXeteRa1a68oiKAjKFvawJWxJSMj+/f0xhxhDAgFycmYy7+fjMY858z3fmXxOTpJ3zvY95pxDREQEICHoAkREJHooFEREpIVCQUREWigURESkhUJBRERaKBRERKSFQkFERFooFEREpIVCQUREWoSDLuBA5eXluaKioqDLEBGJKfPnz9/qnMvfX7+YC4WioiLmzZsXdBkiIjHFzNZ2pp92H4mISAuFgoiItFAoiIhIC4WCiIi0UCiIiEgLhYKIiLRQKIiISIu4CYVlm6v41cwlVNc1Bl2KiEjUiptQKN1Rw33vruLzTZVBlyIiErXiJhSO6p8NwJxV2wKuREQkesVNKBRkpXBsUS+e+2QDzrmgyxERiUpxEwoAFx87iFXl1fxzeXnQpYiIRKW4CoVzx/SjX3YKf3y7JOhSRESiUlyFQlI4gelTh/LRmh18WLI16HJERKJOXIUCwLQJg+ifk8p/vLKEpmYdWxARaS3uQiElMcStZ45kyaZKnpm/PuhyRESiStyFAsDZRxcyblAO//3aMrZX1wddjohI1IjLUDAzfvH1o6isbeD2v30WdDkiIlEjLkMBYFRhFt89eTgvLNjIm59vCbocEZGoELehAHDjScMZ2TeTn7ywiIqahqDLEREJXFyHQlI4gd9eOIatu+r5j1c+D7ocEZHAxXUoABzZP5vvnDiMZ+aX8vaysqDLEREJVNyHAsD3TxnOYX0yuO3ZRVTWajeSiMQvhQKQHA7xmwvGUFZVyy9fWRJ0OSIigVEoeMYMzOG6qUN58qP1vKsB80QkTikUWvnhqYcxND+dn76wmNqGpqDLERHpdgqFVlISQ/zHeUeybnsNM95dFXQ5IiLdTqHQxvHD8/jaUYXc/XYJ67fXBF2OiEi3Uii046dnjyKUYNz+sq5dEJH4olBoR2F2Kt//ygje+HwL/9S1CyISRxQKHbjmhCEMzk3jVzOX6r4LIhI3FAodSAon8KPTD2fZliqe+7g06HJERLqFQmEfvnZUIWMGZPP7N5brFFURiQsKhX0wM249cxSbKmp56MM1QZcjIuI7hcJ+TBqWy1dGFvDHt0s0LpKI9Hi+hYKZDTSzt81siZl9ZmY3t9PHzOxOMysxs0/NbJxf9RyKW756GJW1jTz0wZqgSxER8ZWfWwqNwP9xzo0CJgLfNbPRbfqcCYzwHtOBe3ys56Ad2T+bU0cV8MD7q6nS1oKI9GC+hYJzbpNz7mNvugpYAvRv0+084GEXMRvIMbNCv2o6FDedMoKK3Q08PGtt0KWIiPimW44pmFkRMBaY02ZWf2B9q9el7B0cUeHoATmcfHg+f3pvFbvqGoMuR0TEF76HgpllAM8CP3DOVbad3c5b9rpSzMymm9k8M5tXXh7csNY3n3oYO2saeERbCyLSQ/kaCmaWSCQQHnPOPddOl1JgYKvXA4CNbTs552Y454qdc8X5+fn+FNsJxwzM4YTheTz4wWrqGnXdgoj0PH6efWTAA8AS59zvO+j2EnCFdxbSRKDCObfJr5q6wvSpQymvquPFT/bKLhGRmBf28bOPBy4HFpnZAq/t/wGDAJxz9wIzgbOAEqAGuNrHerrElBF5jCrMYsZ7q7hg/AASEtrbAyYiEpt8CwXn3Pu0f8ygdR8HfNevGvxgZkyfOoQfPrWQt5eVccqoPkGXJCLSZXRF80E4++h+9MtO0d3ZRKTHUSgchMRQAt8+YQhzVm9n4fqdQZcjItJlFAoHadqEQWSmhJnxnrYWRKTnUCgcpIzkMNOOHchrizezuaI26HJERLqEQuEQXD6xiGbneGyOLmYTkZ5BoXAIBuWmccrIPjw+Z51uwiMiPYJC4RBdNbmIbdX1vPJpVF9zJyLSKQqFQ3T88FyGF2Tw0IdriFx2ISISuxQKh8jMuGpyEYs2VPDxOp2eKiKxTaHQBb4+tj+ZKWHdx1lEYp5CoQukJ4e5uHggry7axJZKnZ4qIrFLodBFrphURJNzPD5nXdCliIgcNIVCFxmUm8aJh+Xz5EfraGhqDrocEZGDolDoQpcdN5gtlXW8tWRL0KWIiBwUhUIXOnlkAf2yU3h0tnYhiUhsUih0oVCCccmEQbxfspXVW6uDLkdE5IApFLrYxRMGEk4wHput8ZBEJPYoFLpYQWYKpx/Rl7/OL9V4SCIScxQKPrh04iAqdjfwssZDEpEYo1DwwaShuQzLT+dR7UISkRijUPCBmXHpcYNZsH4nizdUBF2OiEinKRR88s1xA0hJTOAxXeEsIjFEoeCT7LREzjm6Hy8u2EBVbUPQ5YiIdIpCwUeXTRxMTX0Tz3+yIehSREQ6RaHgozEDcziqfzaPzl6rG/CISExQKPjssomDWL5lFx+t2RF0KSIi+6VQ8Nk5Y/qRmRLW6akiEhMUCj5LSwrzzXEDeHXxJrbuqgu6HBGRfVIodIPLJg6iocnx1Efrgy5FRGSfFArdYHhBJpOH5fLY7LU06gY8IhLFFArd5IpJg9lYUctbS8uCLkVEpEMKhW5y6qg+FGan8MgsHXAWkeilUOgm4VAC3/JuwLOyfFfQ5YiItMu3UDCzB82szMwWdzD/JDOrMLMF3uNf/aolWkybMIjEkGlrQUSilp9bCg8BZ+ynz3vOuWO8x+0+1hIV8jOTOfPIQp6dX0p1XWPQ5YiI7MW3UHDOvQts9+vzY9UVkwZTVdfICws0HpKIRJ+gjylMMrOFZvaqmR0RcC3dYvzgXowqzOKRWRoPSUSiT5Ch8DEw2Dk3BvgD8EJHHc1supnNM7N55eXl3VagH8yMKyYNZunmKo2HJCJRJ7BQcM5VOud2edMzgUQzy+ug7wznXLFzrjg/P79b6/TDecdExkN6eNaaoEsREfmSwELBzPqamXnTE7xatgVVT3dKSwpz4fiBvLZ4M2WVtUGXIyLSws9TUp8AZgGHm1mpmV1jZjeY2Q1elwuAxWa2ELgTmObiaCf75ZMG09jseGKuxkMSkegR9uuDnXOX7Gf+XcBdfn39aDckL50pI/J4fO5abjx5GImhoI/5i4gEf/ZRXLv6+CK2VNYxc9GmoEsREQEUCoE66bAChuan88D7q3V6qohEBYVCgBISjG8fP4RPSyt0eqqIRAWFQsC+OW4AOWmJ3P/eqqBLERFRKAQtNSnEpccN4o0lW1i7rTrockQkzikUosAVk4oIJxh//mBN0KWISJxTKESBPlkpnDOmH0/PW0/F7oagyxGROKZQiBLXnDCEmvomnpy7LuhSRCSOKRSixBH9spk0NJeHPlxDfWNz0OWISJxSKESR6VOHsqmilhd1rwURCYhCIYqcdHg+owqzuOedlTQ162I2Eel+CoUoYmZ89+RhrCqv5vXPNgddjojEIYVClDnzyEKG5KVz99slGvpCRLqdQiHKhBKM75w4jM82VvLO8ti+y5yIxJ5OhYKZ3WxmWRbxgJl9bGan+V1cvDp/bH8Ks1P44z9XBl2KiMSZzm4pfNs5VwmcBuQDVwO/9q2qOJcUTmD61KHMXb2dj9ZsD7ocEYkjnQ0F857PAv7snFvYqk18MO3YQeRlJHHHmyuCLkVE4khnQ2G+mf2dSCi8bmaZgK6w8lFqUogbThzG+yVbmbUyLm5dLSJRoLOhcA1wK3Csc64GSCSyC0l8dNnEwfTJSub3byzTmUgi0i06GwqTgGXOuZ1mdhnwU6DCv7IEICUxxPdOHs5Ha3bw3oqtQZcjInGgs6FwD1BjZmOAHwNrgYd9q0paXHTsQPrnpPK7v2trQUT819lQaHSRv0jnAXc45+4AMv0rS/ZIDoe46ZThLCyt4M0lZUGXIyI9XGdDocrMbgMuB14xsxCR4wrSDb4xbgBFuWn89vVlGhNJRHzV2VC4GKgjcr3CZqA/8BvfqpIvSQwl8KPTR7JsSxV/nbc+6HJEpAfrVCh4QfAYkG1mZwO1zjkdU+hGZx3Vl/GDe/G7N5ZTXdcYdDki0kN1dpiLi4C5wIXARcAcM7vAz8Lky8yMn3xtFOVVddz3joa/EBF/dHb30U+IXKNwpXPuCmAC8DP/ypL2jBvUi3PG9GPGe6vYVLE76HJEpAfqbCgkOOdan/qy7QDeK13ox6cfTrOD37y2LOhSRKQH6uwf9tfM7HUzu8rMrgJeAWb6V5Z0ZGDvNK45YQjPfbJBg+WJSJfr7IHmHwEzgKOBMcAM59y/+FmYdOz7XxlOv+wUfvr8YhqaNASViHSdTu8Ccs4965y7xTn3Q+fc834WJfuWlhTm3849gmVbqvjLh2uCLkdEepB9hoKZVZlZZTuPKjOr7K4iZW+nje7DV0YW8D9vLNdBZxHpMvsMBedcpnMuq51HpnMuq7uKlL2ZGf9+zhE0Njtu/9vnQZcjIj2EziCKYYNy07jplBG8ungzr3y6KehyRKQH8C0UzOxBMyszs8UdzDczu9PMSszsUzMb51ctPdn1U4dy9IBsfvbiYrbuqgu6HBGJcX5uKTwEnLGP+WcCI7zHdCLDc8sBCocS+O2FY9hV28i/vthu/oqIdJpvoeCcexfY14n05wEPu4jZQI6ZFfpVT092WJ9Mbj51BDMXbeblTzcGXY6IxLAgjyn0B1oP+VnqtclBuH7qUMYMyOYnzy9mw06djSQiByfIULB22tq9WYCZTTezeWY2r7y83OeyYlM4lMAd08bS2NTMzU98QqMuahORgxBkKJQCA1u9HgC0u+/DOTfDOVfsnCvOz8/vluJiUVFeOr/8xlHMW7uDO95aEXQ5IhKDggyFl4ArvLOQJgIVzjmdV3mIzjumPxeOH8Bdb5fwQcnWoMsRkRjj5ympTwCzgMPNrNTMrjGzG8zsBq/LTGAVUAL8CbjRr1rizc/PO4Jh+Rl8/4lPKN1RE3Q5IhJDzLnYuudvcXGxmzdvXtBlRL2V5bs4/+4PGNgrjWe/M5nUpFDQJYlIgMxsvnOueH/9dEVzDzUsP4M7p41lyeZKfvTMQmIt/EUkGAqFHuzkkQX8+PSRvPzpJu76R0nQ5YhIDAgHXYD464YTh7JiSxW/e2M5fbJTuKh44P7fJCJxS6HQw5kZv/7m0ZTvquO25xaRn5HMySMLgi5LRKKUdh/FgaRwAvdcNp5RhZnc+NjHfLJuR9AliUiUUijEiYzkMH++agL5mclc+eBcPttYEXRJIhKFFApxJD8zmceuPY6M5DCX3T+HZZurgi5JRKKMQiHODOydxuPXTSQxlMCl989hZfmuoEsSkSiiUIhDRXnpPH7dRMDxrT/NZu226qBLEpEooVCIU8MLMnjs2onUNzYzbcZsVmmLQURQKMS1w/tm8vh1kWC4eMZslm/RMQaReKdQiHOjCrN46vqJGDBtxmydlSQS5xQKwvCCTJ6+fhKpiSEumTGbBet3Bl2SiAREoSBA5ODzU9dPJCcticvun8Pc1fu6vbaI9FQKBWkxoFcaT18/iT5ZyVz+wBze/HxL0CWJSDdTKMiX9M1O4enrJ3F430yuf3Q+z8wvDbokEelGCgXZS25GMo9fN5GJQ3vzf/+6kD+9uyrokkSkmygUpF0ZyWEevOpYzjqqL7+YuYRfv7pUN+oRiQMaOls6lBwO8YdLxtErbTH3vrOS7dV1/PLrRxEO6X8JkZ5KoSD7FEow/vP8I8lNT+LOf5Sws6aBOy8ZS0qi7vks0hPpXz7ZLzPjltMO59/OGc3fP9/ClQ/OpWJ3Q9BliYgPFArSaVcfP4Q7ph3Dx+t2cNG9s9i4c3fQJYlIF1MoyAE575j+PHT1BDbs3M03/vghSzdXBl2SiHQhhYIcsOOH5/H09ZNwOC68ZxYfrtwadEki0kUUCnJQRvfL4rkbj6dvdgpXPjiXFxdsCLokEekCCgU5aP1zUnnmhsmMHdSLm59cwIx3V+paBpEYp1CQQ5KdlsjD357A144u5Jczl/Lzv31OU7OCQSRW6ToFOWQpiSH+MG0sfbNSeOD91WyprOV/Lj5G1zKIxCBtKUiXSEgwfnb2aH76tVG89tlmLr5vFmWVtUGXJSIHSKEgXeraKUO597LxrCjbxbl3fcDiDbqTm0gsUShIlzv9iL48c8NkEgwuuPdDZi7aFHRJItJJCgXxxeh+Wbz4vRMYXZjFjY99zB1vrtCZSSIxQKEgvsnPjNyX4Rtj+/M/by7ne49/QnVdY9Blicg++BoKZnaGmS0zsxIzu7Wd+VeZWbmZLfAe1/pZj3S/lMQQv7toDLedOZJXF2/ivLs/oKSsKuiyRKQDvoWCmYWAu4EzgdHAJWY2up2uTznnjvEe9/tVjwTHzLj+xGE8es1x7Kiu59y7PuDlTzcGXZaItMPPLYUJQIlzbpVzrh54EjjPx68nUW7y8DxeuWkKI/tm8r3HP+Hnf/uMhqbmoMsSkVb8DIX+wPpWr0u9tra+aWafmtkzZjbQx3okCvTNTuHJ6ZO4anIRf/5gDRfdN4v122uCLktEPH6GgrXT1vb0k78BRc65o4E3gb+0+0Fm081snpnNKy8v7+IypbslhRP493OP4A+XjKVkyy7OuuM9DagnEiX8DIVSoPV//gOAL+1Ids5tc87VeS//BIxv74OcczOcc8XOueL8/HxfipXud86Yfsy8eQqH9c3k5icXcMtTC6iq1R3dRILkZyh8BIwwsyFmlgRMA15q3cHMClu9PBdY4mM9EoUG9k7jqekT+cGpI3hhwQa+duf7zF+7I+iyROKWb6HgnGsEvge8TuSP/dPOuc/M7HYzO9frdpOZfWZmC4GbgKv8qkeiVziUwA9OPYynr59EU7Pjwns/5Fczl1Db0BR0aSJxx2LtKtPi4mI3b968oMsQn1TVNvDLmUt5Yu46huan85sLxjB+cK+gyxKJeWY23zlXvL9+uqJZokpmSiK/+sZRPHLNBOoamrng3g/5xSufa6tBpJsoFCQqTRmRz+s/nMq3JgziT++t5oz/fZd3l+vMMxG/KRQkamUkh/nF14/i8WuPI8GMKx6cy3cf/5jNFbpPg4hfFAoS9SYPz+PVH0zhlq8exhufb+GU3/2T+99bRaOuhhbpcgoFiQnJ4RA3nTKCN344leKi3vznK0s4+w/v894K7VIS6UoKBYkpg3PTeejqY7nn0nFU1zdy+QNzuerPc1mxRSOvinQFhYLEHDPjzKMKefOWE/l/Z41k/todnHHHe/zk+UWUV9Xt/wNEpEO6TkFi3vbqeu58awWPzl5LYiiBKycXMX3qUHqnJwVdmkjU6Ox1CgoF6TFWle/ijrdW8NLCjaQlhrjq+CKumzKUnDSFg4hCQeLWii1V3PHWCl5ZtIn0pDBXTBrMVZOLKMhKCbo0kcAoFCTuLdtcxZ1vrWDm4k0kJiRw/th+XDdlKCP6ZAZdmki3UyiIeNZsreaB91fz1/nrqW1o5isjC7hychFThueRkNDebT9Eeh6Fgkgb26vreWTWWh6etYZt1fUM6p3GtAkDuXD8QPIzk4MuT8RXCgWRDtQ1NvHa4s08Pmcdc1ZvJzFknDa6LxeMH8AJI/JIDOlMbel5FAoinVBStosn5q7j2Y9L2VnTQO/0JM4+upDzx/Zn7MAczLR7SXoGhYLIAahvbOad5eW88MkG3lyyhbrGZgbnpnHmkYWcdkQfjhmQo+MPEtMUCiIHqbK2gdcWb+alBRuZvWobjc2Ogsxkvjq6D6cd0ZdJQ3NJCmsXk8QWhYJIF6ioaeDtZWW8/tlm3lleTk19E+lJISYNy2XKiHymHpZPUW6adjNJ1OtsKIS7oxiRWJWdlsj5Y/tz/tj+1DY08f6Krby9rIx3V5Tz5pIyAAb0SmXqYflMGZ7HhCG9yc3QmUwSu7SlIHKQ1m6r5t3l5by7YiuzVm5jV10jAMMLMji2qDfHDenNhCG96ZeTGnClItp9JNKtGpqaWbh+J3PXbOej1duZt2YHVV5I9M9JpbioF2MG5DBmYDZH9MsmJTEUcMUSb7T7SKQbJYYSKC7qTXFRbzgJmpodSzdXMnf1dj5as505q7bz4oKNAIQSjMP7ZDJmYDZjBuRw9IAchhdk6OC1RAVtKYh0ky2VtSxcv5NPSytYWLqThet3Ulkb2ZoIJxjDCzIYVZjFyL6ZjCzMYlTfTPIzk3UQW7qEdh+JRDnnHGu21bBoQwVLNlWydFMlSzdXsamitqVP7/QkRvbNZERBBsMKMhiWH3n0yVJYyIHR7iORKGdmDMlLZ0heOueO6dfSvrOmniWbqli6uZKl3vOzH29oOZANkJ4UahUS6QzLz6AoL52BvdPISNavtRw8/fSIRJmctCQmDctl0rDcljbnHGVVdaws28XK8l2sLK9mZfku5qzaxvOfbPjS+3PTkxiUm8ag3pHHwN5fTPfNStGV2bJPCgWRGGBm9MlKoU9WCpOH531pXnVdI6u3VrNmWzXrttewfnsN67bXMH/tDv62cCPNrfYQJ4USGNArlcKcFAqzU+mXk0q/7BQKWz1rSyO+ae2LxLj05DBH9s/myP7Ze81raGpm487drNtew9ptkcBYv6OGjTtreW9FOWVVdbQ9rJiVEqZfTiqFXkgUZqWQn5lMQVYyBZkpFGQm0zs9ibBGk+2RFAoiPVhiKIHBuekMzk1nyoi959c3NrOlspZNFbVsqtjNxp1fPG/cuZsF63eyo6Zhr/clGPROT6YgMzkSGF5o5GckU5CVQl5GJDhy05PITk3ULqsYolAQiWNJ4QQGescdOlLX2ER5VR1lVXWUVdZRvquO8spayqrqWtqXbq5k6656mpr3PpsxlGD0SkukV1pSJCgyIs+9vde9M5LJTU+iV1pkXk5aIslhXdwXFIWCiOxTcjjEgF5pDOjVcXBA5IK9HTX1lFXWsXVXHdur61se26rr2V5dx47qBpZtrmJ7dT07dzfstetqj9TEENmpieSkJZKdmvil6Zy0JLJSE8lp256aRGZKWFslh0ihICJdIpRg5GUkk9fJAQGbmh07a74IjR3e886aeip2N7CzpiHyvLuBddtr+LQ08np3Q1OHn2kGWSlfhEVmSpislNbPiWSltmpL/WJeVkoiGSlhQnEeKgoFEQlEKMHIzUgmNyOZdg53dKiusYmK3Q1U7AmNmkhwRNrqW4KkYncDVbWNlFXuoqq2kcraBmrqOw6UPTKTw3sFxp7X7YZJm36xPq6VQkFEYkpyOERBZoiCzJQDfm9DUzO7ahtbQqJydwOV3nRVbSOVXpDsmVdV28jmylpWlH3Rp73jJq0lhRPI2hMSqYlfTO8Jk+TIc3pymPSkEKlJIdKSwqS1TIdISwyTmhQKZDwsX0PBzM4A7gBCwP3OuV+3mZ8MPAyMB7YBFzvn1vhZk4jEr8RQAr3Sk+iVnnRQ73fOUVPf1H6IeK/bm7eporZlXm1Dc6e/XjjBvgiKpDCXHjeIa6cMPajaO/01/fpgMwsBdwNfBUqBj8zsJefc5626XQPscM4NN7NpwH8BF/tVk4jIoTCzyH/4yWEK974spFPqG5upqm2guq6JmoZGquua2F3fRE19I7sbmqipjzx21ze2TNd40509XnMo/NxSmACUOOdWAZjZk8B5QOtQOA/4d2/6GeAuMzMXa6P0iYh0UlI4wTuWEnQl7fNzh1V/YH2r16VeW7t9nHONQAWQ26YPZjbdzOaZ2bzy8nKfyhURET9Dob3zutpuAXSmD865Gc65YudccX5+fpcUJyIie/MzFEqBga1eDwA2dtTHzMJANrDdx5pERGQf/AyFj4ARZjbEzJKAacBLbfq8BFzpTV8A/EPHE0REguPbgWbnXKOZfQ94ncgpqQ865z4zs9uBec65l4AHgEfMrITIFsI0v+oREZH98/U6BefcTGBmm7Z/bTVdC1zoZw0iItJ5GhBdRERaKBRERKSFxdpxXTMrB9Ye5NvzgK1dWE4s0DLHBy1zfDiUZR7snNvvOf0xFwqHwszmOeeKg66jO2mZ44OWOT50xzJr95GIiLRQKIiISIt4C4UZQRcQAC1zfNAyxwfflzmujimIiMi+xduWgoiI7EPchIKZnWFmy8ysxMxuDbqeg2VmA83sbTNbYmafmdnNXntvM3vDzFZ4z728djOzO73l/tTMxrX6rCu9/ivM7MqOvma0MLOQmX1iZi97r4eY2Ryv/qe8MbYws2TvdYk3v6jVZ9zmtS8zs9ODWZLOMbMcM3vGzJZ663tST1/PZvZD7+d6sZk9YWYpPW09m9mDZlZmZotbtXXZejWz8Wa2yHvPnWbW3mjUHXPO9fgHkbGXVgJDgSRgITA66LoOclkKgXHedCawHBgN/Ddwq9d+K/Bf3vRZwKtEhimfCMzx2nsDq7znXt50r6CXbz/LfgvwOPCy9/ppYJo3fS/wHW/6RuBeb3oa8JQ3Pdpb98nAEO9nIhT0cu1jef8CXOtNJwE5PXk9E7m/ymogtdX6vaqnrWdgKjAOWNyqrcvWKzAXmOS951XgzAOqL+hvUDethEnA661e3wbcFnRdXbRsLxK55ekyoNBrKwSWedP3AZe06r/Mm38JcF+r9i/1i7YHkaHX3wK+Arzs/cBvBcJt1zGRQRgnedNhr5+1Xe+t+0XbA8jy/kBam/Yeu5754qZbvb319jJwek9cz0BRm1DokvXqzVvaqv1L/TrziJfdR525C1zM8TaXxwJzgD7OuU0A3nOB162jZY+178n/Aj8G9tz1PBfY6SJ37IMv19/RHf1iaZmHAuXAn71dZvebWTo9eD075zYAvwXWAZuIrLf59Oz1vEdXrdf+3nTb9k6Ll1Do1B3eYomZZQDPAj9wzlXuq2s7bW4f7VHHzM4Gypxz81s3t9PV7WdezCwzkf98xwH3OOfGAtVEdit0JOaX2duPfh6RXT79gHTgzHa69qT1vD8HuoyHvOzxEgqduQtczDCzRCKB8Jhz7jmveYuZFXohyx9mAAADYUlEQVTzC4Eyr72jZY+l78nxwLlmtgZ4ksgupP8Fcixyxz74cv0d3dEvlpa5FCh1zs3xXj9DJCR68no+FVjtnCt3zjUAzwGT6dnreY+uWq+l3nTb9k6Ll1DozF3gYoJ3JsEDwBLn3O9bzWp9F7sriRxr2NN+hXcWw0Sgwts8fR04zcx6ef+hnea1RR3n3G3OuQHOuSIi6+4fzrlLgbeJ3LEP9l7m9u7o9xIwzTtrZQgwgshBuajjnNsMrDezw72mU4DP6cHrmchuo4lmlub9nO9Z5h67nlvpkvXqzasys4ne9/CKVp/VOUEfcOnGAztnETlTZyXwk6DrOYTlOIHI5uCnwALvcRaRfalvASu8595efwPu9pZ7EVDc6rO+DZR4j6uDXrZOLv9JfHH20VAiv+wlwF+BZK89xXtd4s0f2ur9P/G+F8s4wLMyAljWY4B53rp+gchZJj16PQM/B5YCi4FHiJxB1KPWM/AEkWMmDUT+s7+mK9crUOx9/1YCd9HmZIX9PXRFs4iItIiX3UciItIJCgUREWmhUBARkRYKBRERaaFQEBGRFgoFkW5kZieZN8qrSDRSKIiISAuFgkg7zOwyM5trZgvM7D6L3Mthl5n9zsw+NrO3zCzf63uMmc32xrt/vtVY+MPN7E0zW+i9Z5j38Rn2xX0SHjvg8e5FfKRQEGnDzEYBFwPHO+eOAZqAS4kM0Paxc24c8A7wb95bHgb+xTl3NJGrTve0Pwbc7ZwbQ2QMn01e+1jgB0TG/R9KZGwnkagQ3n8XkbhzCjAe+Mj7Jz6VyABlzcBTXp9HgefMLBvIcc6947X/BfirmWUC/Z1zzwM452oBvM+b65wr9V4vIDK2/vv+L5bI/ikURPZmwF+cc7d9qdHsZ2367WuMmH3tEqprNd2Efg8limj3kcje3gIuMLMCaLl/7mAivy97Ruv8FvC+c64C2GFmU7z2y4F3XOQeF6Vmdr73GclmltatSyFyEPQfikgbzrnPzeynwN/NLIHIaJbfJXKjmyPMbD6Ru3xd7L3lSuBe74/+KuBqr/1y4D4zu937jAu7cTFEDopGSRXpJDPb5ZzLCLoOET9p95GIiLTQloKIiLTQloKIiLRQKIiISAuFgoiItFAoiIhIC4WCiIi0UCiIiEiL/w9tbmpt9irZCgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(hist.history['loss'])\n",
    "plt.xlabel('epoch')\n",
    "plt.ylabel('loss')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "⚙️ 엔지니어\n",
    "\n",
    "> 반복적으로 학습할 수록 손실(Loss)이 0에 가깝게 된다.  \n",
    "> 나이스!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 해결 (Solution)\n",
    "\n",
    "⚙️ 엔지니어  \n",
    "\n",
    "> 고객님~ 원하시는 솔루션입니다.   \n",
    "> test_data에 원하시는 값을 입력하시면    \n",
    "> 원하시는 결과값이 출력 됩니다.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input data is [0, 0, 0, 0, 0, 1, 0, 0, 0, 0] and ouput is 5\n"
     ]
    }
   ],
   "source": [
    "def auto_coding(x):\n",
    "    input_data = np.array(x).reshape(1, 10)\n",
    "    predict = model.predict(input_data)\n",
    "    return np.argmax(predict[0])\n",
    "\n",
    "test_data =  [0, 0, 0, 0, 0, 1, 0, 0, 0, 0]\n",
    "result = auto_coding(test_data)\n",
    "print('input data is {} and ouput is {}'.format(test_data, result))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "⚙️ 엔지니어  \n",
    "\n",
    "> 내가 만든 human_coding과  \n",
    "> 같은 결과가 나온다   \n",
    ">\n",
    "> 이것은  \n",
    "> 히든 레이어에서   \n",
    "> 학습에 의해서  \n",
    "> 자동으로 코딩이 되었다는 것이다.  \n",
    ">\n",
    "> 여기서 문제는  \n",
    "> 히든 레이어에서의 $w$와 $b$ 값들이  \n",
    "> 왜 이런 값을 가지는지  \n",
    "> 인간은 모른다는 것이다.  \n",
    "> 그래서  \n",
    "> Hidden(숨은) 레이어 라고 한다.  \n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}