Classifier in Pytorch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Classifier in Pytorch By Anubhav Natani "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook insted of writing full codes i had made functions for train,test,fit,infrence those are necessary as to keep the code modular as well they are really helpfull when we debug the code i would suggest everyone to follow the same"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Torchvision module contain several boiler plate code for data prepossing and also contains the datasetsm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision \n",
    "import torchvision.transforms as transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Transforms are used for image transformation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "transform = transforms.Compose([transforms.ToTensor(),transforms.Normalize((0.1307,), (0.3081,))])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Downloading the dataset\n",
    "1. Making the loader of the dataset\n",
    "2. Applying the transforms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainset = torchvision.datasets.MNIST(root='./data', train=True, download=True, transform=transform)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "testset = torchvision.datasets.MNIST(root='./data', train=False, download=True, transform=transform)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainloader = torch.utils.data.DataLoader(trainset,batch_size=128,shuffle=True,num_workers=2)\n",
    "testloader = torch.utils.data.DataLoader(testset,batch_size=128,shuffle=False,num_workers=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> loader has dataset so loder gives data in the batches"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Plotting the train and test samples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAADXRJREFUeJzt3W2MXOV5xvHrYlnb2JhgY+xaDtQpmDYIFYdu7EqQyikhctJIJq1AcaXItFVNKlAbiUpFfAn9UAlFDQlUEZUDVpyKkCAFYiux2lCriKR1HRaH8BLT2LVMcG3ZpHaxgcZve/fDHqeL2XlmPXNmzlj3/ydZM3Puc+bcnt1rz5x5ZuZxRAhAPuc13QCAZhB+ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJnd/PnU3z9JihWf3cJZDKL/SWjscxT2XdrsJve6WkByQNSXo4Iu4rrT9Ds7TcN3azSwAF22LLlNft+Gm/7SFJX5b0MUlXS1pt++pO7w9Af3Vzzr9M0q6I2B0RxyV9Q9KqetoC0GvdhH+RpNcm3N5bLXsH22ttj9oePaFjXewOQJ26Cf9kLyq86/PBEbEuIkYiYmRY07vYHYA6dRP+vZIum3D7vZL2ddcOgH7pJvzPSlpi+322p0n6lKRN9bQFoNc6HuqLiJO275T0Txof6lsfES/X1hmAnupqnD8iNkvaXFMvAPqIt/cCSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkCD+QVFez9NreI+mopFOSTkbESB1NAei9rsJf+XBE/LyG+wHQRzztB5LqNvwh6Xu2n7O9to6GAPRHt0/7r4+IfbbnS3rK9isR8czEFao/CmslaYZmdrk7AHXp6sgfEfuqy4OSnpS0bJJ11kXESESMDGt6N7sDUKOOw297lu3Zp69L+qikl+pqDEBvdfO0f4GkJ22fvp+vR8Q/1tIVgJ7rOPwRsVvStTX2AqCPGOoDkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRVxyy9QEsentayNjRvbnHbU4vmFesHPzi7WD/6of9tve/zTxW3befE8XJ05jw9o1i/5OGtXe2/Dhz5gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiCptuP8ttdL+oSkgxFxTbVsrqRvSlosaY+kWyPicO/aRKd+uu6DxfrchW8U64dfnVOsf3jk5WL9uotebVn7zHv+vbjtkMvHplMxVqz38r7bbX+F/qhYv+ThYrkvpnLk/6qklWcsu1vSlohYImlLdRvAOaRt+CPiGUmHzli8StKG6voGSTfX3BeAHuv0nH9BROyXpOpyfn0tAeiHnr+33/ZaSWslaYZm9np3AKao0yP/AdsLJam6PNhqxYhYFxEjETEyrOkd7g5A3ToN/yZJa6rrayRtrKcdAP3SNvy2H5O0VdKv295r+08k3SfpJts7Jd1U3QZwDml7zh8Rq1uUbqy5F3TozVuWt6zt+r2Hurvz67rbvBvtxtoPj7X+vL4krTvcuvmHt99Q3Hb28+VT1EUbdhTrV77xQrE+CHiHH5AU4QeSIvxAUoQfSIrwA0kRfiApvrr7HDB2w9Ji/cHP/12hWv4R7z/1drG+++SFxfrWt5YU60++dm3L2lvHWn+ttyTpXy8uli9/Yl+xfnL3npa1JXquvO82uvvi78HAkR9IivADSRF+ICnCDyRF+IGkCD+QFOEHkmKc/xzwn2vLf6OXTuv8x/j4kd8s1jfe/ZFifcZ3flisv0e7CrXunOxy++w48gNJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUozzV86bWZ5KLE60HlWOE8frbucdLv3n8tdIP7qs9VSJv4jyZ+bPc/nrsb/79w8W67d8/LZifeyFV4p1NIcjP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8k5Ygor2Cvl/QJSQcj4ppq2b2S/lTS69Vq90TE5nY7u8hzY7kHc2bvnRvazEXt1qUlt/24vO3Y4H7L+3mzZxfrF3x3RrH+ox2Li/Wrbn/2bFtCF7bFFh2JQ4Xf1v83lSP/VyWtnGT5FyNiafWvbfABDJa24Y+IZyQd6kMvAPqom3P+O22/YHu97Tm1dQSgLzoN/0OSrpC0VNJ+SV9otaLttbZHbY+e0LEOdwegbh2FPyIORMSpiBiT9BVJywrrrouIkYgYGVb5AyoA+qej8NteOOHmJyW9VE87APql7Ud6bT8maYWkebb3SvqcpBW2l0oKSXsk3d7DHgH0QNvwR8TqSRY/0oNeGvWH15bHo//60tZj+Use+LPitkv+vPzd9mrzXoteGjt6tFj/0Su/Uaw/8LuPFutf1lVn3RP6g3f4AUkRfiApwg8kRfiBpAg/kBThB5Liq7sr31n/oWL9jr/c2rK28/cfKm577aJPF+szN15UrM8bbfO5qoNdfO5q/txieeiN8q/IRy74n2L9wRWtPyo99PT24rboLY78QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4/yVBQ/+W7F+/ZV3tazt/IPyOP+Pl/9DeefLy+Uhl/9G//DYifIdFPzWtKGOtx1X/hXa/ceta0ue7nLX6ApHfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IinH+KXr//ftb1q6c9pnitptXfqlYv2q4PA12O92P1Xfu229dXKzP/El3/zf0Dkd+ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0jK0WZ6aNuXSfqapF+RNCZpXUQ8YHuupG9KWixpj6RbI+Jw6b4u8txY7htraPvcMnRJ+bvxT12xqFh//boL62znHS7477Fi/eKte4v1sUPFH7nG3n77rHtC57bFFh2JQ57KulM58p+UdFdEvF/Sb0u6w/bVku6WtCUilkjaUt0GcI5oG/6I2B8R26vrRyXtkLRI0ipJG6rVNki6uVdNAqjfWZ3z214s6QOStklaEBH7pfE/EJLm190cgN6ZcvhtXyjpW5I+GxFHzmK7tbZHbY+e0LFOegTQA1MKv+1hjQf/0Yh4olp8wPbCqr5Q0sHJto2IdRExEhEjw5peR88AatA2/LYt6RFJOyLi/gmlTZLWVNfXSNpYf3sAemUqQ303SPq+pBc1PtQnSfdo/Lz/cUmXS/qZpFsiojhXdNahPqBfzmaor+3n+SPiB5Ja3RlJBs5RvMMPSIrwA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kFTb8Nu+zPa/2N5h+2Xbf1Etv9f2f9l+vvr38d63C6Au509hnZOS7oqI7bZnS3rO9lNV7YsR8be9aw9Ar7QNf0Tsl7S/un7U9g5Ji3rdGIDeOqtzftuLJX1A0rZq0Z22X7C93vacFtustT1qe/SEjnXVLID6TDn8ti+U9C1Jn42II5IeknSFpKUaf2bwhcm2i4h1ETESESPDml5DywDqMKXw2x7WePAfjYgnJCkiDkTEqYgYk/QVSct61yaAuk3l1X5LekTSjoi4f8LyhRNW+6Skl+pvD0CvTOXV/uslfVrSi7afr5bdI2m17aWSQtIeSbf3pEMAPTGVV/t/IMmTlDbX3w6AfuEdfkBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQcEf3bmf26pFcnLJon6ed9a+DsDGpvg9qXRG+dqrO3X42IS6eyYl/D/66d26MRMdJYAwWD2tug9iXRW6ea6o2n/UBShB9Iqunwr2t4/yWD2tug9iXRW6ca6a3Rc34AzWn6yA+gIY2E3/ZK2/9he5ftu5vooRXbe2y/WM08PNpwL+ttH7T90oRlc20/ZXtndTnpNGkN9TYQMzcXZpZu9LEbtBmv+/603/aQpJ9KuknSXknPSlodET/payMt2N4jaSQiGh8Ttv07kt6U9LWIuKZa9nlJhyLivuoP55yI+KsB6e1eSW82PXNzNaHMwokzS0u6WdJtavCxK/R1qxp43Jo48i+TtCsidkfEcUnfkLSqgT4GXkQ8I+nQGYtXSdpQXd+g8V+evmvR20CIiP0Rsb26flTS6ZmlG33sCn01oonwL5L02oTbezVYU36HpO/Zfs722qabmcSCatr009Onz2+4nzO1nbm5n86YWXpgHrtOZryuWxPhn2z2n0Eacrg+Iq6T9DFJd1RPbzE1U5q5uV8mmVl6IHQ643Xdmgj/XkmXTbj9Xkn7GuhjUhGxr7o8KOlJDd7swwdOT5JaXR5suJ9fGqSZmyebWVoD8NgN0ozXTYT/WUlLbL/P9jRJn5K0qYE+3sX2rOqFGNmeJemjGrzZhzdJWlNdXyNpY4O9vMOgzNzcamZpNfzYDdqM1428yacayviSpCFJ6yPib/rexCRs/5rGj/bS+CSmX2+yN9uPSVqh8U99HZD0OUnflvS4pMsl/UzSLRHR9xfeWvS2QuNPXX85c/Ppc+w+93aDpO9LelHSWLX4Ho2fXzf22BX6Wq0GHjfe4QckxTv8gKQIP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8k9X/oo8fkYeoO8QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAADIBJREFUeJzt3W2MXGUZxvHroixbLRgL2FqxghA0VhKLrlVTXzAIASW2fsDYKKmRuGBolIREGz4oxmgaIr4bzWoba4IgBpGGEIVUFEWCbJHQYlUqKbC0dtFqKEYLZW8/7KlZys6Z6cw5c4be/1+ymZlzn7PnzrTXPjPzzMzjiBCAfI5qugEAzSD8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSOrqfJzvGwzFX8/p5SiCV/+rfejr2u5N9ewq/7fMkfV3SHEnfj4h1ZfvP1Ty9xWf3ckoAJe6JzR3v2/XDfttzJH1b0vmSlkhaZXtJt78PQH/18px/maQdEfFwRDwt6XpJK6ppC0Ddegn/SZIem3F7otj2HLZHbY/bHn9G+3s4HYAq9RL+2V5UeN7ngyNiLCJGImJkSMM9nA5AlXoJ/4SkxTNuv1LSrt7aAdAvvYT/Xkmn23617WMkfUjSpmraAlC3rqf6IuKA7TWSfqHpqb4NEfFgZZ0BqFVP8/wRcaukWyvqBUAf8fZeICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5Lq6xLdqMecJa9pWfvsLdeVHrvmi2tK6yesv7urnjD4GPmBpAg/kBThB5Ii/EBShB9IivADSRF+IKme5vlt75S0T9Kzkg5ExEgVTeHw7H/5cS1rZw5PlR577pq7Sutb1jM+HKmqeJPPuyPi7xX8HgB9xJ91IKlewx+SbrO9xfZoFQ0B6I9eH/Yvj4hdthdIut32nyLizpk7FH8URiVprl7c4+kAVKWnkT8idhWXk5JukrRsln3GImIkIkaGNNzL6QBUqOvw255n+7iD1yWdK2lbVY0BqFcvD/sXSrrJ9sHf86OI+HklXQGoXdfhj4iHJb2hwl5Qg6OY0EEL/M8AkiL8QFKEH0iK8ANJEX4gKcIPJMVXdx/hplT+kd5LT/hdaX30zZ8orce9Ww+7JwwGRn4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIp5/iPA0b/c0rL2+ck3lR77hQX3l9YPHDtUWp9TWsUgY+QHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaSY509uStF0C2gIIz+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJNU2/LY32J60vW3GtuNt3277oeJyfr1toi5HyaU/OHJ1MvL/QNJ5h2xbK2lzRJwuaXNxG8ALSNvwR8SdkvYesnmFpI3F9Y2SVlbcF4Cadfucf2FE7Jak4nJBdS0B6Ifa39tve1TSqCTN1YvrPh2ADnU78u+xvUiSisvJVjtGxFhEjETEyJCGuzwdgKp1G/5NklYX11dLurmadgD0SydTfddJulvSa21P2L5Y0jpJ59h+SNI5xW0ALyBtn/NHxKoWpbMr7gUNaPd5/n+8fm5pfcEdVXaDfuIdfkBShB9IivADSRF+ICnCDyRF+IGk+Oru5Np9bHff2/5TWl/wrSq7QT8x8gNJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUszzH+F+ctvy0vrnP/KH0vo7TttRWt912B1hUDDyA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBSzPMf4Yb3ln9ev93n+ccW/6q0foHedLgtYUAw8gNJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUm3n+W1vkHSBpMmIOKPYdpWkj0t6otjtyoi4ta4m0b2Tr32ktD71yfIluqc0VWU7GCCdjPw/kHTeLNu/GhFLix+CD7zAtA1/RNwpaW8fegHQR708519j+wHbG2zPr6wjAH3Rbfi/I+k0SUsl7ZZ0TasdbY/aHrc9/oz2d3k6AFXrKvwRsScino2IKUnfk7SsZN+xiBiJiJEhDXfbJ4CKdRV+24tm3PyApG3VtAOgXzqZ6rtO0lmSTrQ9Ielzks6yvVRSSNop6ZIaewRQg7bhj4hVs2xeX0MvqMGBicdL66/79cWl9e3vKv+nfvjqt5XWT/303aV1NId3+AFJEX4gKcIPJEX4gaQIP5AU4QeS4qu7k1uwaW5pfepd5R/pXbfy2tL6hm+8o2Wt3TQk6sXIDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJMc+f3Py7Jkrrt/z7hNL6ynn/Kq1f8/bFLWvHXc88f5MY+YGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKeb5kzvwWPk8/9qbPlxaf/9HvlVaf+3lD7as7bq+9FDUjJEfSIrwA0kRfiApwg8kRfiBpAg/kBThB5JqO89ve7GkH0p6uaQpSWMR8XXbx0v6saRTJO2U9MGI+Gd9raIJp974VGl96KI5pfWxxb9qWVty9Zryc7O8d606GfkPSLoiIl4n6a2SLrO9RNJaSZsj4nRJm4vbAF4g2oY/InZHxH3F9X2Stks6SdIKSRuL3TZKWllXkwCqd1jP+W2fIulMSfdIWhgRu6XpPxCSFlTdHID6dBx+28dKulHS5RHx5GEcN2p73Pb4M9rfTY8AatBR+G0PaTr410bET4vNe2wvKuqLJE3OdmxEjEXESESMDGm4ip4BVKBt+G1b0npJ2yPiKzNKmyStLq6vlnRz9e0BqEsnH+ldLukiSVtt319su1LSOkk32L5Y0qOSLqynRTTq91tLy9/858ml9dGX7mhZu/Ccu0qP3cLbUGrVNvwR8VtJblE+u9p2APQLf1qBpAg/kBThB5Ii/EBShB9IivADSfHV3ejJ+u++r7R+6dpvtqx9aeEDpce+590fK63PueO+0jrKMfIDSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFLM86Mnr9i4rbT+7UtPa1l764v+WnrsMX/bV1p/trSKdhj5gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApR0TfTvYSHx9vMd/2DdTlntisJ2Nvq6/afw5GfiApwg8kRfiBpAg/kBThB5Ii/EBShB9Iqm34bS+2fYft7bYftP2pYvtVth+3fX/x89762wVQlU6+zOOApCsi4j7bx0naYvv2ovbViPhyfe0BqEvb8EfEbkm7i+v7bG+XdFLdjQGo12E957d9iqQzJd1TbFpj+wHbG2zPb3HMqO1x2+PPaH9PzQKoTsfht32spBslXR4RT0r6jqTTJC3V9CODa2Y7LiLGImIkIkaGNFxBywCq0FH4bQ9pOvjXRsRPJSki9kTEsxExJel7kpbV1yaAqnXyar8lrZe0PSK+MmP7ohm7fUBS+de4Ahgonbzav1zSRZK22r6/2HalpFW2l0oKSTslXVJLhwBq0cmr/b+VNNvng2+tvh0A/cI7/ICkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0n1dYlu209IemTGphMl/b1vDRyeQe1tUPuS6K1bVfZ2ckS8rJMd+xr+553cHo+IkcYaKDGovQ1qXxK9daup3njYDyRF+IGkmg7/WMPnLzOovQ1qXxK9dauR3hp9zg+gOU2P/AAa0kj4bZ9n+8+2d9he20QPrdjeaXtrsfLweMO9bLA9aXvbjG3H277d9kPF5azLpDXU20Cs3FyysnSj992grXjd94f9tudI+oukcyRNSLpX0qqI+GNfG2nB9k5JIxHR+Jyw7XdKekrSDyPijGLb1ZL2RsS64g/n/Ij4zID0dpWkp5peublYUGbRzJWlJa2U9FE1eN+V9PVBNXC/NTHyL5O0IyIejoinJV0vaUUDfQy8iLhT0t5DNq+QtLG4vlHT/3n6rkVvAyEidkfEfcX1fZIOrizd6H1X0lcjmgj/SZIem3F7QoO15HdIus32FtujTTczi4XFsukHl09f0HA/h2q7cnM/HbKy9MDcd92seF21JsI/2+o/gzTlsDwi3ijpfEmXFQ9v0ZmOVm7ul1lWlh4I3a54XbUmwj8hafGM26+UtKuBPmYVEbuKy0lJN2nwVh/ec3CR1OJysuF+/m+QVm6ebWVpDcB9N0grXjcR/nslnW771baPkfQhSZsa6ON5bM8rXoiR7XmSztXgrT68SdLq4vpqSTc32MtzDMrKza1WllbD992grXjdyJt8iqmMr0maI2lDRHyx703Mwvapmh7tpelFTH/UZG+2r5N0lqY/9bVH0uck/UzSDZJeJelRSRdGRN9feGvR21mafuj6/5WbDz7H7nNvb5f0G0lbJU0Vm6/U9PPrxu67kr5WqYH7jXf4AUnxDj8gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSIrwA0n9D0+kd1YBzy/iAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAADnZJREFUeJzt3X+MHPV5x/HPc+eznTjQ2ALbh+PgQAwxQeDQq93UVXFkkUKEYmgSFCtKnYpyoQotRDQtstSGSElkpQFiGpTqAAcjEUgUQjCFEJAThdBQh4O62KmDTd0LOHZ9GNPaFOFf9/SPG0cX++a7x87szN4975dk3e48MzuPRv7c7N53Zr/m7gIQT0fdDQCoB+EHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxDUpCp3Ntmm+FRNq3KXQChv6P90yA/aWNYtFH4zu1jSGkmdku5w99Wp9adqmhbbsiK7BJCw0TeMed2m3/abWaek2yRdIukcSSvM7JxmXw9AtYp85l8k6QV33+HuhyTdJ2l5OW0BaLUi4Z8j6aURz3dmy36LmfWaWb+Z9R/WwQK7A1CmIuEf7Y8KJ9wf7O597t7j7j1dmlJgdwDKVCT8OyXNHfH8HZJ2FWsHQFWKhP9pSfPN7F1mNlnSxyWtL6ctAK3W9FCfux8xs2sk/VDDQ31r3f0XpXUGoKUKjfO7+yOSHimpFwAV4vJeICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgio0S6+ZDUg6IOmopCPu3lNGUwBar1D4Mx9w970lvA6ACvG2HwiqaPhd0mNm9oyZ9ZbREIBqFH3bv8Tdd5nZTEmPm9kv3f2JkStkvxR6JWmq3lpwdwDKUujM7+67sp+Dkh6QtGiUdfrcvcfde7o0pcjuAJSo6fCb2TQzO+nYY0kflLSlrMYAtFaRt/2zJD1gZsde51vu/mgpXQFouabD7+47JJ1fYi8AKsRQHxAU4QeCIvxAUIQfCIrwA0ERfiCoMu7qQ4t1TJuWrL/y0fNya3uXHSy07wvP2p6s3zH3J8n6kLzQ/ou4+qULc2s//27+MZOk0776s7LbaTuc+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKHOvbhz2ZJvhi21ZZfubKM571pL1L876eUWdnKijwfljSEO5tY9s/3By2/vnr2+qp2NSvT11sDO57Rf+/MpkfdKPnmmqp1bb6Bu03/el/8NkOPMDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFDcz1+BzlNPTda3r5mTrP/zrDuT9b1H8+/ZX/LQ9cltpw2kx7vfed+LyXoRR/97MFn/k9mXJ+t7/nhusv7UF76eW1s85XBy29dndyXrJyer4wNnfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IquE4v5mtlXSppEF3PzdbNkPStyXNkzQg6Qp3f7V1bba3jqlTk/UZD6bHlLecfnuy/oPXfydZv+Uvr86tzX90Y3LbRo4U2rqY/7wqPY7/wJ/e1OAVpjS97xk/3Zms13lcyjKWM/9dki4+btkNkja4+3xJG7LnAMaRhuF39yck7Ttu8XJJ67LH6yRdVnJfAFqs2c/8s9x9tyRlP2eW1xKAKrT82n4z65XUK0lT9dZW7w7AGDV75t9jZt2SlP3MvUPD3fvcvcfde7oK/AEGQLmaDf96SSuzxyslPVhOOwCq0jD8ZnavpKcknW1mO83sSkmrJV1kZtslXZQ9BzCONPzM7+4rckp8AX9m4G8uSNa/f/qaQq//xS+tTNanP/pUodevS8dJJyXrH/3wk8n6GV3pe+6RxhV+QFCEHwiK8ANBEX4gKMIPBEX4gaD46u4SfPNT/5isN5rGelH/J5L12Zv+N1nPnwS7fp3vPTu3tvrhdbk1SVrQcCgvfVy7LP9rybceyv+6c0nS4fRt2BMBZ34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCMrcvbKdnWwzfLFNvDuB/+vL70/WN6+8tdDr70lMwS1Jq3ZeWuj1i+iw9FUGf939WG7t7K709OBFfWLHJbm1A587Lb3xvz5XcjfV2OgbtN/32VjW5cwPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0Exzl+Czrenp9De9ncLkvXPXvJwsr5w6q+S9bd35F8H8O6u1n5lQ6PvKhhKfNvAriPp6xcu7f90sj73y8mybNuL+X0dOJDeeJxinB9AQ4QfCIrwA0ERfiAowg8ERfiBoAg/EFTDcX4zWyvpUkmD7n5utuxGSVdJejlbbZW7P9JoZxN1nL9unWedmVsbvHBmodc+ZUX+WLkkPXT2+mQ9Nc7f87Vrk9ue9g8/S9ZxorLH+e+SdPEoy29x94XZv4bBB9BeGobf3Z+QtK+CXgBUqMhn/mvM7DkzW2tm00vrCEAlmg3/NySdKWmhpN2Sbspb0cx6zazfzPoPq8H8aAAq01T43X2Pux919yFJt0talFi3z9173L2nS1Oa7RNAyZoKv5l1j3h6uaQt5bQDoCoN7/c0s3slLZV0ipntlPR5SUvNbKEklzQgKX3vJYC2w/38wXXOPyNZv+4HDyXrH3jLG8n65dvz5xQ4ctEryW398KFkHSfifn4ADRF+ICjCDwRF+IGgCD8QFOEHgmrt9zqjdtY1OVmfdMfryXqjobx/eaMrWR+66i25NYby6sWZHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCYpx/gnv+1oXJ+i/ffVuynv/F28P+6rark/Xu7Xz9drvizA8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTHOPwH4+8/Prf3oQzc32Do9i9KCx9Pj+PNvZhx/vOLMDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBNRznN7O5ku6WNFvDt3f3ufsaM5sh6duS5kkakHSFu7/aulbjst99b7K+/+9fy62dNik9jn/rq+9J1t/zlQPJ+tFkFe1sLGf+I5Kud/cFkn5f0mfM7BxJN0ja4O7zJW3IngMYJxqG3913u/uz2eMDkrZKmiNpuaR12WrrJF3WqiYBlO9NfeY3s3mS3idpo6RZ7r5bGv4FIWlm2c0BaJ0xh9/M3ibpfknXufv+N7Fdr5n1m1n/YR1spkcALTCm8JtZl4aDf4+7fy9bvMfMurN6t6TB0bZ19z5373H3nq4GN5EAqE7D8JuZSbpT0lZ3H3mL2HpJK7PHKyU9WH57AFplLLf0LpH0SUmbzWxTtmyVpNWSvmNmV0p6UdLHWtMitl2bfse09by7cmtbDnly2x8vnZesH927LVnH+NUw/O7+pCTLKS8rtx0AVeEKPyAowg8ERfiBoAg/EBThB4Ii/EBQfHV3G5jUPTtZ/4sLfpKsdyR+hw8cnp7c1uc0uCVj7yvpOsYtzvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/G1g4M/OSNavm/5wsj6kodzaP135keS2Hf/+b8k6Ji7O/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOP8FeiYOjVd/73/KfT6+47mT4M2ecee5LZHCu0Z4xlnfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IquE4v5nNlXS3pNmShiT1ufsaM7tR0lWSXs5WXeXuj7Sq0fHs+ZvOT9a3Lvp6g1dI/47+gx9+Nrd21q+fbvDaiGosF/kckXS9uz9rZidJesbMHs9qt7j7V1vXHoBWaRh+d98taXf2+ICZbZU0p9WNAWitN/WZ38zmSXqfpI3ZomvM7DkzW2tmo84LZWa9ZtZvZv2HlX8ZKoBqjTn8ZvY2SfdLus7d90v6hqQzJS3U8DuDm0bbzt373L3H3Xu6NKWElgGUYUzhN7MuDQf/Hnf/niS5+x53P+ruQ5Jul7SodW0CKFvD8JuZSbpT0lZ3v3nE8u4Rq10uaUv57QFolbH8tX+JpE9K2mxmm7JlqyStMLOFklzSgKRPt6TDCaDjDSu0/T0HupP1BZ/blls7WmjPmMjG8tf+JyWN9r+XMX1gHOMKPyAowg8ERfiBoAg/EBThB4Ii/EBQ5u6V7exkm+GLbVll+wOi2egbtN/3jenCEs78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxBUpeP8ZvaypF+NWHSKpL2VNfDmtGtv7dqXRG/NKrO309391LGsWGn4T9i5Wb+799TWQEK79taufUn01qy6euNtPxAU4QeCqjv8fTXvP6Vde2vXviR6a1YtvdX6mR9Afeo+8wOoSS3hN7OLzex5M3vBzG6oo4c8ZjZgZpvNbJOZ9dfcy1ozGzSzLSOWzTCzx81se/Zz1GnSaurtRjP7dXbsNpnZh2rqba6Z/djMtprZL8zs2mx5rccu0Vctx63yt/1m1ilpm6SLJO2U9LSkFe7+H5U2ksPMBiT1uHvtY8Jm9keSXpN0t7ufmy37iqR97r46+8U53d3/tk16u1HSa3XP3JxNKNM9cmZpSZdJ+pRqPHaJvq5QDcetjjP/IkkvuPsOdz8k6T5Jy2voo+25+xOS9h23eLmkddnjdRr+z1O5nN7agrvvdvdns8cHJB2bWbrWY5foqxZ1hH+OpJdGPN+p9pry2yU9ZmbPmFlv3c2MYlY2bfqx6dNn1tzP8RrO3Fyl42aWbptj18yM12WrI/yjfcVQOw05LHH3CyRdIukz2dtbjM2YZm6uyigzS7eFZme8Llsd4d8pae6I5++QtKuGPkbl7ruyn4OSHlD7zT6859gkqdnPwZr7+Y12mrl5tJml1QbHrp1mvK4j/E9Lmm9m7zKzyZI+Lml9DX2cwMymZX+IkZlNk/RBtd/sw+slrcwer5T0YI29/JZ2mbk5b2Zp1Xzs2m3G61ou8smGMr4mqVPSWnf/UuVNjMLMztDw2V4ansT0W3X2Zmb3Slqq4bu+9kj6vKTvS/qOpHdKelHSx9y98j+85fS2VMNvXX8zc/Oxz9gV9/aHkn4qabOkoWzxKg1/vq7t2CX6WqEajhtX+AFBcYUfEBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGg/h9kgxFNRvLOHQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAADM9JREFUeJzt3X/MnXV5x/H3RekPqTBpBOwQqGJZJCSr+Ax/oAuG4JCQgH9I7B+sJsaSTbKZmU1CFmR/mDE3cWYYtjoaSoKgiSAs6VTSbGEujFEIEbS4ElKwo7Y6IBTlV9trfzyn5qE85z5Pz7nPuU+53q+kOefc3/vHlZN+nu+5z/c+9zcyE0n1HNV1AZK6Yfilogy/VJThl4oy/FJRhl8qyvBLRRl+qSjDLxV19CQPtiSW5jKWT/KQUikv8SteyZdjIeuOFP6IuBD4GrAI+OfMvK5p/WUs531x/iiHlNTg/tyy4HWH/tgfEYuArwMfA84E1kbEmcPuT9JkjXLOfw7weGY+kZmvALcDl7RTlqRxGyX8JwM/m/N6Z2/Za0TE+ojYGhFbX+XlEQ4nqU2jhH++LxVe9/vgzNyQmTOZObOYpSMcTlKbRgn/TuCUOa/fDjw9WjmSJmWU8D8ArI6Id0TEEuCTwN3tlCVp3IYe6svMfRFxJfB9Zof6Nmbmj1urTNJYjTTOn5mbgc0t1SJpgry8VyrK8EtFGX6pKMMvFWX4paIMv1SU4ZeKMvxSUYZfKsrwS0UZfqkowy8VZfiloiZ662698Tz7qQ80tt/3pa8Pve+LTj576G01mD2/VJThl4oy/FJRhl8qyvBLRRl+qSjDLxXlOL9Gsm9Z1xVoWPb8UlGGXyrK8EtFGX6pKMMvFWX4paIMv1TUSOP8EbED2AvsB/Zl5kwbRWl65Ad/t7H9+j//pwlVora1cZHPRzLzly3sR9IE+bFfKmrU8Cfwg4h4MCLWt1GQpMkY9WP/uZn5dEScCNwTEY9l5r1zV+j9UVgPsIxjRjycpLaM1PNn5tO9xz3AncA586yzITNnMnNmMUtHOZykFg0d/ohYHhHHHnwOfBR4tK3CJI3XKB/7TwLujIiD+/lmZn6vlaokjd3Q4c/MJ4DmQWBNvUXHHdfY/uJfPdfY/uFl+4Y+9lW73ztgjRx63xrMoT6pKMMvFWX4paIMv1SU4ZeKMvxSUd66u7jH/vrdje3bz7xxbMf+9xve39i+gvvGdmzZ80tlGX6pKMMvFWX4paIMv1SU4ZeKMvxSUY7zv8EdverUxvZrzr9zQpW83vLdw/8cWKOz55eKMvxSUYZfKsrwS0UZfqkowy8VZfilohznfwOIxUv6ti295cXGbS8/9udtl/Mat79wQt+2N937WOO2B9ouRq9hzy8VZfilogy/VJThl4oy/FJRhl8qyvBLRQ0c54+IjcDFwJ7MPKu3bAXwLWAVsAO4LDOfHV+ZavLTf1jTt+3x0/9xgpW83nP7j+nbdmDv3glWokMtpOe/GbjwkGVXAVsyczWwpfda0hFkYPgz817gmUMWXwJs6j3fBFzacl2SxmzYc/6TMnMXQO/xxPZKkjQJY7+2PyLWA+sBltH//E/SZA3b8++OiJUAvcc9/VbMzA2ZOZOZM4tZOuThJLVt2PDfDazrPV8H3NVOOZImZWD4I+I24D7gdyJiZ0R8GrgOuCAitgMX9F5LOoIMPOfPzLV9ms5vuRYN6Q8/8J9Db7v6jj9qbL/mgub7+g+6H8D2F09qaPW+/V3yCj+pKMMvFWX4paIMv1SU4ZeKMvxSUd66+w3gvz+0om/b+R++onHb1Zvvb2x/6SeLh6rpoH/93u/1bVvFfSPtW6Ox55eKMvxSUYZfKsrwS0UZfqkowy8VZfilohznfwNougX20s0PNG770sXnNLZ/5rdGu/V3xkiba4zs+aWiDL9UlOGXijL8UlGGXyrK8EtFGX6pKMf5i4vMse5/0UsO9E8re36pKMMvFWX4paIMv1SU4ZeKMvxSUYZfKmrgOH9EbAQuBvZk5lm9ZdcCnwF+0Vvt6szcPK4ideR65007+rY5QXe3FtLz3wxcOM/yr2bmmt4/gy8dYQaGPzPvBZ6ZQC2SJmiUc/4rI+JHEbExIo5vrSJJEzFs+G8ETgfWALuAr/RbMSLWR8TWiNj6Ki8PeThJbRsq/Jm5OzP3Z+YB4BtA37tAZuaGzJzJzJnFLB22TkktGyr8EbFyzsuPA4+2U46kSVnIUN9twHnAWyNiJ/BF4LyIWAMksANongda0tQZGP7MXDvP4pvGUIukCfIKP6kowy8VZfilogy/VJThl4oy/FJR3rq7uKcuGu3v/3d/9ZbG9vz1iyPtX+Njzy8VZfilogy/VJThl4oy/FJRhl8qyvBLRTnOX9ySE3490vY3PPmR5v0/++RI+9f42PNLRRl+qSjDLxVl+KWiDL9UlOGXijL8UlGO8x8BjjrmmOb2FcNPlbh0yWgTZW8849bG9ouu+Yu+baf9zYON2+bLTu82Tvb8UlGGXyrK8EtFGX6pKMMvFWX4paIMv1TUwHH+iDgFuAV4G3AA2JCZX4uIFcC3gFXADuCyzHx2fKUeufafd3Zj+8//pHk8+w9O29bY/uW3/cth19SWU49uvgbh0Stu6Nt2xlv+uHHbd/3Zfw1VkxZmIT3/PuDzmflu4P3AZyPiTOAqYEtmrga29F5LOkIMDH9m7srMh3rP9wLbgJOBS4BNvdU2AZeOq0hJ7Tusc/6IWAW8B7gfOCkzd8HsHwjgxLaLkzQ+Cw5/RLwZ+A7wucx8/jC2Wx8RWyNi66t4rbY0LRYU/ohYzGzwb83MO3qLd0fEyl77SmDPfNtm5obMnMnMmcUsbaNmSS0YGP6ICOAmYFtmXj+n6W5gXe/5OuCu9suTNC4L+UnvucDlwCMR8XBv2dXAdcC3I+LTwFPAJ8ZT4vTbc+UHG9vv+cLfNrYff9Sb2iznsCyK5r//+/PASPu/+fnf7tt2xs3PNW472pE1yMDwZ+YPgejTfH675UiaFK/wk4oy/FJRhl8qyvBLRRl+qSjDLxXlrbtbcNyTzbe/7nIcf5BB4/hX7X5vY/sDfznT2H7M9v/r23Zg+2ON22q87Pmlogy/VJThl4oy/FJRhl8qyvBLRRl+qajIzIkd7LhYke8LfwUsjcv9uYXn85l+P8F/DXt+qSjDLxVl+KWiDL9UlOGXijL8UlGGXyrK8EtFGX6pKMMvFWX4paIMv1SU4ZeKMvxSUYZfKmpg+CPilIj4t4jYFhE/jog/7S2/NiL+NyIe7v27aPzlSmrLQibt2Ad8PjMfiohjgQcj4p5e21cz8+/GV56kcRkY/szcBezqPd8bEduAk8ddmKTxOqxz/ohYBbwHuL+36MqI+FFEbIyI4/tssz4itkbE1ld5eaRiJbVnweGPiDcD3wE+l5nPAzcCpwNrmP1k8JX5tsvMDZk5k5kzi1naQsmS2rCg8EfEYmaDf2tm3gGQmbszc39mHgC+AZwzvjIltW0h3/YHcBOwLTOvn7N85ZzVPg482n55ksZlId/2nwtcDjwSEQ/3ll0NrI2INUACO4ArxlKhpLFYyLf9PwTmuw/45vbLkTQpXuEnFWX4paIMv1SU4ZeKMvxSUYZfKsrwS0UZfqkowy8VZfilogy/VJThl4oy/FJRhl8qKjJzcgeL+AXw5JxFbwV+ObECDs+01jatdYG1DavN2k7LzBMWsuJEw/+6g0dszcyZzgpoMK21TWtdYG3D6qo2P/ZLRRl+qaiuw7+h4+M3mdbaprUusLZhdVJbp+f8krrTdc8vqSOdhD8iLoyIn0bE4xFxVRc19BMROyLikd7Mw1s7rmVjROyJiEfnLFsREfdExPbe47zTpHVU21TM3Nwws3Sn7920zXg98Y/9EbEI+B/gAmAn8ACwNjN/MtFC+oiIHcBMZnY+JhwRvw+8ANySmWf1ln0ZeCYzr+v94Tw+M78wJbVdC7zQ9czNvQllVs6dWRq4FPgUHb53DXVdRgfvWxc9/znA45n5RGa+AtwOXNJBHVMvM+8Fnjlk8SXApt7zTcz+55m4PrVNhczclZkP9Z7vBQ7OLN3pe9dQVye6CP/JwM/mvN7JdE35ncAPIuLBiFjfdTHzOKk3bfrB6dNP7LieQw2cuXmSDplZemreu2FmvG5bF+Gfb/afaRpyODczzwY+Bny29/FWC7OgmZsnZZ6ZpafCsDNet62L8O8ETpnz+u3A0x3UMa/MfLr3uAe4k+mbfXj3wUlSe497Oq7nN6Zp5ub5ZpZmCt67aZrxuovwPwCsjoh3RMQS4JPA3R3U8ToRsbz3RQwRsRz4KNM3+/DdwLre83XAXR3W8hrTMnNzv5ml6fi9m7YZrzu5yKc3lPH3wCJgY2Z+aeJFzCMi3slsbw+zk5h+s8vaIuI24Dxmf/W1G/gi8F3g28CpwFPAJzJz4l+89antPGY/uv5m5uaD59gTru1DwH8AjwAHeouvZvb8urP3rqGutXTwvnmFn1SUV/hJRRl+qSjDLxVl+KWiDL9UlOGXijL8UlGGXyrq/wFlkYzsYfZMUwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAADwVJREFUeJzt3X+QVfV5x/HPw7JAQYkQBAGxCIJKSUWzxXS0KRnFoLWDtsUJrR3SccRptdNMSBuHyYxkpu1Yp2pt2sSulQFmFHViFNqhrQZrjWkGWZCKcVUI3SqC/Bi0CGlYdvfpH3twNrjne3fvr3OX5/2acfbe85xzzzMXP3vu3e8552vuLgDxDCu6AQDFIPxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4IaXs+djbCRPkpj6rlLIJSf6bg6/YQNZN2Kwm9miyQ9JKlJ0j+6+72p9UdpjK60ayrZJYCELb55wOuW/bHfzJok/b2k6yXNkbTUzOaU+3oA6quS7/zzJe129z3u3inpCUmLq9MWgFqrJPxTJb3b5/nebNnPMbPlZtZmZm0ndaKC3QGopkrC398fFT5xfbC7t7p7i7u3NGtkBbsDUE2VhH+vpGl9np8vaV9l7QCol0rCv1XSLDO70MxGSPqSpI3VaQtArZU91OfuXWZ2l6R/U+9Q32p3/3HVOgNQUxWN87v7JkmbqtQLgDri9F4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCqmiWXjPrkPSRpG5JXe7eUo2mANReReHPfMHdD1fhdQDUER/7gaAqDb9Les7MtpnZ8mo0BKA+Kv3Yf5W77zOziZKeN7M33f2lvitkvxSWS9Ioja5wdwCqpaIjv7vvy34elPSMpPn9rNPq7i3u3tKskZXsDkAVlR1+MxtjZmefeizpOkmvV6sxALVVycf+SZKeMbNTr/O4u/9rVboCUHNlh9/d90i6rIq9AKgjhvqAoAg/EBThB4Ii/EBQhB8IivADQVXjqr7who0ala6PO6dOnQxe58zzkvXdt46o2b4XzGtP1l/ccWlFrz/jqe7c2qj29yp67a4Dh9Ir9OTvu1Fw5AeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoMKM8zfNnpmst3/108n6JQ8fy63t+rP0WPhbv746WUeOaf9R2fa/WZ02+nPj9b+brPe89mbtdl4lHPmBoAg/EBThB4Ii/EBQhB8IivADQRF+IKgw4/y/veGHyfqXx+5Lv0ANx4xL6ZEn6395+DO5tW9MSM+jsubolGT9795ekKw3/dO4ZL0SnWMtWf/irT9K1m//9Mu5tZnDf6Gsnk7puHl8sn7BaxW9fF1w5AeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoEqO85vZakk3Sjro7nOzZeMlPSlpuqQOSbe4+we1a7NyZzf9X2H7/r2Oa5P19+6flaxbT3qcf8xz+WP5Cxb9YXLbsa/sTdYn7m3c69J3bL08WV/3ra7c2jfP/a+K9j3jH36SrOfvuXEM5Mi/RtKi05bdLWmzu8+StDl7DmAIKRl+d39J0pHTFi+WtDZ7vFbSTVXuC0CNlfudf5K775ek7OfE6rUEoB5qfm6/mS2XtFySRml0rXcHYIDKPfIfMLPJkpT9PJi3oru3unuLu7c0a2SZuwNQbeWGf6OkZdnjZZI2VKcdAPVSMvxmtl7SjyRdbGZ7zew2SfdKWmhmuyQtzJ4DGELMPT2GXE1jbbxfadfUbX99DZ8xPVnvnJq+Lv0nt+X/nty18JHkthc/fmeyPvNP09elR1Xq3+y+F9Yn65c0l/81c9Z3/yhZn/217cm6n+wse9+V2OKbddSPpG+EkOEMPyAowg8ERfiBoAg/EBThB4Ii/EBQYW7d3bWnI1kfVqJ+8bb8U5Pn35oeypu17tVkvSdZjctHNifrEYfyqokjPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8EFWacv1I9P/1pbm1Ca/qSXMbxc1j6ytM3vz62opef9Uz+bctnr2hLbutdQ+Hm25XhyA8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTHOj8IMn35Bsl7qluhrjk5J1i/5dv6s8d0BxvFL4cgPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0GVHOc3s9WSbpR00N3nZstWSbpd0qFstZXuvqlWTeLMNP/ZXcn6zs6Tyfp3lyxI1nveeHOwLYUykCP/GkmL+ln+oLvPy/4j+MAQUzL87v6SpCN16AVAHVXynf8uM3vNzFab2biqdQSgLsoN/3ckzZQ0T9J+SffnrWhmy82szczaTupEmbsDUG1lhd/dD7h7t7v3SHpE0vzEuq3u3uLuLc0qf2JFANVVVvjNbHKfpzdLer067QCol4EM9a2XtEDSBDPbK+keSQvMbJ4kl9Qh6Y4a9gigBkqG392X9rP40Rr0ghpoGpu+9/3xz19S0esf/kz6f6Ep176bW1s54dnktq+caE7Wh/3vsWSd+RLSOMMPCIrwA0ERfiAowg8ERfiBoAg/EBS37h4CrGVusn5gfv5w3k13vJjc9hsTHi6npSpJT9H9uRInhC5+bnuy/uATN+XWLnx4d3Lb7gMH0zs/A3DkB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgzN3rtrOxNt6vtGvqtr9GMeyX05fNzl3zVrK+ZNwryfpnRzQNuqdTmiz9+7/b0xfGVrL9nx9On7/wL+/NSdZ/eNlTyXrK08fTt538q7e+mKx3vTAhWT/vwf8cdE/VsMU366gfSZ9AkeHIDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBcT1/Hbz/a+OT9X+etK3EK6TH8f/gnQW5tT33X5rc9lNt+5P19lXnJuu7Fj6SrN9z6LLc2vZrJya3PefDjmR93lfvStaPzcqf4vuiGe8nt33liieS9dkf3pasDwUc+YGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gqJLj/GY2TdI6Seepd9bjVnd/yMzGS3pS0nRJHZJucfcPatfq0HXuq8eT9Rd/lp6K+vFDv5qs3zNlU27tjsMXJbf10aOS9W9d/ViyXsqrvzUzt9Z9uKOi155yXwXXzA9LnzvxG83p9/yizh3l77tBDOTI3yVphbtfKulzku40szmS7pa02d1nSdqcPQcwRJQMv7vvd/ft2eOPJLVLmippsaS12WprJeVPjwKg4QzqO7+ZTZd0uaQtkia5+36p9xeEpPS5mgAayoDDb2ZnSXpa0lfc/eggtltuZm1m1nZSJ8rpEUANDCj8Ztas3uA/5u7fyxYfMLPJWX2ypH5nNnT3VndvcfeWZpWYeRFA3ZQMv5mZpEcltbv7A31KGyUtyx4vk7Sh+u0BqJWSt+42s6sl/UDSTvUO9UnSSvV+739K0gWS3pG0xN2PpF4r6q27K3Xy2s8m6ye+lj/CeuLpScltP/il9L//20u+nayveH9+evsvjM6tdR8d8LdHDNBgbt1dcpzf3V9W/kTqJBkYojjDDwiK8ANBEX4gKMIPBEX4gaAIPxAUt+4eApq/n761d/P382tnaU9y23Gbp5XT0sc2bL0iWZ99ND29OIrDkR8IivADQRF+ICjCDwRF+IGgCD8QFOEHgmKc/wzXNGd2sv7NC9cn6zs6u5P1S//2w2Q9vTWKxJEfCIrwA0ERfiAowg8ERfiBoAg/EBThB4JinP8M1/7H5yTrvzIyfYv369p/J1kf/sbbg+4JjYEjPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8EVXKc38ymSVon6TxJPZJa3f0hM1sl6XZJh7JVV7r7plo1inzDp52fW3vhhgdKbD06WT2+Zkqy/im9U+L10agGcpJPl6QV7r7dzM6WtM3Mns9qD7r7X9euPQC1UjL87r5f0v7s8Udm1i5paq0bA1Bbg/rOb2bTJV0uaUu26C4ze83MVpvZuJxtlptZm5m1ndSJipoFUD0DDr+ZnSXpaUlfcfejkr4jaaakeer9ZHB/f9u5e6u7t7h7S7NGVqFlANUwoPCbWbN6g/+Yu39Pktz9gLt3u3uPpEckza9dmwCqrWT4zcwkPSqp3d0f6LN8cp/Vbpb0evXbA1ArA/lr/1WSfl/STjPbkS1bKWmpmc2T5JI6JN1Rkw5R2vCm3NIFw9NDeSveT39gO+fJ7cm6J6toZAP5a//Lkvq76JsxfWAI4ww/ICjCDwRF+IGgCD8QFOEHgiL8QFDcuvsM0PXf/5Nbu2HqFaW2rm4zGDI48gNBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUOZevyuyzeyQpL6D0hMkHa5bA4PTqL01al8SvZWrmr39orufO5AV6xr+T+zcrM3dWwprIKFRe2vUviR6K1dRvfGxHwiK8ANBFR3+1oL3n9KovTVqXxK9lauQ3gr9zg+gOEUf+QEUpJDwm9kiM3vLzHab2d1F9JDHzDrMbKeZ7TCztoJ7WW1mB83s9T7LxpvZ82a2K/vZ7zRpBfW2yszey967HWZ2Q0G9TTOzfzezdjP7sZn9Sba80Pcu0Vch71vdP/abWZOktyUtlLRX0lZJS939jbo2ksPMOiS1uHvhY8Jm9nlJxyStc/e52bL7JB1x93uzX5zj3P3rDdLbKknHip65OZtQZnLfmaUl3STpyyrwvUv0dYsKeN+KOPLPl7Tb3fe4e6ekJyQtLqCPhufuL0k6ctrixZLWZo/Xqvd/nrrL6a0huPt+d9+ePf5I0qmZpQt97xJ9FaKI8E+V9G6f53vVWFN+u6TnzGybmS0vupl+TMqmTT81ffrEgvs5XcmZm+vptJmlG+a9K2fG62orIvz9zf7TSEMOV7n7FZKul3Rn9vEWAzOgmZvrpZ+ZpRtCuTNeV1sR4d8raVqf5+dL2ldAH/1y933Zz4OSnlHjzT584NQkqdnPgwX387FGmrm5v5ml1QDvXSPNeF1E+LdKmmVmF5rZCElfkrSxgD4+wczGZH+IkZmNkXSdGm/24Y2SlmWPl0naUGAvP6dRZm7Om1laBb93jTbjdSEn+WRDGX8jqUnSanf/i7o30Q8zm6Heo73Ue2fjx4vszczWS1qg3qu+Dki6R9Kzkp6SdIGkdyQtcfe6/+Etp7cF6v3o+vHMzae+Y9e5t6sl/UDSTkk92eKV6v1+Xdh7l+hrqQp43zjDDwiKM/yAoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwT1/xfHLjmKIt9OAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "def imshow(img):\n",
    "    #unnormalize\n",
    "    img = img*0.3081+0.1307\n",
    "    a = img.numpy()\n",
    "    #bringing first two shape values at first\n",
    "    a = a.reshape(a.shape[1],a.shape[2])\n",
    "    plt.imshow(a)\n",
    "    plt.show()\n",
    "    \n",
    "dataiter = iter(trainloader)\n",
    "images,labels = dataiter.next()\n",
    "\n",
    "for i in range(0,5):\n",
    "    imshow(images[i])\n",
    "    print(labels[i].item())\n",
    "#imshow(images[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Defining a conv netwrok for the task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Net(\n",
      "  (conv1): Conv2d(1, 32, kernel_size=(3, 3), stride=(1, 1))\n",
      "  (conv2): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1))\n",
      "  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
      "  (dropout1): Dropout2d(p=0.25)\n",
      "  (dense1): Linear(in_features=9216, out_features=128, bias=True)\n",
      "  (dropout2): Dropout(p=0.5)\n",
      "  (dense2): Linear(in_features=128, out_features=10, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net,self).__init__()\n",
    "        self.conv1 = nn.Conv2d(1,32,3)\n",
    "        self.conv2 = nn.Conv2d(32,64,3)\n",
    "        self.pool = nn.MaxPool2d(2)\n",
    "        self.dropout1 = nn.Dropout2d(0.25)\n",
    "        self.dense1 = nn.Linear(64*12*12,128)\n",
    "        self.dropout2 = nn.Dropout(0.5)\n",
    "        self.dense2 = nn.Linear(128,10)\n",
    "    def forward(self,x):\n",
    "        x = F.relu(self.conv1(x))\n",
    "        x = F.relu(self.conv2(x))\n",
    "        x = self.pool(x)\n",
    "        x = self.dropout1(x)\n",
    "        x = x.view(-1,64*12*12)\n",
    "        x = F.relu(self.dense1(x))\n",
    "        x = self.dropout2(x)\n",
    "        x = self.dense2(x)\n",
    "        return x\n",
    "    \n",
    "net = Net()\n",
    "net.to(device)\n",
    "print(net)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Defining a loss function and optimizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.optim as optim\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.Adam(net.parameters(),lr=0.001)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">Writing the functions for training and testing the pytorch models and a utility function like keras for fitting the modelm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# training model in pytorch\n",
    "def train(model, device, train_loader, optimizer, criterion, epoch, train_losses):\n",
    "    model.train()\n",
    "    train_loss = 0\n",
    "    for data, target in train_loader:\n",
    "        data, target = data.to(device), target.to(device)\n",
    "        optimizer.zero_grad()\n",
    "        output = model(data)\n",
    "        loss = criterion(output, target)\n",
    "        loss.backward(retain_graph=True)\n",
    "        optimizer.step()\n",
    "        train_loss += loss.item()*data.size(0)\n",
    "    #calculating the total loss\n",
    "    train_loss = ((train_loss)/len(train_loader.dataset))\n",
    "    train_losses.append(train_loss)\n",
    "    #logging the result\n",
    "    print(\"Train Epoch: %d Train Loss: %.4f.\" % (epoch, train_loss))\n",
    "\n",
    "    \n",
    "# testing model in pytorch\n",
    "def test(model, device, test_loader, criterion, epoch, test_losses, accuracies):\n",
    "    model.eval()\n",
    "    test_loss = 0\n",
    "    correct = 0\n",
    "    with torch.no_grad():\n",
    "        for data, target in test_loader:\n",
    "            data, target = data.to(device), target.to(device)\n",
    "            output = model(data)\n",
    "            test_loss += criterion(output, target)\n",
    "            pred = output.argmax(dim=1, keepdim=True)\n",
    "            correct += pred.eq(target.view_as(pred)).sum().item()\n",
    "    #test loss calculation\n",
    "    test_loss = (test_loss/len(test_loader.dataset))\n",
    "    #calculating the accuracy in the validation step\n",
    "    accuracy = (100*correct)/len(test_loader.dataset)\n",
    "    accuracies.append(accuracy)\n",
    "    test_losses.append(test_loss)\n",
    "    #logging the results\n",
    "    print(\"Test Epoch: %d Test Loss: %.4f Test Accuray: %.2f.\" %\n",
    "          (epoch, test_loss, accuracy))\n",
    "\n",
    "\n",
    "# model fitting in pytorch\n",
    "def fit(model, device, train_loader, test_loader, optimizer, criterion, no_of_epochs):\n",
    "    train_losses = []\n",
    "    test_losses = []\n",
    "    accuracies = []\n",
    "    for epoch in range(0, no_of_epochs):\n",
    "        train(model, device, train_loader, optimizer,\n",
    "              criterion, epoch, train_losses)\n",
    "        test(model, device, test_loader, criterion,\n",
    "             epoch, test_losses, accuracies)\n",
    "    return train_losses, test_losses, accuracies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    ">Training the neural network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train Epoch: 0 Train Loss: 0.2441.\n",
      "Test Epoch: 0 Test Loss: 0.0004 Test Accuray: 98.33.\n",
      "Train Epoch: 1 Train Loss: 0.0963.\n",
      "Test Epoch: 1 Test Loss: 0.0003 Test Accuray: 98.75.\n",
      "Train Epoch: 2 Train Loss: 0.0714.\n",
      "Test Epoch: 2 Test Loss: 0.0003 Test Accuray: 98.83.\n",
      "Train Epoch: 3 Train Loss: 0.0618.\n",
      "Test Epoch: 3 Test Loss: 0.0002 Test Accuray: 99.01.\n",
      "Train Epoch: 4 Train Loss: 0.0562.\n",
      "Test Epoch: 4 Test Loss: 0.0003 Test Accuray: 99.01.\n",
      "Train Epoch: 5 Train Loss: 0.0493.\n",
      "Test Epoch: 5 Test Loss: 0.0002 Test Accuray: 99.05.\n",
      "Train Epoch: 6 Train Loss: 0.0404.\n",
      "Test Epoch: 6 Test Loss: 0.0002 Test Accuray: 99.21.\n",
      "Train Epoch: 7 Train Loss: 0.0378.\n",
      "Test Epoch: 7 Test Loss: 0.0002 Test Accuray: 99.20.\n",
      "Train Epoch: 8 Train Loss: 0.0325.\n",
      "Test Epoch: 8 Test Loss: 0.0002 Test Accuray: 99.17.\n",
      "Train Epoch: 9 Train Loss: 0.0321.\n",
      "Test Epoch: 9 Test Loss: 0.0002 Test Accuray: 99.16.\n"
     ]
    }
   ],
   "source": [
    "train_losses,test_losses,accuracies = fit(net,device,trainloader,testloader,optimizer,criterion,10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> After training and testing the models we can save the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.save(net.state_dict(),\"mnist_cnn.pt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Plotting the losses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x1919ee35940>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD8CAYAAACb4nSYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xt8VPWd//HXJ5P7/TaROwEh0RBuMSJWJKis1bZqa20FpV5qy9qrq+3+yrq2trR9PKz6s4h1W21X21Vb1tW15WdVVlsEXasSEFFASEQgkVsCSUiAkEzy/f0xQ5qEhExuTDLzfj4eeXDmzPec+WSM73PO95zzPeacQ0REIkNUqAsQEZHTR6EvIhJBFPoiIhFEoS8iEkEU+iIiEUShLyISQRT6IiIRRKEvIhJBFPoiIhEkOtQFdJadne1yc3NDXYaIyLCyfv36auect6d2Qy70c3NzKS0tDXUZIiLDipntCqZdUN07ZnaZmW0zs3IzW9LF+3eY2RYz22RmfzGz8e3eazGzjYGflcH/CiIiMtB63NM3Mw/wMPAPQCWwzsxWOue2tGv2DlDsnDtqZl8D7gWuDbx3zDk3Y4DrFhGRPghmT38WUO6c2+GcawJWAFe1b+CcW+2cOxp4+SYwZmDLFBGRgRBM6I8GKtq9rgzM684twIvtXsebWamZvWlmn+1qATNbHGhTWlVVFURJIiLSF8GcyLUu5nU5CL+ZLQKKgZJ2s8c55/aY2UTgr2b2nnPuww4rc+5R4FGA4uJiDfAvIjJIgtnTrwTGtns9BtjTuZGZzQf+FbjSOXf8xHzn3J7AvzuAV4GZ/ahXRET6IZjQXwdMNrMJZhYLLAA6XIVjZjOBR/AH/oF28zPMLC4wnQ1cALQ/ASwiIqdRj6HvnPMB3wRWAVuBp51zm81sqZldGWh2H5AM/FenSzPPBkrN7F1gNXBPp6t+Bkzt0SYefKWM9z+uG4zVi4iEhaBuznLOvQC80GneD9pNz+9muTeAqf0pMFhmxvK/ltHU0kLh6LTT8ZEiMgAOHjzIJZdcAsC+ffvweDx4vf4bS99++21iY2N7XMfNN9/MkiVLyM/P77bNww8/THp6Otdff32/a54zZw6/+MUvmDFj+F2NPuTuyO2rtIQYzhmXwZrtVfzzJ88KdTkiEqSsrCw2btwIwA9/+EOSk5P57ne/26GNcw7nHFFRXXdOPP744z1+zje+8Y3+FxsGwmrAtZJ8L+9/fJgD9Y2hLkVE+qm8vJzCwkJuvfVWioqK2Lt3L4sXL6a4uJgpU6awdOnStrZz5sxh48aN+Hw+0tPTWbJkCdOnT+f888/nwAH/aca77rqLZcuWtbVfsmQJs2bNIj8/nzfeeAOAI0eO8PnPf57p06ezcOFCiouL2zZI3XnyySeZOnUqhYWF3HnnnQD4fD6+9KUvtc1fvnw5AD//+c8pKChg+vTpLFq0aMC/s2CEzZ4+QEmel/tWbeO17dV8/hzdHybSWz/6f5vZsufwgK6zYFQqd18xpU/Lbtmyhccff5xf/epXANxzzz1kZmbi8/m46KKLuOaaaygoKOiwTF1dHSUlJdxzzz3ccccdPPbYYyxZctLoMTjnePvtt1m5ciVLly7lpZde4qGHHmLEiBE8++yzvPvuuxQVFZ2yvsrKSu666y5KS0tJS0tj/vz5PP/883i9Xqqrq3nvvfcAqK2tBeDee+9l165dxMbGts073cJqT79gZCrZybGs2a4bvETCwZlnnsm5557b9voPf/gDRUVFFBUVsXXrVrZsOfm6kISEBC6//HIAzjnnHHbu3Nnluq+++uqT2rz++ussWLAAgOnTpzNlyqk3Vm+99RYXX3wx2dnZxMTEcN1117F27VomTZrEtm3buO2221i1ahVpaf7zjFOmTGHRokU89dRTxMTE9Oq7GChhtacfFWXMzfOy+oMDtLQ6PFFd3VcmIt3p6x75YElKSmqbLisr48EHH+Ttt98mPT2dRYsW0dh4cldu+xO/Ho8Hn8/X5brj4uJOauNc7+4N7a59VlYWmzZt4sUXX2T58uU8++yzPProo6xatYo1a9bwpz/9iZ/85Ce8//77eDyeXn1mf4XVnj74u3hqjjazqTI0h04iMjgOHz5MSkoKqamp7N27l1WrVg34Z8yZM4enn34agPfee6/LI4n2Zs+ezerVqzl48CA+n48VK1ZQUlJCVVUVzjm+8IUv8KMf/YgNGzbQ0tJCZWUlF198Mffddx9VVVUcPXr0lOsfDGG1pw9w4WQvZrBmexUzx2WEuhwRGSBFRUUUFBRQWFjIxIkTueCCCwb8M771rW9xww03MG3aNIqKiigsLGzrmunKmDFjWLp0KfPmzcM5xxVXXMGnP/1pNmzYwC233IJzDjPjZz/7GT6fj+uuu476+npaW1v53ve+R0pKyoD/Dj2x3h7ODLbi4mLX34eoXPXw/xJl8NzXB/6PQkTCl8/nw+fzER8fT1lZGZdeeillZWVERw/9/WMzW++cK+6p3dD/TfpgXp6Xh/5aRs2RJjKSer6xQ0QEoKGhgUsuuQSfz4dzjkceeWRYBH5vhNdvE1CS7+XBv5TxWnk1V04fFepyRGSYSE9PZ/369aEuY1CF3YlcgOlj0klLiGHNNl26KSLSXliGvifKuHByNmu2V9HaOrTOWYiIhFJYhj7AvPwcqhuOs3XfwN5dKCIynIVt6M+dnA3Aq+riERFpE7ahn5MaT8HIVA3JIDLEzZs376QbrZYtW8bXv/71Uy6XnJwMwJ49e7jmmmu6XXdPl4AvW7asw01Sn/rUpwZkXJwf/vCH3H///f1ez0AL29AH/1U8G3bVcLixOdSliEg3Fi5cyIoVKzrMW7FiBQsXLgxq+VGjRvHMM8/0+fM7h/4LL7xAenp6n9c31IV16M/L8+JrdbxRfjDUpYhIN6655hqef/55jh/3P1p7586d7Nmzhzlz5rRdN19UVMTUqVP505/+dNLyO3fupLCwEIBjx46xYMECpk2bxrXXXsuxY8fa2n3ta19rG5b57rvvBmD58uXs2bOHiy66iIsuugiA3NxcqqurAXjggQcoLCyksLCwbVjmnTt3cvbZZ/PVr36VKVOmcOmll3b4nK5s3LiR2bNnM23aND73uc9RU1PT9vkFBQVMmzatbaC3NWvWMGPGDGbMmMHMmTOpr6/v83fblbC8Tv+EovEZJMdFs2b7AS4rHBHqckSGvheXwL73BnadI6bC5fd0+3ZWVhazZs3ipZde4qqrrmLFihVce+21mBnx8fE899xzpKamUl1dzezZs7nyyisx63owxV/+8pckJiayadMmNm3a1GFo5J/+9KdkZmbS0tLCJZdcwqZNm/j2t7/NAw88wOrVq8nOzu6wrvXr1/P444/z1ltv4ZzjvPPOo6SkhIyMDMrKyvjDH/7Ar3/9a774xS/y7LPPnnJ8/BtuuIGHHnqIkpISfvCDH/CjH/2IZcuWcc899/DRRx8RFxfX1qV0//338/DDD3PBBRfQ0NBAfHx8b77tHoX1nn6MJ4oLJmWxZltVr0fPE5HTp30XT/uuHeccd955J9OmTWP+/Pl8/PHH7N+/v9v1rF27ti18p02bxrRp09ree/rppykqKmLmzJls3ry5x8HUXn/9dT73uc+RlJREcnIyV199Na+99hoAEyZMaHtU4qmGbwb/+P61tbWUlJQAcOONN7J27dq2Gq+//nqefPLJtjt/L7jgAu644w6WL19ObW3tgN8RHNZ7+gAleTms2ryf8gMNTD7j9A9uJDKsnGKPfDB99rOf5Y477mDDhg0cO3asbQ/9qaeeoqqqivXr1xMTE0Nubm6Xwym319VRwEcffcT999/PunXryMjI4KabbupxPafaUTwxLDP4h2buqXunO3/+859Zu3YtK1eu5Mc//jGbN29myZIlfPrTn+aFF15g9uzZvPLKK5x11sA9Ajas9/TBfzIX0FU8IkNYcnIy8+bN48tf/nKHE7h1dXXk5OQQExPD6tWr2bVr1ynXM3fuXJ566ikA3n//fTZt2gT4h2VOSkoiLS2N/fv38+KLL7Ytk5KS0mW/+dy5c/njH//I0aNHOXLkCM899xwXXnhhr3+3tLQ0MjIy2o4SnnjiCUpKSmhtbaWiooKLLrqIe++9l9raWhoaGvjwww+ZOnUq3/ve9yguLuaDDz7o9WeeStjv6Y9OT2ByTjKvbqviKxdODHU5ItKNhQsXcvXVV3e4kuf666/niiuuoLi4mBkzZvS4x/u1r32Nm2++mWnTpjFjxgxmzZoF+J+CNXPmTKZMmXLSsMyLFy/m8ssvZ+TIkaxevbptflFRETfddFPbOr7yla8wc+bMU3bldOd3v/sdt956K0ePHmXixIk8/vjjtLS0sGjRIurq6nDOcfvtt5Oens73v/99Vq9ejcfjoaCgoO0pYAMlLIdW7uwnz2/hP/62i413/wOJsWG/nRORCBTs0Mph370D/i6eppZW3tyhSzdFJLJFROifm5tJQoxHo26KSMSLiNCPj/Fw/plZvKqTuSIS4SIi9MH/wPRdB4+ys/pIqEsREQmZiAp90KWbIhLZIib0c7OTyM1KVOiLSESLmNAH/97+Gx9W09jcEupSRERCIrJCP99LY3Mr63YeCnUpIiIhEVTom9llZrbNzMrNbEkX799hZlvMbJOZ/cXMxrd770YzKwv83DiQxffW7IlZxHqidOmmiESsHkPfzDzAw8DlQAGw0MwKOjV7Byh2zk0DngHuDSybCdwNnAfMAu42s4yBK793EmOjOW9ipvr1RSRiBbOnPwsod87tcM41ASuAq9o3cM6tds6dePTMm8CYwPQngZedc4ecczXAy8BlA1N635TkeSk70MDHtX0bFU9EZDgLJvRHAxXtXlcG5nXnFuDEEHa9XXbQtV26qS4eEYlAwYR+V4+o6XKUNjNbBBQD9/VmWTNbbGalZlZaVTW4YTwpJ5lRafGs2X5gUD9HRGQoCib0K4Gx7V6PAfZ0bmRm84F/Ba50zh3vzbLOuUedc8XOuWKv1xts7X1iZpTk5/C/5Qdpbmkd1M8SERlqggn9dcBkM5tgZrHAAmBl+wZmNhN4BH/gt9+FXgVcamYZgRO4lwbmhVRJnpeG4z7W76oJdSkiIqdVj6HvnPMB38Qf1luBp51zm81sqZldGWh2H5AM/JeZbTSzlYFlDwE/xr/hWAcsDcwLqU9MyiI6ynQVj4hEnIh4iEpXvvjI32ho9PHCbb1//JmIyFCjh6j0YF6+ly17D3Pg8KkfjiwiEk4iNvQ16qaIRKKIDf2Ckal4U+IU+iISUSI29M2MuZO9vFZWTUvr0DqvISIyWCI29MHfr193rJl3K2tDXYqIyGkR0aE/Z1I2UQavakgGEYkQER36GUmxTB+brn59EYkYER364L+KZ1NlLYeONIW6FBGRQRfxoT8vPwfn4LUy7e2LSPiL+NCfOjqNjMQYDbUsIhEh4kPfE2VcONnL2rIqWnXppoiEuYgPffD361c3NLFl7+FQlyIiMqgU+sBcDckgIhFCoQ94U+IoHJ3Kq9v0NC0RCW8K/YCSPC8bdtdSd6w51KWIiAwahX5ASV4OLa2ON8qrQ12KiMigUegHFI1LJyU+Wv36IhLWFPoB0Z4o5kzK5tVtVQy1p4mJiAwUhX47JXle9h1uZPv+hlCXIiIyKBT67ZTkn7h0U1fxiEh4Uui3MzItgfwzUtSvLyJhS6HfSUm+l3Uf1XDkuC/UpYiIDDiFficleV6aWlr524cHQ12KiMiAU+h3UpybQWKsR108IhKWFPqdxEV7+MSZWby6/YAu3RSRsKPQ70JJnpeKQ8f4qPpIqEsRERlQCv0ulOTlABp1U0TCj0K/C+OyEpmYnaTQF5Gwo9Dvxtw8L2/uOEhjc0uoSxERGTAK/W6U5HtpbG7lrY8OhboUEZEBo9DvxuwJWcRGR+mB6SISVhT63UiI9TB7YpbG4RGRsBJU6JvZZWa2zczKzWxJF+/PNbMNZuYzs2s6vddiZhsDPysHqvDToSTPy4dVR6g4dDTUpYiIDIgeQ9/MPMDDwOVAAbDQzAo6NdsN3AT8votVHHPOzQj8XNnPek+rEj0wXUTCTDB7+rOAcufcDudcE7ACuKp9A+fcTufcJqB1EGoMmTO9SYxOT1Doi0jYCCb0RwMV7V5XBuYFK97MSs3sTTP7bK+qCzEzY16+lzfKq2nyhdX2TEQiVDChb13M682gNOOcc8XAdcAyMzvzpA8wWxzYMJRWVQ2tveqSPC9HmlpYv6sm1KWIiPRbMKFfCYxt93oMsCfYD3DO7Qn8uwN4FZjZRZtHnXPFzrlir9cb7KpPi09MyiY6ynhVV/GISBgIJvTXAZPNbIKZxQILgKCuwjGzDDOLC0xnAxcAW/pabCgkx0VTnJuh6/VFJCz0GPrOOR/wTWAVsBV42jm32cyWmtmVAGZ2rplVAl8AHjGzzYHFzwZKzexdYDVwj3NuWIU+wLz8HD7YV8/+w42hLkVEpF+ig2nknHsBeKHTvB+0m16Hv9un83JvAFP7WWPIleR5uefFD1izvYovFo/teQERkSFKd+QG4awRKeSkxKmLR0SGPYV+EMyMkjwvr5VV4WvRpZsiMnwp9IM0Lz+Hw40+3q2sDXUpIiJ9ptAP0pxJ2UQZ6uIRkWFNoR+ktMQYZo7L4FUNySAiw5hCvxdK8rxsqqyjuuF4qEsREekThX4vzMv33y38ell1iCsREekbhX4vFI5KIzMpVqNuisiwpdDvhagoY+7kbNZur6K1tTdjzomIDA0K/V4qyfdy8EgT7++pC3UpIiK9ptDvpbmTvZgu3RSRYUqh30tZyXFMHZ2mfn0RGZYU+n1Qkudlw+4a6o42h7oUEZFeUej3QUmel1YHr5fr0k0RGV4U+n0wY2w6qfHRrNHTtERkmFHo90G0J4oLJ3tZs70K53TppogMHwr9PirJ87L/8HE+2Fcf6lJERIKm0O+juXn+IRl0FY+IDCcK/T4akRbPWSNSdL2+iAwrCv1+KMn3UrrrEA3HfaEuRUQkKAr9fijJ89Lc4nhDl26KyDCh0O+H4vGZJMV61K8vIsOGQr8fYqOj+MSkbF26KSLDhkK/n0ryvFTWHGNH9ZFQlyIi0iOFfj+VBC7dfFVX8YjIMKDQ76exmYlM9CapX19EhgWF/gCYl5fDWzsO0tjcEupSREROSaE/AEryvRz3tfLmjoOhLkVE5JQU+gPgvAmZxEVHqV9fRIY8hf4AiI/xMHtiFmvVry8iQ5xCf4DMy/eyo/oIuw8eDXUpIiLdCir0zewyM9tmZuVmtqSL9+ea2QYz85nZNZ3eu9HMygI/Nw5U4UPNiUs315Rpb19Ehq4eQ9/MPMDDwOVAAbDQzAo6NdsN3AT8vtOymcDdwHnALOBuM8vof9lDz4TsJMZmJrBmm56mJSJDVzB7+rOAcufcDudcE7ACuKp9A+fcTufcJqC107KfBF52zh1yztUALwOXDUDdQ46ZUZLn5Y0PD3Lcp0s3RWRoCib0RwMV7V5XBuYFoz/LDjvz8nI42tTC+p01oS5FRKRLwYS+dTEv2NHFglrWzBabWamZlVZVDd8+8fPPzCLGY7o7V0SGrGBCvxIY2+71GGBPkOsPalnn3KPOuWLnXLHX6w1y1UNPUlw05+Zm6np9ERmyggn9dcBkM5tgZrHAAmBlkOtfBVxqZhmBE7iXBuaFrZI8L9v217Nu56FQlyIicpIeQ9855wO+iT+stwJPO+c2m9lSM7sSwMzONbNK4AvAI2a2ObDsIeDH+Dcc64ClgXlh64rpo8hJieOLj/yNJc9u4mDD8VCXJCLSxobawz+Ki4tdaWlpqMvol/rGZh58pYzH39hJclw03700j+vOG48nqqtTHCIi/Wdm651zxT210x25gyAlPoa7PlPAi7ddSMHIVL7/p81c+YvXWb9LV/WISGgp9AdR3hkp/P6r5/GL62ZysKGJz//yDb7z9LtU1avLR0RCQ6E/yMyMz0wbxV++U8KtJWey8t2Pufj+V3ns9Y/wtXS+l01EZHAp9E+TpLhollx+Fi/901xmjEtn6fNb+MxDr/OWxuAXkdNIoX+anelN5j++PItfLSqivtHHtY++yT+teIf9hxtDXZqIRACFfgiYGZcVjuSVO0r41sWTeOG9fVx8/6v8eu0OmtXlIyKDSKEfQgmxHr5zaT7/c/tczpuYxU9f2MrlD77GG+XVoS5NRMKUQn8IyM1O4rGbzuU3NxRz3NfCdb95i2/8fgN7646FujQRCTMK/SFkfsEZvHx7CbfPz+OVLfu5+P41/Nur5TT51OUjIgNDoT/ExMd4uG3+ZF65o4QLJ2dz70vbuGzZWj1/V0QGhEJ/iBqbmcijNxTz25vPpdU5bnjsbf7xiVIqa/QMXhHpO4X+EDcvP4dVt8/lnz+Zz9rt1cx/YA0P/aWMxmY9nUtEek+hPwzERXv4xkWTeOU7JVx8Vg7/9+XtfHLZWv76wf5QlyYiw4xCfxgZnZ7Av11/Dk/ech7RUcaXf1vKV363jt0H1eUjIsFR6A9DcyZn8+Jtc7nzU2fxtw8PMv/na3jg5e3q8hGRHin0h6nY6CgWzz2Tv3xnHpdNGcHyv5Qx/4E1rNq8j6H2jAQRGToU+sPciLR4li+cyYrFs0mKjeYfn1jPTY+v46PqI6EuTUSGID05K4w0t7TyxN928fOXt3Pc18qXzh/PxWflMGNsOklx0aEuT0QGUbBPzlLoh6ED9Y3c8+IHPPfOxzgHnijj7JEpnDMug3NyMyken8Go9IRQlykiA0ihLxxubOad3bWs33mI0l01bKyo5WiT/2TvqLR4isZnUDw+g+LcTM4akUK0R719IsNVsKGvY/4wlhofQ0mel5I8LwC+llY+2FdPaWAjsH5XDc9v2gtAYqyHmePS244GZo5LJzU+JpTli8gg0J5+hPu49hilOw+xYVcNpbtq2Lr3MK0OzCD/jBTOGZ9BcW4GxeMzGZORgJmFumQR6YK6d6RPGo772Li7ltJdh1i/q4Z3dtfScNwHQE5KHOeMzwhsCDKZMiqVGHUJiQwJ6t6RPkmOi2bO5GzmTM4GoKXVsW1fPet3+buESnfW8OL7+wCIj4li2pj0wHmBDIrGZZCeGBvK8kWkB9rTl17bV9fI+l01bUcDm/ccpqXV/3c0OSe5w9FAblaiuoRETgN178hpc7TJx7sVdW1HA+t31VDf6O8SykqKZeqYNMZnJjI2M5FxmYmMy0pkbEai7h0QGUDq3pHTJjE2mvPPzOL8M7MAaG11lB1oaDsa2Lq3ntKdNW3nBk7ITo71bwQCP+03CmekxBMVpSMEkYGmPX05LZxz1B5tZveho3//Ofj36b11x2ht96cYGx3FmIwExnexQdBRgsjJtKcvQ4qZkZEUS0ZSLNPHpp/0fpOvlT21x9o2AhWBf3cdPErpzhrquzhKGJuZ2OVGQUcJIt1T6MuQEBsdRW52ErnZSSe919VRwomNQumuGla+u6fjUYInijGZCR26jsZlJjIhsH5dZiqRTKEvQ15PRwnNLa18XHPyUcLuQ0dZ3+koITrKmJCdRN4ZKUzKSSbvjBQmn5FMblYSsdHaGEj4U+jLsBfjOfVRQt2xZnYdPMqO6ga272+gbH8D7++p44X393LilFZ0lJGbnUTeGclMykkh74xkJuekMCFbGwMJL0GFvpldBjwIeIDfOOfu6fR+HPAfwDnAQeBa59xOM8sFtgLbAk3fdM7dOjCli/TMzEhPjCU98eSjhMbmFsoPNFB+oIHt++spO9DAlj2Heen9fW3dRZ4oIzcr0X9EkJPM5MCRwYTsJOKiPSH4jUT6p8fQNzMP8DDwD0AlsM7MVjrntrRrdgtQ45ybZGYLgJ8B1wbe+9A5N2OA6xbpt/gYD4Wj0ygcndZhfmNzCzuqjlB2oJ6y/f4NwrZ99aza3HFjMD4rkbwc/0ZgcmCjMNGrjYEMbcHs6c8Cyp1zOwDMbAVwFdA+9K8CfhiYfgb4hek2TBmm4mM8FIxKpWBUaof5jc0tfFR9hO3769uODrYfqOflrfvb7kiOMsjNSvJvCE5sEHJSmOhNIj5GGwMJvWBCfzRQ0e51JXBed22ccz4zqwOyAu9NMLN3gMPAXc651/pXskhoxMd4OHtkKmeP7LgxOO47sTFooDzQTbR9fz2vbD3QYWMwPiuJSTnJFI5KY8a4dGaMSSctUcNXy+kVTOh3tcfe+Y6u7trsBcY55w6a2TnAH81sinPucIeFzRYDiwHGjRsXREkiQ0dctIezRqRy1oiOG4MmXysfVfu7ibbvb6D8gL+b6JWt+9tOIE/ITmLG2HRmjE1n+th0zh6Zou4hGVTBhH4lMLbd6zHAnm7aVJpZNJAGHHL+232PAzjn1pvZh0Ae0OGWW+fco8Cj4L8jtw+/h8iQExsdRf6IFPJHpHSYX9/YzHuVdWysrGXj7lr+t7ya59752L+MJ4qCUaltG4IZY9MZr0HrZAAFE/rrgMlmNgH4GFgAXNepzUrgRuBvwDXAX51zzsy8+MO/xcwmApOBHQNWvcgwlBIfwycmZfOJSf7hq51z7K1r5N2KWjZW1PJORS3/ua6C376xE4D0xBimjwlsBALdQhlJGsJa+qbH0A/00X8TWIX/ks3HnHObzWwpUOqcWwn8O/CEmZUDh/BvGADmAkvNzAe0ALc65w4Nxi8iMlyZGaPSExiVnsDlU0cC/kdblh1oYGOF/2hgY0Uty8vK2rqFxmcldugWKhiZqhPFEhQNuCYyTDQc9/m7hSpq2VhRw7sVdew73AhAjMcoGJnK9HbdQrlZSRqDKIJoPH2RCLCvrpGNFTVsrKhjY0UNmyrrONrUAkBqfDTTx6YzM9AtNH1MOlnJcSGuWAaLQl8kArW0OsoO1P/9/MDuWrbvr2+7qWxsZgIzxmYwbXQaOalxpMbHkJoQTUp8TNt0QoxHJ46HIYW+iABw5LiP9z8+0S3k/9lb19hte0+UkRofTWpCDCnx0f6NQXxgOiGm04aiU7uEGFLiotWtFAIaT19EAEiKi+a8iVmcNzGrbd6hI00cOtJEfWMzhxt9HD7WTH2jj8ONzV1O76hu4PAxH/WNzRxupHEgAAAHRklEQVQJdB+dSkpc541Bx43EiY1IemIMo9MTGZuZQFpCjI4wTgOFvkgEykyKJbOPl336Wlqpb/R12DAc7nKD4QtsVJrZU9tI/fH6tnmtXXQwpMRFMzbTvwEYm5HYYXpMRiIJsbo6aSAo9EWkV6I9UW3PN+gL5xxHmlo4fKyZQ0eaqKw5RmWN/zkIFTXH2FF1hDXbq2hsbu2wXHZyXLsNgv/fE09NG5kWT7QejhMUhb6InFZmRnJcNMlx0YxKTzhplFPwbxiqGo5TcajdBuHQMSpqjvJORQ1/fm9v27hG4D8PMTItvsMGof2RgjclTl1HAQp9ERlyzIyclHhyUuI5Z3zGSe/7WlrZW9cYODr4+wah4tBRVm+roqr+eIf2cdFRjMlI8G8I2o4QEhgT2DikJUTOwHcKfREZdqI9UYE9+cQu3z/W1MLHtScen3msw8Zh/a4a6ht9HdrHRUfhiTI8ZphBVNu0EWX+I4mowHsdpu3k+VGB5dumA208UYF1B6ajzL9x85gRFeWfnpidxHcuzR/c725Q1y4iEgIJsR4m5aQwKSely/frjja3HRnsPnSUQ0eaaGl1tDpoda7dD7S2dj3d4hzOubbl2k+3Ld/qn/a1ttLUEpjfrk1Lq8OdmHbu5PGLB4FCX0QiTlpiDGmJJz81LRLodLeISARR6IuIRBCFvohIBFHoi4hEEIW+iEgEUeiLiEQQhb6ISARR6IuIRBCFvohIBFHoi4hEEIW+iEgEUeiLiEQQhb6ISARR6IuIRBCFvohIBFHoi4hEEIW+iEgEUeiLiEQQhb6ISARR6IuIRJCgQt/MLjOzbWZWbmZLung/zsz+M/D+W2aW2+69fwnM32Zmnxy40kVEpLd6DH0z8wAPA5cDBcBCMyvo1OwWoMY5Nwn4OfCzwLIFwAJgCnAZ8G+B9YmISAgEs6c/Cyh3zu1wzjUBK4CrOrW5CvhdYPoZ4BIzs8D8Fc654865j4DywPpERCQEooNoMxqoaPe6EjivuzbOOZ+Z1QFZgflvdlp2dJ+rPZUj1XDfmYEXBmZBTgde9zR9YrFerbuzLuad1C6YNv1s14u3g1tHUCsZ4k7xO5zy9+vmvb4sc7oNyH+3IfK79JsbgFUMwDpGFMIXftv/9ZxCMKHf1X/Vzr9dd22CWRYzWwwsBhg3blwQJXUhJgFKvhf44gMf0dV023+Y7qbpYv4p1neq6fa6/INwvW/TzazgP7PHFQ3OOobyhuGUv+Mp3ut2ub4sc7oNkZDrNiZCYChsBDMnDkANpxZM6FcCY9u9HgPs6aZNpZlFA2nAoSCXxTn3KPAoQHFxcd/+kmKT4KI7+7SoiEikCKZPfx0w2cwmmFks/hOzKzu1WQncGJi+Bvirc84F5i8IXN0zAZgMvD0wpYuISG/1uKcf6KP/JrAK8ACPOec2m9lSoNQ5txL4d+AJMyvHv4e/ILDsZjN7GtgC+IBvOOdaBul3ERGRHpgbMn2MfsXFxa60tDTUZYiIDCtmtt45V9xTO92RKyISQRT6IiIRRKEvIhJBFPoiIhFEoS8iEkGG3NU7ZlYF7OrHKrKB6gEqZ7jTd9GRvo+O9H38XTh8F+Odc96eGg250O8vMysN5rKlSKDvoiN9Hx3p+/i7SPou1L0jIhJBFPoiIhEkHEP/0VAXMITou+hI30dH+j7+LmK+i7Dr0xcRke6F456+iIh0I2xCv6eHt0cSMxtrZqvNbKuZbTaz20JdU6iZmcfM3jGz50NdS6iZWbqZPWNmHwT+Rs4PdU2hZGa3B/4/ed/M/mBm8aGuaTCFRegH+fD2SOIDvuOcOxuYDXwjwr8PgNuAraEuYoh4EHjJOXcWMJ0I/l7MbDTwbaDYOVeIf/j4BaGtanCFRegT3MPbI4Zzbq9zbkNguh7//9SD82ziYcDMxgCfBn4T6lpCzcxSgbn4n4GBc67JOVcb2qpCLhpICDz1L5Eunu4XTsIl9Lt6eHvEhlx7ZpYLzATeCm0lIbUM+D9Aa6gLGQImAlXA44Hurt+YWVKoiwoV59zHwP3AbmAvUOec+5/QVjW4wiX0g3oAe6Qxs2TgWeCfnHOHQ11PKJjZZ4ADzrn1oa5liIgGioBfOudmAkeAiD0HZmYZ+HsFJgCjgCQzWxTaqgZXuIR+UA9gjyRmFoM/8J9yzv13qOsJoQuAK81sJ/5uv4vN7MnQlhRSlUClc+7Ekd8z+DcCkWo+8JFzrso51wz8N/CJENc0qMIl9IN5eHvEMDPD32e71Tn3QKjrCSXn3L8458Y453Lx/1381TkX1ntyp+Kc2wdUmFl+YNYl+J9hHal2A7PNLDHw/80lhPmJ7R4fjD4cdPfw9hCXFUoXAF8C3jOzjYF5dzrnXghhTTJ0fAt4KrCDtAO4OcT1hIxz7i0zewbYgP+qt3cI87tzdUeuiEgECZfuHRERCYJCX0Qkgij0RUQiiEJfRCSCKPRFRCKIQl9EJIIo9EVEIohCX0Qkgvx/UJIjj+I5yeEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(train_losses, label='Training loss')\n",
    "plt.plot(test_losses, label='Validation loss')\n",
    "plt.legend(frameon=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'Epocs')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl4VeW59/HvnZkQEiEJMxiQQRCZjIDiVLXqAStq69Bqax2rtU6n9Zy2x/a85/RtTx3aak/7alWqtirVqiAOxVLrUKuiwQRBBpmHAJIwh4SM9/vH3kDABHZgr6wk+/e5rn1l75W19r6zr2T/sp7nWc9j7o6IiMihJIVdgIiItA8KDBERiYkCQ0REYqLAEBGRmCgwREQkJgoMERGJiQJDRERiosAQEZGYKDBERCQmKWEXEE95eXleUFAQdhkiIu3G3Llzy909P5Z9O1RgFBQUUFRUFHYZIiLthpmtjnVfNUmJiEhMFBgiIhITBYaIiMREgSEiIjFRYIiISEwUGCIiEhMFhoiIxCTQ6zDM7DbgesCAR9z9fjMbBTwEZAGrgCvcfccBx/UD/gD0BBqAh939gSBrFZGDa2hwnpu7jq2VNeR0St17y95zPzOVrLQUkpIs7FIlIIEFhpmNIBIW44AaYJaZvQI8CnzP3d8ys2uAO4EfHXB4HfBdd//IzLoAc81strsvDKpeEWmeu/PfLy/k8XdXHXS/JGNfgBwYKAe5ZXdKpUu6wqatC/IMYxjwvrtXApjZW8BFwFDg7eg+s4HXOCAw3H0DsCF6f6eZLQL6AAoMkRD8cvanPP7uKq4/dQC3nz2E7VW1n7vtaGLb9qpaSrdV7f1ebb03+xpJBl0ymg6TA7d1zUxlSM8u5GWlt+K7IEEGxgLgp2aWC1QBk4Ci6PYLgBeBS4B+B3sSMysAxgBzmvn+DcANAP37949P5SKy18NvL+d//76My0/sxw8nDcPM6JyeQu+jOrXoedydqtr6fWFSeejA2bC9iu1VdeyoqqWmvuFzz9k7J4MRfXI4vk8OI/pGvipEgmPuzSf+ET+52bXAzUAFkbODKuB3wK+BXGAmcKu75zZzfBbwFvBTd3/hUK9XWFjomktKJH6enrOGH06fz/kje/HA5WNIDqnJyN3ZXduwN0jKK6pZuH4H80u3s6B0OyvKd+3dt1ejEDm+Tw4j+uSQ30Uh0hwzm+vuhTHtG2Rg7PdCZj8D1rn7/2u0bQjwpLuPa2L/VOBl4DV3/2Usr6HAEImfF0tKuf2ZEs4Yks/vvl5IWkrbHVS5c3ctn6zfwYLS7cyP3laW72LPx1vP7EYh0jebEX1y6N4lI9yi24iWBEbQo6S6u/smM+sPXAyc1GhbEnAXkRFTBx5nwFRgUaxhISLx8/qiz/jus/MYV9CNB688oU2HBUT6PiYMzGXCwH2NFTt31+53FjK/dDuvL/5sb4j0yE7feway52yke7ZC5GCCnt78+WgfRi1ws7tvNbPbzOzm6PdfAB4DMLPewKPuPgmYCHwdmG9mJdF9f+jurwZcr0jCe3d5OTc99RHH9c7m0asKyUhNDrukw9IlI5XxA3MZ3yhEKqrr9guRj9dt4/XFm/aGSPcuB4RI3xx6KET2arUmqdagJimRI1OydhtXPPI+fbp24pkbTqJr57SwSwrcgSEyv3Q7y8sq9oZI/oEh0ieHHtnpRBpC2r820yQlIu3Hko07uer3H5Cblc6T145PiLAAyEpPYdyAbowb0G3vtl3VdSzcsIP56/aFyJtLNtEQDZG8rHSO75PNsb2y6ZaZ9vnhv5mRr53TkjtMsIACQ0SAVeW7uHLqHDqlJvPUdeMTvi2/c3oKJxZ048SCfSFSWbPvTGTP2cjbS8upb2i+lSYlyfYGyf7Xk6Qc8nqTrPSUNhc2CgyRBLdhexVXPDqHuvoGpt14Ev26ZYZdUpuUmZZCYUE3ChuFSEODU1FTt/eakuYuXtx37UkNazbviuy7u+6gYZOcZGRnpBz0AsY9t26d0/brqwmKAkMkgW2uqObKR+ewo6qWp6+fwKDuXcIuqV1JSjKyM1LJzkg9+BXITXB3KqrrYrhaft8+67ZW7b3fOGzystIpuuvs+P5wTVBgiCSo7VW1fOP3H1C6rYo/XDOe4/vmhF1SQjEzumSk0iUjlb5dW3asu7Orpn7vFfPVdfXBFHkABYZIAqqsqePaxz/k08928sg3Cvfr8JW2z8zISk8hKz2FPi2couVItO2rcUQk7qrr6vnWH+fy0ZqtPHD5GM4Y2j3skqSd0BmGSAKpq2/gtmkl/GNpOfd8ZSSTju8VdknSjugMQyRBNDQ4339hPrM+2ciPzx/OpYUt7aaVRKfAEEkAexZAem7uOu44ewjXnDIg7JKkHVJgiCSAX0UXQLr2lAHcetagsMuRdkqBIdLBPfL2Cn7992VcVtiPuyYPa3NXD0v7ocAQ6cCmfbCGn766iMkje/Gzi49XWMgRUWCIdFAz563nh9Pnc8bQfH516ejQVsuTjkOBIdIB/X3xZ/zrMyWcWNCNB69o+wsgSfug3yKRDua95Zu56cmPGN47m6lXFdIprX0ugCRtjwJDpAOZt3Yb1z3xIf27ZfLE1ePokpEadknSgSgwRDqIJRt3ctVjH9AtK40nr0ucBZCk9SgwRDqA1ZsjCyClpyTx1LUTtA61BEKBIdLONV4A6clrx9M/VwsgSTA0+aBIO7ZnAaRtlbVMu34Cg3toASQJjs4wRNqpHbsjCyCt21rF1KsKtQCSBE6BIdIOVdXU710A6aGvn9Aq6zmLKDBE2pnqunq+9eRc5q7eyv2XjeELWgBJWon6METakbr6Bm7/Uwlvf1rGPV8eyeSRWgBJWo/OMETaiT0LIP1lwUbumjyMS0/UAkjSuhQYIu2Au/OTVyILIN121mCuO3Vg2CVJAlKTlEgbVlvfwMfrtvHc3FKmfbCGayYO4PazB4ddliQoBYZIG+LuLNtUwTvLyvnnsnLeX7GFiuo6zODrE47mR+drASQJjwJDJGQbt+/mn9GAeGdZOZt2VgNwdG4mF4zuzSmD8jhpYK7mhpLQKTBEWtmO3bXMWbFlb0As21QBQLfOaZx8TC6nDMpj4qA8+nXTFB/StigwRAJWU9dA8ZqtewNi3rrt1Dc4GalJjBuQy6WFfZk4KI9hPbNJ0qp40oYpMETizN1ZvHHn3oCYs2ILVbX1JBmM7HsUN51+DBMH5TH26KNIT9HiRtJ+KDBE4qB0WxX/XBoJiHeXl1NeUQPAwPzOXBI9g5gwMJecTlrQSNovBYbIYdheWct7K8qjo5k2s7J8FwB5Wel7+yAmDsqj91GdQq5UJH4UGCIx2F1bz0ert+4d7jq/dDsNDplpyUwYmMuVE47mlEF5DOmRpWGv0mEFGhhmdhtwPWDAI+5+v5mNAh4CsoBVwBXuvqOJY88DHgCSgUfd/edB1ipty5KNO5n2wRrqGzzUOhxn9eZKPli5heq6BpKTjDH9juKWMwdzyuA8RvU9irQUTZggiSGwwDCzEUTCYhxQA8wys1eAR4HvuftbZnYNcCfwowOOTQZ+C3wRWAd8aGYz3X1hUPVK27F+WxVXTp3DjqpaOqeHfxKcl5XG18b355RBeYwb0I0uGeqHkMQU5F/jMOB9d68EMLO3gIuAocDb0X1mA69xQGAQCZll7r4ieuyfgCmAAqODq6yp47oniqiqqeelW05hiFaQE2kzgjyXXgCcZma5ZpYJTAL6RbdfEN3nkui2A/UB1jZ6vC667XPM7AYzKzKzorKysrgVL62vocG545kSFm/cwf9+bYzCQqSNCSww3H0RcDeRs4hZwDygDrgGuNnM5gJdiDRXHaipXsMmG7Pd/WF3L3T3wvz8/LjULuH4xewlvPbJZ/zH5OFaFEikDQq0t87dp7r7WHc/DdgCLHX3xe5+jrufAEwDljdx6Dr2P/PoC6wPslYJ1/Tidfz2jeV8dVw/rplYEHY5ItKEQAPDzLpHv/YHLgamNdqWBNxFZMTUgT4EBpvZADNLAy4HZgZZq4Rn7uqt/Ptz85kwsBv/PWWEhqWKtFFBjwd83swWAi8BN7v7VuCrZvYpsJjIWcNjAGbW28xeBXD3OuA7RDrEFwHPuvsnAdcqIVi3tZJv/bGI3kdl8OAVJ5CarCGqIm2VuYc7zj2eCgsLvaioKOwyJEYV1XV85cF3Kd1WxfRvT2RQ96ywSxJJOGY2190LY9k3/EHukpDqG5zb/1TC0k0VPH71iQoLkXZA5/8SintmLeZviz7jP780nFMHa3SbSHugwJBW92zRWn739gq+PuFovnFSQdjliEiMFBjSqj5YuYX/mD6fUwbl8eMvDQ+7HBFpAQWGtJo1myMjovp1zeS3XxurEVEi7Yz+YqVV7Nxdy7VPfEiDw9RvnkhOpibwE2lvFBgSuPoG55Zpxaws38WDV45lQF7nsEsSkcOgYbUSuJ+9uog3l5Tx04tGcPIxeWGXIyKHSWcYEqhpH6xh6jsruXpiAVeMPzrsckTkCCgwJDDvLd/Mj2Ys4PQh+fzHpGFhlyMiR0iBIYFYVb6Lm56aS0FeZ/73a2NI0YgokXZPf8USd9urarnmiQ8xYOpVhWRrSVORDkGd3hJXdfUNfOfpj1i7pZInrx3P0bkaESXSUSgwJK5+8vJC/rG0nHu+PJLxA3PDLkdE4khNUhI3f3xvFU+8t5obThvIpSc2tVS7iLRnCgyJi3eWlvN/XlrIWcd259/POzbsckQkAAoMOWLLyyr49lNzGZSfxQNfHUNykpZYFemIFBhyRLZV1nDdE0WkJifx6FWFZKWrW0yko9Jftxy22voGvv3UR5RureLp68fTr1tm2CWJSIAUGHJY3J3/nPkJ7y7fzC8uGUVhQbewSxKRgKlJSg7L4++u4uk5a7jpjGP48gl9wy5HRFqBAkNa7M0lm/jJyws5Z3gP7jxnaNjliEgrUWBIiyz9bCe3PF3MsT2z+dVlo0nSiCiRhKHAkJht2VXDtU8UkZ6azKNXFdJZI6JEEor+4iUmNXUN3PjkXDbu2M0zN0yg91Gdwi5JRFrZIc8wzOw7Zta1NYqRtsnduWvGfD5YuYV7vzKSMf316yCSiGJpkuoJfGhmz5rZeWamRusEM/WdlTxbtI5bzxzElNF9wi5HREJyyMBw97uAwcBU4JvAUjP7mZkdE3Bt0ga8vugzfvrqIiYd35Pbzx4SdjkiEqKYOr3d3YGN0Vsd0BV4zszuCbA2CdmSjTu5dVoxI3rn8ItLNCJKJNEdstPbzG4FrgLKgUeBO9291sySgKXAvwVbooShvKKaax7/kKyMFB75RiGd0pLDLklEQhbLKKk84GJ3X914o7s3mNn5wZQlYaquq+dbf5zL5l3VPPutk+iZkxF2SSLSBsTSJPUqsGXPAzPrYmbjAdx9UVCFSTjcnR+8MJ+5q7fyi0tGM7LvUWGXJCJtRCyB8SBQ0ejxrug26WCKVm3hst+9zwsflfKvXxzC5JG9wi5JRNqQWJqkLNrpDextitIFfx3I4o07uO+1Jfxt0SbystL5yYUjuHJ8/7DLEpE2JpYP/hXRju89ZxXfBlYEV5K0lrVbKvnV3z5lenEpWWkp3HnuUK6eWEBmmv4fEJHPi+WT4Ubg18BdgAOvAzcEWZQEq2xnNb99YxlPzVlNkhk3nDqQG08/hq6d08IuTUTasEMGhrtvAi4/nCc3s9uA6wEDHnH3+81sNPAQkEHkmo5vu/sHTRx7DzCZSD/LbOC2xk1j0nI7d9fyyNsrePSdlVTXNXBpYV9uPWswvXI0L5SIHFos12FkANcCxxH5kAfA3a85xHEjiITFOKAGmGVmrwD3AP/l7n8xs0nRx2cccOzJwERgZHTTO8DpwJux/FCyv9219Tz5/mp++8YytlbWMnlkL777xSEMzM8KuzQRaUdiaZL6I7AYOBf4b+AKIJbhtMOA9929EsDM3gIuItKslR3dJwdY38SxTiSc0oicnaQCn8XwmtJIXX0DLxSXcv/sT1m/fTenDs7jznOHaqisiByWWAJjkLtfYmZT3P0JM3saeC2G4xYAPzWzXKAKmAQUAbcDr5nZfUSam04+8EB3f8/M3gA2EAmM3zR3zYeZ3UC0T6V/f43sgci1FK998hn3/XUJyzZVMKpvDvdeMoqJg/LCLk1E2rFYAqM2+nVbtJlpI1BwqIPcfZGZ3U2k/6ECmEekz+Im4A53f97MLiUyqeHZjY81s0FEzlD2LBY928xOc/e3m3idh4GHAQoLCxO+j+Pd5eXcPWsJ89Zu45j8zjx05VjOPa4nmmRYRI5ULIHxcHQ9jLuAmUAW8KNYntzdpxIJBMzsZ8A64H+A26K7/JnI/FQHuohIc1ZF9Ni/ABOAzwWGRMxft517XlvMP5aW0ysng3u+PJKLx/YhJVmLKopIfBw0MKITDO5w961EPqwHtuTJzay7u28ys/7AxcBJwC3s68A+k8gEhgdaA1xvZv9DpEnqdOD+lrx2olhRVsEvZn/KKx9voGtmKndNHsaVE44mI1WTBYpIfB00MKJXdX8HePYwn//5aB9GLXCzu281s+uBB6JXi+8m2v9gZoXAje5+HfAckTCZT6QDfJa7v3SYNXRIG7fv5oHXl/Js0VrSU5K49cxBXHfaQLIzUsMuTUQ6KDvUpQ1m9iMindbPEJlHCgB339LsQSEpLCz0oqKisMsI1LbKGh58azmP/3MVDe5cMf5obv7CIPK7pIddmoi0Q2Y2190LY9k3lj6MPddb3Nxom9PC5ik5MpU1dTz2z1U89NZyKqrruGh0H+744hD6dcsMuzQRSRCxXOk9oDUKkabV1jfwpw/X8uvXl1K2s5qzh3Xne+cO5die2Yc+WEQkjmK50vsbTW139z/EvxzZo6HBeenj9fxy9qes3lzJiQVdefCKsRQWdAu7NBFJULE0SZ3Y6H4GcBbwEaDACIC78+anZdwzawmLNuzg2J5deOybJ3LG0HxdSyEioYqlSeqWxo/NLIfIdCESZx+t2crP/7KYD1ZuoX+3TB64fDRfGtmbpCQFhYiE73AWPqgEBse7kES3bFMFlzz0Hl0z0/jJlOO47MT+pKXoojsRaTti6cN4icioKIjM/TScw78uQ5oxvXgdAK/eegrdszMOsbeISOuL5Qzjvkb364DV7r4uoHoSUkODM6N4PacMylNYiEibFUtgrAE2uPtuADPrZGYF7r4q0MoSyNw1WyndVsWd5w4NuxQRkWbF0kj+Z6Ch0eP66DaJk+nFpWSmJXPOcT3CLkVEpFmxBEaKu9fseRC9r8Wf46S6rp5XPt7AOcN7kJl2OGMQRERaRyyBUWZmF+x5YGZTgPLgSkosby4pY3tVLReO6RN2KSIiBxXLv7Q3Ak+Z2W+ij9cBTV79LS33YkkpeVlpnKLV8ESkjYvlwr3lwAQzyyIyu+3O4MtKDNuravnbok1cMb6/FjoSkTbvkJ9SZvYzMzvK3SvcfaeZdTWz/9saxXV0sxZsoKaugQtHqzlKRNq+WP6t/Rd337bnQXT1vUnBlZQ4pheXMjCvMyP75oRdiojIIcUSGMlmtnd1HjPrBGi1niO0flsVc1Zu4cIxfTSpoIi0C7F0ej8JvG5mj0UfXw08EVxJiWHmvPW4o+YoEWk3Yun0vsfMPgbOBgyYBRwddGEd3YziUsb2P4r+uVoxT0Tah1iH5mwkcrX3l4msh7EosIoSwKINO1i8cScX6doLEWlHmj3DMLMhwOXAV4HNwDNEhtV+oZVq67BmFJeSkmRMHtk77FJERGJ2sCapxcA/gC+5+zIAM7ujVarqwBoanBdL1nPG0Hy6ddYMKyLSfhysSerLRJqi3jCzR8zsLCJ9GHIE3l+5mY07dmsqEBFpd5oNDHef7u6XAccCbwJ3AD3M7EEzO6eV6utwZhSXkpWewtnDNDOtiLQvh+z0dvdd7v6Uu58P9AVKgO8HXlkHtLu2nr/M38h5I3qSkZocdjkiIi3SogmM3H2Lu//O3c8MqqCO7O+LN7Gzuk6jo0SkXdKMd61oenEpPbLTmTAwN+xSRERaTIHRSrbuquHNJZu4YFRvkpM0dkBE2h8FRit5Zf4Gautdo6NEpN1SYLSSF0tKGdIji+G9ssMuRUTksCgwWsHaLZV8uGqrZqYVkXZNgdEKXiwpBeCCUZoKRETaLwVGwNyd6cWljBvQjb5dNTOtiLRfCoyAfbJ+B8vLdunaCxFp9xQYAZteXEpachKTRvQKuxQRkSOiwAhQXX0DM+et5wvH5pOTmRp2OSIiRyTQwDCz28xsgZl9Yma3R7eNNrP3zazEzIrMbFwzx/Y3s7+a2SIzW2hmBUHWGoR3l2+mbGe1mqNEpEMILDDMbARwPTAOGAWcb2aDgXuA/3L30cCPo4+b8gfgXncfFn2OTUHVGpQZJaVkZ6RwxtDuYZciInLEDrmm9xEYBrzv7pUAZvYWcBHgwJ6r13KA9QceaGbDgRR3nw3g7hUB1hmIypo6XluwkQtG99bMtCLSIQTZJLUAOM3Mcs0sE5gE9ANuB+41s7XAfcAPmjh2CLDNzF4ws2Izu9fMmvzUNbMbok1bRWVlZQH9KC03e+Fn7KqpZ8poNUeJSMcQWGC4+yLgbmA2MAuYB9QBNwF3uHs/IosyTW3i8BTgVOB7wInAQOCbzbzOw+5e6O6F+fn58f4xDtuM4lJ652QwrqBb2KWIiMRFoJ3e7j7V3ce6+2nAFmApcBXwQnSXPxPpnzjQOqDY3Ve4ex0wAxgbZK3xVF5RzdtLy5kypg9JmplWRDqIoEdJdY9+7Q9cDEwj0mdxenSXM4mEyIE+BLqaWX6j/RYGWWs8vfLxBuobXKOjRKRDCbLTG+B5M8sFaoGb3X2rmV0PPGBmKcBu4AYAMysEbnT369y93sy+B7xukdn65gKPBFxr3EwvLmV4r2yG9OgSdikiInETaGC4+6lNbHsHOKGJ7UXAdY0ezwZGBllfEFaW76Jk7TZ+OOnYsEsREYkrXekdZzOKSzGDC0apOUpEOhYFRhy5Oy+WlHLyMbn0zMkIuxwRkbhSYMRRydptrNpcyYW69kJEOiAFRhzNKC4lPSWJ80b0DLsUEZG4U2DESW19Ay99vIGzh/egS4ZmphWRjkeBESfvLC1ny64aLlJzlIh0UAqMOJleXErXzFROG9J2picREYknBUYcVFTX8deFG5k8shdpKXpLRaRj0qdbHLy2YCO7axs0FYiIdGgKjDiYUVJKv26dGNu/a9iliIgERoFxhDbt2M0/l5Vz0eg+RKa9EhHpmBQYR2jmvPU0OExRc5SIdHAKjCM0o6SUkX1zOCY/K+xSREQCpcA4Ass27WRB6Q5NBSIiCUGBcQRmFK8nOcn40qjeYZciIhI4BcZhamhwZpSUMnFQHvld0sMuR0QkcAqMwzR3zVbWba3iojE6uxCRxKDAOEzTi0vplJrMOcM1M62IJAYFxmGoqWvglY83cO5xPeicHvSy6CIibYMC4zC8uWQT26tquVDXXohIAlFgHIYZJaXkZaVxyqC8sEsREWk1CowW2rG7lr8t2sT5I3uTkqy3T0QShz7xWmjW/I3U1GlmWhFJPAqMFppeXMrAvM6M7JsTdikiIq1KgdEC67dV8f7KzUzRzLQikoAUGC0wc9563OFCXawnIglIgdECM4pLGdv/KI7O7Rx2KSIirU6BEaNFG3aweONOdXaLSMJSYMRoRkkpKUnG5JFqjhKRxKTAiEFDgzOzZD2nD8mnW+e0sMsREQmFAiMGc1ZuYcP23ZoKREQSmgIjBjOKS8lKT+HsYT3CLkVEJDQKjEPYXVvPq/M3cO5xPemUlhx2OSIioVFgHMLfF29iZ3WdRkeJSMJTYBzCjOJSundJ56RjcsMuRUQkVAqMg9hWWcMbSzYxZXRvkpM0FYiIJLZAA8PMbjOzBWb2iZndHt022szeN7MSMysys3EHOT7bzErN7DdB1tmcV+ZvoLbemTJazVEiIoEFhpmNAK4HxgGjgPPNbDBwD/Bf7j4a+HH0cXN+ArwVVI2HMqO4lMHdsziud3ZYJYiItBlBnmEMA95390p3ryPywX8R4MCeT+AcYH1TB5vZCUAP4K8B1tistVsq+XDVVi4co5lpRUQg2MBYAJxmZrlmlglMAvoBtwP3mtla4D7gBwceaGZJwC+AOw/1ImZ2Q7Rpq6isrCxuxc+cF8mxKaM1FYiICAQYGO6+CLgbmA3MAuYBdcBNwB3u3g+4A5jaxOHfBl5197UxvM7D7l7o7oX5+fnxqp0XPlrHuAHd6Ns1My7PKSLS3gXa6e3uU919rLufBmwBlgJXAS9Ed/kzkT6OA50EfMfMVhE5C/mGmf08yFob+2T9DpaX7eJCdXaLiOwV9Cip7tGv/YGLgWlE+ixOj+5yJpEQ2Y+7X+Hu/d29APge8Ad3/36QtTY2vbiUtOQkJh/fq7VeUkSkzUsJ+PmfN7NcoBa42d23mtn1wANmlgLsBm4AMLNC4EZ3vy7gmg6qvsGZOW89Xzg2n5zM1DBLERFpUwINDHc/tYlt7wAnNLG9CPhcWLj748DjAZTXpHeXl1O2s1pTgYiIHEBXeh9genEpXTJSOGNo97BLERFpUxQYjVTV1PPago1MPr4XGamamVZEpDEFRiOzF33Grpp6LZQkItIEBUYjM4pL6Z2TwbiCbmGXIiLS5igwojZXVPPWp2VcMLoPSZqZVkTkcxQYUS9/vIH6BtfoKBGRZigwomaUlDKsVzZDe3YJuxQRkTZJgQGsKt9F8ZptXDRGEw2KiDRHgUHk7MIMLhil5igRkeYkfGC4OzOKSzlpYC49czLCLkdEpM0Kei6pNq+qtp7xA3KZODgv7FJERNq0hA+MzLQU7v7KyLDLEBFp8xK+SUpERGKjwBARkZgoMEREJCYKDBERiYkCQ0REYqLAEBGRmCgwREQkJgoMERGJibl72DXEjZmVAasP8/A8oDyO5bRnei/2p/djf3o/9ukI78XR7p4fy44dKjCOhJkVuXth2HW0BXov9qf3Y396P/ZJtPdCTVIiIhITBYaIiMREgbHPw2EX0Ibovdif3o/96f3YJ6HeC/VhiIhITHSGISIiMUn4wDCz88xsiZktM7Pvh13E8AJ7AAAEBUlEQVRPmMysn5m9YWaLzOwTM7st7JrCZmbJZlZsZi+HXUvYzOwoM3vOzBZHf0dOCrumMJnZHdG/kwVmNs3MOvySnQkdGGaWDPwW+BdgOPBVMxseblWhqgO+6+7DgAnAzQn+fgDcBiwKu4g24gFglrsfC4wigd8XM+sD3AoUuvsIIBm4PNyqgpfQgQGMA5a5+wp3rwH+BEwJuabQuPsGd/8oen8nkQ+EPuFWFR4z6wtMBh4Nu5awmVk2cBowFcDda9x9W7hVhS4F6GRmKUAmsD7kegKX6IHRB1jb6PE6EvgDsjEzKwDGAHPCrSRU9wP/BjSEXUgbMBAoAx6LNtE9amadwy4qLO5eCtwHrAE2ANvd/a/hVhW8RA8Ma2Jbwg8bM7Ms4HngdnffEXY9YTCz84FN7j437FraiBRgLPCgu48BdgEJ2+dnZl2JtEYMAHoDnc3synCrCl6iB8Y6oF+jx31JgNPKgzGzVCJh8ZS7vxB2PSGaCFxgZquINFWeaWZPhltSqNYB69x9zxnnc0QCJFGdDax09zJ3rwVeAE4OuabAJXpgfAgMNrMBZpZGpNNqZsg1hcbMjEgb9SJ3/2XY9YTJ3X/g7n3dvYDI78Xf3b3D/wfZHHffCKw1s6HRTWcBC0MsKWxrgAlmlhn9uzmLBBgEkBJ2AWFy9zoz+w7wGpFRDr93909CLitME4GvA/PNrCS67Yfu/mqINUnbcQvwVPSfqxXA1SHXExp3n2NmzwEfERldWEwCXPWtK71FRCQmid4kJSIiMVJgiIhITBQYIiISEwWGiIjERIEhIiIxSehhtSItZWb1wPxGm/7k7j8Pqx6R1qRhtSItYGYV7p4Vdh0iYVCTlEgcmNkqM7vbzD6I3gZFtx9tZq+b2cfRr/2j23uY2XQzmxe9nWxmnc3slejjBWZ2Wbg/lcj+FBgiLdPJzEoa3Rp/qO9w93HAb4jMdEv0/h/cfSTwFPDr6PZfA2+5+ygiczJ9ApwHrHf3UdE1Fma1xg8kEis1SYm0QHNNUtFJCs909xXRCRw3unuumZUDvdy9Nrp9g7vnmVkZ0Nfdqxs9xxAi09Q8C7zs7v9olR9KJEY6wxCJH2/mfnP77P8N90+BE4h0qv+Pmf04jrWJHDEFhkj8XNbo63vR+++yb+nOK4B3ovdfB26CveuGZ5tZb6DS3Z8ksjhPIk8fLm2QmqREWqCJYbWz3P370Sapx4BJRP4R+6q7L4uuXPh7II/IinVXu/saM+tBZHbTgUA9kfDIBu4lssJfLXCTuxe1xs8lEgsFhkgcRAOj0N3Lw65FJChqkhIRkZjoDENERGKiMwwREYmJAkNERGKiwBARkZgoMEREJCYKDBERiYkCQ0REYvL/AZJXAqFL7ugnAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(accuracies)\n",
    "plt.ylabel(\"Accuracy\")\n",
    "plt.xlabel(\"Epocs\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Using model for infrence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "odict_keys(['conv1.weight', 'conv1.bias', 'conv2.weight', 'conv2.bias', 'dense1.weight', 'dense1.bias', 'dense2.weight', 'dense2.bias'])\n"
     ]
    }
   ],
   "source": [
    "state_dict = torch.load('mnist_cnn.pt')\n",
    "print(state_dict.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> We can save model with state dict for no furthur problem as loading only the state dict requires the model too"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "checkpoint = {\n",
    "    'input_size':(1,28,28),\n",
    "    'output_size':10,\n",
    "    'hidden_layers':[each.out_features for each in net.layers],\n",
    "    'state_dict':net.state_dict()}\n",
    "torch.save(checkpoint,'checkpoint.pth')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "net.load_state_dict(state_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "__main__.Net"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(net)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataiter = iter(trainloader)\n",
    "images,labels = dataiter.next()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def infrence(model,data,label):\n",
    "    model.eval()\n",
    "    model.to(\"cpu\")\n",
    "    with torch.no_grad():\n",
    "        output = model(data)\n",
    "    ps = torch.argmax(torch.nn.functional.softmax((output),dim=1))\n",
    "    \n",
    "    print(\"Actual Label: \"+ str(label.item())+\" Predicted Label: \"+str(ps.item()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Actual Label: 9 Predicted Label: 9\n"
     ]
    }
   ],
   "source": [
    "infrence(net,images[1].reshape(1,1,28,28),labels[1])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}