Univariate_regression_by_hand.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b8fe7377",
   "metadata": {},
   "source": [
    "# Univariate linear regression coded by hand\n",
    "It only requires a single line of code to run linear regression in languages like R or Python, which makes it all too easy to run an analysis without really understanding it. During my university training, I learned how to perform linear regression with only a pencil and paper. This was excruciatingly slow and required an extremely simple data set, but the point of the exercise was to develop a better understanding of how the algorithm functioned. I've been doing a refresher course on some of the basics over the past few weeks with a fantastic machine learning course through Coursera. We just finished coding an algorithm for simple univariate regression using only basic mathematical functions in python. It was a fantastic learning experience, and even though I've been regularly performing regression via gradient descent for more than a decade, I have an entirely new appreciation for the method and a much better sense of how to troubleshoot issues (e.g., model non-convergence, feature scaling, learning rate optimization) when they arise. **This code is heavily based on a series of tutorials in the Coursera supervised machine learning module taught by Andrew Ng.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "cc428b41",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Import libraries and test data\n",
    "import math, copy\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "fb729f62",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x2438d05a730>"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGUCAYAAAAIz0nfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAAArhElEQVR4nO3df3BV9Z3/8de5+WUSLgVdlIR1GzXEAKkwE4i/Al0MFYILbqPtLNJ+hcI6OrDOV6et7IJdtgsO/Yq1FSm6RcPoUraMBSvgV4wRKLV8m5Ed2kKSDSIpEhr5UcAbCLlJ7vn+cSUlJPee5ObknnvueT5mmJjP55PwjrScl59fxzBN0xQAAIADfE4XAAAAvIsgAgAAHEMQAQAAjiGIAAAAxxBEAACAYwgiAADAMalOF9CbUCikUCjUrc0wDBmG4VBFAADAimmauvpWEJ/PJ58v8rxHwgaRCxcuOF0GAAAYoOzs7KhBhKUZAADgGIIIAABwDEEEAAA4JiH3iPS2KdVqjQkAADirtz2eVgdNXBNErHbdAgCAxGMVRHiyAwAAxxBEAACAYwgiAADAMQQRAADgmH4HkYcfflj5+fkqLCxUaWmpDh06JElqbm5WeXm5CgoKNGHCBO3Zsyfi92hpadFDDz2kgoICFRUVafPmzbH/BAAAwLX6HUTuv/9+1dfXq76+Xt/61rf0yCOPSJIWLlyo8vJyNTQ0aOPGjZo7d67a2tp6/R5LlixRTk6OGhoaVFVVpSeffFLNzc0D+0kAAIDr9DuIVFRUKDU1fOq3uLhYTU1N6ujoUHV1tRYsWCBJGjdunPLz87V3795ev8fbb7+thQsXSpJycnI0bdo0bd++PdafAQAAuNSA9oi8/PLLuu+++3Ty5Emlp6crOzu7qy8nJ0dNTU29ft3x48c1cuTIPo0FAACD56qX5cZdzBearVu3Th988IH27t2rlpYWpaSkdOs3TVPBYDDi11853mosAACwTyAoLa2RtjVK7SEpzSfNypNWlkj+9PjWElMQefbZZ/Xzn/9c7733noYOHarMzEy1trbqwoULXbMizc3Nys3N7fXrc3NzdeLECQ0dOrRr7KRJk2L8EQAAQF8FgtKdW6S6s1Loiva1B6X3j0v7KuIbRvq1NNPZ2alFixapurpau3bt0ogRIyRJaWlpmjp1qiorKyVJdXV1qq2t1eTJkyVJ58+f15EjR7q+z8yZM7V+/XpJ4RBSVVWl6dOn2/IDAQCAyJbW9AwhkhQypbpz0rKa+NZjmGbfV4caGxt10003KT8/v9vSSmVlpb74xS9q/vz5Onr0qDIzM/XDH/5QZWVlkqQNGzZo+fLlamxslCR99tlneuSRR/Tf//3fSk1N1dNPP605c+Z0fb9QKKRAINDt9/b7/bxrBgCAAbrpP6XGQOT+PL909Buxfe9Ynt/9CiLxQhABAMB+pind+LrUdCHymFHZ0ifflCzeVderWJ7fPNkBAPAIwwhvTI0mzRdbCIkVQQQAAA+5/fro/XdY9NuNIAIAgIf84kj0/jcs+u1GEAEAwEM6BthvN4IIAAAe0dHHlNHXcXYgiAAA4BF93YTKZlUAAGC7q97GMuBxdiCIAADgEX29OSyeN4wRRAAA8AjDkHKzoo/JzWJpBgAADJIHbone/6BFv90IIgAAeMjKEmnccMl31ayHzwi3ryiJbz0EEQAAPMSfLu2rkBYXhV9wNyo7/HFxUbjdnx7fenjpHQAAHmaa9u0J4aV3AACgX+K5MbU3BBEAAOAYgggAAHAMQQQAADiGIAIAABxDEAEAAI4hiAAA4GFOX+KR6uxvDwAA4i0QlJbWSNsapfaQlOaTZuWFb13lQjNxoRkAAIMlEJTu3CLVnZVCV7T7DGnMsIHdrsqFZgAAIKqlNT1DiCSFTKnunLSsJr71EEQAAPCQbY09Q8hlIVN6qzGOxYggAgCAZ5hmeE9INO2h+G5gJYgAAOARhhHemBpNmi++758hiAAA4CGz8sIbU3vjM6TZefGshiACAEBMEu/Mad+sLAmfjrk6jFw+NbOiJL71cI8IAAB9lEj3b8TKnx4+orusJrwx9fLPMTsvHEK4R0TcIwIASDyDef+Gk0zTvj0h3CMCAMAgSbT7N+wSz42pvSGIAADQB4l2/0ayIIgAAGDBNKW2zuhj2jrdu4HVSTEFkZaWFpWWlmr79u2SpGAwqMLCwm6/rr/+ei1fvrzXr9+wYYOGDh3abXxlZWXMPwQAAIPJMKSW9uhjWtqdX+Zwo36fmtmwYYOWLFmiM2fOdLWlp6ervr6+27i77rpLJSWRzwDde++9euONN/r72wMAgCTS7xmRefPmqbm5WXfffXfEMb/4xS+UkpKimTNnDqg4AAASgWlKQ9KijxmSxtJMLGzfI9LR0aF/+Zd/0apVq6KOq6qq0ujRozVhwgStXbvW7jIAALCNYUgZKdHHZKSwNBML24PIT3/6UxUUFESdMZkzZ47OnTunw4cP680339SPfvQjvfPOO3aXAgCAbe7Jjd5fZtGP3tkaRFpaWvTv//7veuaZZ6KOy8jIkPF5bMzLy1NZWZlqa2vtLAUAAFv99tTA+tE7W4PIc889p7KyMn3pS1/q1n7+/HkdOXKk6/OqqipduHBBktTU1KSqqiqVlpbaWQoAALaqPzuwfvSu30Fk06ZNmjhxovbv368nnnhCU6ZMkSSdPHlSL7zwgr7//e/3+JqtW7eqrKys6/M9e/aoqKhIBQUFmjFjhp5++umoJ2wAAHBSKBS+tCyaTjM8Dv3Du2YAAOgDY531GPOxwa8jkfGuGQAABkmaxRPTqh+9418bAAAWTFMabvFm3eHp3CMSC4IIAAAWDENqtXjXTGsn94jEgiACAEAfMNkxOAgiAABYME3rUzMhk6WZWBBEAACwYBhSm8XSzCWWZmJCEAEAwIJpSlYZwxAzIrEgiAAAYMEw+rY0w4xI/xFEAACwYJrSNRZv370mhRmRWBBEAACwYBiSz2K2w2cwIxILgggAAH1gNdvBbEhsCCIAAFgwzfCpmGgudRJGYkEQAQDAgmFIFjlEnWJpJhYEEQAALIRC9o7DXxBEAACw4PNJKRazHSlGeBz6h39lAAD0wbhro/cXWfSjdwQRAAD64P/OlDIiPDUzfNLbM+NbT7IgiAAA4sbNp0pyh0gfz5XGXyelGuEHaKoR/vzjueF+9F+q0wUAAJJbICgtrZG2NUrtISnNJ83Kk1aWSP50p6vrn9wh0oGvh/85FGJPiB0M00y8fBoKhRQIBLq1+f1++fgTBwBXCQSlO7dIdWelKw+U+AxpzDBpX4X7wggii+X5zZMdADBoltb0DCFS+AVxdeekZTVOVIVEQhABAAyabY09Q8hlIVN6qzGOxSAhEUQAAIPCNKXWjuhjWjvcvYEVA0cQAQAMCsOQzlyKPubMJa5F9zqCCABg0HRYzHZY9SP5EUQAAIOC97OgLwgiAIBB0dclF5ZmvI0gAgAAHEMQAQAMCsOQUizGpIgZEa8jiAAABk2mxYtErPqR/AgiAIBBYZrW17f707lHxOsIIgCAQWEYUobF2kxGCkszXkcQAQAMmll5kR80Pkmz8+JXCxITQQQAMGgWj5MirbyYkhaNi2c1SEQxBZGWlhaVlpZq+/btXW3Lly/Xddddp8LCwq5f7777bsSvf+ihh1RQUKCioiJt3rw5tuoBAAnta1XRg8jXq+JZDRJRv/crb9iwQUuWLNGZM2d69M2fP1+rV6+2/B5LlixRTk6OGhoa9Kc//UmTJk3SlClTNHLkyP6WAwBIYIf+HL3/oEU/kl+/Z0TmzZun5uZm3X333TH/pm+//bYWLlwoScrJydG0adO6za4AANwvFJI6LU7EdJpc8e51tu4Rqays1OjRo1VSUhJ1ueX48ePdZj9ycnLU1NRkZykAAIf5+viE6es4JCfb/vifeuopnTlzRocPH9ZPf/pTLVq0SLW1tRHHp6T85UyXaZoKBoN2lQIASACmKaVaHM1NNbhHxOtsCyKZmZld/zx+/HgVFxervr6+17G5ubk6ceJE1+fNzc3Kzc21qxQAQAIwDGlkVvQxI7O4R8TrbAsiO3bs6JrVqK2t1YEDBzRp0iRJ0vnz53XkyJGusTNnztT69eslhUNIVVWVpk+fblcpAIAE8dWbo/dXWPQj+fU7iGzatEkTJ07U/v379cQTT2jKlCmSpC1btig/P1+33nqrvvnNb+qVV17RjTfeKEnaunWrysrKur7HqlWrdPz4cRUUFOiee+7R6tWrlZ+fb9OPBABIFCtLpHHDJd9Vsx4+I9y+osSZupA4DNNMvNW5UCikQCDQrc3v98vHjiYAcJ1AUFpWI73VKLWHpDRf+EbVFSXW76KBu8Ty/CaIAADixjTZE5LMYnl+82QHAMQNIQRXI4gAAADHEEQAAHGTeJsB4LR+v2sGAID+CASlpTXStsa/bFadlRc+UcNmVbBZFQAwaAJB6c4tUt1Z6cpXyvgMacwwaV8FYSSZsFkVAJBQltb0DCGSFDKlunPhY73wNoIIAGDQbGvsGUIuC5nhu0XgbQQRAHCBxFtEt2aaUltn9DFtne782WAfNqsCQIJy+yZPw5Ba2qOPaWnnbhGvI4gAQAKKtMlz7UHp/eNs8kTyYGkGABJQMmzyNE1pSFr0MUPSWJrxOoIIACSgZNjkaRhSRkr0MRkpLM14HUEEABKMaYb3hETTHnLHTMKsvPCdIb3xGeG38MLbCCIAkGAMI7wxNZo0nztmElaWhC8uuzqMXL7QbEWJE1UhkRBEACABTb8xev8Mi/5E4U8Pb6xdXCTl+aVR2eGPi4vYcIswTs0AAAaVP136cWn4l2m6YyYH8cOMCAAkoJ2fRO9/x6I/URFCcDWCCAAkmGTarApYIYgAQIJJps2qgBWCCAAkII69wisIIgCQgDj2Cq8giABAAvKnS1umS8OvOt46/PN2jr0iWRBEACABnWiRbtssnWnr3n6mLdx+osWZugC7EUQAIAGVvy21RTg50xaSZr4d33qAwUIQAYAEdOjPA+sH3IIgAgAJJtSHO0JCZngc4HYEEQBIML4+3BFiGOFxgNvxP2MASECFw6P3j7HoB9yCIAIACej2EQPrB9yCIAIACej9E9H7qy36AbcgiABAgjFN6WJ79DEX23npHZIDQQQAEoxhSKcuRR9z6hIvvUNyIIgAQAKymuxgMgTJIqYg0tLSotLSUm3fvr2r7Z//+Z9VUFCgwsJCFRcX61e/+lXEr9+wYYOGDh2qwsLCrl+VlZWxlAIASaez095xQCJL7e8XbNiwQUuWLNGZM2e6td9111363ve+p8zMTL3//vuaM2eOmpqaIn6fe++9V2+88Ub/KwaAJJeSYu84IJH1e0Zk3rx5am5u1t13392tfdasWcrMzJQkFRcX69NPP1UncR0A+s00JavtH4bYrIrkMCh7RF5++WVNnz5dKVHielVVlUaPHq0JEyZo7dq1g1EGALiSYUg5WdHH5GSxWRXJod9LM1a2bduml156KeoekTlz5ujhhx+WYRhqbGzUV77yFd1yyy2aMWOG3eUAgCs9cIu05g+R+x+8JX61AIPJ1hmRn/3sZ/rOd76jd999V3/9138dcVxGRoaMz6N8Xl6eysrKVFtba2cpAOBqK0ukccMl31WzHj4j3L6ixJm6ALvZFkSeeeYZPf/889q1a5fy8/O79Z0/f15Hjhzp+ryqqkoXLlyQJDU1NamqqkqlpaV2lQIArudPl/ZVSIuLpDy/lJsV/ri4KNzuT3e6QsAe/Q4imzZt0sSJE7V//3498cQTmjJliiRp6dKlOnXqlKZOndp1JPfyqZitW7eqrKys63vs2bNHRUVFKigo0IwZM/T000+rpIR4DwBXu3o/KvtTkWwM00y8fdehUEiBQKBbm9/vl493XgPwiEBQuv0XUt257u2GpMJh0m8fYFYEiSeW5zdPdgBJLfH+U6tvvrOvZwiRwjMideek7+6Lc0HAICGIAEg6gaD0+K+lm/5TuvH18MfHfx1ud4ufHY7ev9GiH3AL24/vAoCTAkHpzi1S3VkpdEX72oPS+8fdsdHTNKWLHdHHXOz4/OIz7hKByzEjAiCpLK3pGUIkKWSGlzSW1ThRFYBICCIAksq2xp4h5LKQKb3VGMdiYmQYUpbFfHVWKrMhSA4EEQBJwzSl9kgp5HPtIXdsYH1odPT+uRb9gFsQRAAkDcOQ0iz+VkvzuWMm4dk7w8d0e1M4TPo/d8azGmDwEEQAJJVZeT2vRb/MZ0iz8+JZTez86VLNA9LjX+p+s+rjXwq3J/qGW6CvuNAMQFLpOjVzLrwn5DKfIY0Z5o5TM73hhAzcgAvNAHje1e9oGZWdHO9oIYQgWTEjAiCphUISf3UA8RHL85sLzQAknUAwfJ/ItsbwKZk0X3jvyMoS986IAMmKGREASSXSzapu3yMCuAF7RAB4HjerAu5CEAGQVJLhZlXASwgiAJJGMt2sCngFQQRA0kimm1UBryCIAEgqs/KkSDnDkHtuVgW8giACIKksmSClR/ibLd0nPTUhntUAsEIQAZBUVh2IvE+kPST94EA8qwFghSACIKlEPTUjTs0AiYYgAiBpcGoGcB+CCICkwakZwH0IIgCSytTc6P33WPQDiC+CCICk8v8+jd6/z6IfQHwRRAAklbpz0fvrLfoBxBdBBEDSCFlsVJUks4/jAMQHQQRA0ujrJlQ2qwKJgyACIGlwagZwH4IIgKQyeujA+gHEF0EEQFJp6RhYP4D4IogASBqmKXVa3JraaXKzKpBICCIAkgZ7RAD3IYgASCqz8iRfhKDhM6TZefGsBoCVmIJIS0uLSktLtX379q625uZmlZeXq6CgQBMmTNCePXuifv1DDz2kgoICFRUVafPmzbGUAQA9rCyRxgzrGUZ8Rrh9RYkTVQGIpN9BZMOGDcrPz9dvf/vbbu0LFy5UeXm5GhoatHHjRs2dO1dtbW29fo8lS5YoJydHDQ0Nqqqq0pNPPqnm5ubYfgIAuII/XdpXIS0ukvL80qjs8MfFReF2f7rTFQK4kmGasW3b+tu//Vt9+9vf1t/93d+po6NDfr9fp0+fVnZ2dlf/smXLNG3atB5fe/PNN2vHjh0aM2aMJGnevHkqLS3VwoULJUmhUEiBQKDb1/j9fvl8rCQB6B/TZE8IEC+xPL9tebKfPHlS6enpXSFEknJyctTU1NTr+OPHj2vkyJF9GgsAA0EIARKbbVMMKSkp3T43TVPBYLBP463GAgCA5GRLEBkxYoRaW1t14cKFrrbm5mbl5ub2Oj43N1cnTpzo01gAAJC8bAkiaWlpmjp1qiorKyVJdXV1qq2t1eTJkyVJ58+f15EjR7rGz5w5U+vXr5cUDiFVVVWaPn26HaUAsAmXfgGIh34HkU2bNmnixInav3+/nnjiCU2ZMkWStH79em3btk0FBQX6h3/4B23atElDh4Zf6rB161aVlZV1fY9Vq1bp+PHjKigo0D333KPVq1crPz/fph8JQKwCQenxX0s3/ad04+vhj4//OtwOAIMh5lMzg4lTM0D8BYLSnVukurNS6Ir2y/dvcPQVgBXHTs0AcL+lNVLtVSFEkkJmuH1ZjSNlAUhyBBEAkqRfHpUiTY+akt48Gs9qAHgFQQSATFM63Rp9zOlWNrACsB9BBIAMQ2rtjD6mtZPLwQDYjyACoM8zHcyIALAbQQSAJOvZDmZDAAwGgggAGYaUnRp9THYqYQSA/QgiACRJD94cvf9rFv0AEAuCCABJUnrKwPoBIBYEEQCSpJ2fRO9/x6IfAGJBEAEg05Tar75S9SrtIU7NALAfQQSADENKs/jbIM3HZlUA9iOIAJAkzcoLv+CuNz5Dmp0Xz2oAeAVBBIAkaWWJVPCF3vsKviCtKIlvPQC8gSACoEtvEyJGhHYAsANBBIAkaWmN9D/nerabkv7nvLSsJt4VAfACgggASdK2RinSwZmQKb3VGMdiAHgGQQQAx3cBOIYgAoDjuwAcQxABIInjuwCcQRABICl8fHfMsJ5hxGeE2zm+C2AwEEQASJL86dK+CmlxkZTnl0Zlhz8uLgq3+9OdrhBAMjJMM/G2n4VCIQUCgW5tfr9fPh+5CYiXjg4pNdXpKgC4SSzPb/6aAdCl4ax011bpTNtf2q7LkH7zValguHN1AUheTDEAkBQOIYX/1T2ESOHPC/8r3A8AdiOIAJAUngmJtE5rft4PAHYjiACQ1HMmpL/9ABALgggAdXbaOw4A+oogAkApKfaOA4C+IogAkGlKVre3G+JdMwDsRxABIMOQcrKij8nJ4l0zAOxHEAEgSXrglsizIoakB2+JZzUAvIIgAkBS+F0zY4f3/q6ZscN51wyAwUEQASCJd80AcIZt75o5evSoysvLu7V9+umnev755zVv3rxu7cuXL9eaNWs0YsSIrrYXXnhB9957ryTeNQMkAtNkTwiA/nH0XTM33XST6uvruz5va2vT2LFjNXHixF7Hz58/X6tXr7brtwdgM0IIgHgYtCmGF198UXfffbeKiooG67cAAAAuNyhB5Ny5c1q9erX+7d/+LeKYyspKjR49WiUlJdq8efNglAE4grs2AKDvbNsjcqWnnnpKFy9e1Jo1a3rtb21tVWZmpiTpd7/7naZNm6Y9e/Zo7NixktgjAvcJBKWlNdK2RinYKaWnSLPywidR2OQJwCsc3SNy2fHjx/Xqq6/q4MGDEcdcDiGSNH78eBUXF6u+vr4riABuEghKE9+QGs53b1/zB2nnMenDBwkjABCJ7VMMTz/9tB599FHdcMMNXW2nT5/WsWPHuj7fsWOHgsGgJKm2tlYHDhzQpEmT7C4FiIv//eueIeSyhvPSE7+Obz0A4Ca2zoj84Q9/0I4dO/TRRx91a3/xxRe1e/du7d69W5K0ZcsWPfbYY8rMzNSQIUP0yiuv6MYbb7SzFCBuXj8cvf+1w9L6e+JTCwC4zaDsERko9ojALUxT8r1kPS70KMdhASS/WJ7fPNmBAehrjE+8uA8AiYEgAgxAX2c5mA0BgN4RRIABIIgAwMAQRIABME0pxWJMiliaAYBICCLAABiGJKvZDoMZEQCIhCACDIBpStdYTIlck8KMCABEQhABBsAwpOEZ0ccMz2BGBAAiIYgAA3TtNdH7r7PoBwAvI4gAA3SuLXr/WYt+APAygggwAKYpdVrs/+g02SMCAJEQRIABMAwpzeL/RWk+9ogAQCQEEWCAplu8r3EG73MEgIgIIgAAwDEEEWCAdn4Svf8di34A8DKCCDAApim1h6KPaQ+xWRUAIiGIAAPAZlUAGBiCCDBAs/IkX4Sg4TOk2XnxrAYA3IUgAgzQyhJpzLCeYcRnhNtXlDhRFQC4A0EEGCB/urSvQlpcJOX5pVHZ4Y+Li8Lt/nSnKwSAxGWYZuJtowuFQgoEAt3a/H6/fD5yExKfabInBIA3xfL85skO2IwQAgB9RxABAACOIYggYSTeIiEAYLClOl0AvC0QlJbWSNsapWCnlJ4SPg67soRNngDgBWxWhWMCQankF1L9uZ59hcOkmgcIIwDgJmxWhat8Z1/vIUQKt393XzyrAQA4gSACx/zscPT+jRb9AAD3I4jAEaYpXeyIPuZiBxtYASDZEUQAAIBjCCJwhGFEflHcZT6Dy8EAINkRROAYy2UXlmUAIOkRROAI05RCFkGj02SPCAAkO4IIHGO17MKyDAAkP4IIHGEYUpbFvb5ZqYQRAEh2tgWRefPmaeTIkSosLOz6dejQoV7HNjc3q7y8XAUFBZowYYL27NljVxlwkYdGR++fa9EPAHA/W2dEli1bpvr6+q5f48aN63XcwoULVV5eroaGBm3cuFFz585VW1ubnaXABb5XLKVHmPFIN6Sni+NbDwAg/uK+NNPR0aHq6motWLBAkjRu3Djl5+dr79698S4FDlt1QOqIsBm1w5R+cCCe1QAAnGBrEFmxYoVGjx6tKVOmqLq6utcxJ0+eVHp6urKzs7vacnJy1NTUZGcpcIFtjVIoQl9I0luN8asFAOAMi+2Cfbdu3TplZmZKkt5//3199atf1R//+EcNGzasx9iUlJRun5umqWAwaFcpcAHTlFotrnhv/fyKdzasAkDysm1G5HIIkaR77rlHOTk5+vjjj3uMGzFihFpbW3XhwoWutubmZuXm5tpVClzAMKQ/X4o+5s+XCCEAkOxsCyK//OUvFQqFJ9p/9atf6bPPPlNhYaEk6fTp0zp27JgkKS0tTVOnTlVlZaUkqa6uTrW1tZo8ebJdpcAlIu0P6Ws/AMD9bFua+clPfqJ/+qd/UlZWlq677jpt2bJFWVlZkqQXX3xRu3fv1u7duyVJ69ev1/z58/XCCy8oMzNTmzZt0tChQ+0qBS4QCkmGot/ibnw+zsdtNwCQtAzTTLxLtEOhkAKBQLc2v98vH0+kpOJb14cg8li8qgEADFQsz2+e7HBMqsX+D6t+AID7EUTgCNOUhmVEHzMsg5feAUCyI4jAEYYhXeqMPuZSJ6dmACDZEUQAAIBjCCJwhGlKQ9KijxmSxtIMACQ7gggcYRhSRkr0MRkpLM0AQLIjiMAxs/IkX4Sg4TOk2XnxrAYA4ASCSBJw6/LFyhLp1i+E7wu5kqFw+4oSJ6oCAMSTbTerIr4CQWlpTfgNtu0hKc0XnmFYWSL5052uru9M9bzUrLc2AEByYkbEhQJB6c4t0to/SI0BqelC+OPag+H2gEteZLy0Rmo413tfw3lpWU1cywEAOIAg4kJLa6S6s1LoqvaQGW53ywN8W2PPn+GykCm91RjHYgAAjiCIuNAvj0Z5gH/en+hMM7ykFE17yL37XwAAfUMQcRnTlE5fij7m1KXEf4AbRnhfSzRpPo7vAkCyI4i4jGFIFzuij7nY4Y4HOMd3AQAEEZfp60xHos+ISOETPmOG9QwjPiPczvFdAEh+BBGXSaYg4k+X9lVIi4ukPL80Kjv8cXFRuN1Nx5ABALHhHhGX8fUxOvZ1nNP86dKPS8O/TNMdS0oAAPu45HGFy0xTSrV4WKca7pgRuRohBAC8hyDiMoYhjcyKPmZkFg91AIA7EERc6Ks393w/y2WGpIqb41kNAACxI4i40OJx0fsXWfQDAJAoCCIu9LWqyC+FMyV9vSqe1QAAEDuCiAsd+nP0/oMW/QAAJAqCiMuEQlKnxYmYTjM8DgCAREcQcZlku0cEAOBtPK5cxjSllCS9RwQA4D0EEZcxDOmalOhjMlK4RwQA4A4EEReK9MbavvYDAJAoCCIuY5rSkLToY4aksTQDAHAHgojLGIaUZvGnluZjaQYA4A4EERfyp0fvH2rRDwBAoiCIuNAfP4ve32jRDwBAoiCIuIxpShc7o4+52MkeEQCAOxBEXMgqZBBCAABuYWsQefjhh5Wfn6/CwkKVlpbq0KFDvY5bvny5rrvuOhUWFnb9evfdd+0sJWn1dRMqm1UBAG5gaxC5//77VV9fr/r6en3rW9/SI488EnHs/Pnzu8bW19fr3nvvtbOUpGWa1heaXZPCrAgAwB1sDSIVFRVKTU2VJBUXF6upqcnObw+FZzqsrnhPMZgRAQC4w6DtEXn55Zd13333ReyvrKzU6NGjVVJSos2bNw9WGUnJKmQQQgAAbmGYpv2T+OvWrdNLL72kvXv3aujQoT36W1tblZmZKUn63e9+p2nTpmnPnj0aO3asJCkUCikQCHT7Gr/fLx+vlJVpSqNek/50MfKYnCyp6X8RSAAA8RXL89v2J/uzzz6rV155Re+9916vIURSVwiRpPHjx6u4uFj19fV2l5KUDCP8UrtoeOkdAMAtbAsinZ2dWrRokaqrq7Vr1y6NGDGiq+/06dM6duxY1+c7duxQMBiUJNXW1urAgQOaNGmSXaUkvVl5kV9s5zOk2XnxrAYAgNjZtjTT2Niom266Sfn5+UpJ+ct/sldWVmrnzp3avXu3du/eLUlasGCBqqqqlJmZqSFDhuj73/9+t/0kLM1EFwhKd26R6s5JoSv+9HyGNGaYtK/C+hp4AADsFsvze1D2iAwUQcRaICgtq5HeapTaQ+EX3c3Ok1aUEEIAAM4giHiUabInBADgvITYrOomiRfBYkMIAQC4VarTBcRbICgtrZG2Nf5lSWNWnrSSJQ0AAOLOU0szXZs8z0qhK9rZ5AkAwMCxNGNhaU3PECKFT57UnQtv/gQAAPHjqSCyrbFnCLksZIZPoAAAgPjxTBAxTamtM/qYts7k2cAKAIAbeCaIGIbU0h59TEs7J1AAAIgnzwQRAACQeDwTRExTyrI4rJyVytIMAADx5JkgYhjSxY7oYy52sDQDAEA8eSaISBKTHQAAJBbPBBHTlDojnd39XGeIpRkAAOLJM0HEMKRWi+O7FztZmgEAIJ48E0RMU7LKGIaYEQEAIJ48E0Sk8DtlBtIPAADs5ZkgYhhSpsXx3cxUlmYAAIgnzwQRScobMrB+AABgL08Fkc8srni36gcAAPbyTBAxTanTYiNqp8lmVQAA4skzQcQwpDSLnzbNxx4RAADiyTNBRJJm5UU+GeMzpNl58awGAAB4KoisLJFu/ULP+0QMhdtXlDhRFQAA3uWpICJ9vg/kqjZT1vtHAACA/TwVRL6zT2o433tfw3npu/viWw8AAF7nqSDys8PR+zda9AMAAHt5JoiYpnSxI/qYix0c3wUAIJ48E0SknntD+tsPAADs5ZkgYhhSisWYFHGPCAAA8eSZICJJGRZJxKofAADYyzNBxDQlf3r0Mf509ogAABBPngkihtG3GRGWZgAAiB/PBBFJmn5j9P4ZFv0AAMBengoiAAAgsdgaRBoaGlRaWqqCggLdcccdOnToUK/jmpubVV5eroKCAk2YMEF79uyxs4yIdn4Svf8di34AAGAvW4NIRUWFvvvd76qhoUErV67UnDlzeh23cOFClZeXq6GhQRs3btTcuXPV1tZmZyk9mKbUHoo+pj3EZlUAAOLJtiBy7NgxnTp1SrNnz5YklZWV6bPPPtNHH33UbVxHR4eqq6u1YMECSdK4ceOUn5+vvXv32lVKrwxDSrP4adN8bFYFACCebAsix48f18iRI7u15eTkqKmpqVvbyZMnlZ6eruzs7KjjBsOsPMkXIWj4DGl23qCXAAAArmDr0kxKSvfzsaZpKhgMxjzObitLpDHDeoYRnxFuX1Ey6CUAAIAr2BZEcnNzdeLEiW5tzc3Nys3N7dY2YsQItba26sKFC1HHDQZ/urSvQlpcJOX5pVHZ4Y+Li8LtVheeAQAAe9kWRPLy8nTttddq+/btkqRdu3bJMAyNHTtWp0+f1rFjxyRJaWlpmjp1qiorKyVJdXV1qq2t1eTJk+0qJSp/uvTjUunoN6RPvhn++ONSQggAAE4wTNO+cyL19fVasGCBTp06peHDh+s//uM/NH78eC1fvly7d+/W7t27JUknTpzQ/PnzdfToUWVmZuqHP/yhysrKur5PKBRSIBDo9r39fr98Pq49AQAgUcXy/LY1iNiFIAIAgPvE8vzmyQ4AABxDEAEAAI4hiAAAAMcQRAAAgGMIIgAAwDEEEQAA4JhUpwvoTW8nikMhi1fnAgAAR/X2rLa6JcQ1QeTKK+EBAIA7WAURlmYAAIBjCCIAAMAxBBEAAOCYhH3XzNUbXgzDkGEYDlUEAACsmKbZY0+Iz+dz30vvAACAN7A0AwAAHOPJINLQ0KDS0lIVFBTojjvu0KFDh5wuydMefvhh5efnq7CwUKWlpfx5JIiDBw9q6NCh2r59u9OlQNL69es1YcIEjR49WgsXLnS6HE/buXOniouLNWbMGI0fP17btm1zuiRXS8h7RAZbRUWFnnnmGc2ePVvV1dWaM2eOfv/73ztdlmfdf//9euWVV5SamqpXX31VjzzyiD744AOny/K0U6dO6Rvf+Ib+6q/+yulSIOlHP/qRtm7dqp07d+qGG25QZ2en0yV51qVLl/T1r39d+/fvV35+vg4ePKjbb79dp06dUlZWltPluZLnZkSOHTumU6dOafbs2ZKksrIyffbZZ/roo48crsy7KioqlJoazsTFxcVqampyuCJvCwaDevDBB7Vq1Sr9zd/8jdPleF5nZ6dWrlyp1157TTfccIMkKSUlxeGqvKu9vV3BYFCnT5+WJI0aNUrp6ekcphgAzwWR48ePa+TIkd3acnJyePgliJdffln33Xef02V42j/+4z/qgQce0IwZM5wuBZI++eQTBYNBffvb31ZRUZHGjx+v1157zemyPMvv9+v111/Xl7/8Zd1///362te+po0bNyozM9Pp0lzLk0szV//XhGmaCgaDDlWDy9atW6cPPvhAe/fudboUz1qzZo2ysrL0+OOPO10KPnfixAmNGDFCa9eu1fXXX6+PP/5YkydP1m233aYJEyY4XZ7nXLx4Uc8995zeffddZWRk6MUXX9QPfvADfeUrX1FaWprT5bmS54JIbm6uTpw40a2tublZubm5DlUESXr22Wf185//XO+9956GDh3qdDme1dDQoOrqahUWFkoKL2U+9thjamxs1OLFix2uzpsu79O5/vrrJUk333yz7rrrLv3+978niDjgnXfe0bBhw/TlL39ZknTHHXdozJgx+s1vftPVhv7x3NJMXl6err322q6TALt27ZJhGBo7dqzDlXlTZ2enFi1apOrqau3atUsjRoxwuiRPW7NmjRoaGlRfX6/6+nqVlJRo3bp1hBAHjR49WhkZGdq8ebMk6dNPP9WHH36o22+/3eHKvCk/P18ffvihDh8+LEn6+OOPdfbsWeXn5ztcmXt5bkZEkrZs2aIFCxboySef1PDhw/Xmm2+y0cghn3zyiX7yk58oPz9fkyZN6mqvrKzUnXfe6WBlQGIwDENvvvmmHn30Uf3rv/6rrrnmGj3//PO69dZbnS7Nk2677TY999xz+vu//3uFQiFlZ2fr1Vdf1ahRo5wuzbW4WRUAADjGc0szAAAgcRBEAACAYwgiAADAMQQRAADgGIIIAABwDEEEAAA4hiACAAAcQxABAACOIYgAAADHEEQAAIBjCCIAAMAx/x9fff7kRK+rpAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Create and visualize some training data\n",
    "x_train = np.random.randint(10, size=(100)) #Random float numbers from [0,100)\n",
    "y_train_noise = np.random.rand(100) #A small amount of noise to add to y\n",
    "y_train = 2*(x_train + y_train_noise) # Y = 2x + noise\n",
    "plt.scatter(x_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a68416ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Define a cost function\n",
    "def compute_cost(x, y, w, b):\n",
    "   \n",
    "    m = x.shape[0] #Get number of observations\n",
    "    cost = 0 #Initialize starting value at zero\n",
    "    \n",
    "    for i in range(m):\n",
    "        f_wb = w * x[i] + b #This calculates the target value (model output)\n",
    "        cost = cost + (f_wb - y[i])**2 #Squared difference between predicted and observed target\n",
    "    total_cost = 1 / (2 * m) * cost #Cost divided by twice the sample size (twice by convention - doesn't affect output)\n",
    "\n",
    "    return total_cost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "efcdb0f9",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Define a function to compute the gradient for dj_dw and dj_db\n",
    "def compute_gradient(x, y, w, b): \n",
    "    \"\"\"\n",
    "    Computes the gradient for linear regression \n",
    "    Args:\n",
    "      x (ndarray (m,)): Data, m examples \n",
    "      y (ndarray (m,)): target values\n",
    "      w,b (scalar)    : model parameters  \n",
    "    Returns\n",
    "      dj_dw (scalar): The gradient of the cost w.r.t. the parameters w\n",
    "      dj_db (scalar): The gradient of the cost w.r.t. the parameter b     \n",
    "     \"\"\"\n",
    "    \n",
    "    # Number of training examples\n",
    "    m = x.shape[0] #Number of observations    \n",
    "    dj_dw = 0 #Initialize at zero\n",
    "    dj_db = 0 #Initialize at zero\n",
    "    \n",
    "    for i in range(m):  \n",
    "        f_wb = w * x[i] + b #Compute predicted target value\n",
    "        dj_dw_i = (f_wb - y[i]) * x[i] #This is the partial derivative of j w.r.t. w\n",
    "        dj_db_i = f_wb - y[i] # This is the partial derivative of b w.r.t b\n",
    "        dj_db += dj_db_i #This stores the current value of dj_db\n",
    "        dj_dw += dj_dw_i #This stores the current value of dj_dw\n",
    "    dj_dw = dj_dw / m #This is still just part of partial derivative calculation\n",
    "    dj_db = dj_db / m #This is still just part of partial derivative calculation\n",
    "        \n",
    "    return dj_dw, dj_db"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "1e816404",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Create a gradient descent algorithm that uses the previously defined cost and gradient functions\n",
    "def gradient_descent(x, y, w_in, b_in, alpha, num_iters, cost_function, gradient_function): \n",
    "    \"\"\"\n",
    "    Performs batch gradient descent to fit w,b. Updates w,b by taking \n",
    "    num_iters gradient steps with learning rate alpha\n",
    "    \n",
    "    Args:\n",
    "      x (ndarray (m,))  : Data, m examples \n",
    "      y (ndarray (m,))  : target values\n",
    "      w_in,b_in (scalar): initial values of model parameters  \n",
    "      alpha (float):     Learning rate\n",
    "      num_iters (int):   number of iterations to run gradient descent\n",
    "      cost_function:     function to call to produce cost\n",
    "      gradient_function: function to call to produce gradient\n",
    "      \n",
    "    Returns:\n",
    "      w (scalar): Updated value of parameter after running gradient descent\n",
    "      b (scalar): Updated value of parameter after running gradient descent\n",
    "      J_history (List): History of cost values\n",
    "      p_history (list): History of parameters [w,b] \n",
    "      \"\"\"\n",
    "    \n",
    "    w = copy.deepcopy(w_in) # avoid modifying global w_in\n",
    "    # An array to store cost J and w's at each iteration primarily for graphing later\n",
    "    J_history = [] #This allows us to track the change in cost over iterations\n",
    "    p_history = [] #This allows us to track the change in parameter values\n",
    "    b = b_in\n",
    "    w = w_in\n",
    "    \n",
    "    for i in range(num_iters):\n",
    "        # Calculate the gradient and update the parameters using gradient_function\n",
    "        dj_dw, dj_db = gradient_function(x, y, w , b)     \n",
    "\n",
    "        # Update Parameters using equation (3) above\n",
    "        b = b - alpha * dj_db                            \n",
    "        w = w - alpha * dj_dw                            \n",
    "\n",
    "        # Save cost J at each iteration\n",
    "        if i<100000:      # prevent resource exhaustion by setting a max number of iterations\n",
    "            J_history.append( cost_function(x, y, w , b))\n",
    "            p_history.append([w,b])\n",
    "        # Print cost every at intervals 10 times or as many iterations if < 10\n",
    "        if i% math.ceil(num_iters/10) == 0:\n",
    "            print(f\"Iteration {i:4}: Cost {J_history[-1]:0.2e} \",\n",
    "                  f\"dj_dw: {dj_dw: 0.3e}, dj_db: {dj_db: 0.3e}  \",\n",
    "                  f\"w: {w: 0.3e}, b:{b: 0.5e}\")\n",
    " \n",
    "    return w, b, J_history, p_history #return w and J,w history for graphing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "f469591a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration    0: Cost 3.20e+01  dj_dw: -6.160e+01, dj_db: -1.009e+01   w:  6.160e-01, b: 1.00923e-01\n",
      "Iteration 1000: Cost 1.90e-01  dj_dw:  1.388e-03, dj_db: -8.553e-03   w:  1.996e+00, b: 6.80604e-01\n",
      "Iteration 2000: Cost 1.89e-01  dj_dw:  1.315e-04, dj_db: -8.101e-04   w:  1.991e+00, b: 7.13418e-01\n",
      "Iteration 3000: Cost 1.89e-01  dj_dw:  1.245e-05, dj_db: -7.672e-05   w:  1.991e+00, b: 7.16526e-01\n",
      "Iteration 4000: Cost 1.89e-01  dj_dw:  1.180e-06, dj_db: -7.266e-06   w:  1.991e+00, b: 7.16820e-01\n",
      "Iteration 5000: Cost 1.89e-01  dj_dw:  1.117e-07, dj_db: -6.882e-07   w:  1.991e+00, b: 7.16848e-01\n",
      "Iteration 6000: Cost 1.89e-01  dj_dw:  1.058e-08, dj_db: -6.518e-08   w:  1.991e+00, b: 7.16851e-01\n",
      "Iteration 7000: Cost 1.89e-01  dj_dw:  1.002e-09, dj_db: -6.173e-09   w:  1.991e+00, b: 7.16851e-01\n",
      "Iteration 8000: Cost 1.89e-01  dj_dw:  9.491e-11, dj_db: -5.847e-10   w:  1.991e+00, b: 7.16851e-01\n",
      "Iteration 9000: Cost 1.89e-01  dj_dw:  8.992e-12, dj_db: -5.537e-11   w:  1.991e+00, b: 7.16851e-01\n",
      "(w,b) found by gradient descent: (  1.9905,  0.7169)\n"
     ]
    }
   ],
   "source": [
    "#Run the gradient descent algorithm on training data\n",
    "#The true value of w=2 and b=0, let's see how we did\n",
    "# initialize parameters\n",
    "w_init = 0\n",
    "b_init = 0\n",
    "# some gradient descent settings\n",
    "iterations = 10000\n",
    "tmp_alpha = 1.0e-2\n",
    "# run gradient descent\n",
    "w_final, b_final, J_hist, p_hist = gradient_descent(x_train ,y_train, w_init, b_init, tmp_alpha, \n",
    "                                                    iterations, compute_cost, compute_gradient)\n",
    "print(f\"(w,b) found by gradient descent: ({w_final:8.4f},{b_final:8.4f})\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "4096ff2f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABLsAAAGbCAYAAAAskpJqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABmTElEQVR4nO3de1xVdb7/8fcGRAxBJC2hDFTYIKJdxKQCzWuZjZqmYpZ5/+XUaZqZtJqadOxkZ7yc6eLUcdS8g91IR2xyUsiUysxGK4kQL5kgqaGoKKLw/f3htCdk7xTFfX09Hw8eynetvdZn7T1j38d7fy8WY4wRAAAAAAAA4AX8XF0AAAAAAAAAUF8IuwAAAAAAAOA1CLsAAAAAAADgNQi7AAAAAAAA4DUIuwAAAAAAAOA1CLsAAAAAAADgNQi7AAAAAAAA4DUIuwAAAAAAAOA1CLsAAAAAAADgNQi7AB/w6aefaujQoYqMjFTDhg3VsmVL9ejRQy+//LLKysrq9V5ffPGFpkyZoiNHjtTrdevThx9+KIvFonnz5tnaXn/9dS1cuNBlNTm6/5QpU2SxWFRYWOj8ogAAgA39qZroT124wsJCNWnSRPfee6+qq6slSbt27dKUKVO0Z8+ei7qmvWeyWCx65plnbL8vWrRIfn5+Wr58+SXVD3giizHGuLoIAJfP//7v/+rxxx9XcnKyRo0apWuuuUYlJSV677339O677+qPf/yjpkyZUm/3mzdvnsaNG6fdu3crOjq63q5bnw4fPqwtW7YoISFBkZGRkqSUlBQFBAToww8/dElNju6/a9cu7dq1S7fddpsaNWrkktoAAPB19Kdqoz91Yc6cOaOkpCQFBgZq48aNCgwMlCStXbtWvXr1Uk5Ojm6//fY6X9feM1ksFj399NP67//+b9t5zz77rF588UVt3bpVrVu3rpdnAjxBgKsLAHD5fPTRR3r88cc1YsQIvf766/Lz+89gztGjR+uLL77Qt99+68IKXaNp06bq2bOnq8u4IK1bt6ZjAgCAC9Gfso/+1IWZP3++tm/frs8//9wWdNWHC32mZ555Ru+8846eeeYZpaen19v9AbdnAHitu+66yzRp0sQcO3bsgs7funWrufPOO03jxo1NcHCw6d27t/nXv/5V45xdu3aZ4cOHm2uvvdZcccUVpm3btmbChAnmyJEjZsGCBUZSrZ/du3fbvd/nn39uJJl58+bVOjZ69GgTGRlpzpw5Y4wx5sMPPzQ9evQwV155pQkLCzNJSUlm2rRpdXo/frJ7924jycydO9cYY0zXrl1r1dy1a9caz5yWlmbCw8NN48aNTceOHc0bb7xR45pRUVFm2LBh5tVXXzU33nijadiwoZk0aZL57rvvTGpqqomIiDCBgYHmqquuMn379jVbtmyxvfaX7v/Te7pjxw7b+eXl5eZ3v/udueaaa0xgYKCJj483f/3rX2vUk5OTYySZhQsXmtGjR9vet6FDh5rS0tKLet8AAPBF9Kfsoz91Yf2pqKgok5aWZve65/7k5OQYY4z5r//6L9OmTRsTHBxsGjdubDp06GBeffXVGtew90ySzNNPP12rhsWLFxuLxWIKCwsvqGbAGzCyC/BSVVVV+vDDD3XXXXepcePG5z3/m2++UUpKitq2batXX31VFotFs2fPVmpqqj777DO1bdtWp06dUo8ePdSkSRP9z//8j5o0aaIvvvhCc+bM0a9//Wvdcccd+t3vfqf//d//1dKlS3X11VdLklq0aGH3nh07dtSNN96oBQsWaMyYMbb2EydO6K233tJ//dd/yd/fX19//bXuuOMO/epXv9K8efNUVVWl7Oxsvfbaa3rqqacu+b2aNWuWRo4cKT8/P82aNUvS2W8rJen7779X586dlZycrNmzZys4OFiffPKJ7rvvPgUFBalfv36267zxxhv65ptv9Jvf/EZXX321rrzySp06dUqxsbF68MEHFRERoUOHDumll17SnXfeqby8PDVr1uwX738uY4z69++vTZs26emnn1ZcXJxycnL0yCOP6Mcff9Qf//jHGuePGTNGo0eP1uuvv66CggI988wzCg0N1d/+9rdLft8AAPB29KcuHP2p2v71r3/pu+++02uvvVaj/frrr9f06dM1adIkzZw5U9dff72tXZL8/f310EMPKSYmRhaLRe+//75+/etf64orrtCDDz54oR+JzT333KOHHnpIq1at0mOPPVbn1wMeydVpG4DL44cffjCSzBNPPHFB5w8dOtQ0b97cHD9+3NZ24sQJc/XVV5vBgwcbY4zZsmWLkWQyMzNrvLaiosKcOHHCGGPM3Llzf/Hbx3PNnj271rdSP337tHPnTmOMMbNmzTKSan2Ddvjw4Qu6x7nO/SbSGGNuu+22Gt8+/mT06NGmZ8+e5uTJkzV+RowYYfr06WM7LyoqynTr1s1UVlae9/6ffvqpkWTefvvt897/3G/t/vGPfxhJ5s0336xx3qOPPmqCgoLMjz/+aIz5zzeGS5YsqXHeoEGDzDXXXHPeGgEAAP2pX0J/6vz9qb/85S8mKCjInD59utaxDz74oMZorvNp27atufvuux0+kzGOR3YZY8ydd95p+vfvf0H3ArwBuzECXsrUce+JnJwc9enTR8HBwba2Ro0aqW/fvsrJyZEkRUVF6YorrtDkyZO1ZMkS7dy5U8YYNWzY8KIX+xw+fLiCgoJq7JyzcOFCde/e3bYOQdu2bSVJo0aNUlZWln744QdJUlhY2EXdsy7ef/99rV27Vo0aNarxs3jx4lo7+kRGRqpBgwa1rvHGG2/orrvuUsuWLRUcHKzk5GRJ0vHjx+tcT05Ojvz9/XXPPffUaL/33ntVUVGhTz75pEb7ud8Ct2rVSsXFxXW+LwAAvoj+VP3w1f5UcXGxrFarAgLqNqHq22+/1bhx45SQkKCwsDD5+/vrm2++uahn/UlcXJz2799/0a8HPA1hF+ClrrzySjVs2FC7du26oPN//PFHXXXVVbXar7rqKv3444+2a7733nsKCQnRgw8+qJiYGLVo0ULPPfdcnTuDPwkLC9OgQYO0ePFiVVdXa+/evcrJydG4ceNs5/Tp00evvfaatm/frl/96ldq0aKFEhIStGLFiou6Z10cOHBAgwcP1ieffFLr54033jjv66dNm6a0tDQdPXpUTz75pDIyMrRs2TJJde9AS2c/p6ZNm9bqNP302R06dOgXX9+wYcOL/qwAAPA19Kfqh6/2p44cOaLQ0NA61fbtt9+qU6dOWrFihYYMGaK//e1vWrNmjTp06HBJfbjGjRvr8OHDF/16wNOwZhfgpQICApSSkqI1a9aorKxMTZo0+cXzr7zySh04cKBWe0lJicLDw22/d+3aVbm5uTp69Kj+9a9/aeHChXr22WeVkJCgQYMGyWKx1LnWMWPGaNmyZcrOztYnn3yi8PBwDRgwoMY5Dz30kB566CHt379fmzdv1p///GcNHTpUe/bsUURERJ3veS5HdTdt2lTHjh2zfXtYVy+//LL69u2rrKwsW9u532D+0v3PdeWVV+rw4cM6ffp0jW89S0pKbMcBAED9oD9VN/SnagoLC1N5ebndY45qXbBggSoqKpSfn6/IyEhbe0hIyCXVcvz4cYdrmAHeiJFdgBf7/e9/r6NHj+rhhx9WdXV1rePffvut3n//fUlSt27dtHr16hrDo48fP67Vq1ere/fuks4OxS4rK5MkhYaGqmvXrnr55ZclSXv27JH0n6HwBw8evOA6b7/9drVp00YLFy7UokWLNGLECDVs2NB2vKCgQFVVVZKkiIgI9evXT08++aQqKyttQ8hPnz6t/Px87d2794Lv+3NhYWF2a+7Zs6c++OADbdmypdaxC/mWt6KiQq1atarRZu/bQkf3P1e3bt1UVVWld955p0b78uXL1bBhQ916663nvQYAALhw9KcuHP2pmiIjI23TVO3VKtX+jCsqKtSoUaMa4ePp06d19OjRS6rl22+/rZdAE/AUjOwCvFifPn30hz/8QdOmTVNBQYHGjBmj6667TocOHdIHH3yg5cuXa/r06brzzjv17LPPKisrS926ddOjjz4qi8Wil156SSdOnNDkyZMlSZ999pnGjBmj//f//p+Sk5NVVVWl9PR0XXHFFbrrrrskSUlJSQoICNCkSZP0yCOPqKSkRMOGDavxbea5LBaLRo8erT/+8Y+qrq7W2LFjaxx/9dVXtWbNGo0bN07x8fEqLS3V//7v/8pqtap9+/aSpKKiIrVt21Zdu3bVhx9+WOf3Kjk5WVlZWZo8ebI6dOigkydP6v7779fUqVO1Zs0a9ejRQxMnTlSHDh104MABvfHGG2rSpIneeuutX7xu3759tXjxYrVt21ZRUVHKzc3VSy+9dMH3P9cdd9yh7t27a9y4cdqzZ4/i4+O1du1azZkzR88+++wvvs8AAKDu6E9dOPpTNXXt2lVHjx7Vhg0b1KVLlxrH4uPjFRYWpueee06SVFZWpm7duqlv37566aWXNHbsWN1zzz3av3+/XnrpJW3fvl1du3a9qDqOHj2q9evXa9q0aZf8TIDHcM26+ACc6f333zd33323ad68uQkICDDNmjUzd911l5k/f74pLy+3nbdlyxbTu3dvExwcbK644grTs2dPs2XLFtvx4uJi8+tf/9okJiaa4OBg06xZM3PHHXeYTz75pMb9Fi5caFq3bm0aNGhgoqKizP79+89bY1FRkfH39ze33HJLrWObNm0yQ4YMMW3atDFBQUHm2muvNaNGjTLff/+97ZyfdgSytwPPueztHnT8+HEzYsQI07RpU9OoUSMzcuRI27GdO3eaESNGmBYtWpgGDRqYli1bmnvuucd88MEHtnOioqLM8OHDa92rtLTUDBs2zISGhprw8HAzaNAg89ZbbxlJZsGCBee9v72ddo4fP24ee+wxExkZaRo0aGCsVqt5+eWXa9z3p92Dfl6jMcY8/fTThn/6AQCoO/pTNdGfurD+VFRUlBk1apTdY6tXrzYJCQkmMDDQREREmC+++MIYY8yrr75qoqKiTKNGjUzHjh3NvHnzzC233FLjc6nLbozz5s0zFovFFBYWXlDNgDewGMNKxQAAAAAA1Lc5c+boN7/5jb766ivFxsY6/f4nTpxQ+/bt1blzZ6Wnpzv9/oCrEHYBAAAAAHAZnDlzRklJSWratKnWrl0rf39/p97/N7/5jV5//XVt27ZNrVu3duq9AVdigXoAAAAAAC6DgIAAvf322/riiy/00EMPOfXeL7/8sl5++WXNnTuXoAs+h5FdAAAAAAAA8BqM7AIAAAAAAIDXCHB1ARerurpa1dXVNdosFossFouLKgIAAJ7MGKNzB7z7+fnJz8813w0WFBRo9OjROnDggMLDwzV//ny1a9fO7rmVlZV64IEHlJqaqkceecTWvnPnTj300EMqKirSmTNnNHLkSP3hD3+QJO3fv18PP/yw8vPz5efnp//6r//S//t//0+StGfPHsXExCgmJsZ2rW7duum1115TdXW1nnrqKa1cuVLV1dWKi4vT3Llz1aJFC9u59NMAAEB9qms/zaPDrvLycleXAQAAvFhwcLDLwq6BAwdq2rRp6tevn9atW6dhw4bpyy+/rHXe2rVrNWLECJWWlio1NbXGsfHjx+uBBx7QyJEjVVZWpptvvlnJycnq3r27Ro8erbvvvluZmZk6fPiwbr31ViUmJuq2226TJIWFhSk/P7/W/ZYtW6YvvvhCX331lRo0aKAnn3xSkyZN0uLFi23n0E8DAACX2y/105jGCAAA4Gb27t2rgwcPql+/fpKkHj166OjRoyosLKx1bs+ePVVcXKy0tLRaxyoqKnTgwAFJUpMmTRQcHKzAwEBJ0saNGzV48GBJUtOmTTV27FgtX778vLVVVFTo6NGjqqiokCRFRkbargkAAOAOCLsAAADczL59+2pMC5SkiIgIFRUV1ek68+fP16xZs9S5c2elpaWpb9++SklJkSS1b99eS5culTFGVVVVOnz4sEpLS22vLSsrU2xsrOLj4/Xoo4/awq2RI0eqZcuWiomJ0UMPPaSVK1fqhRdeuMQnBgAAqD+EXQAAAG7I39+/xu/GGFVWVtbpGjNnztTEiROVnp4uq9WqjIwM7dq1S5KUkZGhdevWKS4uTqmpqdqwYYMiIiIkSS1btlR5ebl27Nihzz77TEVFRZoyZYok6aOPPtKRI0f0+eefq0uXLvrhhx/09ttvX/oDAwAA1BOPXbPL3gKnrlxXAwAAeDZ760y5akH1yMhIFRcX12grKSlRZGTkBV+jtLRU6enpOnHihCRp6tSpOnjwoJYsWaLJkycrKipKq1evtp3fv39/dejQQdLZoO2nsC00NFSDBw9Wenq6JOmll17SuHHj1LJlS913331q06aN0tLSNGHCBNu16KcBAID6VNd+mleFXa7cMQkAAHgfV4Vd0dHRCg8PV1ZWlu6++27l5OTIYrEoISFBhw4d0okTJ3Tdddf94jXCwsIUEhKizMxMDRw4UOXl5dq2bZt69uwpSaqqqrIFWnPnzlVhYaGGDBkiSdq8ebMiIyN1zTXX6OTJk1q+fLlt8Xur1aq3335b99xzjwIDA/Xxxx8rMTGxxr3ppwEAgMvtl/pp9DgAAADcUGZmpl544QVZrVY9+eSTWrFihSwWi2bPnq0RI0bYzlu/fr2SkpKUlZWlGTNmKCkpSWVlZfLz89OqVas0a9YsxcXFqVOnThowYIAGDRokSdq0aZOioqLUpk0b5ebmKjs7W0FBQZLOLpDfs2dPxcTE6MYbb1RMTIwee+wxSdKf/vQnhYaGKiEhQW3bttXGjRs1d+5cp78/AAAAjliMMcbVRVyM6upqHTt2rEZbSEgI3xgCAICLQt+i/vBeAgCA+lTXvgU9DgAAAAAAAHgNwi4AAAAAAAB4DcIuAAAAAAAAeA3CLgAAAAAAAHgNwi4AAAAAAAB4DcIuAAAAAAAAeA3CLgAAAAAAAHgNwi4AAAAAAAB4jQBXF+CuzlRLxtRss1ikAOJBAAAAt2CMdPKMdOKMdLpaigh2dUUAAMAdEHY5kJwpbTlYsy01QvpogEvKAQAAwL/dv1Z6d/fZkOsnrUOlncNdVxMAAHAfjFMCAACARzljagZdUu3fAQCA7yLsAgAAgEe5ws7chPLTzq8DAAC4J8IuAAAAeBR7YdeJM7XXWwUAAL6JsAsAAAAeJdhO2FVlzi5SDwAAQNhVB3xbCAAA4Hr2RnZJrNsFAADOIuxywOLqAgAAAGCXo7CLdbsAAIBE2AUAAAAPc0UD++2M7AIAABJhFwAAADyMvTW7JMIuAABwFmEXAAAAPIrDaYyEXQAAQIRdAAAA8DAsUA8AAH4JYVcdsBkjAACA6wU7WrOLBeoBAIAIuxyysB0jAACAW2JkFwAA+CWEXQAAAPAorNkFAAB+CWEXAAAAPAojuwAAwC8h7AIAAIBHcbhmF2EXAACQi8Ou3/3ud2rTpo3i4uLUtm1bLV++XJJUUlKiPn36yGq16oYbbtD69etdWSYAAADciMNpjCxQDwAA5OKw684771ReXp6+/fZbvf322xo7dqyqq6s1duxY9enTRwUFBVq2bJmGDx+uU6dOubJUSezGCAAA4A4a+dtvZ2QXAACQXBx29e7dWw0bNpQk5eXlKTExUdXV1Vq3bp3GjBkjSWrXrp1iYmK0YcMGp9bGZowAAADuqYG/1MBOL5awCwAASJKDQeDOk5+fr+7duyswMFCrVq3SgQMHFBgYqODgYNs5ERERKioqcmGVAAAAcCfBAdKRyppthF0AAEBygwXq4+PjVVxcrMzMTN111106ffq0/P1rjk03xqiystLBFQAAAOBrrrCzSD1rdgEAAMkNwq6f3HTTTWrcuLFKSkp08uRJlZeX246VlJQoMjLShdUBAAA4V0FBgVJSUmS1WpWcnKzt27c7PLeyslJDhw7V7Nmza7Tv3LlTvXr1UkJCgqxWq6ZNm2Y7tn//fg0cOFAJCQlKTEzUnDlzbMf27NmjgIAAxcfH234mTJhgO/7jjz/qoYceUkxMjKKjo/XBBx/U45NfGHuL1DOyCwAASC4Mu06ePKm33npLVVVVkqQNGzaotLRU7dq1U7du3bRgwQJJ0jfffKO8vDylpqa6qlQAAACnGzhwoCZNmqSCggI9//zzGjZsmN3z1q5dq+joaK1cubLWsfHjx2v48OHKy8vT5s2btWjRImVnZ0uSRo8erR49eigvL08bNmzQiy++qNzcXNtrw8LClJ+fb/t57bXXJElnzpzRHXfcoVatWqmgoEB79uxR9+7dL8M78MuCCbsAAIADLgu7LBaLXn/9dbVu3VpxcXGaOHGiMjMz1bhxY82bN0+rVq2S1WpVWlqaMjIyFBoa6qpSbQzbMQIAACfYu3evDh48qH79+kmSevTooaNHj6qwsLDWuT179lRxcbHS0tJqHauoqNCBAwckSU2aNFFwcLACAwMlSRs3btTgwYMlSU2bNtXYsWO1fPny89aWlZWlkJAQPfHEE/LzO9uVPHcJCmdgZBcAAHDEZQvUBwUF6R//+IfdY5GRkVqzZo2TK6qJ3RgBAICr7Nu3Ty1atKjR9tOGPTExMRd8nfnz56tr165655131KpVK/Xt21cpKSmSpPbt22vp0qX67W9/q+rqah0+fFilpaW215aVlSk2Nlb+/v7q3bu3pk+frqCgIG3dulWnTp1SamqqDhw4oNatW+vll19WbGxs/Tz8BbIXdrFmFwAAkNxozS4AAAD8R31s2DNz5kxNnDhR6enpslqtysjI0K5duyRJGRkZWrduneLi4pSamqoNGzYoIiJCktSyZUuVl5drx44d+uyzz1RUVKQpU6ZIkoqLi5WcnKycnBzl5+drwIABGjJkyKU/cB0xsgsAADhC2AUAAOBmIiMjVVxcXKOtrhv2lJaWKj09XY8//rjatGmjqVOnqlevXlqyZIkkKSoqSqtXr1ZBQYE+/vhjhYWFqUOHDpLOBm0/TXcMDQ3V4MGDlZeXJ0lq1qyZwsPDFRAQIIvFovvvv19ff/21qqur6+PRL1iwnd0YCbsAAIBE2AUAAOB2oqOjFR4erqysLElSTk6OLBaLEhISdOjQIe3du/e81wgLC1NISIgyMzMlSeXl5dq2bZsSExMlybZJkCTNnTtXhYWFthFamzdvVlFRkaSzmwotX77ctllQ//79tWDBAttaYO+++65uvfVW2/pdzmJ3GiNhFwAAEGFXnbA+PQAAcJbMzEy98MILslqtevLJJ7VixQpZLBbNnj1bI0aMsJ23fv16JSUlKSsrSzNmzFBSUpLKysrk5+enVatWadasWYqLi1OnTp00YMAADRo0SJK0adMmRUVFqU2bNsrNzVV2draCgoIknV0gv2fPnoqJidGNN96omJgYPfbYY5Kkzp07a/LkyerWrZsSEhL0xhtvaNmyZU5/fxxNY2RDIQAAYDHGM7sE1dXVOnbsWI22kJCQevtW8ZZM6dMfarYlXy19MrBeLg8AANzM5e5b+BJnvJdPfCJN31q7vWK81ND5m0MCAIDLqK59C3pvAAAA8Dj21uySWLcLAAAQdgEAAMAD2ZvGKEnlp51bBwAAcD+EXQAAAPA4jsIuRnYBAADCLgAAAHicYMIuAADgAGFXHXjmUv4AAADe5woHa3YxjREAABB2OWBxdQEAAABwiGmMAADAEcIuAAAAeBzCLgAA4AhhFwAAADwOa3YBAABHCLsAAADgcRyN7GLNLgAAQNgFAAAAj+NogXpGdgEAAMKuOmAzRgAAAPfgaBpjOWEXAAA+j7DLAQvbMQIAALitYAcju5jGCAAACLsAAADgcRyt2XWcsAsAAJ9H2AUAAACP42exP5WRsAsAABB2AQAAwCM1tjOV8ThrdgEA4PMIuwAAAOCR7IZdjOwCAMDnEXbVAbsxAgAAuA97i9QTdgEAAMIuB9iMEQAAwL3ZG9nFbowAAICwCwAAAB6JaYwAAMAewi4AAAB4pMbsxggAAOwg7AIAAIBHYmQXAACwh7CrDgwr1AMAALgNu2HXGfpsAAD4OsIuB1igHgAAwL3Z243xTLVUWe38WgAAgPsg7AIAAIBHsjeyS2JHRgAAfB1hFwAAADySo7CLdbsAAPBthF0AAADwSIRdAADAHsIuAAAAeKTGAfbbCbsAAPBthF11wMY+AAAA7oORXQAAwB7CLgcsbMcIAADg1uztxigRdgEA4OsIuwAAANxQQUGBUlJSZLValZycrO3btzs8t7KyUkOHDtXs2bNrtO/cuVO9evVSQkKCrFarpk2bZju2f/9+DRw4UAkJCUpMTNScOXNsx/bs2aOAgADFx8fbfiZMmFDrvlOmTFGzZs3q4WkvDiO7AACAPQ5WOgAAAIArDRw4UNOmTVO/fv20bt06DRs2TF9++WWt89auXasRI0aotLRUqampNY6NHz9eDzzwgEaOHKmysjLdfPPNSk5OVvfu3TV69GjdfffdyszM1OHDh3XrrbcqMTFRt912myQpLCxM+fn5Dut788039fe//71+H7qOHIVd5WecWwcAAHAvjOwCAABwM3v37tXBgwfVr18/SVKPHj109OhRFRYW1jq3Z8+eKi4uVlpaWq1jFRUVOnDggCSpSZMmCg4OVmBgoCRp48aNGjx4sCSpadOmGjt2rJYvX35B9X3++eeaPn26Fi5ceDGPV28Y2QUAAOwh7AIAAHAz+/btU4sWLWq0RUREqKioqE7XmT9/vmbNmqXOnTsrLS1Nffv2VUpKiiSpffv2Wrp0qYwxqqqq0uHDh1VaWmp7bVlZmWJjYxUfH69HH31UFRUVkqSioiKNGTNGb775pkJDQy/xSS8NYRcAALCHsKsODNsxAgAAJ/H396/xuzFGlZWVdbrGzJkzNXHiRKWnp8tqtSojI0O7du2SJGVkZGjdunWKi4tTamqqNmzYoIiICElSy5YtVV5erh07duizzz5TUVGRpkyZIkm699579corr6h169aX/pCXiLALAADY49Kw68EHH1RMTIzi4+OVkpJiW3h1ypQpuvLKK2ssivrPf/7TqbWxGSMAAHCVyMhIFRcX12grKSlRZGTkBV+jtLRU6enpevzxx9WmTRtNnTpVvXr10pIlSyRJUVFRWr16tQoKCvTxxx8rLCxMHTp0kHQ2aPtpumNoaKgGDx6svLw8SWcXrx8/frzi4+PVo0cPHTlyRPHx8XUedVYfAv0kfzudNsIuAAB8m0vDrv79+ys/P1/5+fkaPXq0xo8fbzs2atQo27H8/Hz17t3bhZUCAAA4T3R0tMLDw5WVlSVJysnJkcViUUJCgg4dOqS9e/ee9xphYWEKCQlRZmamJKm8vFzbtm1TYmKiJKmqqsp27ty5c1VYWKghQ4ZIkjZv3mwLr06ePKnly5fbFr/fv3+/rX+2bt0620L211xzTf29ARfIYrE/uouwCwAA3+bSsGvgwIEKCDi7IWTHjh1d8o0gAACAO8rMzNQLL7wgq9WqJ598UitWrJDFYtHs2bM1YsQI23nr169XUlKSsrKyNGPGDCUlJamsrEx+fn5atWqVZs2apbi4OHXq1EkDBgzQoEGDJEmbNm1SVFSU2rRpo9zcXGVnZysoKEjS2QXye/bsqZiYGN14442KiYnRY4895oq34bzshV3sxggAgG+zGOMeK1H9+te/lsVi0V//+ldNmTJFr7zyisLDw9W0aVM9/vjjtm8af1JdXa1jx47VaAsJCZGfX/3kd11XSB/tr9l2UzNpy+B6uTwAAHAzl7tv4Uuc+V7GZ0jfHqnZdkdL6f276/1WAADAReratwhwRlHn89prryk3N1cbNmyQJD3xxBO2RVC3bdumnj17KjExUQkJCS6sEgAAAO6GaYwAAOBcLv+qcsaMGZo/f77Wrl1r2766UaNGtuPXX3+9OnbsqPz8fFeVaOMWQ+AAAABgQ9gFAADO5bKwq6qqSg8//LDWrVunnJwcNW/e3HZs9erVtq218/LytHXrVnXq1Mmp9VnYjhEAAMDtBduZp0DYBQCAb3PZNMbvv/9er776qmJiYmoEWQsWLFBmZqYmTJigRo0aqXHjxpo/f75atmzpqlIBAADgphjZBQAAzuWysCs6OlqO1sa/5ZZbnFwNAAAAPBG7MQIAgHO5fM0uAAAA4GLZDbtOS9UstgoAgM8i7KoD+kwAAADuxV7YZSSdZHQXAAA+i7DLAdanBwAAcH/2wi5JOsa6XQAA+CzCLgAAAHisEEdhV6Vz6wAAAO6DsAsAAAAeKzTQfvtRRnYBAOCzCLsAAADgsRjZBQAAzkXYBQAAAI/lcGQXYRcAAD6LsKsODNsxAgAAuBVHI7uYxggAgO8i7HKA3RgBAADcn6ORXUxjBADAdxF2AQAAwGOxQD0AADgXYRcAAAA8FgvUAwCAcxF2AQAAwGMFN7C//AQjuwAA8F2EXQAAAPBYfhYpxM5URnZjBADAdxF21QGbMQIAALgfe1MZmcYIAIDvIuxywMJ2jAAAAB7B3iL1TGMEAMB3EXYBAADAozGyCwAA/BxhFwAAADwaI7sAAMDPEXYBAADAo4XaGdnFAvUAAPguwi4AAAB4NHu7MR5jZBcAAD6LsKsODNsxAgAAuB170xiPVdJ3AwDAVxF2OcBmjAAAAJ7B3jRGI6n8jNNLAQAAboCwCwAAAB7N3jRGiXW7AADwVYRdAAAA8Gj2RnZJhF0AAPgqwi4AAAB4NEcju1ikHgAA30TYBQAAAI/GyC4AAPBzhF11wIY+AADAWQoKCpSSkiKr1ark5GRt377d4bmVlZUaOnSoZs+eXaN9586d6tWrlxISEmS1WjVt2jTbsf3792vgwIFKSEhQYmKi5syZYzu2Z88eBQQEKD4+3vYzYcIESdLXX3+tW2+9VbGxsYqNjdWECRN05oxrV4K3txujxMguAAB8FWGXAxa2YwQAAC40cOBATZo0SQUFBXr++ec1bNgwu+etXbtW0dHRWrlyZa1j48eP1/Dhw5WXl6fNmzdr0aJFys7OliSNHj1aPXr0UF5enjZs2KAXX3xRubm5tteGhYUpPz/f9vPaa69Jkho2bKi//vWv2rFjh7Zv364dO3ZowYIFl+EduHAsUA8AAH6OsAsAAMDN7N27VwcPHlS/fv0kST169NDRo0dVWFhY69yePXuquLhYaWlptY5VVFTowIEDkqQmTZooODhYgYFnk6GNGzdq8ODBkqSmTZtq7NixWr58+Xlri42N1Y033ihJCgwMVPv27VVUVHRxD1pPmMYIAAB+jrALAADAzezbt08tWrSo0RYREVHnUGn+/PmaNWuWOnfurLS0NPXt21cpKSmSpPbt22vp0qUyxqiqqkqHDx9WaWmp7bVlZWWKjY1VfHy8Hn30UVVUVNS6/pEjR/T222/rrrvuuoinrD9MYwQAAD9H2AUAAOCG/P39a/xujFFlZd2GKs2cOVMTJ05Uenq6rFarMjIytGvXLklSRkaG1q1bp7i4OKWmpmrDhg2KiIiQJLVs2VLl5eXasWOHPvvsMxUVFWnKlCk1rn3y5Endc889GjNmjG6++eaLf9B6EMLILgAA8DMBri7AkxhWqAcAAE4QGRmp4uLiGm0lJSWKjIy84GuUlpYqPT1dJ06ckCRNnTpVBw8e1JIlSzR58mRFRUVp9erVtvP79++vDh06SDobtP0UtoWGhmrw4MFKT0+3nXv48GENGDBAXbp0qRWCuUKjAMnfIlWd01cj7AIAwDcxsssB1qcHAACuEh0drfDwcGVlZUmScnJyZLFYlJCQoEOHDmnv3r3nvUZYWJhCQkKUmZkpSSovL9e2bduUmJgoSaqqqrKdO3fuXBUWFmrIkCGSpM2bN9umTJ48eVLLly9XamqqJGn37t3q0qWL0tLS9Nxzz9XfQ18Ci8X+6C6mMQIA4JsIuwAAANxQZmamXnjhBVmtVj355JNasWKFLBaLZs+erREjRtjOW79+vZKSkpSVlaUZM2YoKSlJZWVl8vPz06pVqzRr1izFxcWpU6dOGjBggAYNGiRJ2rRpk6KiotSmTRvl5uYqOztbQUFBks4ukN+zZ0/FxMToxhtvVExMjB577DFJ0qJFi7R792699NJLio+PV3x8vG6//XZnvz212Fu3i5FdAAD4Josxnjk5r7q6WseOHavRFhISIj+/+snveq+SPthXs61dU+nr2hsdAQAAL3C5+xa+xBXvZYc3pK9Ka7bdHinl9L9stwQAAE5S174FvTcAAAB4vCZ2RnYdYWQXAAA+ibALAAAAHi+sYe22slPOrwMAALgeYVcdeOR8TwAAAB9gL+xiZBcAAL6JsMsBC9sxAgAAeIwwO9MYyyqlar6tBADA5xB2AQAAwOPZW7Or2kjHTzu/FgAA4FouDbsefPBBxcTEKD4+XikpKdq+fbskqaSkRH369JHVatUNN9yg9evXu7JMAAAAuDl70xils6O7AACAb3Fp2NW/f3/l5+crPz9fo0eP1vjx4yVJY8eOVZ8+fVRQUKBly5Zp+PDhOnWKFUYBAABgn71pjJJ0hC4kAAA+x6Vh18CBAxUQECBJ6tixo4qKinTmzBmtW7dOY8aMkSS1a9dOMTEx2rBhgytLBQAAgBtzNLKLsAsAAN/jNmt2zZkzR3379tWBAwcUGBio4OBg27GIiAgVFRW5sLqzWN8UAADAPdlbs0tiGiMAAL4owNUFSNJrr72m3NxcbdiwQcePH5e/v3+N48YYVVY6t6fCZowAAACew+HILsIuAAB8jsvDrhkzZuiNN97Q2rVrFRoaqkaNGunkyZMqLy+3je4qKSlRZGSkiysFAACAu2LNLgAA8BOXTWOsqqrSww8/rHXr1iknJ0fNmzeXJDVo0EDdunXTggULJEnffPON8vLylJqa6qpSAQAA4OYcTWNkZBcAAL7HZSO7vv/+e7366quKiYlRp06dbO0LFizQvHnzNGrUKL388stq1KiRMjIyFBoa6qpSAQAA4OaaOJjGWMbILgAAfI7Lwq7o6GgZ43jJ9zVr1jixGgAAAHiyhv5SowDp5Jma7YzsAgDA97jNboye4BeyOQAAALiYvXW7WLMLAADfQ9jlALsxAgAAeBZ763aVMbILAACfQ9gFAAAArxBmZ90upjECAOB7CLsAAADgFZjGCAAAJMIuAAAAeAm7I7sIuwAA8DmEXXXA+vQAAADuy9GaXWwyBACAbyHscsDCCvUAAAAexd7IrspqqaLK+bUAAADXIewCAACAV7C3ZpfEVEYAAHwNYRcAAAC8gr1pjNLZqYwAAMB3EHYBAADAK9ibxigxsgsAAF9D2AUAAACv4HAaIyO7AADwKYRddcBOPgAAAO7L0ciuw4zsAgDApxB2OcBmjAAAAJ4l3EHYVUrYBQCATyHsAgAAgFcID7LfXlrh3DoAAIBrEXYBAADAKzRlZBcAABBhFwAAgFsqKChQSkqKrFarkpOTtX37dofnVlZWaujQoZo9e3aN9p07d6pXr15KSEiQ1WrVtGnTbMf279+vgQMHKiEhQYmJiZozZ47t2J49exQQEKD4+Hjbz4QJE2zH//a3v9muOW7cOJ065R5pUoCfFGpnkXpGdgEA4FsIuwAAANzQwIEDNWnSJBUUFOj555/XsGHD7J63du1aRUdHa+XKlbWOjR8/XsOHD1deXp42b96sRYsWKTs7W5I0evRo9ejRQ3l5edqwYYNefPFF5ebm2l4bFham/Px8289rr70mSfr88881ffp05ebm6ttvv9WpU6dqhWyuZG/dLkZ2AQDgWwi76oDNGAEAgDPs3btXBw8eVL9+/SRJPXr00NGjR1VYWFjr3J49e6q4uFhpaWm1jlVUVOjAgQOSpCZNmig4OFiBgWeHPm3cuFGDBw+WJDVt2lRjx47V8uXLz1vbe++9p0GDBqlp06ayWCwaPXq03n333Yt+1vpG2AUAAAi7HLCwHSMAAHCRffv2qUWLFjXaIiIiVFRUVKfrzJ8/X7NmzVLnzp2Vlpamvn37KiUlRZLUvn17LV26VMYYVVVV6fDhwyotLbW9tqysTLGxsYqPj9ejjz6qiooKu7VdTF2Xk71F6pnGCACAbyHsAgAAcEP+/v41fjfGqLKysk7XmDlzpiZOnKj09HRZrVZlZGRo165dkqSMjAytW7dOcXFxSk1N1YYNGxQRESFJatmypcrLy7Vjxw599tlnKioq0pQpU+zWdjF1XU6M7AIAABcVdk2dOlVHjx6t1b59+3b93//93yUXBQAA4Inqq48UGRmp4uLiGm0lJSWKjIy84GuUlpYqPT1djz/+uNq0aaOpU6eqV69eWrJkiSQpKipKq1evVkFBgT7++GOFhYWpQ4cOks6GWT9NdwwNDdXgwYOVl5dnt7a61nW5OQq7DOtRAADgMy4q7PrTn/6kI0eO1GqvrKzUE088cak1AQAAeKT66iNFR0crPDxcWVlZkqScnBxZLBYlJCTo0KFD2rt373mvERYWppCQEGVmZkqSysvLtW3bNiUmJkqSqqqqbOfOnTtXhYWFGjJkiCRp8+bNtqmJJ0+e1PLly5WamipJuuuuu/TWW2+prKxMxhgtXLhQd9111wU/2+VmbxrjmWrp2Gnn1wIAAFzjosIuY4wsdha12rp1q+1bQAAAAF9Tn32kzMxMvfDCC7JarXryySe1YsUKWSwWzZ49WyNGjLCdt379eiUlJSkrK0szZsxQUlKSysrK5Ofnp1WrVmnWrFmKi4tTp06dNGDAAA0aNEiStGnTJkVFRalNmzbKzc1Vdna2goLOJkV79+5Vz549FRMToxtvvFExMTF67LHHJEk333yzHn/8cSUnJysuLk7+/v566qmnLvIdq3/2RnZJrNsFAIAvsRhz4YO6W7VqJYvFou+++07XXHONAgICbMdOnTqlkpISPfXUU/rv//7vy1Lsz1VXV+vYsWM12kJCQuTnVz/LkN39nrT6u5ptMU2kHffVy+UBAICbuZS+hTv1kdzB5e6n/ZKF+dKonNrtW+6Vbmp+2W8PAAAug7r2LQLstjrwzDPPyBij8ePH65FHHtGVV15pOxYUFKSEhATdeOONF1G2+2EzRgAAcKF8qY/k7hyO7GKRegAAfEadwq4xY8ZIknr16qWWLVs65ds5AAAAd0cfyX3YW7NLYhojAAC+5KJ6YtnZ2frNb34j6ezaFCNHjlRwcLC6dOmi77//vl4LBAAA8BT0kVyPkV0AAOCiwq4XX3zRtpNPVlaWMjMz9be//U0tW7bUb3/723otEAAAwFPQR3I9RnYBAIA6TWP8yc6dO3XLLbdIktauXavevXtr+PDh6tChg7p3716vBbqTC1/KHwAA+CJf7SO5k6aM7AIAwOdd1Miu5s2b6/Dhw5KknJwc3XbbbZLOro5fVVVVf9UBAAB4EPpIrtfQXwq283UuYRcAAL7jokZ2PfDAA3rggQcUGxurnTt3asiQIZKkv//972rbtm29Fugq7MYIAADqyhf6SJ4gPEgqP16z7UemMQIA4DMuKuz605/+pGuvvVZffvmlpkyZomuuuUZVVVUqLCzU448/Xt81AgAAeAT6SO4hvKH0/TlhF2t2AQDgOyzGeOZKVNXV1Tp27FiNtpCQkHrb6rvfe9Kq72q2tQmVCofXy+UBAICbudx9C1/i6vey+0opp7hmW0JTaXuaU24PAADqWV37Fhfd4/jyyy81YsQIJSUlqVOnThoxYoS2bdt2sZcDAADwCvSRXM/ejoys2QUAgO+4qLBrxYoV6tSpk06fPq3hw4dr2LBhOn36tDp37qx33323vmt0Gx45BA4AADiNr/aR3E24nR0ZD1WwszYAAL7iotbsevrpp/XnP/9Zjz32WI32v/zlL3r66ad1zz331EdtLmVhhXoAAFBHvtBH8gTNG9VuO1MtHa2UmtgJwgAAgHe5qJFdO3fuVK9evWq19+7dW7t27brkogAAADwRfST30NzONEZJOsgi9QAA+ISLCrtat26t7OzsWu3r1q1Tq1atLrkoAAAAT0QfyT3YG9klSQdPOrcOAADgGhc1jXHq1Km6//779dlnnykpKUmStHnzZr311ltatmxZvRYIAADgKegjuYdmjkZ2EXYBAOATLmpk17333qvc3FxVV1dr0aJFWrRokYwx+vjjj3XvvffW6VrHjx9XSkqKsrKybG1TpkzRlVdeqfj4eNvPP//5z4spFQAAwGnqs4+Ei+dwZBfTGAEA8Al1Crs++ugjderUSbt27VLHjh21bNkyffHFF/riiy80c+ZM/eEPf9CWLVsu+HoLFy5UTEyMNm3aVOvYqFGjlJ+fb/vp3bt3XUq9LNjBBwAA2FPffSRcGqYxAgDg2+oUdr366qvq0KGDWrduXetYRESEYmNj9corr1zw9UaOHKmSkhLddtttdSnDKdiMEQAAXKj67iPh0jhaoP4QI7sAAPAJdQq7Nm/erOHDhzs8PnjwYK1bt+6Si5KkBQsWKDY2VjfffLPefPPNerkmAADA5eDMPhLOLyhAatygdjsjuwAA8A11CruKiop07bXXOjweERGhAwcOXHJRTzzxhH788Uft2LFDc+fO1cMPP6y8vLxLvi4AAMDl4Kw+Ei6cvdFdrNkFAIBvqFPY1axZM+3cudPh8fz8fIWHh19yUY0a/Wehheuvv14dO3ZUfn7+JV8XAADgcnBWHwkXzt66XYzsAgDAN9Qp7Ordu7f+8pe/qLq6utaxM2fO6LnnnlP37t0vuajVq1ersrJSkpSXl6etW7eqU6dOl3xdAACAy8FZfSRcuGaM7AIAwGfVKeyaPHmytmzZotTUVGVkZOhf//qXvv32W73xxhu64YYbVFBQoMmTJ1/w9TIyMpSUlKQtW7bot7/9rbp06SJJyszMVExMjOLi4vTAAw9o/vz5atmyZd2e7DJgM0YAAGBPffeRcOnsjew6xMguAAB8QkBdTo6KitLHH3+sRx99VPfff7+t3RijlJQULVu2TFar9YKvN2zYMA0bNqxW+/z58+tS1mVhYTtGAABwgeq7j4RLZ2/NrvIz0skzUqM69YABAICnqfN/6uPi4rRmzRodPnxYhYWFMsaoVatWat68+eWoDwAAwCPQR3Iv9kZ2SWfX7bouxLm1AAAA57ro77WaNm3KOloAAADnoI/kHhyGXRWEXQAAeLs6rdkFAAAAeAJ70xgldmQEAMAXEHYBAADA6/zSNEYAAODdCLvqgN0YAQAAPEMzByO7DlU4tw4AAOB8hF0OsBkjAACA52JkFwAAvouwCwAAwA0VFBQoJSVFVqtVycnJ2r59u8NzKysrNXToUM2ePbtG+86dO9WrVy8lJCTIarVq2rRptmP79+/XwIEDlZCQoMTERM2ZM8futUeNGqWkpCTb79XV1XriiScUHx8vq9WqX/3qVyopKbnEp61/IQ2khv612w8QdgEA4PUIuwAAANzQwIEDNWnSJBUUFOj555/XsGHD7J63du1aRUdHa+XKlbWOjR8/XsOHD1deXp42b96sRYsWKTs7W5I0evRo9ejRQ3l5edqwYYNefPFF5ebm1nj99OnTtXXr1hpty5Yt0xdffKGvvvpKBQUFateunSZNmlQ/D12PLBapxRW120sIuwAA8HqEXQAAAG5m7969OnjwoPr16ydJ6tGjh44eParCwsJa5/bs2VPFxcVKS0urdayiokIHDhyQJDVp0kTBwcEKDAyUJG3cuFGDBw+WJDVt2lRjx47V8uXLba9dtWqV1qxZo+nTp9e65tGjR1VRcXbxq8jISNs13U0LO1MZS044vw4AAOBchF11YFihHgAAOMG+ffvUokWLGm0REREqKiqq03Xmz5+vWbNmqXPnzkpLS1Pfvn2VkpIiSWrfvr2WLl0qY4yqqqp0+PBhlZaWSpK+/PJLPfvss3rrrbfUoEGDGtccOXKkWrZsqZiYGD300ENauXKlXnjhhUt42svH7sguwi4AALweYZcDLFAPAABcyd+/5oJTxhhVVlbW6RozZ87UxIkTlZ6eLqvVqoyMDO3atUuSlJGRoXXr1ikuLk6pqanasGGDIiIidPz4cY0YMUIZGRkKDw+vdc2PPvpIR44c0eeff64uXbrohx9+0Ntvv33xD3oZ2Qu7fjgpVfMFJgAAXi3A1QUAAACgpsjISBUXF9doKykpUWRk5AVfo7S0VOnp6Tpx4uxQpqlTp+rgwYNasmSJJk+erKioKK1evdp2fv/+/dWhQwcdPnxYhw4d0oABAyRJJ06c0MGDBxUfH6/8/Hy99NJLGjdunFq2bKn77rtPbdq0UVpamiZMmHDpD17P7IVdZ6ql0gqpmYPdGgEAgOdjZBcAAICbiY6OVnh4uLKysiRJOTk5slgsSkhI0KFDh7R3797zXiMsLEwhISHKzMyUJJWXl2vbtm1KTEyUJFVVVdnOnTt3rgoLCzVkyBC1bNlS+/btU35+vvLz87V48WK1a9dO+fn5kiSr1aq3337bNsrs448/tl3T3dgLuySmMgIA4O0IuwAAANxQZmamXnjhBVmtVj355JNasWKFLBaLZs+erREjRtjOW79+vZKSkpSVlaUZM2YoKSlJZWVl8vPz06pVqzRr1izFxcWpU6dOGjBggAYNGiRJ2rRpk6KiotSmTRvl5uYqOztbQUFB563rT3/6k0JDQ5WQkKC2bdtq48aNmjt37mV7Hy6Fw7CLHRkBAPBqFmM8c9n16upqHTt2rEZbSEiI/PzqJ78b9L6Uubtm23WNpe8eqJfLAwAAN3O5+xa+xF3ey09LpFverd2+pId0v9WppQAAgEtQ174Fvbc68MhUEAAAwEcxjREAAN9E2OWAhe0YAQAAPNrVhF0AAPgkwi4AAAB4pUYBUpPA2u2EXQAAeDfCLgAAAHgte1MZCbsAAPBuhF0AAADwWoRdAAD4HsIuAAAAeC27YddJ59cBAACch7CrDgzbMQIAAHiUFo1qt/1YIVVWOb8WAADgHIRdDrAZIwAAgOezN7JLkg4wugsAAK9F2AUAAACv5SjsKi53bh0AAMB5CLsAAADgta4Jtt9eRNgFAIDXIuwCAACA13IUdu0j7AIAwGsRdgEAAMBrXdvYfvu+486tAwAAOA9hVx2wGSMAAIBnCQ2UQhrUbmcaIwAA3ouwywEL2zECAAB4BXtTGZnGCACA9yLsAgAAgFezN5WRaYwAAHgvwi4AAAB4NXsju4rKJcMaFQAAeCXCLgAAAHi1a+2EXRVVUukp59cCAAAuP8KuOuDLPwAAAM/DjowAAPgWwi4HWJ8eAADAO9ibxiixIyMAAN6KsAsAAABezd40RokdGQEA8FaEXQAAAPBqTGMEAMC3EHYBAADAqzULkgLt9HqZxggAgHci7AIAAIBXs1jsr9vFNEYAALwTYVcdGLZjBAAA8Ej2pjLuPeb8OgAAwOXn8rDr+PHjSklJUVZWlq2tpKREffr0kdVq1Q033KD169c7vS4L2zECAAB4jevshF3fHefLTAAAvJFLw66FCxcqJiZGmzZtqtE+duxY9enTRwUFBVq2bJmGDx+uU6dOuahKAAAAeLrokNptJ89IB086vxYAAHB5uTTsGjlypEpKSnTbbbfZ2s6cOaN169ZpzJgxkqR27dopJiZGGzZscFWZAAAA8HD2wi5J2sNURgAAvI7LpzGe68CBAwoMDFRw8H9WEY2IiFBRUZELqwIAAIAniw613/7dcefWAQAALj+3C7skyd/fv8bvxhhVVla6qBoAAAB4Oocju446tw4AAHD5uV3Y1bx5c508eVLl5f/ZC7qkpESRkZEurOos1i8FAADwTC0bS/b2H2IaIwAA3sftwq4GDRqoW7duWrBggSTpm2++UV5enlJTU51aB5sxAgAAVyooKFBKSoqsVquSk5O1fft2h+dWVlZq6NChmj17do32nTt3qlevXkpISJDVatW0adNsx/bv36+BAwcqISFBiYmJmjNnjt1rjxo1SklJSTXafvzxRz300EOKiYlRdHS0Pvjgg0t4Uudo6C9FBtduJ+wCAMD7uDTsysjIUFJSkrZs2aLf/va36tKliyRp3rx5WrVqlaxWq9LS0pSRkaHQUAcLLQAAAHihgQMHatKkSSooKNDzzz+vYcOG2T1v7dq1io6O1sqVK2sdGz9+vIYPH668vDxt3rxZixYtUnZ2tiRp9OjR6tGjh/Ly8rRhwwa9+OKLys3NrfH66dOna+vWrTXazpw5ozvuuEOtWrVSQUGB9uzZo+7du9fPQ19m9qYyEnYBAOB9XBp2DRs2TJ9//rmOHTumHTt26KOPPpIkRUZGas2aNSooKNC2bdvUo0cPV5YJAADgVHv37tXBgwfVr18/SVKPHj109OhRFRYW1jq3Z8+eKi4uVlpaWq1jFRUVOnDggCSpSZMmCg4OVmBgoCRp48aNGjx4sCSpadOmGjt2rJYvX2577apVq7RmzRpNnz69xjWzsrIUEhKiJ554Qn5+Z7uS56636q6i7IRd3x2TDGtVAADgVdxuGiMAAICv27dvn1q0aFGj7WJ2p54/f75mzZqlzp07Ky0tTX379lVKSookqX379lq6dKmMMaqqqtLhw4dVWloqSfryyy/17LPP6q233lKDBg1qXHPr1q06deqUUlNTFRcXpz59+mjHjh2X8LTOY29kV/kZ6ccK59cCAAAuH8IuAAAAN1Qfu1PPnDlTEydOVHp6uqxWqzIyMrRr1y5JZ5eTWLduneLi4pSamqoNGzYoIiJCx48f14gRI5SRkaHw8PBa1ywuLlZycrJycnKUn5+vAQMGaMiQIRf/oE7kcEdGpjICAOBVAlxdgCdhiDsAAHCGyMhIFRcX12ir6+7UpaWlSk9P14kTJyRJU6dO1cGDB7VkyRJNnjxZUVFRWr16te38/v37q0OHDjp8+LAOHTqkAQMGSJJOnDihgwcPKj4+Xvn5+WrWrJmuuOIKBQSc7Ubef//9euSRR1RdXW2b1uiufinsSrrKubUAAIDLx717JC7EbowAAMBVoqOjFR4erqysLElSTk6OLBaLEhISdOjQIe3du/e81wgLC1NISIgyMzMlSeXl5dq2bZsSExMlSVVVVbZz586dq8LCQg0ZMkQtW7bUvn37lJ+fr/z8fC1evFjt2rVTfn6+pLOh2IIFC2xrgb377ru69dZb3T7okhyHXbuOOrcOAABwebl/rwQAAMAHZWZm6oUXXpDVatWTTz6pFStWyGKxaPbs2RoxYoTtvPXr1yspKUlZWVmaMWOGkpKSVFZWJj8/P61atUqzZs1SXFycOnXqpAEDBmjQoEGSpE2bNikqKkpt2rRRbm6usrOzFRQUdN66OnfurMmTJ6tbt25KSEjQG2+8oWXLll2296E+RYVIfna+0Swsc34tAADg8rEY45mT86qrq3XsWM0FFkJCQurtW8X7PpAyztnw6OpGUsnIerk8AABwM5e7b+FL3Pm9bL1U2n3OGl23R0o5/V1TDwAAOL+69i1c3+MAAAAAnCQ2rHbbDkZ2AQDgVQi7AAAA4DNim9RuKyqXTpx2fi0AAODyIOyqA4+c7wkAAACbmFD77TtZpB4AAK9B2OWAhe0YAQAAvI69aYwSUxkBAPAmhF0AAADwGfamMUqEXQAAeBPCLgAAAPiM6BDJz84I/kLCLgAAvAZhFwAAAHxGoP/ZwOtcjOwCAMB7EHbVgWGFegAAAI9nbyojYRcAAN6DsMsB1qcHAADwTvbCruJy6fhp59cCAADqH2EXAAAAfIo1zH57/mGnlgEAAC4Twi4AAAD4lISm9tvzCLsAAPAKhF0AAADwKY7Cru2lzq0DAABcHoRdAAAA8CktrpCaNqzdzsguAAC8A2FXHbAZIwAAgOezWOyP7iLsAgDAOxB2OWBhO0YAAACvZS/s2n1UOsGOjAAAeDzCLgAAAPgce2GXkZR/xNmVAACA+kbYBQAAAJ+TEG6/namMAAB4PsIuAAAA+Jx27MgIAIDXIuwCAACAz4kMlkIDa7d/RdgFAIDHI+yqA3ZjBAAA8A4Wi5RoZyrj1kPOrwUAANQvwi4H2IwRAADAu93YrHZbUbl08KTzawEAAPWHsAsAAAA+yV7YJTG6CwAAT0fYBQAAAJ/kKOz6F2EXAAAejbALAAAAPqlduBRgpzdM2AUAgGcj7AIAAIBPaugvJTSt3c40RgAAPBthVx0YtmMEAADwKvamMn57RCo/7fRSAABAPSHscoDdGAEAALzfDVfWbjOStv3o9FIAAEA9IewCAACAz7qpuf32z35wbh0AAKD+EHYBAADAZ93UXPKzM6T/0wPOrwUAANQPwi4AAAA3VFBQoJSUFFmtViUnJ2v79u0Oz62srNTQoUM1e/bsGu07d+5Ur169lJCQIKvVqmnTptmO7d+/XwMHDlRCQoISExM1Z84cu9ceNWqUkpKS7B6bMmWKmjWzs+iVB2ncQGofXrv9U0Z2AQDgsQi76oD16QEAgLMMHDhQkyZNUkFBgZ5//nkNGzbM7nlr165VdHS0Vq5cWevY+PHjNXz4cOXl5Wnz5s1atGiRsrOzJUmjR49Wjx49lJeXpw0bNujFF19Ubm5ujddPnz5dW7dutXvfN998U3//+98v7SHdRPLVtdu+OybtL3d+LQAA4NIRdjlgYYV6AADgInv37tXBgwfVr18/SVKPHj109OhRFRYW1jq3Z8+eKi4uVlpaWq1jFRUVOnDg7Hy8Jk2aKDg4WIGBgZKkjRs3avDgwZKkpk2bauzYsVq+fLnttatWrdKaNWs0ffr0Wtf9/PPPNX36dC1cuPCSn9Ud3GIn7JKkTUxlBADAIxF2AQAAuJl9+/apRYsWNdoiIiJUVFRUp+vMnz9fs2bNUufOnZWWlqa+ffsqJSVFktS+fXstXbpUxhhVVVXp8OHDKi0tlSR9+eWXevbZZ/XWW2+pQYMGNa5ZVFSkMWPG6M0331RoaOglPKX7sDeyS5I+KXFuHQAAoH4QdgEAALghf3//Gr8bY1RZWVmna8ycOVMTJ05Uenq6rFarMjIytGvXLklSRkaG1q1bp7i4OKWmpmrDhg2KiIjQ8ePHNWLECGVkZCg8vPZiVvfee69eeeUVtW7d+uIfzs3EhklNG9ZuZ90uAAA8U4CrC3Bk5MiRev/99xUWFmZre+edd9SuXTvXFQUAAOAEkZGRKi4urtFWUlKiyMjIC75GaWmp0tPTdeLECUnS1KlTdfDgQS1ZskSTJ09WVFSUVq9ebTu/f//+6tChgw4fPqxDhw5pwIABkqQTJ07o4MGDio+PV35+vvbs2aPx48dLkk6fPq0jR44oPj5e69at0zXXXHOJT+4afhap81XS+9/XbP/sgHSqSmrob/91AADAPbn1yK5nnnlG+fn5th+CLgAA4Auio6MVHh6urKwsSVJOTo4sFosSEhJ06NAh7d2797zXCAsLU0hIiDIzMyVJ5eXl2rZtmxITEyVJVVVVtnPnzp2rwsJCDRkyRC1bttS+ffts/a/FixerXbt2ys/Pl3R2F8efjq1bt05hYWHKz8/32KDrJ7e0qN1WUSVtYnQXAAAex63DLndj2I4RAAA4SWZmpl544QVZrVY9+eSTWrFihSwWi2bPnq0RI0bYzlu/fr2SkpKUlZWlGTNmKCkpSWVlZfLz89OqVas0a9YsxcXFqVOnThowYIAGDRokSdq0aZOioqLUpk0b5ebmKjs7W0FBQa56XJe73cGguZy6LZMGAADcgMUY94xwfprGGBISooiICE2ePFk9evSwHa+urtaxY8dqvCYkJER+fvWT343KlhZ+W7MtLFA6PKZeLg8AANzM5e5b+BJPfC9PVUlh88+O5vq5rpHSh/1dUxMAADirrn0Lt12z67XXXlOjRo0kSdnZ2brnnnv03Xff1VjDCwAAAKgPDf2l21pI684ZyfVJiXTyjNTIbXvNAADgXG779dpPQZckde/eXREREbbdgwAAAID61t3OsmOV1WcDLwAA4DncNuxauXKlqqurJUkfffSRjh49qvj4eBdXBQAAAG/VzcEa+9ms2wUAgEdx27Dr1VdfVXR0tOLj4/XUU08pMzNTV1xxhavLAgAAgJdKai4F25mu+I/zb34JAADciNuuPrBmzRpXl1CLW67kDwAAgHrRwP/s6K6s72q2f3FI2l8uRQS7pi4AAFA3bjuyy9UsFldXAAAAAGfrG2W//T1GdwEA4DEIuwAAAIB/cxR2nTvaCwAAuC/CLgAAAODfWjaWOlxZu/2D76VTVc6vBwAA1B1hFwAAAPAzfa+r3VZ+Rsre5/xaAABA3RF2AQAAAD9zd7T99jd2OrUMAABwkQi76oDdGAEAALxf56ukiCtqt6/YzVRGAAA8AWGXA2zGCAAA4Jv8/aTBbWq3l1VKa9iVEQAAt0fYBQAAAJxjaIz99uWFzq0DAADUHWEXAAAAcI7kq8/uzHiulXukslNOLwcAANQBYRcAAABwDj+LNMTOVMYTZ6QMRncBAODWCLvqwLBCPQAAgM94MM5++7xvnFsHAACoG8IuB1igHgAAwLe1v/Lszozn2nJQ+tdB59cDAAAuDGEXAAAA4MC4BPvtf/3auXUAAIALR9gFAAAAODA0RmrcoHb7kgKp5ITz6wEAAOdH2AUAAAA40LiB9IC1dntltfTKV86vBwAAnB9hFwAAAPALftvB/nqur34tHat0ejkAAOA8CLvqgM0YAQAAfE9smHRPq9rtRyqllxjdBQCA2yHscsDCdowAAAD4t4k32G+f/i/p0EmnlgIAAM6DsAsAAAA4j+QWUrfI2u3HTkvTvnB+PQAAwDHCLgAAAOACvJBsv33219I3h51bCwAAcIywCwAAALgAna+2v3bX6WrpofWSYYFXAADcAmEXAAAAcIGe7yz521nb9aP90oJ859cDAABqI+xywN769NV8WwcAAODT2jaVftvB/rHf5EqFZc6tBwAA1EbY5YCfnbSLsAsAAABTOknXNa7dfvy0dN9aqbLK+TUBAID/IOxywN7w9CrCLgAAAJ8X3ECanWr/2OYD0sMbWL8LAABXIuxywN/OO0PYBQAAnKWgoEApKSmyWq1KTk7W9u3bHZ5bWVmpoUOHavbs2TXad+7cqV69eikhIUFWq1XTpk2zHdu/f78GDhyohIQEJSYmas6cOXavPWrUKCUlJdl+//rrr3XrrbcqNjZWsbGxmjBhgs6cOXOJT+t5fhUtjWtr/9i8b6RZ25xaDgAA+BnCLgfsjeyqNnxLBwAAnGPgwIGaNGmSCgoK9Pzzz2vYsGF2z1u7dq2io6O1cuXKWsfGjx+v4cOHKy8vT5s3b9aiRYuUnZ0tSRo9erR69OihvLw8bdiwQS+++KJyc3NrvH769OnaunVrjbaGDRvqr3/9q3bs2KHt27drx44dWrBgQf08tIf5y21SfJj9YxM/kebmObUcAADwb4RdDtgLuyTW7QIAAJff3r17dfDgQfXr10+S1KNHDx09elSFhYW1zu3Zs6eKi4uVlpZW61hFRYUOHDggSWrSpImCg4MVGBgoSdq4caMGDx4sSWratKnGjh2r5cuX2167atUqrVmzRtOnT69xzdjYWN14442SpMDAQLVv315FRUX18NSeJ7iB9EZvKTjA/vHx66X/czwgDwAAXCaEXQ44CruYyggAAC63ffv2qUWLFjXaIiIi6hwqzZ8/X7NmzVLnzp2Vlpamvn37KiUlRZLUvn17LV26VMYYVVVV6fDhwyotLZUkffnll3r22Wf11ltvqUGDBg6vf+TIEb399tu666676viE3qPDlVJGL/s7eUvShI+kJz7hC1MAAJyJsMsBwi4AAOBK/v7+NX43xqiysrJO15g5c6YmTpyo9PR0Wa1WZWRkaNeuXZKkjIwMrVu3TnFxcUpNTdWGDRsUERGh48ePa8SIEcrIyFB4eLjDa588eVL33HOPxowZo5tvvrnuD+hFfhV9dkqjI9O3Sne/J/1wwlkVAQDg2xwMuoa9Beolwi4AAHD5RUZGqri4uEZbSUmJIiMjL/gapaWlSk9P14kTZxOWqVOn6uDBg1qyZIkmT56sqKgorV692nZ+//791aFDBx0+fFiHDh3SgAEDJEknTpzQwYMHFR8fr/z8fEnS4cOHNWDAAHXp0kVTpky5tIf1Er/pIJ04I/1hk/3j/9grtX9DerWLNKi1ZHE0FAwAAFwyRnY54HBkV7Vz6wAAAL4nOjpa4eHhysrKkiTl5OTIYrEoISFBhw4d0t69e897jbCwMIWEhCgzM1OSVF5erm3btikxMVGSVFVVZTt37ty5Kiws1JAhQ9SyZUvt27dP+fn5ys/P1+LFi9WuXTtb0LV792516dJFaWlpeu655+r70T3aUzdJL3R2fPxghTT4n1LPVdK2Q86rCwAAX0PY5QDTGAEAgCtlZmbqhRdekNVq1ZNPPqkVK1bIYrFo9uzZGjFihO289evXKykpSVlZWZoxY4aSkpJUVlYmPz8/rVq1SrNmzVJcXJw6deqkAQMGaNCgQZKkTZs2KSoqSm3atFFubq6ys7MVFBR03roWLVqk3bt366WXXlJ8fLzi4+N1++23X663weM8eZO0uLsU+Au97Owi6Ya3pH7vSbn72e0bAID6ZjHGM//zWl1drWPHjtVoCwkJkZ9f/eR307ZIT39Wu/3ASKl5o3q5BQAAcCOXu2/hS3gvpQ3F0pAPpJILWKcroak0Ol4aEiO1bHz5awMAwNPUtW/hOz2OOmLNLgAAAFys1EjpqyHSgFbnPzfvsPT4J9J1S6Qb35L++Jm0dp90rG77EQAAgH9jgXoHWLMLAAAAl6JZIynzDmnZDmnSJ9L+CxjltfXQ2Z//3iL5WaTrr5RuaCa1ayq1Cz87CuyaYMdfzAIAAMIuh1izCwAAAJfKYpHut0r9o6VpX0gvf3V218YLUW2kfx06+/Nz/hbp2sZSy2DpuhCpxRXSlQ2lK4OkZkFn/wwPkho3kIIDpOAG0hUBZ8MzAAB8AWGXA47CrmFrpTahZ4/7W852Gn7602KRfnrZT9tJW+SgrQ7HHbXBc7C9uOfhIwPc14g4KaaJq6sA6iYkUHohWfr99dJLX0mzv5KOXOQ0xSojfXfs7I9KLvx1jQL+E3419JMa+EkN/P/9p9/ZRfUbnPMT4Pfvfq7O/vnzv9tt+/fvfqr9d0d+6b+5F/26+r7eRb4OAHzNvW2kDle6ugrCLoccDQ3/uOTsDwAAcI3UCMIueK5mjaTnbpb+cJOUuUt6Pf/s7ozOcPLM2Z9DFc65HwDA98SHuUfY5baz/QsKCpSSkiKr1ark5GRt377dqfcPbeDU2wEAAMCHNAqQhluldf2kohHS37pKv4o6O90QAABcGrf9z+nAgQM1bdo09evXT+vWrdOwYcP05ZdfOu3+na5y2q0AAADgwyKDpXEJZ39OV0lbf5Ry90u5JdK2H6WdR8+u3wUAAC6MW4Zde/fu1cGDB9WvXz9JUo8ePXT06FEVFhYqJibGKTXEN5XuaSW9u9sptwMAAADUwP/sl66drpIeu/5sW8UZKf+ItL1U2nNM2ntc2vvvP78/Lh077dKSAQBwO24Zdu3bt08tWrSo0RYREaGioiKnhV2StKi71GGblLHjbCei2pxdEPSnP3/+92oj/fSFm3HwdwAAAKCuggKkG5qd/bHnVJVUWnF2La4f//1z+JRUfkYqP23/z9PVZ38qq/7z93N/Kqulquqzfdmf+rrVpubfjZGqHfydPjAAwFXcMuySJH9//xq/G2NUWXmRW9ZcpJBAaUqnsz/1xfz7v/rnC8bMOeee+zoAlw//PwPcW5D/+c8BfElDfyki+OyPuzH//mL4l447PPZL1z3PPevrNZdSBwD4ooZusjK8W4ZdkZGRKi4urtFWUlKiyMhIF1VUf37atti2RTF7FQMAAMBLWSxSAP1dAICTuUnmVlN0dLTCw8OVlZUlScrJyZHFYlFCQoKLKwMAAAAAAIA7c8uRXZKUmZmpMWPG6He/+52aNm2qFStWyGLhayEAAAAAAAA45rZhV3x8vHJzc11dBgAAAAAAADyIW05jBAAAAAAAAC4GYRcAAAAAAAC8BmEXAAAAAAAAvAZhFwAAAAAAALwGYRcAAAAAAAC8htvuxng+xphabdXV1S6oBAAAeAN7/Qh7/Q2cH/00AABQn+raT/OqsKu8vNwFlQAAAG9F2HVx6KcBAIDL7Zf6aUxjBAAAAAAAgNcg7AIAAAAAAIDXIOwCAAAAAACA17AYD12Morq6utYCZRaLRRaLxUUVAQAAT2aMqbX2g5+fn/z8+G6wruinAQCA+lTXfprHhl0AAAAAAADAufiqEgAAAAAAAF6DsOsXFBQUKCUlRVarVcnJydq+fburS/IaDz74oGJiYhQfH6+UlBTbe1tSUqI+ffrIarXqhhtu0Pr1622vOX78uO677z5ZrVYlJibqzTfftB07ffq0HnnkEVmtVrVt21Yvv/yy05/JU3399dcKDQ1VVlaWJD4DZ5s3b55uuOEGxcbGauzYsZL4DJxpzZo16tixo9q2bavrr79eq1atksRn4AzHjx9XSkqK7d8e6fK878YYTZ06VfHx8YqLi9Mf/vCHX9ymGrhc6Pt4HvpInoU+leeg/+X+vKKfZuBQu3btzMqVK40xxqxdu9a0b9/exRV5j3feececPn3aGGPM/Pnzza233mqMMaZv377mpZdeMsYY8/XXX5trrrnGVFRUGGOMefjhh83vfvc7Y4wxxcXF5pprrjH79+83xhgzY8YMM2jQIFNVVWWOHj1q2rZtaz7//HNnP5bHOXDggLn++utNq1atzKpVq4wxfAbO9Je//MV06dLFlJSUGGOMOXPmjDGGz8BZTp48aUJDQ82OHTuMMcZ89dVX5oorrjDl5eV8BpfZggULzNVXX20CAgJs//YYc3n+t//WW2+Z5ORkU1FRYU6fPm26detm3n77bWc+LmCMoe/jaegjeRb6VJ6D/pf785Z+GmGXA99995256qqrarRFRUXZ/k+J+rN161YTFRVlTp8+bYKCgszx48dtx7p27Wo++OADY4wxrVq1Mnl5ebZjDz74oJk7d64xxphu3bqZ9957z3Zs8uTJ5umnn3bSE3imU6dOmS5duph//OMfpmvXrmbVqlV8Bk505swZ06xZM7Nnz54a7XwGznP06FETFBRkPvnkE2OMMaWlpSYsLMzWzmdw+f30b48xl+9/+6NGjTKvvvqq7diCBQvM8OHDL99DAReAvo97o4/kWehTeRb6X57D0/tpTGN0YN++fWrRokWNtoiICBUVFbmoIu81Z84c9e3bVwcOHFBgYKCCg4Ntx37+np/7mVzoMdg3btw4DRo0SHfeeaetjc/Aeb7//ntVVlbq8ccfV2Jioq6//notXryYz8CJQkJCtGTJEnXt2lX9+/fX4MGDtWzZMh07dozPwAUu1//2+Uzgjuj7uDf6SJ6FPpVnof/lmTyxnxZQL1fxUv7+/jV+N8aosrLSRdV4p9dee025ubnasGGDjh8/ft73/OfH63IMNb3yyiu64oor9Oijj9Y6xmfgHMXFxWrevLn++te/6qqrrtKuXbuUmpqqmJgYPgMnOXHihGbNmqV//vOfatiwoWbPnq0///nPWrx4MZ+Bi1yu953PBO6Evo97o4/keehTeRb6X57L0/ppjOxyIDIyUsXFxTXaSkpKFBkZ6aKKvM+MGTM0f/58rV27VqGhoWrevLlOnjyp8vJy2zk/f8/P/Uwu9BhqKygo0Lp16xQfH6/4+Hh99tlnmjBhgt555x0+Aydp1qyZJOmqq66SJLVu3Vq33nqrduzYwWfgJO+//77CwsLUtWtXJScna+nSpTpw4ID27NnDZ+ACl+u/AXwmcCf0fdwffSTPQ5/Ks9D/8kwe2U+rl8mQXqpt27a2OarZ2dkmOjraVFdXu7gqz3fmzBnz61//2txxxx3m6NGjNY716dPHvPLKK8YYY/Ly8kzz5s1NWVmZMcaYCRMmmN///vfGGGP2799vIiMjbWuo/fnPfzb33nuvqa6uNsePHzcJCQkmOzvbiU/l2X4+H5vPwDmqq6tNQkKCeeONN4wxxpSUlJjo6GiTn5/PZ+Ak27ZtM82aNTMFBQXGGGN27txprr76arNv3z4+Ayf5+b89xlyef3/eeOMNc8stt5jKykpz+vRp06NHD/P666878zEB+j4ejD6S+6NP5Vnof3kOT++nEXb9gm+++cbceuutJjY21tx8881m69atri7JK+zevdtIMjExMSYuLs728/HHH5uioiLTu3dvExsbazp06GDWrl1re11ZWZkZOnSoiY2NNW3btjXp6em2Y6dOnTITJkwwsbGxxmq1mpkzZ7ri0TzWz/8h4zNwnoKCAtO9e3cTHx9vbrjhBvPuu+8aY/gMnGnRokUmISHBxMfHm44dO5rVq1cbY/gMLrf09HTTsWNH07hxYxMTE2NSU1ONMZfnfa+urjbPPvussVqtJjY21kyaNMlUVVU572EBQ9/Hk9FH8gz0qTwL/S/35i39NIsxxtTPGDEAAAAAAADAtVizCwAAAAAAAF6DsAsAAAAAAABeg7ALAAAAAAAAXoOwCwAAAAAAAF6DsAsAAAAAAABeg7ALAAAAAAAAXoOwCwAAAAAAAF6DsAsAAAAAAABeg7ALwCWJjo7WM888I0lau3atLBaLU+77zDPP6Pbbb6/R9uGHH8pisaiwsNApNZxr3rx5io6Odsm9AQAAzkU/7T/opwG+JcDVBQDwbBs3blTjxo1dXYYkKTk5Wbt379a1117r6lIAAABcjn4aAF/FyC4Al+T+++/XzJkzJUm9evWSJFksFlksFo0cOVKSdOLECT388MNq3ry5wsLC9Ktf/Urff/+97RrR0dH64x//qKFDhyo8PFyDBg3Sxx9/rPbt2ys0NFSNGzfWzTffrA8++EDS2W8Gn3/+ea1fv952r4ULF6qkpEStWrXSnj17bNd+9dVX1apVKzVs2FBJSUn66KOPbMd++oZx2bJlSkpKUkhIiHr16qWSkhKHz1tdXa1nn31W0dHRCg4OVocOHTR//nzt2bNH48aN03fffWeracqUKZKkgwcPavjw4QoLC1Pz5s31wAMPqLS01HbNn85NSUlRUFCQ2rdvry1btlzS5wIAAEA/jX4a4KsIuwDUm6VLl0qSdu/erd27d9s6Vw899JDy8/O1cuVKffjhh2rQoIGGDh1a47WvvPKKevbsqU8//VT/8z//I39/f40bN045OTnatGmTOnbsqEGDBqm0tFTJycl65JFH1LlzZ9u97r333lr1ZGZm6oknntCUKVO0efNm9e7dW3fddZf2799f47y//e1vmjlzpnJyclRcXKxnn33W4TPOmzdPr7/+ul5//XXl5uZqwoQJ+uKLL3Tttdfqf/7nf3TNNdfYanrsscckSffee6+qqqqUnZ2trKws7dmzRw8//HCN67755pv6zW9+o08//VTx8fG67777ZIyp82cAAABgD/00+mmATzEAcAm6du1qnn76aWOMMR988IE595+V3bt3m8DAQFNaWmpr27dvn5FkvvvuO2OMMVFRUeb//u//fvE+R44cMZLMhx9+aIwx5umnnzZdu3atdS9JZseOHcYYY2655Rbzhz/8ocY5ycnJtnpzcnKMJFNZWWk7Pm3aNNOhQweHdfz61782t99+u91jc+fONVFRUTXa1q9fb1q0aFHjHhs3bjQNGjQwZ86cMcYYI8l88MEHtuM//PCDsVgs5tNPP3VYBwAAwPnQT/sP+mmAb2FkF4DLavv27aqsrFRERISCgoIUFBSkNm3aSJL27t1rO8/f37/G606cOKFp06bptttuU8uWLRUVFSVJOn369AXf+5tvvlFSUlKNtk6dOikvL69G288Xa23WrJnKysocXnPo0KH69NNP1b59ez366KP6+9//rurqaofnf/XVV/rhhx8UEhJie/7u3bvr9OnTtb65/MlVV12lZs2a1RjmDwAAUN/op9FPA7wVC9QDuKyOHz+uRo0a6Ysvvqh17LrrrnP4uvvvv1+ff/65nnjiCXXo0EGhoaG64YYbLrkeY8wv7kR0vl2KunTpooKCAr355pvKzc3Vfffdp4EDB2rx4sV2zz9+/Lhat26trKysWseuvvpqh/eprKxUo0aNfrEWAACAS0E/jX4a4K0IuwDUm6CgIEnSqVOn1LBhQ0lSYmKiTp48qePHj9f69u6XvP/++1qyZIkGDRrk8F6nTp36xWu0bdtWmzdv1j333GNr27x5s3r27HnBdZzrzJkzatmypX7/+9/r97//vZYuXapx48Zp8eLFdmtKTEzU999/r9DQUEVGRl7QPb799luVlZWpXbt2F10nAADAz9FPo58G+BLCLgD1JjY2VgEBAZoxY4Z+9atf6fTp00pKStLdd9+tYcOGafr06UpISNDu3bs1Z84cLViwQGFhYXavFRcXp3feeUdt27ZVSUmJ5s+fX+N4QkKCnnvuOa1atUotWrTQlVdeKT+/mjOzf//732vkyJGKi4vTTTfdpIyMDH311VfKzMy86GccPXq0YmJi1K9fP/n7++u9997TTTfdZKuppKREr7/+um644QYFBgbqzjvvVEJCgvr166fnn39e0dHRysvL06JFi7RixQrbdX96jrKyMv3ud7/TkCFDbNMIAAAALhX9NPppgC9hzS4A9ebqq6/Wiy++qNmzZ+uWW27RO++8I0lavny57r77bj3yyCPq0KGDHnnkEV133XUKDg52eK0FCxbo22+/1U033aRHH3201rd8/fv31/Dhw3X//ferd+/e+vrrr2tdY9CgQZo2bZomT56sjh076v3339fq1asv+Js7e/r3769//OMfuv3223X77beroqJC6enpkqSbbrpJTz31lCZOnKiuXbvqo48+kr+/v/75z3+qffv2uv/++3XDDTfo6aef1o033ljjutu3b1fPnj3Vu3dvxcfH1+o0AgAAXAr6afTTAF9iMYY9UwHAlSwWiz744INLGrYPAACA+kc/DfBMjOwCAAAAAACA1yDsAgAAAAAAgNdgGiMAAAAAAAC8BiO7AAAAAAAA4DUIuwAAAAAAAOA1CLsAAAAAAADgNQi7AAAAAAAA4DUIuwAAAAAAAOA1CLsAAAAAAADgNQi7AAAAAAAA4DUIuwAAAAAAAOA1/j+O7rH25bOHpwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1200x400 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Plot cost vs iteration to see how well the algorithm is performing\n",
    "# plot cost versus iteration  \n",
    "fig, (ax1, ax2) = plt.subplots(1, 2, constrained_layout=True, figsize=(12,4))\n",
    "ax1.plot(J_hist)\n",
    "ax2.plot(1000 + np.arange(len(J_hist[1000:])), J_hist[1000:])\n",
    "ax1.set_title(\"Cost vs. iteration\");  ax2.set_title(\"Cost vs. iteration (tail)\")\n",
    "ax1.set_ylabel('Cost')            ;  ax2.set_ylabel('Cost') \n",
    "ax1.set_xlabel('iteration step')  ;  ax2.set_xlabel('iteration step') \n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "534c5328",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The predicted value is off by 6.696004010796237 percent\n"
     ]
    }
   ],
   "source": [
    "#We can also make a prediction based on model. X=5 should produce something very close to 10 as the prediction\n",
    "test_data = 5\n",
    "print(f\"The predicted value is off by {((w_final*test_data + b_final)-10)/10*100} percent\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01819df9",
   "metadata": {},
   "source": [
    "# Summary\n",
    "Using only basic mathematical functions, this script can use gradient descent to find the optimal solution for a univariate regression. There are some important limitations with this code. First, there is no flexibility in the model structure and it cannot accomodate multiple features. Second, there is no threshold set for when to stop the gradient descent algorithm. It just goes until the maximum number of iterations is reached. Third, it's incredibly inefficient code when there are plenty of libraries that can perform this task in a single line. Yet, it's a pretty neat exercise."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}