Navigation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Navigation\n",
    "\n",
    "---\n",
    "\n",
    "This notebook contains the front end code for my navigation project of the [Deep Reinforcement Learning Nanodegree](https://www.udacity.com/course/deep-reinforcement-learning-nanodegree--nd893).\n",
    "\n",
    "Nothing between here and section 4.1 has substantially changed from what was assigned, in either the markdown, or the code.  The assignment starts in section 4.1 with training the agent.\n",
    "\n",
    "### 1. Start the Environment\n",
    "\n",
    "We begin by importing some necessary packages.  If the code cell below returns an error, please revisit the project instructions to double-check that you have installed [Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Installation.md) and [NumPy](http://www.numpy.org/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from unityagents import UnityEnvironment\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will start the environment!  **_Before running the code cell below_**, change the `file_name` parameter to match the location of the Unity environment that you downloaded.\n",
    "\n",
    "- **Mac**: `\"path/to/Banana.app\"`\n",
    "- **Windows** (x86): `\"path/to/Banana_Windows_x86/Banana.exe\"`\n",
    "- **Windows** (x86_64): `\"path/to/Banana_Windows_x86_64/Banana.exe\"`\n",
    "- **Linux** (x86): `\"path/to/Banana_Linux/Banana.x86\"`\n",
    "- **Linux** (x86_64): `\"path/to/Banana_Linux/Banana.x86_64\"`\n",
    "- **Linux** (x86, headless): `\"path/to/Banana_Linux_NoVis/Banana.x86\"`\n",
    "- **Linux** (x86_64, headless): `\"path/to/Banana_Linux_NoVis/Banana.x86_64\"`\n",
    "\n",
    "For instance, if you are using a Mac, then you downloaded `Banana.app`.  If this file is in the same folder as the notebook, then the line below should appear as follows:\n",
    "```\n",
    "env = UnityEnvironment(file_name=\"Banana.app\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:unityagents:\n",
      "'Academy' started successfully!\n",
      "Unity Academy name: Academy\n",
      "        Number of Brains: 1\n",
      "        Number of External Brains : 1\n",
      "        Lesson number : 0\n",
      "        Reset Parameters :\n",
      "\t\t\n",
      "Unity brain name: BananaBrain\n",
      "        Number of Visual Observations (per agent): 0\n",
      "        Vector Observation space type: continuous\n",
      "        Vector Observation space size (per agent): 37\n",
      "        Number of stacked Vector Observation: 1\n",
      "        Vector Action space type: discrete\n",
      "        Vector Action space size (per agent): 4\n",
      "        Vector Action descriptions: , , , \n"
     ]
    }
   ],
   "source": [
    "env_path = \"C:\\\\Users\\\\scotth\\\\Google Drive\\\\deep-reinforcement-learning\\\\p1_navigation\\\\Banana_Windows_x86_64\\\\Banana.exe\"\n",
    "env = UnityEnvironment(file_name=env_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Environments contain **_brains_** which are responsible for deciding the actions of their associated agents. Here we check for the first brain available, and set it as the default brain we will be controlling from Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the default brain\n",
    "brain_name = env.brain_names[0]\n",
    "brain = env.brains[brain_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Examine the State and Action Spaces\n",
    "\n",
    "The simulation contains a single agent that navigates a large environment.  At each time step, it has four actions at its disposal:\n",
    "- `0` - walk forward \n",
    "- `1` - walk backward\n",
    "- `2` - turn left\n",
    "- `3` - turn right\n",
    "\n",
    "The state space has `37` dimensions and contains the agent's velocity, along with ray-based perception of objects around agent's forward direction.  A reward of `+1` is provided for collecting a yellow banana, and a reward of `-1` is provided for collecting a blue banana. \n",
    "\n",
    "Run the code cell below to print some information about the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of agents: 1\n",
      "Number of actions: 4\n",
      "States look like: [1.         0.         0.         0.         0.84408134 0.\n",
      " 0.         1.         0.         0.0748472  0.         1.\n",
      " 0.         0.         0.25755    1.         0.         0.\n",
      " 0.         0.74177343 0.         1.         0.         0.\n",
      " 0.25854847 0.         0.         1.         0.         0.09355672\n",
      " 0.         1.         0.         0.         0.31969345 0.\n",
      " 0.        ]\n",
      "States have length: 37\n"
     ]
    }
   ],
   "source": [
    "# reset the environment\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "\n",
    "# number of agents in the environment\n",
    "print('Number of agents:', len(env_info.agents))\n",
    "\n",
    "# number of actions\n",
    "action_size = brain.vector_action_space_size\n",
    "print('Number of actions:', action_size)\n",
    "\n",
    "# examine the state space \n",
    "state = env_info.vector_observations[0]\n",
    "print('States look like:', state)\n",
    "state_size = len(state)\n",
    "print('States have length:', state_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Take Random Actions in the Environment\n",
    "\n",
    "In the next code cell, you will learn how to use the Python API to control the agent and receive feedback from the environment.\n",
    "\n",
    "Once this cell is executed, you will watch the agent's performance, if it selects an action (uniformly) at random with each time step.  A window should pop up that allows you to observe the agent, as it moves through the environment.  \n",
    "\n",
    "Of course, as part of the project, you'll have to change the code so that the agent is able to use its experience to gradually choose better actions when interacting with the environment!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# env_info = env.reset(train_mode=False)[brain_name] # reset the environment\n",
    "# state = env_info.vector_observations[0]            # get the current state\n",
    "# score = 0                                          # initialize the score\n",
    "# while True:\n",
    "#     action = np.random.randint(action_size)        # select an action\n",
    "#     env_info = env.step(action)[brain_name]        # send the action to the environment\n",
    "#     next_state = env_info.vector_observations[0]   # get the next state\n",
    "#     reward = env_info.rewards[0]                   # get the reward\n",
    "#     done = env_info.local_done[0]                  # see if episode has finished\n",
    "#     score += reward                                # update the score\n",
    "#     state = next_state                             # roll over the state to next time step\n",
    "#     if done:                                       # exit loop if episode finished\n",
    "#         break\n",
    "    \n",
    "# print(\"Score: {}\".format(score))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When finished, you can close the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#env.close()  #commented out because it doesn't appear possible to re instantiate an environment below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. It's Your Turn!\n",
    "\n",
    "Now it's your turn to train your own agent to solve the environment!  When training the environment, set `train_mode=True`, so that the line for resetting the environment looks like the following:\n",
    "```python\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "```\n",
    "\n",
    "Nothing above here is substantially changed from what was provided in the lesson.  The code below and the two python files model.py and dqn_agent.py are the code either written by me or borrowed from previous lessons."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.1\n",
    "Here we train the network.  There are two supporting files, model.py and dqn_agent.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "device = cuda:0\n",
      "<bound method Module.parameters of QNetwork(\n",
      "  (fc1): Linear(in_features=37, out_features=81, bias=True)\n",
      "  (fc2): Linear(in_features=81, out_features=69, bias=True)\n",
      "  (fc3): Linear(in_features=69, out_features=4, bias=True)\n",
      ")>\n",
      "Episode 287\t elapsed: 0:14:56.768321\tAve. Score: 13.04 \tscore 19.0 \tscores >= 13: 54 \teps.: 0.03terminating at episode : 287 ave reward reached +13 over 100 episodes\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x13ff634c390>]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsvXmcZFldJ/o9d4k1syqzKquqq9eqXtiapZtuWhFRkBEaGQcXYGQUGUcH3/B8M/je8w06I6LOKKMiMqMPbIQRcEdwQERbaZbuhm6gGnqvbrq7KnupNbMq14iMuNuZP+79nXvOuefGlpmVGVXn+/nUJ7MyIu49cTPje773+9sY5xwWFhYWFucvnK1egIWFhYXF5sISvYWFhcV5Dkv0FhYWFuc5LNFbWFhYnOewRG9hYWFxnsMSvYWFhcV5Dkv0FhYWFuc5LNFbWFhYnOfoS/SMscsYY19kjB1mjD3EGPsP2c/fzRg7xhi7N/v3A5u/XAsLCwuLYcH6VcYyxvYD2M85/yZjbBLAPQB+CMCbAKxyzn9n0JPNzMzwAwcOrGO5FhYWFhce7rnnnnnO+Z5RX+/1ewLn/ASAE9n3K4yxwwAuGeVkBw4cwKFDh0Z5qYWFhcUFC8bYk+t5/VAePWPsAIDrAXwt+9HPMcbuZ4x9hDE2vZ6FWFhYWFhsDgYmesbYBIBPAngH53wZwAcAXAXgOqSK/70lr3sbY+wQY+zQ3NzcBizZwsLCwmIYDET0jDEfKcn/Kef8UwDAOT/FOY855wmADwG4yfRazvktnPMbOec37tkzssVkYWFhYTEiBsm6YQA+DOAw5/x3pZ/vl572wwAe3PjlWVhYWFisF32DsQBeBuAtAB5gjN2b/eyXALyZMXYdAA5gFsDPbsoKLSwsLCzWhUGybu4EwAwPfW7jl2NhYWFhsdGwlbEWFhYW5zks0VtYjDlOLnVw2+FTW70Mi20MS/QWFmOOP/v6U/g//uSerV6GxTaGJXoLizFHECUIY45+7UwsLlxYorewGHPESZJ9tURvYYYleguLMUeUEXxsFb1FCSzRW1iMOZKM6DNhb2FRgCV6C4sxh1X0Fv1gid7CYsxB3rz16C3KYInewmLMEVmit+gDS/QWFmOOxBK9RR9YorewGHOQok+sR29RAkv0FhZjDuvRW/SDJXoLizFHZAumLPrAEr2FxZgjzvLnrXVjUQZL9BYWYw7bAsGiHyzRW1iMOWwwduvw3297DB+7a3arl9EXlugtLMYcRPCxbYFwznHrQyfxxUdOb/Uy+sISvYXFmCOKU6KPbLObc46EA/EY3EhZorewGHPEtqnZloFzLgrWtjMs0VtYjDlsU7OtQ8L5WMRGLNFbWIw5co9++xPO+YaEj8d1t0RvYTHmII9+HJTl+Qar6C0sLM4JbAuEFE+fbeMfHjwx0msfObmML397bujXcavoLSwszgXImx+HoOBm4k+/9hTe8Zf3jvTam3/vDrz1I18f+nWpoh/plOcUlugtLMYcsQ3GAgA6YYzwHOc6WuvGwsLinMA2NUsRxgnihIOfQ+JNkvG47pboLSzGHHFsPXogJXoA59RKOW+sG8bYZYyxLzLGDjPGHmKM/Yfs57sYY//EGHss+zq9+cu1sLDQEdv0SgAQts25vA7JeVQwFQH4fzjnzwXwnQD+T8bY8wC8E8BtnPNrANyW/d/CwuIcI7ZNzQAAQXzuLay0BcL2v+59iZ5zfoJz/s3s+xUAhwFcAuD1AD6aPe2jAH5osxZpYWFRjnw4+BYvZIsRRBnRn0Pi5QMEY08udc7RasoxlEfPGDsA4HoAXwOwj3N+Akg3AwB7N3pxFhYW/WGzblKEIyr69QRvE947rfWfHj6F7/zN23D7CDn6G4mBiZ4xNgHgkwDewTlfHuJ1b2OMHWKMHZqb29o3a2FxPiJvamaJHhie6LvR6LdCCec9N9jZ+RYA4Atb3Mp4IKJnjPlISf5POeefyn58ijG2P3t8PwDjO+Gc38I5v5FzfuOePXs2Ys0WFhYSIlsZCwAIo9Guw2o3GvmcScJ7dg3dP1UDkFbtbiUGybphAD4M4DDn/Helhz4D4K3Z928F8OmNX56FhUU/WOsmxajB2HY3HvmcnPcOgjuMAQCe2mKi9wZ4zssAvAXAA4wxqi/+JQDvAfBXjLGfBvAUgDduzhItLCzKwDm31k2GUYOxrWAdil66/ibQY9ue6DnndwJgJQ+/amOXY2FhMQxkjokucKIXBVNDXofWeqwb3rtAi4i+GyXgnIOxMirdXNjKWAuLMYY8PvBCz6Mnoh92w2sFuXUzbAZOv1438lrmVrtDHXsjYYnewmKMIQcCST0+fbaNG//LPw0dAPzmUwv4iT/6miDMcUOvytgwTvCWD38N33pqofCYrOg/c99xvPb9dwxM+P3aFMfSL2h2fuvsG0v0FhZjDFnRE+F88pvPYH41wCfueWaoY9339CLufHweC+1gQ9d4rtArGLvQCnDHY/O47+nFwmMy0b/jL+/F4RPLaAeDBWj7KXraM7/3WXswWRskJLo5sERvYTHGkEmNCMd304/1sMqcjhWd41a/GwURjDUpenpvhsdkop+opGS8uBYOdM5+vW5I0b/3TS/Cc/fvGOiYmwFL9BYWYwyZuIjXfTcN+IVDFgKNez5+3r2yuP5eHT5lj34iU90Lrf53NTzrXNkry4euqedsTRCWYInewmKMkWyCoh9fj748GBsm5amXbSm9cqKaKfp2f0VPhxok68axRG9hYTEqVEWvEn0wpAUzzrNnOec9g7HivRmuSUsqmGoS0a/1V/S0sfa2bqyit7CwWCdiA9FXRlT0kVD040f08ppN1k0vta949BnRLwyg6OlQg1g3riV6CwuLUWEiet/LPPqhrRsiw96vu+3wKWP2yjD40qOn8U1DqiMALLVDfOTOo31THO94bA6HZs8CyDNuAHMwmX4mbwJffPQ0vvXUglIZ26y62RpyRR8nHLfc/kShJw4di/Py/Hv6nbhbVChFsERvYTHGUKwbjWyGJ/riMU349c8+jD+8/Ymhjq3jPX//CD7wJfMxfu2zD+PXPvswvvrEmZ7HeMuHv443fPAuAGrg2aTo6T3J7+03P3cYf/jlI+iE+WtZ1gRAVvRffWIev/G5R/CuTz+oHFM+Tdkli62it7CwWC+U9EptAEkQDevRZ4q+j3Wz2o2w0hm9bQAAtIO4dCPqRKlnPj9EJal8LNNGFRly7LtRgihJlOfT3YwcjCXf/uHjand2eUMpi2vECYfrsC1rfUCwRG9hMcYwFUwRYY/q0Ud9XtfqxgMXFJVhLYxLN5TJjFiHaR8sWzem4Gj+3vLHwiglebl6lbz+Rcm6IYrWK41loi8rmooyot9qWKK3sBhjKC0QuKro+3ntOmKDvWF6zloYr6sRGAB0wrh0fVRBOsxdgxyMNalrek8yIQdx2nlSIf/s4i1oHj2g5tunx5K/L1P0yZb784AleguLsYbS1EzrSx8Obd0Q0ZdvEJRzvp7WvgDQDZNSRT9R9QEAq0MQfRANZt3I7y2IUvtI3hhoTXJlrHw8OejKB7Jutj61ErBEb2Ex1lCybkjRZ6QWjFwwVb5BkGWznmEdccIRxIloS6Cj5qe0NIx1I9tU5vTKYo59mCl6eR10zWSPXn7N6ZU8bqAqevO64iSB61qit7C4YNHqRji90lnXMXQ1+vTZNoinR/XoexVMEfmuZ/xeJ4yz85jXRxvWcmewfjNhnCibWhgnBT/d1McnjIsefR6MDXD4xLLyWgA4ms2ABTSPvuSaRQm3it7C4kLG+297DP/qQ19b1zFkEvr7B0/i+977JZGtshktEEjJd6Okb9C2DGsZ0ZdZN1S9Oqh10+7GSnrl3953Aq9675expNgvaguEJOEZyasefZ5vD7z2/XdgfrWrXONnFtbE90rWTY88esd69BYWFy7mV7s4s85hFLJ/vLQWIoy5sB2GrXAdpHul7M3rwclBQYq+bEOh9zRoMHY1iJT3enqlgyBOlLsO3boJpVRS1c5JcPXeCfz8P3uWWIN8jbuRPKQkX0N5MNYqeguLCxphzNfdbsBks1BgMhiye+UgvW7kbJv2iAFZKlDqlXsOYOC++O1upGwawhpSMnHUFghyS2OZjMOYo+I6uGpvM/u/Gqwt+74sfh0n3Hr0FhYXMsIoGTpgqsNElqQ6h/fos/z7Hlk3soofNcUyV/TlvjYAxXrphdVuhK60qdH3oSE/PhH2VK7so4SLRnBRnMBxAM/J+wXpAVzCINZNlHCbXmlhcSEjiBME2dDoUWFKJRSKftQWCL2sG4ncWyNm3uTB2BJyNOSy65CvmV5l2zHEAPQaAXo+KfqKl3f8dBlDJesXFMVcScmU4xKKddOnMnarYYnewmKLMOowaxlmRZ8elwhv8GP1X49C9Ou0bsry9en8nTApfQ/yElsF66Z4fJnYgXwzJCIXij5J4DisVNHL12aQytjUFtp6mt36FVhYXKAgslnPoA+yDHzJBw4E0Q93tzBIC4S2Yt2MpujX+lg3MrGWDQCRr1krMHv0SjZNH0Vf9RzxGpcxeG7u2ZsKqgB1sym9O7GK3sLiwgaRzbAVrDJIhVMPekC1bLoDBGT/9r7j+NmPH1LsjdMrHbz2/XcU8tEHDcY+MbeKa/7T5zAr5Z0Dacri73/x8WztvT16oNy+kZ/T6sbKkBV6z6ahLGSxBNLdVBjn1k0Yp4peVviqojcXZiUc+OhXZ/HCd9+Kf/Whu6XzJpboLSwuZJCiXU9AlhQmERWgpgAOYt/c8+QCvvjonJJeeWSuhcMnlkXREEG2a3oVTX36W8cQxhz/695jys8Pn1gWvezL7mTkAqa1kvXLGTWtbmTMMJLvTHKbTN1c44zI5WEtLmPKOMaoxLrhmnXzxUdPY7kT4RtZj3wAiPnWtygGLNFbWGwZRg2aytDHBwJpHxmC3Gu9DFT8JPe6oQ1E71LZ6sbY1aykj/Wwbqq+K45dhrJYgJK3XrJ+OaOmVdLyODJYLrGu6Mmjz4KvSUbMcrqlat3Iij4/V5xwUTUbxlxsAnGS2Dx6C4sLGbl1s36PXlX0Rb+6F7pRjISrwWEiUj3g2upGmJlIib6XoifPWyZqPV4QJ9wYQ5CJtWwTlL3ydjcyXkOTEtc9eqqOla0vxiClW+oFVeYAbBAlStWsfHc0FoqeMfYRxthpxtiD0s/ezRg7xhi7N/v3A5u7TAuL8w9EYusKxmaEUpWIXrYxyqwPZR1S8BZIVSsRqZ4r3woiTNZ81H23p0efK/r8/Maukn1+Vlb0FSmKPjIremW8YPp9ohF9nLVCkO+IXIeJ4LacdVNxndIiqdkzLcQJxzV7J7LX5XcQY0H0AP4YwM2Gn7+Pc35d9u9zG7ssC4vzH0Q2G+PRu+JngSEDpRdEOmaUZ8PQ2vTMmlY3RqPioll1sdrLuiFFLxG1mYwNRB8noBojeaMoex0FYyueA5lTwx5ZN7Jtxrl6R1Tm0Vc9pzQY+8Rcats8a99k+jqpt85YED3n/HYAZ/s9z8LCYjiI6sx1tEEgsqlI6ZXdUA7G9t9EAi3vPk5koldVezuI0Kx4aFa9noqeFLG80ZjepymXPk44mhVPWVuv11EwtuI6Ss66KVumrHGbbN04Tp5eGSVcXOOq7yjvQXadnphbBQBcnSl6OSYw7h79zzHG7s+snekNW5HFQDi93MGHbj+yrqrK8w3dKMb7P/9YXxV7z5ML+IcHTxgfC+ME7//8Y+tqw9sPdzw2hy8+enrgnjQfv2sWT51pGx8jtSkr0uEVfZw9N89MEdaNIRjbrHpoVDy0ujHuefIsPvfACTx+ehV//vWnxPNoCd2s8veP7jiCZxaK70FW5kfnW/jYXbOIEo5GpTyY+6lvPoP7n1nK15RZN77LINcmqRWtmqLXNp2eij4mRe+KjeITh57Gg8fzNRyZa2FHzcOeyWp2vjzYOxaKvgQfAHAVgOsAnADw3rInMsbexhg7xBg7NDc3N+LpLHTc+tBJ/NfPHcb86mCNny4EfPPJRbzv89/G14/2vgH9yFeO4rdufdT42OcfPoX3ff7b+I3PHd6MJQIAfv8Lj+P9n39sII++E8b45U8/hM/cd8z4eGwgell1nmn1//vQFb1s3ciqPUk45le72NX0MVn1sNwJ8aMfuAtv/9Nv4q/veQa/9DcPKNkmdMyFdoj/8neH8Zl7jxfOLfvxP/D+O/CuTz+EbpgIojdtgu/+zEP46Fdnxf/XghhBlFa3yn1l5OtQ6HWjHVcheofBF5Wxab96xlLrJiX+BL/w1/fjFz/1gHjNqeUOLtpZE3cGgRQDGFui55yf4pzHnPMEwIcA3NTjubdwzm/knN+4Z8+eUddpoWGQIREXGvq1vyXEWiaFjMlaOsbusVMrG7s4Ca0gQieMB/Lo8+eU9YUpplcCwBW7G3AdVihYMoFUM32Vpy7J1s3J5Q66UYIDM01MNXwsSVWr3SgG51JDsWxd3SjJbSCD1RMZcubXwhiNzLrRPXrOOVpBLO40Kq6DbtYcruo7CqnK+fh6ewf9b8Qvs24yj95lDK7DECccxxbXoKMbxvBdR3pd9vnkY9wCgTG2X/rvDwN4sOy5FpuDQeZ7XmgYmOh5OdETT8ipchuNdjfGajcSHm+v9EoizLK2BHkLBPWjXPUcXDpdx9Ez/YleV82kWgE1GEt54gd3p0QvV62KzUJrQdyNEnF8PSc/fV/F30MnjEsVfTdKs2Da2QbUqLoIsnNUPVchekXRa8JI31xlRe9I6ZVhnIiAquemHv1Rw+YZxAk811Eqaul820HRe/2ewBj7cwCvADDDGHsGwK8AeAVj7DoAHMAsgJ/dxDVaGEABIsvzOShrpF/Zf5Lw0m6DRAAnltY34q8XVrtR3yAlIepj71BBjt4K12EMB3Y3cXRuAKLXjh2VWDdEcAdmmphuVJQB2oGUubMTvlDO3TAW35sKrEz9ddpBjJmJKhgrEj3dYZCib1Y8dKME3ShGxXXgSurZlF6pNzUjyMHY1KOXCqbiNKDquwxRkoi7pEbFFZtX2sc+f10Q5UJsLIiec/5mw48/vAlrsRgCxFNW0edYC1TboAxpJkXJY0pWBQfbhF7i7SBWbIwgLg+YhsJqKK8idRxWIBPPZTg408Q3Zs/2fR969Sn1fwHUoqjZ+RZqvoOLdtQw1agoZEmb61qgzoMNpJGDRusme0y+g2oHMTyXCVtGBhErEX6j4mKhHaIbkXWjvg9CsU1xeTDWcRhYZtVEWb96usZxwjGbBcan6r5yl+I5edYPfS6TxLZAsFgHRJMmm3UjMKh1k3DeY1BE/tqFks6J60HqMUdKal6vpmYie6NkQ0+y9D2dTFyWEn07iDG30ntcYUHRl7RAmD3TwoHdTTgOw1TDV15D157uqvJWw7E4/prBuiHClS2mThjDcxxUvSLR08ZDx29UPQRRjG6WXinf2chEnxcwme+Q5O6fdAzfZaIFgpcFaMM4yVsdaGrBc9Wul+kabAsEi3UgEZ7jFi9kixFECY5kOcydAScrxT2tGzXdbz14/PRqwVtfC2Poe8wgwdiyzYDa4BaI3mE4MJOOwzs638KTZ1pGogXUvHtAtW5kRX90PiV6AJjWiJ4GeYsxgVIw1pSqWc8qZ+OEY7Ed4IuPnBaPrYUxPIeh4rkGRa/eFTQrrogDVH1XGdsnX/uywSOEipsXnDnZtSRiTz36NNAaxRyz2aak/24rrqM0R6PzWkVvMTJIkV7o1s2n7z2Gm3/vDqx0QnQyIumXlx4nPRS99OHVW/QOgzOrXbzm927HPz58Svm5qYd7r41JHnlnfjxVjA4rEv3luxoAgKfOtvG6/34nPn73rPEYJkVP52sHsUiZfGZhDZfvTo851agor1nppnc/tJnIip7+Rtckkr5sV128r//2D4/iY3c9KR6jOatVzyn8LvVq3EbFQxAnAyh6tQVCwaNX0ivTr0Ts5NG7DkOYcGEz6fZPqujzHjm0Bkv0FiOD/mAvcJ7HYjtEECdYWgvRGbAbZNIj60b26M8OkINehpVOhDjhhX7qpjmrvYheEEaJR98O0lRELekGDmNoVlOVutgOsdqNjDn1SVIcUB7FXKns7EYJkuwrZcPo1s0KKXph3WQbRRgLVU6K/iP/+kb8x5ufI851armDK/c08Z4feYE4nuewzLpRib2tXb9m1QXnqdKvemp6Za/BI72ybnLrxhEtENysR303jEuraz0pvTIct6wbi+0Jq+hT0Ae21Y1zj77PII/e1k1+PRcHHE5tgrAI9KwRQ0Cy1x1Iv6KqdjdGs+oag7Fkj9BmYzqPaVOMEq7c2bS6EVBNqYIIcVpX9BnRkw1E75/z/DFS+7ua1XwKVJJgoR1g/84aLpmui+O5DkPFqOjV60f59iudItHLd0F63UnRuslfJ6ybLJ2Shoe4DlN+f3pDNsW6kWoSrEdvMTLo7/RCD8bm/m8kiKR/Hj0Gsm4Wewyn7ruukvmrJuumrBhKXk9Z1k0riNCsegXrxmEMtYzoacMypZ2afhYlibKmdpCrciKynXVd0WfWDfXLkV5PhVVEkp7U7z2KOZbaIaYaFaUWgBS9vhHpufjN7A5jpRNmTc2kgilZ0WvzeXUxoObRp8fwsnTKmEOkV8opovpdoaf1yKGvjiV6i1GRl5pv8UK2GMIi6MbCuumbddMrvTJ7YGaisq6sG9qAdKIyKfqe1k2fwrhWN20yZgrG+m6qcBd7KHpTd8jUupEUfZBPcKL2wzXfFXcM6XsgT764wS2upeenvbXiOcLLJntrqu5rPrmDiucUUj/169fI7jTCOJ376rlmj16Z9So1bSPobYrpZ2mb4iQbRuL0HIjuSU3V5GCsVfQWI8NWxqYgIpWLkPp59INYNzMT1XUqerO3bioa6lUZ28+6aWXWja7oiVxqnoOFVrmiN47gy3q05+eIxIZQlQhRz7wB1A6YhCXNApMVfTdK4yvTjYpStOS7DFXPLW6UmnUzUc03m4qn97qRrRvVxuldGcvEOsMsXkFTp3p1A61kuf9A+nvnWSzIHdcWCBZbj9hWxgKAlO8tEX2/ytheefTZ8fZMVrG4LkWfKF8JowZje1o3BkVPZFWvuMK6CQzq3WjdxIlChK1uLCn6nDJ2aj49UKLoteso94RZaAdIeBrc1VV1xRCM1a0v8uiBtLukbJOobYrV4in9b0Q9d/6zKE6QiBYIvZW5EoyNE3HXaBW9xchIrKIHkBNpS1L0y50IN//e7Tg0a+5iGSUcnBdH29HxGEsVvZ4xMwzywd8c3/Ebn8eH7zyartMUjJVI9fHTK3jRr/4jXvjuW/HtUyulGwahHcRolARjgZT8liTr5j/8xbfEWuhnprXLwz9a3UisUVbdu5tFohcevfR3qQe1fcnimM+KuVKPPn8PwqMvaYFAaGqKXibVMOaYW+ni+37nS3hSavMc86J1Y866YeLuxnPy1MkyKAVTCRefze2QdWOJfkwhFP0FHowNRDA2Fmry2EIbj5xcwb1PLxpfkxebFa9dmHD4jlPozjgs5J4qp5a7+PXPPgzA3NgrkAKDT8y1sLQWYrkT4Z4nF/Ih1iVW02o3DcaWKfqa74hYQzdKcNcTZ3CfdF1Mij5OUruikXnwQZwIr1xW9P/fzc/GL//z5ymvFVk30h0IFVMRPKknzPxqSvTTBUWfefQlLRAIqqLXgrFJWsV6RCt8i+M0pVR2u2RLiu4KPDfdaChFsp8y9x1Hsm4Scbdtid5iZJAgueCDsZKiJzVJ6Xxlipw2SZN9E0YJPJdhql7BStc8i3QQUGrfWqiSnGmgiXwOudnZ7HwrD+oalXdaEdqseIWmZkQu9YorrksQJUoBU9lx06ybBPUsoyWMuaTocwX9wkun8LKrdyuv7WjplUBRhftOngZJ8xSmGhVl7m2poi9UxuZEX9GCsWHCjYNXxPvz1bsBQkHRC48+f478WvG+3DzInObfJ+K9bDUs0Y8pclV6YTM9EYqcR7+cpfqVZc2IPkGGS0eDoqebaaBxVJ8+H67dp+Cn4ipET8q5UXFxdL5VmqaZHivr4Fj1Cil8rgjG5oTUjRJ0okTx+4tedR6AzIk+V/QyIQKqlQPI1g1HLVP/+pQq38snOM2tknVj9uj7WTe0RiDz6OXK2DhRiJ7uImhUokzWej96+lkktSmWraVGpUj0npurfuqRQ+9lq2GJfkyR2PRKALm/nQZj0++X11IyKLNeRFWxSdFnI+moxH9pbTSfPpZaCBCW1kKsdmNMVHMV2qh6CtETUT53/w4cnW8Ja8rk0ZO6bVbcoqIX1k1OSGthGlRVNhYt2Enj8qIkJ8I0OJtl3WhEX9WUrTyOkNS2Ts5pl0fduqnALyj6Yq8beeMk1U+oePrgES6uJ703IL2TC6JE2STMWTcOAtHUTD12zajo8w0sivPMJUv0FiOjl/1wIYEIcLUbCX+YPtwjWTdxAs9xMJUVBI2aSx9K2UCE2fkW2kGE3RN5EJOachE6gugn8eTZtlC0pqwb0aq3mrdAIOITil4iJCpqinoo+pqf5o4HMUe9kueoD6ro5fRKItKCdePmNgh11txR89RgrOuYFX0QCW/dc4tErwdj5Tx8usOgpm1lij7PumFiwpSjBWNNip7qFhyWZd1YordYL6x1k0Jup7um+bFltouoKjbYIVHM4XtMlPgvjNjvhm7bW1qb31YQY6pREWTV1BQ9KeJnX7QDQZSIxmqmWAEde6KapxUSAeVEn3/EKZ9dVfTqcaueK1ogUDA2lNItdUWvE/+a5NELRS9tdl7W65289JVOhB01L53O5KiKvuKmlbGJ4vfHYhNOWxnL1o0ajI2SRPTeofcGkHXDVUXvFhW97zqIEi6Gu/hOP+vGEV/DrI89vZethiX6MQWJss20bv7k7ieVDI1zgTjh+N1/fLRnD/VDs2fxiUNPA8itm6W1sOBjywVPf/OtZ3D3kTMAZNurd9YNkKYGfvzuJ/HQ8aWh3geRqdyx8chcC61uhImqK7zzZsVT1PpamE5KumpP2g740ZPp7Fr5vd360El84ZFT+Tg9KRhLWSgmRU+biEz0umKueo7IuiEijJJyRa8TvzxKkF4vFxkRwcvff7fRAAAgAElEQVQqdzpL03QkH9x1mMjwUXP6I/F8z2XKevRgbBRzpTUzrTXK8ugVRe9J/ehF1g3Lsm7Sn8ktkOsmRS/aGzM8fbaN9/3Tt9P3tQnDa4aFJfoxxblQ9P/5fz2I1//BVzbt+Cbc8dgc/vsXHsev/u1Dpc95wwfvwi/89f0AckVv6jQp2y4//5f34cduuRtATvC9sm7IR291I/zG3x3GX9/zzFDvQw4SE04udbDSCTFR9QRRNKpuIeum6jvYvzNt8HViaU2si/DBLz+BW24/IjJ4Jqoebjq4C697wX4RRDYpen1tgMGjz8hvLYzVYKxQ9CrBFawbaSZAxXUUOwaAUO2yVTIl9c3xhSrOq0zlu45uFGNHTVb0+XGKip4rm0xV6oG/0A6wS6oD8BxH3GWJFgiOk/a6EYpetm6K/SBp7b7n4HMPnMQnsr+ZfoVW5wKW6McUgqzOM+eG7JZBR/hRVgoF9WSshbExva5Xi+coSeC7jrBC4oQbKyn7r4uLNRAW1wIstNNy/5qXkmDFVX3obhQrfWSIzOVOjGGcoBMmItDbqLi4/vJp/MGPv1iQIyl8UxqgfAehNyujjYHuLFyHZVk36bl0Re84OSH7LlM8epmsCb4UQ6BfsdzbnsgyVfTqgHDqUURFUp6jKvpim2LVuqH3dnxpDe0gxtV7J8Rj8txdpalZFlR1tMpYk6Knxz2t5YFtgWAxMs7XYOyKpFIHAWWlrHTMzaaW1sIC2fcOxvKsOVVO9FGS9B04roOCxHIwdqEdik6NNd9NFa/WobETJqj5jiCl5ex96QHUThiLTaApXStPIkrAnB2iWDfZ90SepJA72ZQnPyO7Mo8eyMl/Z91XPHqZrMX6JCKm7+Xe9kLRO0wUMelFY3Xfy94r0xR9XiGczntVrRtS5E+cTieSKUTv5sNbZI+eYgRyf550DeZgbPrV3HdoK2GJfkwhuleeA0m/nuZew4KqKCdrgxG9nnaoE9FCOyg01YqEoi9Jr5RG89Eg8aEVvdbNcWfdx/HFNQRxgumGj6rvwvfSSkolvTKIUfddQdB03rRtQ973phPGYhNRiF4iOsBM9PI1y/P202OQNbMWxPA9JxunJ3n0hjYAdM131P18lGBGjgVFr7QiLva2rwiP3hEefVe6S0jfryuO5bkOiEflpmYN30UUc8WaIvvp8Yzor9k7qayFhLeadcPFpiVvovqdjfzefO09W4/eYmTkPvPmHF/uA7Pe2anDYDUbSTc5oKLXu0NO1tSOiovtsJBm2SuPPop51osl/XAS0Zra+fZcl7aJ7J2s4thi6rdPNXzUfQd+5mHLvdE7mXVjUs6if06UWjc0Vq8hkbk7ANErBVOZl07EJVIQEw4/syso68Z3mbG3uqzoRWVszEUbAxlqCmVR0dPzfcn2yRW9uinRe6TNSbZuqr6LMEkUj55GTT5+ehUV1xHjDOlYuqL3XPLo1QHsFdcxqnRh3VhFb7FRyLNuNkfRyzx1LomeLBiTYgKKaYb6/3fU1Q1isR0U0izjnlk3aTCWPtRk2Qyv6NXn75msin7sinWjKfpOGKPmuWCMFQKpRHRh5j23uxEaFbVjI5GK3OumsLZEVfRVKf9cDrZ62fqiJBu+7RU3DcBM9ESO+oblKYo+I3pDMFbOupEzeYB82Ai9ns4vF0zVK2n2kGzdkK30+NwqrtjdUP7GZI9e7UefK3rapKq+Y1TpItCse/Q2GGsxKvKsm9GPEcYJHjympg0+eGwpKxLJDzx7Doiec457n14U1o3snz91pi3y2XXS1odmU0YGfQ4X26GwniarHjjngnB7KXrG0sKXXNHn5zm90hHqvAyhQdETpup+SvReVhSkVMYmqFXy4R7KMaVhJmtBjFY2L1YGBf7yfvT9FH2MiucIglX6zWSVnmFmgZRtvsK6qfmi9D9KErhu0eJQPPrsnNPNYjA2tX3ypmpAfpdEOez6mmWPvp5ZN3IwloLXi+0QB2aawjpiLA0qF7Nuss0+jJVeNxSk1kEbgZyqqb/nrYIl+jFF3MN+GBS3PnQS/+L37xQZK6dXOvjB378Ttz50SslIeaYPqW0E/v7Bk/ihP/gKPvWtYwDUMXD/5qPfwO9mOclyvIDzNN9bbpd7yVR6O767mRLr4loo0iwna56i4k2bJLVAAKgEvtjj/tf+9mG84y++1fP96Hdae3fUxPfTzQouna7joh011H0Xa0E+cLobxqiRjeLpRJ8p+igNDqedK83BTkco297B2LUgQc13xXtuaEVEwrqJEqOdBOSKmtIVl7OaBpOil4mfiFQeS5hn5Tilir6uWTdmRe9l2UkxZibSv4VXPHuPOM8VuxrC29fjGrJ1AwCdKFHaNlT9MuuGNilN0W8Dj94OBx9TiOHg6zDpF9shEp4GQGcmqljpROA8DWD263C40dCVuuxxn20FOJsRvNzbnJpTveb5F+FtL78SFc/BX34jLaSabviYX+2iE8ZCwU/UPOVOwWjdxIn4wDpOTjKy6jYFeHXovxdd0f/n1z0PCef4m28dQ5RwHF9cw2W7Gql1kyl5naT1ISQLrUDp3gjIKX6qdeOw3I6T17a0FmBn3RfveVczXyeRW/r8pIeiT9dJWSxPnm0jitP+ML0UPVkapmCs5zCx0ekdMfNgLNlNubIXPX6ywq+1MMFz90/iD378e/H46VVRD7FropJW6Ereu1OwbvI4jcPyYGxFSr+VIRS9ZtXYFggWI0NYN+tQ9OQji1tjKdinTOfZrIivhD0SEQIqCXfCWATS5JYEUVbKXnEdHJhp4uKpuiCWRtVDxXWwFsbiLsB3HeVOxWjdZEFIICU6smzknilrQdz3mui9aeT3t7ORzket+S4OzqQVsBQHofRKoJhBFMYJOM9THedXu6WKXu9eKQepZUW/2A6zzpHp8+U+PLl100fRZwT4nIvSLJbZ+Zbk0Wvrk8cFmrJupDx7ug5kv9BdHqU2CivFc8EYFNKuZ11Bu2GaxbSj5iubzFS9Is5D63AE4Wfrk+MJ0lCRqqc2kZPvAOWv8mu3GpboxxR5QHF0tU0KKW+clfvR6hi2zVf0OumS98552oGQPuyy8o+znizyB5g+dDUvzUXvhonIuokTdYSgsXtllChpdHRt9Fx3PTagQ/+9ENE3Kq5CfkT0s2dSol/LiAkwefSJcqdzphUoqZXpmknB59kngGqPyKmaC+0g7RyZveeZCVld58HibtRD0WeEfPXeCTCGrL0yN3r0stolUt5ZkkdP758CqvR3WMmKzYhAK1maKmNM9eizfvR0HKXtQnZO33XEnQU9LBdMyWuVg7/ysXILiTz6MSyYYox9hDF2mjH2oPSzXYyxf2KMPZZ9nd7cZVrooM/7eoKx+uBp+qor+uAcKHrdRomlwCPneT764lpR0SsqUVR4prnonTAWm0OUcM2jL+l1I5ENXaOuVHTViWIlJdKEYjA29eintTmreyerovc8ANW60YieerQQjNaNo5JU3UD0dCwgLSjb2cjVLvnZdAwvG74RRIkxhx7IFf1kzcfFO+uYPdMSbQP0uwBVJaeEuUOqmchbIDjiOnSkWgIgy2N35UyhvBVCkejzOyRZaVM1rkzgpqwbsVY5GKsRva8pebn5mXzcrcQgW80fA7hZ+9k7AdzGOb8GwG3Z/y3OIXrlgg8K3fOlD1I3ihXlWDavdCOhp0nS+Yngyadd0BR9mCTC1wXyD2fdQPRxwpUiKXPWTR6MdRwm8udlRd8Nk753OXGJdSPnjANpq4crdjcxO99CknB0o0SocD01MozVXvJRwgtdFHN1qnr0etop2UCL7RDTjdyjl9fnKYo+7pleSbbJwZlmruilwqJGRbVbgFTd76z7SruLinQ3VdMKpujvlbx/WnPVc1DxtK6dFRdRnCh3SDI50/uU7R5aRz54JH++w5gSE5CPpefP61bNNuD5/kTPOb8dgD5l+fUAPpp9/1EAP7TB67LoA1Kj6/HPI0nBA+WK/lx49AVFL4he7TEvWzdBlKp9WdHLhT81P/XoybqJEtX6MGfdcMlrza0bxaMPY2N/eOU42kYwUfXQqLgFogeAKzNypHgAEZwpvTLQFq1bN0LRawVTuqIPY46VboQo4ZhuVATBykRcyWa7hnGm6HsEY0lRH5hppEQfqx59nhKpWh769cj97qJ1Q38TdFyjopd6/CQ8/V2ZrRtZ0at3A3QM+VrIG0KR6PMgbfoe1Oukt8/eCoxqHu3jnJ8AgOzr3o1bksUgIDW6HkUfCEWvDrfQPXqdXAh3PjaPV733S8bGYYPgtsOn8Jr33V7wngFZ0afHJqKVJz7RB0hWUBXJuqn7LjphIjJ14pgr10t+j195fB4H3vl3WO1GIg/adVgejNWKmuiaffSrszjwzr8rtFOQN0c3I4ndExUlq4Vwxe4Gnl5YE33bdY+eFGEUJ4UNphiMVQOLZO3ollEUJ2ICV5p1k7/nfLNw4DnpOL1uj2Bss+qKlMcDu5tY6aTze12pu2Rdy30H0kCxnBorP54WKKV3CmfbAV75O1/CXUfm08dchkY1bxPRqHhiI6n6aRdK+n8g3SF5fRQ9PWyyblxpepQcjGVM2hhKWyAYL9s5xaanVzLG3gbgbQBw+eWXb/bpLhgIRb+eYGwh6yZX9Ip1U3KOwyeW8cRcC8trobHUvh8eObmCR0+tYHktFO/nF17zbPzPrxwVwUzdupFH89HPKiUefTWzbqgIS/foZdL/q6y/PZB7rkowNkpEADNt+JU+5/23PQYAOLHcETn8gLqJkEp97xuvU7JaCJM1H3HCsZxtSDWN6Bu+i1aQ3kWEUR9F76qqdGfDxwd/4gZcMlXHn37tKfG8MObiTme6UcnvYlyGiZqHxXaoFEz1UvT/9uVX4uZrLwKgZu34LgPn6Wtow5E35f/0uucWjkXnoOfVfBdPnmnj6HwLh0+kvfk9h+G//egLRd7+v3/V1eJO70dffCkOzkzgiblVcUyTdSNUvjTnNc+6Sb/KsQPPUYO/ckqm3L+engsAv/WGF6LqOXjx5VsfwhxV0Z9ijO0HgOzr6bIncs5v4ZzfyDm/cc+ePWVPsxgSedbN6McgMi8q+ngg60ao7RHz7IlE20EeE3jTjZehlgXSgFy101c5GCkUvSnrJvPo5clTcVIejJW9blndyucLsp4vCU+tGc45LtvVAFCsHpa9dNp8bjq4C1ftmYAOWjN1o8ytG1LDKeFESVK0bgqVsUWf+ObnX4R9O9Q7iTBOBDlONXyRMeI6TByT+smHQtGbN/PLdjXwXVfPAFD7tMsevVD0kh3y/Et24vmX7NSuhVbZ6zs4kxX00d+b6zC85EB+La/eO4kbD+wCAOyeqOL7n7dP+ZvIg7GGIidHJe30a/qYnA3kMKZYRfJdAL0lYd14eQbT66+7ZOCW25uJUYn+MwDemn3/VgCf3pjlWAyKvKf6eqwb1Zsn5R5ILRDkzBMdRKCmMXeDgF7XCiLRhZPaweoefVr4xNX5qgFZN5Kilzz6uu8oA0nSTpRmRU+tbwGVbOTzUTMxAOA83Sgum05VvN4PSL4jKstWIRCB0J2Hbt3UK+nrKZ9dhh6M1XvdiDVoajxKckU/JSl6l+VDV7ysO2SUBYnLFL2MCa2TJr134dF7vUkvz6PP78zOZL/DfGPvvw7FIsquo6nISelKqWXdyHZXWdZN2ipDVfKUdWMaTrJVGCS98s8B3AXg2YyxZxhjPw3gPQC+nzH2GIDvz/5vcQ4hqhzXQfRlwdhumIjCIuoZYkI+mm60NQiiz4KCQHor7YpqzJzoEw5hIYjzZxkxvsGjp6ybM61UDdb81Gsus25krzsfZ+do040SJc0ySrhIR+xJ9H0I0s3WTL34yVOuC+smH9Ktb6p63369eyVBV+ORpujloRlisIcIxiYIorjUo5chbzxyGwMivX4k7Wv2R813RYsOWdH3g/ycusGjJ8h95mlvdAxE77r5mEJd0esbBG0cg85UOBfouxLO+ZtLHnrVBq/FYghsRK8bUQmrpVnKir7qu6XplfLYuFFA52t1YyWjwstGuAHqvNG1MEY3ikU5Pz3mG7Nu0rmscj/4tOWDOetGVt15FgYQSE2xgkhNbwzjRBxPt24ig3VTBl9T9FTNmls32ezWmBc21UZJ1o1OhnL7X8reEURf9/MgqMty358j60ffOxgrQ++NT8TdNGTdmCCPEgTSa5DHafK7vn6Qz0PX0bRByJ1K9awbOb3VZZqil1oa05QtJoKypOiHj1ttFra+ZMtiJPQacD0ohHVDBSmGrJua75QWTJF1Umbt9AOpZVnRe06qnHTrBkjzqYMoER7ymsm6kUbiyb1idtb9zKPPzy9fO8VTl+wDXdHLGw+1sAWAo2d0oi8GY8tA6y969KqyN1k3E1rWjeuoNkT+85TQqBVClAVjJ2teli+fkxwNfVntRmKcXhAPZt3IRC9PmKINqd+mJ7dAANTGbh1DllUZ5DuHqiEYm6/RKdhdel59ejwpGOu6iq/vMHPmlx4o30pYot8CzM638N5/fFQZ7pEkHL/594fFMGgA+POvP4WvPD5vPEY+M3YjCqbKWyCkgdHeir6s6dkffvmJQhtkGblHH0vFMOmttJ5emX6fEh0RuLBuSoKxVUmR7az7PbNu5PTJvNeNyaOXrJs4Ecr96bNtRcXL16wfuQmPPiP6utamuCEN6dbvnnQfWBC2gQyrniPshChJsLQWSmmGOcHSRtrqRvBdB+0gbQw3kKLXgtrCoyf7pB/Ra/n88mY9jEevzHclj94QFDWlV8rPkzN2KJBc9WWPXg3UyuvbTkS/fVZyAeEfHz6J//GFx/FTLzsoUsQeOr6MP/zyEdwzu4C//nffBQD4xU89AACYfc/rCsfYCEUvD7KQvwaaou/v0ZuJ/rdvfRSnV7qFzAqC7NHHSSL6gruGYCxA1k2CHXUfWOkKRS8T6ZUzE3jNtftwwxXTODKXq2zqUy+vVb52ckGUHpwTz4lipfglTLiUucSx3InE73MYj55IqZURvVwLAED0p48SQ8FUSdaNqez+X910ORyH4ZbbjyCIuOhzAwDf86wZnF7pwHcZfv77n4W51S5+4IX78cTcqnjPg6TQyhuPp3j0Lt580+V4+TUzPV//0qt240euv0QUeMmxBfp9D+LRv+CSnbj+8in4joNn7ZvMjuXgR158Cd54w2Xief/iRRfnaZUG22uy5mEtjOG56Z3Oj774UrzsqhnRm4jSK+Xf8XdfM4Pji2vKprfVsES/BVgL1NxwIL9dpywR3sd7z/Po118wJc8lBdQWCDXPLc+6Cco9+iQjQXk4tg4564b6lwPIinQovVItVAqiRKgs0618veLiD99yo/ieQMQhK3RZ0QeKp272uXVFH+qFZZFq6xD6Z9042XtVNy7qS9/wyxV9v+6VMv7zP38evjF7FrfcfgRRkmChHYrrcsMVu3DDFWmK4sVTdfzxT92UHk9a+yAKVZ/aJFeL/uaPvKDv66+9eCd+919eJ/4v++QdQzptGa7cM4G/efvLlJ8xxvC7b7pO+dlbv+uA+F5PrwRSoj+90hVe/Hvf9CIAwNMLbfFcR/LvAeCGK6ZxwxVbnzsvw1o3WwCyHGR1SLRAt++yF2zCxgRjKZ0yU6WRSdH3D8YGhgZfgVDr5VWz9Lp2FoyVg2ImRd/JFD2RmykYK6Mmkc6OjNCCMqKPilaL7gXrHr3eUkE+BjX16rU+Aj2voxN9wbopZt2Uda8sU710rijmWJIUfRlkW2zY4KLs0fezbMogN3aj5mab1d9dBFgVRZ9X0MpwtGDsdmhF3AuW6LcAcm44QbYxABQGWusgjtpQj14oetW6Sbg5X7+XdaO/HxNoM1jtkqLPCVZk3cij4MIYQZwIe0AnRh2y1SCIPs6PJy9bVvplqrig6GOuNC+TB4hHMRckpbet1UEkkQeX0/8Lr15k3eTBWErxK8xk7aHogfxaBXGq6E29d0zPB4ZPF5Q9+n4B6TLIv0N675tFqmKUIJOJPn3Peg5+npLJ4LL+m/lWY3uv7jxFTvTFW/0WDdjoQ/S9BlwPikLBlKEFAikyU/91yik3BWPz99PDuhGVsVFB0YtgrNTygAaHk6I39bqRIVs3VM6uKHpFjRfvPIoevU70qqKXN4so4YKgKwOmFBatGz2PPhF3XzvqPhoVt1B1qVd5lp0riBIsd0LRrrcMshIftABInr1KHv2oRGgabr5pir7EoweKf+NKZazU2XK7whL9FkDv3wIUWwEvtXuPqtuQrJskJ3ZAzrrJ89rz1L7ieXpVxoYDWDfyc6JEndWaF0zlx6bxfVTFKqybkiwMCuTRNKf0vUn2CjeTdLsk6NeN4qKilzZAhejjPJbQj+RcYd2QFaXmceeVsXmvm8maZ1TYer+VssfPrHbBeT6AowwygenxgDLQBiV79KOqcFMAeJCsm1HgGKwbCuKTyCAo6ZUO27Q1bRS29+rGHI+eXMEn73mmUExjtm5UIl3oQ/QbkXVDwzNyZZ8HZ2kTILKZnW/h6bNt5fWdHkRPm4es6I8vruFJKd9cCcbGuaJXWiBEsSALavqVe/S9FT2tfaLqCeJTiL7EX6cKVf24QZQoweEoLnr0nHPc9cQZhAkfmOh93bqRyv/p9a7D8NDxZRzLBrXvqPlGz1xvuVs4V/b4XFZtOox1M2i6oNzGgKylss24H4xEv0nqWWTdGKyblY76edQrY7e7ordZN5uIn/uzb+Kx06t4+TUz+PhPf4f4+ZrRulHJkqwb060rICn6dQRjw0QleCL3hOeESKr4nZ+6H1P1Cv7kZ/L3Qes3FVTR5tGWFP27P/MQzrQCfDJLH9ULpgRJyR59GGOq4eP0SjdX9BXVuikj0roUzKQPZlkwVt4AXpilg+oj4IrWDRd919N+MDHuPnIWb/7Q3QDS7BVggBYIhaybdK27mhVMVj1cOt3ARTtq+PzhU+I1V++dELNwZVw8VUPFdQozeAnUa2ZuhYi+n3UjKfoBrRsid89lmJmoouY7uGS63udVZpgV/SYRPeXRS7+u73vOPnzojqO4XutASeqfMYaLp+ojt+o+V7BEv0mI4kT0P1nU1Lk+TANQC2zihAtSM/minHNplOAGBGO1ylggV5e00cxlKWbyGgazbnJFv9gOcWq5Y3iOmnUjK/q1MMF0o5ISfXYdm4VgbJmiT0liouoJwgp6KPpXPnsPPvATN+QtbLXDBlGiBIfDJA1aNyouljsRgijBU9JdT31Af1rOuvEcJnz3ZtXDoV/+Z6i4Dl561W68/vfvxBNZbcB7fuQFMP3mr798Gg/86qtLO03SZjq/mjU0q/dR9BLrNQa1bqTeMtPNCu5916sHKrYyH+scevRSJg3hpVftxiO/fnNhw6G/J8dJ2xH3S4fealjrZpPwzMKauK3XA5Imj162bpbXQixk+fSmP2mZ29dl3eiFUhJhk09NfutqJ1KKivSK0cKxo/y9iz7uUazEHug9t7U8eiUYG8aiXexyhzY/3brpnXWTKvo824QgXzoalSd/oI2KPpDjKhxRkmcBBXGiFGnRHcWgwdh2EBdsiaqXBlwnqp5Q6TRGr2wDKSP59Fzp8U+vpBtu3/RKb3RFL9oY+MWg8aDQ5+YCm+fR0xr1ILzprsKRnit3wNyu2N6rG2NQ75Mr9zQLKYa90isBYHEtFFORTL3ey6YkDQu5iRmgK3rqu5L+kbeCuDA31bR2Aj1XtoE6YYyVblTYYFa7MeIkMSr6bhijUXFR8ZzCXc6aFrzUQSTRlDz68qybYi8X3SKQ2xTT+knR0zWZlWIQpqlKJriSou/13Kl6Ssr9CrB6gY5P1k0/os/jBc7ASjpX9OunFzPJrvuwRtBlNbUzLj63d3bTdoMl+k0CBWCff/FOxacGzEQvk+xCOxD+q0ktlw3PGBZ61o1sH7U16waA0qJXtp1MlbMy+etFYGRl5YNH1GCs6zjK4JGa56ImET0FY2k9ZYE+Wnuz4oljy7nuMe9N9PoHvhvFqnWTBWPJ0gjiRAm81wbMo6eNqh/RTzd95fmjgO4Y5lcDOCwPNvZb2zA59KToN8LO0K0b2draaOhNzXpBb2283WGJfpNwdL6FyaqHy3c1FPsCkK0bsypebAci6yaIk0KxkqLo1xOMLQwFz4/VEkQvFaxoc1Pz4xiCsXK6YleNSdDcVxGwDdIZrPLAD0o37YRJNujbldIrB8ujzzsn5oOk5TsRpdeNoQ2vSdGvBbE4fxSnTdIoz73VjfD0Qt6ULrdu+hVM5cHYXoFGCpyuxyagTTFOOHbW/b7qlVT5MEM0KHd+1MljMmqaDbVZ/jxgzropfa7Bz9/OsES/STg638KBmSaaVU/pnQ7IWTfqEAvCqeWu8OiBomJWOjAOoOg552h1o8JzqRJWz7oBitYNoJKkrmx1mBU9FYOF4jn0uV3pRMoYPNmjr1dc1CuulF5J1s1gWTdpMNbg0fexbvThFUGcoBMlQgVT7xlS9E/MtZTfjSD6fpWx2XkS3tvmocBpWUuKQUBN44D+GTdAfjcyTPsDihFsCNFr593MCtQ8j77/c0Uw1hL9hY2nzrZxxe6GsBlaQYSP3HkUr/+DrxizbmRi/MVPPYAjkgXwi596AL/wifvE/+UiVVNTM845Xvv+O/Andz8JAHjHX96La3/lVvzsn9wDAPjEoafx6vd9WRBGaPDoybqRVW43TvCBLz2BH/+ju5WNy1wZK9tAKdHTRvH2P/0mrn3XPyCMuWhNsLQWSk3NpKybIAuSemlmC5B73/1aIPhu2iJgZ903evR//vWn8Jr33S5GFOpBTFldNyouumGaXjlBRJ+1PaYg5WOnVpTXN6seGDMHFMvO08uWIT+9VxHaIPAE0ffOuAHyXjfDWDdX7WkCGLzAqhcmq/n8WmBzFT1d+mEU/ZjwvE2v3Cy0uhF21n2lt/dXnziDB55ZFNkephYIv5QdLOAAACAASURBVPHDL8BqNwQDw3yriz/88hF886kFxUuN+wRjz7YCHD6xjIeOp73gHz6+DAA4NHsWAPDgsSV8+9SqeD6Rn6x2aROS2wgEUYLDJ5Zx71OLIv0yXbspGKvaQLHUYpcCgUDqES+2Q7S6eYtf8uiX2iFWuhEumapjRz1//3l6ZaL0E9fBGMP//KmX4Jq9k3hiblV5rwBwfKkDLHUwt9I1DtZQFH0lVfTdKBHkQwVTVc+Bw4DT2fu6ZKqOY4tr2FH38cc/dROuu2zKuD6CbMX0smWImEcd9ELw3XSgyqXTjb7PpfXok6x64Rde8xzccMU0Xnrl7pHXSLh67wQ++BMvxmfvP4HP3n9i03LogeHsmHELxlqi3yTQMGWh6LsxZs+0lJS+jtIEK/3w/tD1Fws/9K++8TSAlLhl1Vo2PIOg5+9TBs9CO8RSO8/oIeRZNwkaFRftIBaZQnrWw2o3QiuIhR0jv15GKBFqqxuVFpRMVH0Aa2iHsVK+HydcZC4dmGkqNoNsI/RTy991Vdr/nCpyu4a1Ppopcd2jl4dL1HwX3Shtk0wWCmXdeG7aj5zstot21nBscQ2ew/C9z9rTc330fgk9rZsBrJZhcHB3f6LXRwEOgorn4Obn7x95XTIYY7j5+ftx2+HTAM6NRz9I1o2YSjUmnsiYLHP8QFYA+ckrnRBPnVFbCHQN1o06As3JXqsSpRzYNVk3RPQL7QCccyy2A1y9dyJ97Eyr0F5BWDcJF+sl60YPJFJA9KQ0CatXeiXQm+hJHbe7sdJiN044js6nKvzgTEMp7JEn/FT7ED3BVBlLePSkmehlK6niOgiiBN0oFteIRgm6joOK64jWCft2VJVz9oMcTO5t3fS3WgYBbdIHZpp9n0sbz1ZPS/KkQP1mwWGD/86cMVP0lug3AZxzEdwjdf7Y6dWC8lU9+pSw5Q+6TLIyUZJ14znMGIylXO7FdphltHBhH8zOt7Cklc7LPW4mpEBnOtzZTPQnltKCm8mqZ2x4FmpEv1ZC9E0pNVEmVgB4/PQqHAZctquB6WauZqvSzM6yFhE6aAPtRfRl6ZXUhbEbJdk1UrNuPKnvOgDsnawBQOnmpkNOD+1FZDs3iOgJBwcgem8ERb8ZoKIz04jEjYLrsIH8eUBtUzwOsES/CSBCl2d0kl8uQx9ioecIy8TTMaQF+q5jVPSz8+mdw2I7FD1zXnTpTjAGHJlXFX3Fc5QJU0S87SBO52RqH6xc0adEv6PuG8lT/lkriEsHqUzUcvISlbFuTvQXT9VR9VwlcFj184HOg4y3A3or+m+XWDfyxKuKmxN9veLCYVkefZwWetGmXPdd0RJZ73hYBsdhIqg3SMHURmEQot9+in7zKIsxNrAVY5pGtZ1hiX4TIA+HID/5wWPLhefpLRBM5e+m51LWTcVzjIr+iGTdkE+/d0cNl0zVMTvfUpphNSqu5tGTlRIZS7uXdUVf80oKpuQMnl4evTRjVGux+9jpVUFGMslV3Ny66efR68fuGvrOU2C6GIxN/++w1CLqRmkwtuKmLQjCbMKUJ/Vdb1ZdMZVotcfQFR2k6nsRfb80zWExUHrlNhl0Tb+/TfXo2WAZN/I6rHWzjXFiaQ3v+vSDI+UjP3JyGb/xucM4sbSGX/zUAzjbCvDOT96PM6t5JgnlD6fB2PQD8ujJFWX3n6h6hfRKvcJT/mBHST5Gjqwb33UKBVOcczx5pgWHpesg5T1V93FwponHT6+KNEUgzWAJogTv/OT9ONsKBPG2w1TR61YCvbeTWXOyHTW/MHD73Z95CI+fXhXvs9XN+7jrqlnOJiJiJdV2ZK4liF72px1pXYNbN0T0xbYJa2Jt5vRKz3WERx9ECaq+C99hWa8bLh4HUkKk9Mte83IL66NNbpu1u6X1DDtGcKNROQcevZv1lh/ouc54KfoLMuvmjsfm8bG7nsRPvvSACFIOip/7s28JEvvzrz+FqYaPv/jG07jxwC684YZLAaiKXvagn71vEt8+vQLO01S5rtLbnBdK5XVSpBJ5sm4qLsOypuhPr3TRDmI8d/8OHD6xLAKz080KDuxu4qtPnFGeTx/gv8gyfIjoOc+CkCUq8qmzbdR9FxM1TzTIAoBnFtr446/OYrLmwWHpUO7ltVBYN2++6XKcWFrDrQ+lLXflRlk5seafnn07Ur9bV5+u6MEyoKLXPHrfdRDGqrrXA8+6Rx9EMbpZf3zfcxBR1o3k0TcqHl5/3cW4+8gZvOOfPWugtdE5aF298Es/8BzR/nhU/MoPPk8M1OiHvZNVvOnGS/Hya/pnD20m5DGTm4Xvf95Foq6jH/KMrPFg+guS6EmBjtJDmqyC2789BwD44iNp2pfc40RW9BXXEf3Kr947gZPLHSythZhuVHB8Uc1c0dWKTrKdMMFkLU+pNFk3ROzXXz6VEn0WmJ1qpIpez7vXldqEprDLFFSccBzY10TVc5QWCGQVrXQi1HwH000fi2uhuNY/8uJL0Ki4gujV8xVvz2nj0Yt7hvboqU2xyG7KsnY8J++9b+irAqRKr+o6WAtiJDx9jec4CLJgrOukj6frddGoeHj/j10/0LoIRPD9+ti87XuuGuq4JvzUyw4O/FzPdfBbb3jRus+5XuTWzeaZEN99zQy++5qZwdYzZor+grRuKMfb5Nf2w96sVewjWaYGfT0qdS3MFX3anpXsmwMzDWFBTDX8QtaNruZMih7Iid4UjBVEL2XZAKnHbQq+6T1Mqp4jVSH2LuA5ONOA7zqKRy/PuvVdB1P1ChbagXivNd9VzjlZNSh6rSIVKHZZpA/+4IpeDcbSJnrNvvyOTlf0rqboKbha8Rz4LhPpsbJHP0xPGNP6tvtIuq3CubBuhoHtdTMGoEBhWSZIL5R1+1MVvZqDTqlpB3Y3sTMjrKlGBZ0wFjnx8sxUQlHRp8eVs24AtWfL7HwLFdfBc/fvEP9vZm1+TXnTepm67zqCtCjbpAwHZ5op0UuZLPKQlYrrYKrhY6mdK/q67yqBPbOiz8/ZV9EPGKDUs27o2l2xqyk21LLKWM9lSp581Ut7wVPBm+syyaMfzcv2BrRuLlSci2DsMLiggrGMsVnG2AOMsXsZY4c2alGbDVKgchn/oGiXvObofEuQtq4aqXz84EwzV/R1HwnPN50wTgrq2WTdAHnWDXn6ckD26HwLl+9uYPdEuqEcX+oIf/vS6Xrhg1LXFKjnsHzCksN6eqIHdjdR8Vih8ybBz4h+oR2gI9kjsl0kk75vUG107XTl7g5p3ciK3mH5B7TmuziwO90A9WCsrOjl30XFc+G5TJrxmj8+6HCOwvoGtG4uVJyLgqlhQMsYNHi71dgI+fBKzvl1nPMbN+BY5wRExJ0RrBuZ6Kk3y65mBe0gFj1culIwFoBk3TSFBUGEn4/j44U/Yp146LlE7FREIvvus2da2YaSWx2khn3XwWXZ7E4i2zhR72p81xFxCM+QdSODFH2oWDe5ovc9hulGBUtrodIN03cdQYwTkgI2efRlhTpkcdQHzAYR/eizvHf6f3qn0xDfq+eQPHrpd1HxHPiOIzZeeQj2qGmIpkC0RY7KNlP0LJsstU2W0xfnxX3iHY/NGQthypAHY4e3buTc6FdkfUzo69H5Fu558izms1TLXOW5mKx52N2sYGedPPqUiD/3wAlwzhEZmmrpHv1TZ9t48NhSnnVDij77f5JwzJ5p4+BMEzXfFamHsu1xcKYJ12G4KMtmOb3clU8B32XidU5WwKWrTPrjPjjTRMV1lJz5pTXduqkg4fl55NbBQOpp092vyaMvI06h6Ae0buSsG4flRF/1HBycmRDfm86hK/qq58D3mDLMW/yuR7VuXGvd9ILIutlGzOoydmFYNwA4gH9kjN3DGHub6QmMsbcxxg4xxg7Nzc2t83RFPHZqBW/58Nfxrk8/OPBriOjLyvJ7oR1E8F2GyaqHN73kMlRcB//8RWkDpyfmWvixW+7GR74yCyBX5FftmcCNV0yDMYbnXDSJi3fWcGXWyvUXP/UADp9YMSp63R//rX94BD/78XuUYCyQ97uZb3URRIlQ7VftmVC+AsCNB3bhmr0TgkBf8ex0k6INyHMdZbhz+lVdxwsu2YmDM03salbge6pHXwzGpsc9udzJKm0paJn3atdb0MoWlmyFXDJVx4GsERcR46C9bmSlLCuxqufgJQemMVnzlDYL6nqYsglUsqwbiju4Tq74Rw/G9i+YupCR1xlsn+szTop+vemVL+OcH2eM7QXwT4yxRzjnt8tP4JzfAuAWALjxxhs3fFQ6kdwdj80P/BpSoN0RiL7VjfGDL7oYv/2GF8F1GA7/+s1C5R+dX0UYc5zOiolI5f3a668F2ej/8iWX4V++5DIwxvBHP3kjfuZjh7C0Fho9eidrQUDrPb3SxWTVk/Lo1WAs9SmnysxPvf27sNQOMTNRFcd8+yuuwttfcRV+7Ja7AQDf95x9ePsrrsbPfOwQvvDI6YJHD6QfMifKh2n/u1dcjVc/b1+m9tOsG845GGOF9go0/u7EUkdR30TgFdcRKY5mRZ8T+Z3/8ZXi+2ErY+WKR1mJVTwHr3ruPtz3rlcX/NZ8WLlTVPQuw2Lb5NGvT9FvJ8W6nWCK32w1XGfzxhpuNNa1PXLOj2dfTwP4GwA3bcSihgENyzghdVPsh2AdefStIMJE1VNu63fU0v8fzXrMnMna1ZIKZFLFHWP5H8eeLFWzHUSIEm7McNF9+k4U54peC8ZSa2FSy1XPxd4dNYXA6PxknVS8dG1N8X+nQPQV11EKSWq+I45J3iltuEuaot+ZtS44udRR/HQi8NSvp/Opyj59Xq5F5GvnD1kw5Ujqy3GYuIb0OzIF1cR6GIoevbs5Hr1V9Gb4bvFvY6sxTop+5L8qxliTMTZJ3wN4NYDB/ZMNQhBTbvngr6E8+pGsm25c+DAzxrCz7ouukYEWjC1DPn0qzhR98a+mkg21EGuPuVD4tDHEiUr0g0wDomwWsgzoNYwxobxlRb9TInpZRdMHkOwwJRjrMhF0PrncUciSrqEv2SImVVt2DYftXim/V2qDDPTuH0Oc6xUUvQvPdfKZtbKiH5XobdZNT2zHO56U6LfPenphPdbNPgB/kyksD8Cfcc7/YUNWNQT0Yc+D7PjBiMHYIEoQxInx9nyq4Rf6zetqXAeRQqsbZR69SdE72NWsYH41V8rtjNB9LeumlWW2DDINiLJd6ANE/293I6NH36h4osJXVtFEcGHEgYrBo8+CznHCFVIW1o2Xq2FPuksilN0aD1swJY4bp6mVtFn2+h3JdxgF68Zhike/bqI3xCcscvjid7F9ro8zRsHYkYmec34EwJbXRssTg44vruGyXf2n5ozaAoGaVJkCbtONCo7MtZSf9es22JDGDKZZN2ZFP1nzFKJvBeqs1Fjz6AfxiRuSRw7kBLXajYRil1MQ636qaqMgVsiV1tCNY0Sxi5VOJDYEmtfKWNo7R7ZuGop1U7yD6IdhPXpAzeoZRNHLKY9Vjeg9l4k0Wk8Kxo7u0dtgbC/421DRp23Ft3oVg2Hs/6pkRT97ptXjmTmCaPjKWM65CLqarJEpQzOkfkRPpNDqZtaNQa1UXAc7ar5yS08WjZ5eSRvRIKqSzk0EMyERPSlvVyLGmu+K88nkWhHWDReplRftTFM3/aydMDXQqsnWTcUTzykq+v5/lqJXzRDWjSvlYlNMoZe9Jlc/qgVTjkLInrRZjZp1Q0O4rXVjBv2dbubgkWFxoVg32wJyn5XZM228/Br18V/724fx9EIbH/rJvJ6rn6J//e/fiZufvx8PHl/CjpqPHTUPDx5fwq/84LUAcjUqQ++u2GtotXhORnLtIDL2owfSlgtTjQpqnoswTomcLBoi2YeOL+MVv/MlvOo5ewEMVp051aikfdYzgqLq0L2TNVHdSoTbrHrYUfPFc2ULhgiuE+atiC+equOZhTVBhrsn0qIpud3BdMNH3XeV1ER3iMyKYStj5eM6TtpyQl5/r3Okil316OXAuZcF5IFiq4ZBkZ9r7LXXpmA7Kvp6xR0qRrSVGHuil1Mkl7Wh10Ca8vjUWdU770X0nHM8fGIZV+2dwGOnVtCsepis+Tg0uyCaWpkUs/4BH3RIRLPqoRVEiJLEmHXznh99Iaqegzd88C7Ra6XdzRuEAcDdR9LWw7dlnTRNG5GON954KZ67f4d4L6967l78z3/9Erz8mhn8jy88DiAvjPqtN7wQNc/Fj384TcmUc9dJvR9bWBMBWyrGIqJ/7xtfhAeOLeGlV+4Wr/vXLzuI781y+Cuaoh/EuhGVsUMRPWXRMJGt1auXj9LUTFP0NcmicR2G11x7ET72b24auYXwoN0rL1Tk6ZXbh1jf/2PXFZrtbVeMPdHLit406CGMufBSxWt6tEBoZTNW2904s1TSVrTdKMETWR96k2LWBzf3y7ghNKuuOI+J4J61bxKAqqJJ0ZPnLd856LZCGSZrPl56VU68jDG8MrsjoA2ELCFaAylvmVypI+bsmZbo7U8dPinmcP3l07j+8mnl/LuaFexq7sqOW7SK+mHkYCzSVEra7HsVXHkK0efPk0dE0vNqvovvedboPdttZWxvCEW/jTbCay/eudVLGBjjT/TybNJukbiDOCm0Rwh7NDVbyHLgW0GEdhChFeS3+Q9mc19N03Z062ZgRV/xsqwbs0dPkP1tUvR60DQ93vonAdGmol+3ipumesqqc+9kFY2KiyNzLVy8M1Wze3dUxfMHga7oB/HoR0qvJI+esULRmQnkvxabmqmN2TYit3uYQPSFCM9QY2ExOMZePpBa31n3RZBSRpDN+VR+1qNNMQUUW90IrW6MpbUQC630Zw8dT+e+GoOxBUU/GOEK6ybmPTcHWbkKRe/ng7zl460XdFy5WRmQF1PJKY+MMRzY3cTsmZZY195J1brpB0H0mkc/SEZMbcDrDKhWjAjG9tgo5NztvPgt/b9p1u164Dvbz5rYTvA1MWAxHMb+r4pUJ3WQ1BH2UPQm64bywBfXwqy0P29k9uCxTNEbPHDdqxtU0TcqrpR1U/5HLNsl9D7Julnp5LGJUdvkyqBNRR9qUvUcoyd+cKaJ2fmWuKMi60YfjViGsjz63hkxmUc/xB1MPhVIsrp6bEamFghVzwFjTMmu2YjcbtosTCm2FnlWklX0o2Hsib4bxaINwapB0Ydxovj49DMA6Bg2BhqcQS2H1XOlrzORKQUiZ7I+8AN79Jl1Q0OmyyArT3qfRLry+x61e6KMUutGao8g4+BME08vrGFxLd0kqbXDsIpeH+bQ667IG2Az0GFqsdDL+smfn28IpgEjG6Ey7YSp3thu/ejHDeeFR19x08rNsmBsnKRtgB88voy/ve+4FIwtWjeUWkgZNgQqm2fMnOlBnQ8vnqpjfjUYKuuG7CK/xx8xtR3uhIl4n7mil4l+/b/Saol1U/Uco9VxIJtF+2g2VpGIvjKgpUGETh9iOq8pFkKgTJhhmkrl6ZWyou9xDpYrerom9FUZar4B1o0gMuvRG3EuZsaezxh7ou9GCaq+g2bVw7HFYmMzIvUgTvD5h0/hw3ceFXNKTemVcq8WGT/50ivw7VMrePa+HcYGWPt31PDWl16By3Y1cP8zSwMHIptVF4tE9D02hzfecCmuu2wKv33ro4X0Stm66UWOg6JeYt284YbLMLdavNO5OEuxPDrfQt13sbPu423fcyW+7zn7Bjqfrugvmarjp152AD/+HZeXvuZ1L9wvBr8MijfeeCkmax5e+/yLcN/Ti8q5TXAlj15X9I1NUvSD/t1caJisevi3Lz+I78sywyyGw9gTPSn6ZtU1KnqybbphIiwOykenma2yKlwsIfofvv4SvPDSqdJ1OA7Dr77++fjG7FkAg/dJb1Y9sRn1IoxXX3sRXvmcBL9966OFYOxGK3rh0ccq0d/8/IuMz6eMo2MLa2hW02DtL/3Acwc+X+7Rp18dh4nitDK85MAuvOTAroHPAQA/+dID+MmXHgAA/PKnH1LObYIcM/DdtNyd7mjkYOxGevS2140ZjDH8p9c9b6uXMbYY+7+qIEqydDfPmF5JNkAQJ4WNIOEo+PfyzFMZU/XB1KMYqDGoopcUeD9P23cdeA4Tir5hsm42JBibefRx0doygTKOzrSCkTYaXdGfS/RS9HJ6JWOpqheKvqLm0a8X23GCksX5g7En+m6UZAUsrjG9kloSd8NE2QhIDesplouG6loAmGoOVtpOSm/QHiwyYQwSvKz5bqFgSg3Gbl56ZRnkjKNRer2QR78VVaGDKHo58Et3aqqi30DrZojgsoXFoBj7v6pulKDiuWhUPKyFsTIoG8inSQVxLAgSSHvIAMUpUwsGRe86TPj6/UBEVx1Q0Q+bj13zHdF735QBszEFU2brpgz1ipsP3hjh/Fup6HtZJXITNACoeK74vW64R2+zSiw2EecB0cdKSbpsz3DOld7zsuInoteHjyy2Q4WsGhUXU3V/4OwOSrsbWNFXZetmEKJXCUZ/zSC96PuewxtO0QO5fTPKHYXu0W8XmBV9RvTy72EDfHXftkCw2ESM3V9VO4hw2+FT4v/Co6fBGVlu/EPHl/B41psGSP1m2bqh0XidMMGtD50UAdHFdoBLpvPGVJdM1YfqSFj3XTA2hEdfHd66Ieg9WIB8gMh6QGQ2DNGTfTNKHr/e62a7gDx6RyL6ikiDzH9XG9kCwRK9xWZg7P6q/vqeZ/DTHz2EJ7Pe8+TRUxCS/Opf+MT9+PW/Oyxe1w3VYOzuLDXvK4/P42c/fg8+98AJdKMYi2shLpeGl9x0cBdecMngzYsYY7j24h2iwVc/XL6rAd9NWxoP0vmQAqUOU0fY7az7mG74uHrv5MBrLQMR78+98uqBX0MFY6N49FfOTGCq4WPfjmr/J28QfvTFl/Z9TsV1cHCmKRq3PWvfJK7eV/y9boTdcuWeCVy0o6aMbLSw2CiMXXoldZA8MtfCFbubCIjoybrJVPvcaheyExDECVYlRX/V3gl8/vBp3PZIenfwxNwqnj7bBufA8/bvwOcPn0az4uK//vALhl7jZ/+vlw/83Kv2TOCBd7+mMIGpDBQovXS6IfrZA2lr4Ft//nuGXqsJjDHMvud1Q72GFP0g82p1vODSnbj3Xa8e+nXrwXvf9CK89029B6Q5DsMX/99XiP9/8C03GJ+3EYr+e5+1B3f/0qvWfRwLCxPGTtEfzeayHpnPBnHHqXUjpjUFETjnWGwHStphEKmK/uDuJjyH4etH07z3o/MtHJ1Pj31tpuA3wu8eBDXfHbhnC1k3BzKVKXqwbPEAhOkmKfr1W0fjBhtAtdjuGDuin80Inr6mwVhXGbTdznrKy4NI1sJYaXpWr7i4fHdDZOXMnmnh6Hx6t3DtxTsAjKZONxvkyV+ZET0p+q2uqNxZJ49++12zzcZ2iy1YWOgYK6IPogTPLKSqm+bDypWxQDo4ZMHQr2ZJS5v0XQcHs/F5ADA738bR+TamGz72Z33Vt6M6Xe2mm9eB3WkcYdsoesq62YbXbLMxTL8dC4utwFgR/VNn20h4Sm5H56VgrO+IIGCrG4k2BnKvlrMttRCq4jqK/bHajfCN2bM4MNOE6zDUfXdDqkw3GqeW014zYu3u9lD0lJl0ruwuCwuLwTE2RM85x/3PpI2ovvPK3Ti2uIZOGEuKvkj0MvRCKN/Lif47s1mmj59eFSq/WfU2pOXvRuPEUtq47aCwbrKWC1tcUTm1jmCshYXF5mJsiP4jX5nF//1X94Ex4JXP3gPOgWcW1vLulVI7AFN169mWbt0wXJOlQL5WatZ1VfazXU1ftB7eTnju/jR+cEmWipkPxNjaTWlfNhB8XIYlbwT2Z107LSy2O8ZGfh2dX8VE1cMHfuLForhpuRMiTjgqrgvPTatjl9ZCY78aIn+Hpc3MKq6DG66Yxp/89HfgZVfvxr4dVSy0Qnz/tWlr3f//x1+Mydr2y2n+yFtfgmOLa6JgRwRjt1jRv+jSnfizn/kOfOeVw3WUHGf83b9/OU4udbZ6GRYWfTE2RL/QDrF3soqXX7MHX3l8HgBEVg0FInfWfSy1Qyy2yhX97okq5la68N10aMV3XzMDAIXe6RtReLQZmG5WlDsNecTdVoIxhu+6emZL13CusatZGbonvoXFVmBsrJuldigCflQdSpOZKBA53fSx0A6Mip58+z0T2fSj86RL4Hbx6C0sLLYv1sUOjLGbGWOPMsYeZ4y9c6MWZcJCOxABPyI3UvREclP1ChbXwp4e/d4dw80z3e7YLh69hYXF9sXIbMcYcwH8AYDXAngegDczxjZtBMyipOipipQUPdkWUw0fi+3QmHVDXSr3inmm5wfRbxeP3sLCYvtiPexwE4DHOedHOOcBgL8A8PqNWVYRi+1ATHmiNgBLmqKfblSw2A5Kp0T97/buL0aqs4zj+PeZ2ZnZZUEWuksl/OkuiGmpmkpJ07SGC21qi0lpTS+4siYmJNUmemEiSjR4YYwmemFibGokqcbYKmrkokabWuOV0FqBQhC71ZoiWLQIhUoKlMeL857Zw3b+wOzCmffd3yfZ7JkzB+Z58u4++573vOc9MPXg6tpAGje5NPpkjF5E+tdMqsMy4NXC6yNh36w7d+Eib557u3n35WAoam+cze58LfboT509z4k3zzUX/yq+Xx+oNFcHTG/oJo18RGT2zWTWTasu8TseSWRmW4AtACtXruzpg06ezXro04duToSe+1C4g3VkXp2Lnt1B+97rF/CXf50GsoeMvHXmHGPzG9y99t3893/nm8sUx049ehHpZibV4QiwovB6OXB0+kHu/pi7r3f39WNjYz19UD7mnl+MzZ+A9PqZbDmA/GapvMd/MSw1nMvvmp0YHWZ8dJgv3HNjMuuT1DVGLyJdzKQ6PAesMbMJM6sDm4FdsxPWpfJCn991WakY9WqlOZMmL+TFJ0HdVCz09alCuz3cAAAABqxJREFUnxpNrxSRbnoeunH3C2b2CPAboArscPeDsxZZQT5dsljIB2sVXj8TCn1h6CZ349KpG57yR+KNJ1joNb1SRLqZ0Z2x7v4U8NQsxdLWqebQTbHQVzl+Ohu6yZ8XOxIutDYGKowXliDOj5sYnXpEYCr6ZfVKEelfUVSHqR79VI+9+JDsfMXEfGhn/Lrh5lry9WqlOQ2zWPxTkS//UPZ69CLSv6KoDifPnqdWtUseapFPn6zY1IyTdw3VMIPx0XnNPwS16tRF1xWL1aMXkbkniupw6ux5Fg7VL5kpk693M9wYaO6vVoxbVy7izveMThX6gQqfvGOcxcP1ZObOF61eMp8lCxrckODZiojMjihWr/za/e/jyx+7dHWFRijk058CtfPhO5rb9WqFWrXC9vtuZvt9N1/9QEuwemw+e7bdVXYYItLHoujimlnzJqlc3mPv9BSoRq2iIQ0RmfOirYJDhaGb9sdUNb9cROa8aKtg3qOfV2/fox+sVS+5GCsiMhfFW+jDDUKdHkY9WKskeQFWRORKRHExtpV8zH5evfPQTSpr2oiI9CraQt+4jDH6RcN1/B3raYqIzC3RFvp86Ga4wxj91z/+fhV6EZnzoi30+dBNpx790oVD1yocEZG+Fe2VyvwpU53m0YuISMyFvtb9YqyIiCRQ6DtNrxQRkQQKfacbpkREJOpC3316pYiIRFzob5tYzJYNq7j1hkVlhyIi0tei7Q7Pqw/wpY03lR2GiEjfi7ZHLyIil0eFXkQkcSr0IiKJU6EXEUmcCr2ISOJU6EVEEqdCLyKSOBV6EZHEmV/DJ3OY2b+Bf/T4z0eB/8xiOP0gxZwgzbyUUxxSzWnY3cd6/Q+uaaGfCTN73t3Xlx3HbEoxJ0gzL+UUB+XUmoZuREQSp0IvIpK4mAr9Y2UHcBWkmBOkmZdyioNyaiGaMXoREelNTD16ERHpQRSF3szuMbPDZjZpZlvLjqdXZvaKmb1oZnvN7Pmwb7GZPW1mL4Xvff0kFTPbYWbHzexAYV/LHCzzndBu+81sXXmRt9cmp+1m9s/QVnvNbGPhvS+GnA6b2UfLibozM1thZs+a2SEzO2hmnw37o22rDjnF3laDZrbHzPaFvL4a9k+Y2e7QVk+aWT3sb4TXk+H98a4f4u59/QVUgZeBVUAd2AesLTuuHnN5BRidtu+bwNawvRX4RtlxdslhA7AOONAtB2Aj8GvAgNuB3WXHfwU5bQc+3+LYteFnsAFMhJ/Natk5tIhzKbAubC8A/hpij7atOuQUe1sZMD9s14DdoQ1+CmwO+x8FHg7bnwYeDdubgSe7fUYMPfrbgEl3/5u7nwOeADaVHNNs2gQ8HrYfB+4vMZau3P0PwIlpu9vlsAn4oWf+CIyY2dJrE+nla5NTO5uAJ9z9LXf/OzBJ9jPaV9z9mLu/ELZPA4eAZUTcVh1yaieWtnJ3PxNe1sKXAx8Gdob909sqb8OdwEfMzDp9RgyFfhnwauH1ETo3bj9z4Ldm9icz2xL2Xe/uxyD7QQaWlBZd79rlEHvbPRKGMXYUhtSiyymc2n+QrKeYRFtNywkibyszq5rZXuA48DTZ2cdJd78QDinG3swrvH8KuK7T/x9DoW/1lyrWqUJ3uvs64F7gM2a2oeyArrKY2+57wGrgFuAY8K2wP6qczGw+8HPgc+7+RqdDW+zry7xa5BR9W7n72+5+C7Cc7Kyj1QOx89ivOK8YCv0RYEXh9XLgaEmxzIi7Hw3fjwO/JGvQ1/JT5PD9eHkR9qxdDtG2nbu/Fn75LgLfZ+qUP5qczKxGVhB/7O6/CLujbqtWOaXQVjl3Pwn8nmyMfsTMBsJbxdibeYX3F9Jl6DGGQv8csCZcga6TXXzYVXJMV8zMhs1sQb4N3A0cIMvloXDYQ8CvyolwRtrlsAv4RJjRcTtwKh826HfTxqcfIGsryHLaHGY+TABrgD3XOr5uwpjtD4BD7v7twlvRtlW7nBJoqzEzGwnbQ8BdZNcfngUeDIdNb6u8DR8EfufhymxbZV9xvsyr0hvJrrC/DGwrO54ec1hFNgNgH3Awz4NsbO0Z4KXwfXHZsXbJ4ydkp8fnyXoWn2qXA9kp5ndDu70IrC87/ivI6Uch5v3hF2tp4fhtIafDwL1lx98mpw+Rnc7vB/aGr40xt1WHnGJvqw8Afw7xHwC+EvavIvvDNAn8DGiE/YPh9WR4f1W3z9CdsSIiiYth6EZERGZAhV5EJHEq9CIiiVOhFxFJnAq9iEjiVOhFRBKnQi8ikjgVehGRxP0fjv/a/XJ3QOsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import time\n",
    "import math\n",
    "import datetime\n",
    "import torch                       \n",
    "from dqn_agent import Agent\n",
    "from collections import deque\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline \n",
    "run = 0\n",
    "\n",
    "agent = Agent(state_size=37, action_size=4, seed=1, fc1_units = 81, fc2_units = 69)\n",
    "\n",
    "def dqn(n_episodes=2000, max_t=1000, eps_start=.99, eps_end=0.01, eps_decay = .996, run = 0):\n",
    "    \"\"\"Deep Q-Learning.\n",
    "    \n",
    "    Params\n",
    "    ======\n",
    "        n_episodes (int): maximum number of training episodes\n",
    "        max_t (int): maximum number of timesteps per episode\n",
    "        eps_start (float): starting value of epsilon, for epsilon-greedy action selection\n",
    "        eps_end (float): minimum value of epsilon\n",
    "        eps_decay (float): multiplicative factor (per episode) for decreasing epsilon\n",
    "    \"\"\"\n",
    "    start_time = time.time()                               #just grabbing the start time so cause it will\n",
    "                                                           #make me feel better when I ask are we done yet.\n",
    "    scores = [] #scores from each episode\n",
    "    scores_window = deque(maxlen=100)                      #last 100 scores\n",
    "    eps = eps_start #epsilon\n",
    "\n",
    "    for i_episode in range(1, n_episodes+1):               #main for loop\n",
    "        env_info = env.reset(train_mode = True)[brain_name]\n",
    "        state = env_info.vector_observations[0]            # get the current state \n",
    "        score = 0                                          # reset the score counter\n",
    "        done = False                                       # are we done yet?\n",
    "        while not done:                                    # play time\n",
    "            action = agent.act(state,eps)                  # what should we do, I'm surrounded by bananas\n",
    "            env_info = env.step(action)[brain_name]        # send the action to the environment\n",
    "            next_state = env_info.vector_observations[0]   # get the next state\n",
    "            reward = env_info.rewards[0]                   # get the reward\n",
    "            done = env_info.local_done[0]                  # see if episode has finished\n",
    "            agent.step(state,action,reward,next_state, done) # what can we learn from our actions?\n",
    "            score += reward                                # count our rewards\n",
    "            state = next_state                             # needed to make q update work - the whole temporal differnce thing\n",
    "            if done:                                       # are we done yet?\n",
    "                break                                      # yes, with the game anyway\n",
    "        scores_window.append(score)                        # add score to the buffer so we can calc mean over 100\n",
    "        scores.append(score)\n",
    "        eps = max(eps_end,eps_decay*eps)                   # make epsilon a wee bit smaller\n",
    "\n",
    "        count = 0                                          #note terribly useful.  Merely used to tell us how many times we've reached 100\n",
    "        for j in range(len(scores_window)):                #are we there yet? no idea, but we've hit 13 x times.\n",
    "            if scores_window[j] >= 13:\n",
    "                count+=1\n",
    "        elapsed = datetime.timedelta(seconds = time.time()-start_time)  # are we there yet?  No idea, but we started 5 minutes ago\n",
    "        print('\\rEpisode {}\\t elapsed: {}\\tAve. Score: {:.2f} \\tscore {} \\tscores >= 13: {} \\teps.: {:.2f}'.format(i_episode, elapsed,np.mean(scores_window),score, count,eps), end=\"\")\n",
    "        if np.mean(scores_window) >=13:  #this is where we check to see if we have met the completion criteria.\n",
    "            print(\"terminating at episode :\", i_episode, \"ave reward reached +13 over 100 episodes\")\n",
    "            break\n",
    "    torch.save(agent.qnetwork_local.state_dict(), 'checkpoint'+str(run)+'.pth') # save the weights so we don't have to train again to see our agent in action\n",
    "    return scores,i_episode\n",
    "\n",
    "scores, episodes = dqn(eps_start = .1,run=run)           #i'm not sure I actually used episodes in the end\n",
    "plt.plot(scores)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.2 Loop through some training with randomly selected hyper parameters.\n",
    "The code below was used to loop through some hyper parameters.\n",
    "\n",
    "Note: this is an infinite loop.  You will have to interrupt the kernel when you get bored with it.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "done = False\n",
    "run = 1\n",
    "run_stats = []\n",
    "while not done:\n",
    "    fc1_u = np.random.randint(37,256)                        #generate random number of nodes\n",
    "    fc2_u = np.random.randint(8,fc1_u)                       #more random nodes\n",
    "    eps_s = np.random.rand()                                 #randomly initialize epsilon, (this turned out to be the most intersting of the three)\n",
    "    agent = Agent(state_size=37, action_size=4, seed=1, fc1_units = fc1_u, fc2_units = fc2_u)\n",
    "    print('run', run, 'eps_s',eps_s)\n",
    "    scores, episodes = dqn(n_episodes = 2000, eps_start = eps_s,run=run)  #start learning with new parameters\n",
    "    print(\"\\n-------------------------\\n\")\n",
    "    plt.plot(scores)\n",
    "    plt.show()\n",
    "    print(\"-------------------------\\n\")\n",
    "    run +=1\n",
    "    run_stats.append([run,episodes,fc1_u,fc2_u,eps_s])       #This could be useful if you let this cell run for a long time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.3 Finally see a trained agent work.  \n",
    "The weights are saved by run number.  If you want to see how a particular agent performs, change the checkpoint file name to the run you want to see."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score: 11.0\n"
     ]
    }
   ],
   "source": [
    "agent.qnetwork_local.load_state_dict(torch.load('checkpoint0.pth'))\n",
    "\n",
    "env_info = env.reset(train_mode=False)[brain_name] # reset the environment\n",
    "state = env_info.vector_observations[0]            # get the current state\n",
    "score = 0                                          # initialize the score\n",
    "while True:\n",
    "    action = agent.act(state,.05)                  # select an action\n",
    "    env_info = env.step(action)[brain_name]        # send the action to the environment\n",
    "    next_state = env_info.vector_observations[0]   # get the next state\n",
    "    reward = env_info.rewards[0]                   # get the reward\n",
    "    done = env_info.local_done[0]                  # see if episode has finished\n",
    "    score += reward                                # update the score\n",
    "    state = next_state                             # roll over the state to next time step\n",
    "    if done:                                       # exit loop if episode finished\n",
    "        break\n",
    "    \n",
    "print(\"Score: {}\".format(score))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "drlnd",
   "language": "python",
   "name": "drlnd"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}