who-said-that.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "People have their respective traits and looks by which we can recognize who they are; what if we took all of it away and were left only with their messages? Would we be able to recognize them? Would AI be able to recognize them?\n",
    "\n",
    "Answering this question, I will create an ML pipeline for classifying the sender of a message based on other messages they sent. I will also compare the results with humans to see if the signal is strong enough for at least humans to recognize each other."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data\n",
    "\n",
    "## Dataset\n",
    "\n",
    "The dataset used in this assignment is a collection of all the messages from one of the group chats, where I and 4 of my friends exchange messages. The group chat was created during the quarantine in Seoul, South Korea, in early February 2022 and has been used until now.\n",
    "\n",
    "### What is in the dataset?\n",
    "\n",
    "Messages from the five people in the context of this group. Messages related to everyday life, funny situations, planning trips or outings together. \n",
    "\n",
    "### What does the dataset not include?\n",
    "\n",
    "Any form of text or written communication is not in this group chat's context. What I mean by this is that people have different writing styles in different contexts. For example, an email will have a different format, tone, word choice, etc. The same goes for different people. For instance, in the same context of telegram direct messages, I would formulate my messages differently based on who is on the receiving end due to differences in communication goals and relationship with the person. Therefore, I want to emphasize that this dataset includes only texts from the context of the group chat with those specific people.\n",
    "\n",
    "## Data Generation\n",
    "\n",
    "Data was generated by texting the group chat without any prior knowledge that one might use it as a dataset for a machine learning project, so there should be no substantial problems with any biases.\n",
    "\n",
    "## Data Gathering\n",
    "\n",
    "The `telethon` package was used to gather and save all the data to the JSON file. The names of the people were removed, and IDs were used instead to identify them. The study has three main components of interest: texts (text content of the message), timestamps for the message, and sender's ID. The model will make predictions solely from the text content of the message and features that will be engineered before training the model.\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"id\": 133146,\n",
    "    \"text\": \"Just ran up my leg\",\n",
    "    \"from\": 483161515,\n",
    "    \"timestamp\": 1706106108.0\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Converting to Data readable Python\n",
    "\n",
    "In this case, it is pretty simple as we can use `json` module to load data to a dictionary, which can be directly used to create a `Pandas` `DataFrame`. I use `DataFrame` as it allows for easier data manipulation in the pre-processing step compared to a `np.array`, but both can be used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>from</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>133146</td>\n",
       "      <td>Just ran up my leg</td>\n",
       "      <td>483161515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>133145</td>\n",
       "      <td>It was</td>\n",
       "      <td>483161515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>133144</td>\n",
       "      <td>Hahahah</td>\n",
       "      <td>483161515</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>133143</td>\n",
       "      <td>aggressive much?</td>\n",
       "      <td>1798750378</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>133142</td>\n",
       "      <td>omg</td>\n",
       "      <td>1798750378</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       id                text        from\n",
       "0  133146  Just ran up my leg   483161515\n",
       "1  133145              It was   483161515\n",
       "2  133144             Hahahah   483161515\n",
       "3  133143    aggressive much?  1798750378\n",
       "4  133142                 omg  1798750378"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import json\n",
    "import pandas as pd\n",
    "\n",
    "# Open file with data\n",
    "with open('data.json', 'r') as f:\n",
    "    # Read data with json module\n",
    "    data = json.load(f)\n",
    "\n",
    "# Create a dataframe based on the data\n",
    "df = pd.DataFrame(data)\n",
    "\n",
    "# Print the first 5 examples\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pre-processing\n",
    "\n",
    "## Cleaning Up\n",
    "\n",
    "We should clean up the data to make training the model easier and more efficient.\n",
    "\n",
    "### Remove Links\n",
    "\n",
    "It is often the case that someone would send a link to a TikTok video or Airbnb. Those links, while they might indicate who the author of the message is, are not intended to be used for classification. \n",
    "\n",
    "### Non-alphanumerical Characters\n",
    "\n",
    "At some point, we want to vectorize the data so that it is in a readable format for the model. Punctuation signs and non-alphanumerical characters would create additional tokens and unnecessary complexities for this classification.\n",
    "\n",
    "### Transform all Words to Lowercase\n",
    "\n",
    "Like the previous point, the model with limited data would have hardships recognizing the differences between the lowercase and uppercase tokens, so I will transform all words to lowercase.\n",
    "\n",
    "### Remove stopwords\n",
    "\n",
    "In english there are multiple words that do not convey any meaning. For example, 'a', 'about', 'your', 'an', 'because', etc. We remove those words to further simplify the model.\n",
    "\n",
    "### Target Classes\n",
    "\n",
    "Currently, the target classes are the IDs from Telegram; however, to simplify the analysis further, I will change them to numbers from 0 to 4.\n",
    "\n",
    "### Messages with 0 length\n",
    "\n",
    "Telegram can send messages other than simple text, in which case the message content is empty. Messages with a length of 0 will be removed from the dataset to prevent situations in which the message's content is empty. \n",
    "\n",
    "### Lemmatization\n",
    "\n",
    "Words can have different forms. For example, one might say \"he likes swimming\" or \"I like to swim.\" Both have similar meanings, yet would be represented as two different tokens with the vectorizer. To prevent that we use the lemmatization, which reduces different words to their base form. \n",
    "\n",
    "## Feature Engineering\n",
    "\n",
    "After playing a game of guessing who said what, I found some features that might help the model recognize the people better.\n",
    "\n",
    "### Starts with a Capital Letter\n",
    "\n",
    "Some people start their messages with capital letters, which can be a good indicator of whether or not the message comes from that group of people.\n",
    "\n",
    "### Number of Punctuation Signs\n",
    "\n",
    "Again, some people like their exclamation marks while others do not, so having information about the number of punctuation signs can be helpful.\n",
    "\n",
    "### Use of Numbers\n",
    "\n",
    "In the group chat, we have different majors—some like to quantify their claims while others don't, so the number of digits in a text can be another good indicator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# some messages have a different type than text\n",
    "# for example files, geo locations, videos, etc,\n",
    "# we do not include them in the analysis;\n",
    "# thus, we drop them here\n",
    "df.dropna(inplace=True)\n",
    "\n",
    "# Get rid of any unnecessary whitespaces\n",
    "df['text'] = df['text'].map(lambda x: x.strip())\n",
    "\n",
    "# Drop any links as we do not want the model\n",
    "# to learn based on links, but rather actual\n",
    "# messages\n",
    "df = df[\n",
    "    [(\"http\" not in s) and (\"tg://\" not in s) and bool(s) for s in df[\"text\"]]\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import string\n",
    "\n",
    "# add a feature to the dataset\n",
    "df['starts_with_capital'] = df['text'].map(lambda s: 1 if s[0].isupper() else 0)\n",
    "# add another feature counting the number of numbers in the text\n",
    "df['number_count'] = df['text'].map(lambda s: sum([1 if ch.isnumeric() else 0 for ch in s]))\n",
    "# count the number of punctuations signs in a string\n",
    "df['punctuation_count'] = df['text'].map(lambda s: sum([1 if ch in string.punctuation else 0 for ch in s]))\n",
    "\n",
    "additional_features = ['starts_with_capital', 'number_count', 'punctuation_count']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import nltk\n",
    "from nltk.corpus import stopwords\n",
    "from nltk.stem import WordNetLemmatizer\n",
    "\n",
    "stp_wrds = set(stopwords.words(\"english\"))\n",
    "wnl = WordNetLemmatizer()\n",
    "\n",
    "# remove all non-alphanumeric characters and non-spaces\n",
    "df[\"text\"] = df[\"text\"].map(\n",
    "    lambda s: \"\".join(filter(lambda x: x.isalnum() or x.isspace(), s))\n",
    ")\n",
    "\n",
    "# remove the common stopwords\n",
    "df[\"text\"] = df[\"text\"].map(lambda s: ' '.join(filter(lambda w: w in stp_wrds, s.split())))\n",
    "\n",
    "# send all characters to lower\n",
    "df[\"text\"] = df[\"text\"].map(lambda x: x.lower())\n",
    "\n",
    "# lemmatize using wordnet\n",
    "df[\"text\"] = df[\"text\"].map(lambda x: wnl.lemmatize(x))\n",
    "\n",
    "# Drop messages with less than 5 words\n",
    "# We need to give our model something to work with\n",
    "df = df[[len(s.split()) > 0 for s in df[\"text\"]]]\n",
    "\n",
    "# Transforms the labels from telegram ids to simple\n",
    "# numbers from 0 to 5\n",
    "labels = df[\"from\"]\n",
    "lton = {l: i for i, l in enumerate(set(labels))}\n",
    "df[\"from\"] = df[\"from\"].map(lton)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>from</th>\n",
       "      <th>starts_with_capital</th>\n",
       "      <th>number_count</th>\n",
       "      <th>punctuation_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>133146</td>\n",
       "      <td>up my</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>133145</td>\n",
       "      <td>wa</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>127647</td>\n",
       "      <td>is a</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>122273</td>\n",
       "      <td>do it</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>122272</td>\n",
       "      <td>or</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id   text  from  starts_with_capital  number_count  punctuation_count\n",
       "0   133146  up my     1                    1             0                  0\n",
       "1   133145     wa     1                    1             0                  0\n",
       "9   127647   is a     1                    1             0                  0\n",
       "14  122273  do it     1                    1             0                  0\n",
       "15  122272     or     0                    0             1                  0"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAGrCAYAAAAmWFaFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA1rklEQVR4nO3deXRU9f3/8dcEsrDNxACZEEkAlTaksjWhMEAVMRIgWpVYBSNgRfmJgQpBhLQsBREo35ZFWVIBIa1Sq1atjbLEqGAhLAZERDYRTSxMAtJkBEwCyfz+8DB1CrTcZJKZXJ6Pc+45mc/nM/e+r9fjvPzczeJ2u90CAAAwqSB/FwAAAFCXCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUGvu7gEBQXV2tY8eOqUWLFrJYLP4uBwAAXAG3261vvvlG0dHRCgq6/PwNYUfSsWPHFBMT4+8yAABADRQVFalt27aX7SfsSGrRooWk7/5hWa1WP1cDAACuhMvlUkxMjOd3/HIIO5Ln1JXVaiXsAADQwPyvS1C4QBkAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJhaY38XAPhD+ylv+buEWvtiXoq/SwCABoGZHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGp+DTtVVVWaNm2aOnTooCZNmuj666/XU089Jbfb7Rnjdrs1ffp0tWnTRk2aNFFSUpIOHz7stZ5Tp04pLS1NVqtV4eHhGjVqlE6fPl3fuwMAAAKQX8POb3/7Wy1fvlxLlizR/v379dvf/lbz58/Xs88+6xkzf/58PfPMM8rKytL27dvVrFkzJScnq7y83DMmLS1N+/btU25urnJycrR582aNHj3aH7sEAAACjMX9/WmUenb77bfLbrdr1apVnrbU1FQ1adJEL7zwgtxut6KjozVx4kQ98cQTkqSysjLZ7XatWbNGQ4cO1f79+xUfH6+dO3cqMTFRkrR+/XoNHjxYX331laKjo/9nHS6XSzabTWVlZbJarXWzswgoPGcHABq+K/399uvMTu/evZWXl6dDhw5Jkvbs2aN//OMfGjRokCTp6NGjcjqdSkpK8nzHZrOpZ8+eys/PlyTl5+crPDzcE3QkKSkpSUFBQdq+ffslt1tRUSGXy+W1AAAAc/LrE5SnTJkil8uluLg4NWrUSFVVVXr66aeVlpYmSXI6nZIku93u9T273e7pczqdioyM9Opv3LixIiIiPGP+09y5czVz5kxf7w4AAAhAfp3Zefnll/Xiiy9q7dq12rVrl7Kzs/W73/1O2dnZdbrdzMxMlZWVeZaioqI63R4AAPAfv87sTJo0SVOmTNHQoUMlSZ07d9aXX36puXPnauTIkYqKipIkFRcXq02bNp7vFRcXq1u3bpKkqKgolZSUeK33/PnzOnXqlOf7/yk0NFShoaF1sEcAACDQ+HVm5+zZswoK8i6hUaNGqq6uliR16NBBUVFRysvL8/S7XC5t375dDodDkuRwOFRaWqqCggLPmHfffVfV1dXq2bNnPewFAAAIZH6d2bnjjjv09NNPKzY2Vj/60Y+0e/duLViwQA899JAkyWKxaPz48Zo9e7Y6duyoDh06aNq0aYqOjtZdd90lSerUqZMGDhyoRx55RFlZWTp37pzGjh2roUOHXtGdWAAAwNz8GnaeffZZTZs2TY899phKSkoUHR2t//f//p+mT5/uGfPkk0/qzJkzGj16tEpLS9W3b1+tX79eYWFhnjEvvviixo4dq1tvvVVBQUFKTU3VM888449dAgAAAcavz9kJFDxn5+rDc3YAoOFrEM/ZAQAAqGuEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGp+DTvt27eXxWK5aElPT5cklZeXKz09XS1btlTz5s2Vmpqq4uJir3UUFhYqJSVFTZs2VWRkpCZNmqTz58/7Y3cAAEAA8mvY2blzp44fP+5ZcnNzJUk///nPJUkTJkzQ3//+d73yyivatGmTjh07piFDhni+X1VVpZSUFFVWVmrr1q3Kzs7WmjVrNH36dL/sDwAACDwWt9vt9ncRF4wfP145OTk6fPiwXC6XWrdurbVr1+qee+6RJB04cECdOnVSfn6+evXqpXXr1un222/XsWPHZLfbJUlZWVmaPHmyTpw4oZCQkCvarsvlks1mU1lZmaxWa53tHwJH+ylv+buEWvtiXoq/SwAAv7rS3++AuWansrJSL7zwgh566CFZLBYVFBTo3LlzSkpK8oyJi4tTbGys8vPzJUn5+fnq3LmzJ+hIUnJyslwul/bt23fZbVVUVMjlcnktAADAnAIm7LzxxhsqLS3Vgw8+KElyOp0KCQlReHi41zi73S6n0+kZ8/2gc6H/Qt/lzJ07VzabzbPExMT4bkcAAEBACZiws2rVKg0aNEjR0dF1vq3MzEyVlZV5lqKiojrfJgAA8I/G/i5Akr788ku98847eu211zxtUVFRqqysVGlpqdfsTnFxsaKiojxjduzY4bWuC3drXRhzKaGhoQoNDfXhHgAAgEAVEDM7q1evVmRkpFJS/n3BZUJCgoKDg5WXl+dpO3jwoAoLC+VwOCRJDodDe/fuVUlJiWdMbm6urFar4uPj628HAABAwPL7zE51dbVWr16tkSNHqnHjf5djs9k0atQoZWRkKCIiQlarVePGjZPD4VCvXr0kSQMGDFB8fLyGDx+u+fPny+l0aurUqUpPT2fmBgAASAqAsPPOO++osLBQDz300EV9CxcuVFBQkFJTU1VRUaHk5GQtW7bM09+oUSPl5ORozJgxcjgcatasmUaOHKlZs2bV5y4AAIAAFlDP2fEXnrNz9eE5OwDQ8DW45+wAAADUBcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNb+HnX/+85964IEH1LJlSzVp0kSdO3fWhx9+6Ol3u92aPn262rRpoyZNmigpKUmHDx/2WsepU6eUlpYmq9Wq8PBwjRo1SqdPn67vXQEAAAHIr2HnX//6l/r06aPg4GCtW7dOn376qX7/+9/rmmuu8YyZP3++nnnmGWVlZWn79u1q1qyZkpOTVV5e7hmTlpamffv2KTc3Vzk5Odq8ebNGjx7tj10CAAABxuJ2u93+2viUKVO0ZcsWffDBB5fsd7vdio6O1sSJE/XEE09IksrKymS327VmzRoNHTpU+/fvV3x8vHbu3KnExERJ0vr16zV48GB99dVXio6O/p91uFwu2Ww2lZWVyWq1+m4HEbDaT3nL3yXU2hfzUvxdAgD41ZX+fvt1ZufNN99UYmKifv7znysyMlLdu3fXihUrPP1Hjx6V0+lUUlKSp81ms6lnz57Kz8+XJOXn5ys8PNwTdCQpKSlJQUFB2r59+yW3W1FRIZfL5bUAAABz8knYKS0trdH3Pv/8cy1fvlwdO3bUhg0bNGbMGP3yl79Udna2JMnpdEqS7Ha71/fsdrunz+l0KjIy0qu/cePGioiI8Iz5T3PnzpXNZvMsMTExNaofAAAEPsNh57e//a3+8pe/eD7fe++9atmypa699lrt2bPH0Lqqq6v14x//WHPmzFH37t01evRoPfLII8rKyjJaliGZmZkqKyvzLEVFRXW6PQAA4D+Gw05WVpZnJiQ3N1e5ublat26dBg0apEmTJhlaV5s2bRQfH+/V1qlTJxUWFkqSoqKiJEnFxcVeY4qLiz19UVFRKikp8eo/f/68Tp065Rnzn0JDQ2W1Wr0WAABgTo2NfsHpdHrCTk5Oju69914NGDBA7du3V8+ePQ2tq0+fPjp48KBX26FDh9SuXTtJUocOHRQVFaW8vDx169ZN0ncXI23fvl1jxoyRJDkcDpWWlqqgoEAJCQmSpHfffVfV1dWG6wFQ/7hYHEBdMzyzc80113hO+6xfv95z8bDb7VZVVZWhdU2YMEHbtm3TnDlz9Nlnn2nt2rV67rnnlJ6eLkmyWCwaP368Zs+erTfffFN79+7ViBEjFB0drbvuukvSdzNBAwcO1COPPKIdO3Zoy5YtGjt2rIYOHXpFd2IBAABzMzyzM2TIEN1///3q2LGjvv76aw0aNEiStHv3bt1www2G1tWjRw+9/vrryszM1KxZs9ShQwctWrRIaWlpnjFPPvmkzpw5o9GjR6u0tFR9+/bV+vXrFRYW5hnz4osvauzYsbr11lsVFBSk1NRUPfPMM0Z3DQAAmJDh5+ycO3dOixcvVlFRkR588EF1795dkrRw4UK1aNFCDz/8cJ0UWpd4zs7Vh1MngYNjAaCmrvT32/DMTnBwsOcBf983YcIEo6sCAACoczV6zs6f/vQn9e3bV9HR0fryyy8lSYsWLdLf/vY3nxYHAABQW4bDzvLly5WRkaFBgwaptLTUc1FyeHi4Fi1a5Ov6AAAAasVw2Hn22We1YsUK/frXv1ajRo087YmJidq7d69PiwMAAKgtw2Hn6NGjnouSvy80NFRnzpzxSVEAAAC+YjjsdOjQQR999NFF7evXr1enTp18URMAAIDPGL4bKyMjQ+np6SovL5fb7daOHTv05z//WXPnztXKlSvrokYAAIAaMxx2Hn74YTVp0kRTp07V2bNndf/99ys6OlqLFy/W0KFD66JGAACAGjMcdiQpLS1NaWlpOnv2rE6fPq3IyEhf1wUAAOATNQo7FzRt2lRNmzb1VS0AAAA+ZzjsdO/eXRaL5aJ2i8WisLAw3XDDDXrwwQd1yy23+KRAAACA2jB8N9bAgQP1+eefq1mzZrrlllt0yy23qHnz5jpy5Ih69Oih48ePKykpiacpAwCAgGB4ZufkyZOaOHGipk2b5tU+e/Zsffnll9q4caNmzJihp556SnfeeafPCgUAAKgJwzM7L7/8soYNG3ZR+9ChQ/Xyyy9LkoYNG6aDBw/WvjoAAIBaMhx2wsLCtHXr1ovat27dqrCwMElSdXW1528AAAB/Mnwaa9y4cXr00UdVUFCgHj16SJJ27typlStX6le/+pUkacOGDerWrZtPCwUAAKgJw2Fn6tSp6tChg5YsWaI//elPkqQf/vCHWrFihe6//35J0qOPPqoxY8b4tlIAAIAaqNVDBS+nSZMmNS4IAADAlwxfswMAANCQGJ7Zqaqq0sKFC/Xyyy+rsLBQlZWVXv2nTp3yWXEAAAC1ZXhmZ+bMmVqwYIHuu+8+lZWVKSMjQ0OGDFFQUJB+85vf1EGJAAAANWc47Lz44otasWKFJk6cqMaNG2vYsGFauXKlpk+frm3bttVFjQAAADVmOOw4nU517txZktS8eXOVlZVJkm6//Xa99dZbvq0OAACglgyHnbZt2+r48eOSpOuvv14bN26U9N2zdkJDQ31bHQAAQC0ZDjt333238vLyJH33gMFp06apY8eOGjFihB566CGfFwgAAFAbhu/Gmjdvnufv++67T7GxscrPz1fHjh11xx13+LQ4AACA2qrRQwW/z+FwyOFw+KIWAAAAnzN8Gis7O9vrQuQnn3xS4eHh6t27t7788kufFgcAAFBbhsPOnDlzPK+DyM/P15IlSzR//ny1atVKEyZM8HmBAAAAtWH4NFZRUZFuuOEGSdIbb7yhe+65R6NHj1afPn3Ur18/X9cHAABQK4Zndpo3b66vv/5akrRx40bddtttkqSwsDB9++23vq0OAACglgzP7Nx22216+OGH1b17dx06dEiDBw+WJO3bt0/t27f3dX0AAAC1YnhmZ+nSpXI4HDpx4oT++te/qmXLlpKkgoICDRs2zNC6fvOb38hisXgtcXFxnv7y8nKlp6erZcuWat68uVJTU1VcXOy1jsLCQqWkpKhp06aKjIzUpEmTdP78eaO7BQAATMrwzE54eLiWLFlyUfvMmTNrVMCPfvQjvfPOO/8uqPG/S5owYYLeeustvfLKK7LZbBo7dqyGDBmiLVu2SPruDewpKSmKiorS1q1bdfz4cY0YMULBwcGaM2dOjeoBAADmYnhmZ/369frHP/7h+bx06VJ169ZN999/v/71r38ZLqBx48aKioryLK1atZIklZWVadWqVVqwYIH69++vhIQErV69Wlu3bvW8cHTjxo369NNP9cILL6hbt24aNGiQnnrqKS1dulSVlZWX3WZFRYVcLpfXAgAAzMlw2Jk0aZInHOzdu1cTJ07U4MGDdfToUWVkZBgu4PDhw4qOjtZ1112ntLQ0FRYWSvrutNi5c+eUlJTkGRsXF+d5YrP03a3vnTt3lt1u94xJTk6Wy+XSvn37LrvNuXPnymazeZaYmBjDdQMAgIbBcNg5evSo4uPjJUl//etfdfvtt2vOnDlaunSp1q1bZ2hdPXv21Jo1a7R+/XotX75cR48e1U9/+lN98803cjqdCgkJUXh4uNd37Ha7nE6npO/ewP79oHOh/0Lf5WRmZqqsrMyzFBUVGaobAAA0HIav2QkJCdHZs2clSe+8845GjBghSYqIiDB8OmjQoEGev7t06aKePXuqXbt2evnllz0PLqwLoaGhvKEdAICrhOGZnb59+yojI0NPPfWUduzYoZSUFEnSoUOH1LZt21oVEx4erh/84Af67LPPFBUVpcrKSpWWlnqNKS4uVlRUlCQpKirqoruzLny+MAYAAFzdDIedJUuWqHHjxnr11Ve1fPlyXXvttZKkdevWaeDAgbUq5vTp0zpy5IjatGmjhIQEBQcHKy8vz9N/8OBBFRYWel486nA4tHfvXpWUlHjG5Obmymq1ek61AQCAq5vh01ixsbHKycm5qH3hwoWGN/7EE0/ojjvuULt27XTs2DHNmDFDjRo10rBhw2Sz2TRq1ChlZGQoIiJCVqtV48aNk8PhUK9evSRJAwYMUHx8vIYPH6758+fL6XRq6tSpSk9P5zQVAACQVIOwI0lHjhzR6tWrdeTIES1evFiRkZFat26dYmNj9aMf/eiK1/PVV19p2LBh+vrrr9W6dWv17dtX27ZtU+vWrSV9F6CCgoKUmpqqiooKJScna9myZZ7vN2rUSDk5ORozZowcDoeaNWumkSNHatasWTXZLQAAYEIWt9vtNvKFTZs2adCgQerTp482b96s/fv367rrrtO8efP04Ycf6tVXX62rWuuMy+WSzWZTWVmZrFarv8tBPWg/5S1/l1BrX8xL8XcJPsGxAFBTV/r7bfianSlTpmj27NnKzc1VSEiIp71///6eh/0BAAAECsNhZ+/evbr77rsvao+MjNTJkyd9UhQAAICvGA474eHhOn78+EXtu3fv9tyZBQAAECgMh52hQ4dq8uTJcjqdslgsqq6u1pYtW/TEE094HjAIAAAQKAyHnTlz5iguLk4xMTE6ffq04uPjddNNN6l3796aOnVqXdQIAABQYzV6XcSKFSs0bdo0ffLJJzp9+rS6d++ujh071kV9AAAAtVKj5+xI3z1cMDY21pe1AAAA+JzhsON2u/Xqq6/qvffeU0lJiaqrq736X3vtNZ8VBwAAUFuGw8748eP1hz/8QbfccovsdrssFktd1AUAAOAThsPOn/70J7322msaPHhwXdQDAADgU4bvxrLZbLruuuvqohYAAACfMxx2fvOb32jmzJn69ttv66IeAAAAnzJ8Guvee+/Vn//8Z0VGRqp9+/YKDg726t+1a5fPigMAAKgtw2Fn5MiRKigo0AMPPMAFygAAIOAZDjtvvfWWNmzYoL59+9ZFPQAAAD5l+JqdmJgYWa3WuqgFAADA5wyHnd///vd68skn9cUXX9RBOQAAAL5l+DTWAw88oLNnz+r6669X06ZNL7pA+dSpUz4rDgAAoLYMh51FixbVQRkAAAB1o0Z3YwEAADQUhq/ZAQAAaEgIOwAAwNQIOwAAwNSuKOx8/PHHqq6urutaAAAAfO6Kwk737t118uRJSdJ1112nr7/+uk6LAgAA8JUrCjvh4eE6evSoJOmLL75glgcAADQYV3TreWpqqm6++Wa1adNGFotFiYmJatSo0SXHfv755z4tEAAAoDauKOw899xzGjJkiD777DP98pe/1COPPKIWLVrUdW0AAAC1dsUPFRw4cKAkqaCgQI8//jhhBwAANAiGn6C8evVqz99fffWVJKlt27a+qwgAAMCHDD9np7q6WrNmzZLNZlO7du3Url07hYeH66mnnuLCZQAAEHAMh51f//rXWrJkiebNm6fdu3dr9+7dmjNnjp599llNmzatxoXMmzdPFotF48eP97SVl5crPT1dLVu2VPPmzZWamqri4mKv7xUWFiolJUVNmzZVZGSkJk2apPPnz9e4DgAAYC6GT2NlZ2dr5cqV+tnPfuZp69Kli6699lo99thjevrppw0XsXPnTv3hD39Qly5dvNonTJigt956S6+88opsNpvGjh2rIUOGaMuWLZKkqqoqpaSkKCoqSlu3btXx48c1YsQIBQcHa86cOYbrAAAA5mN4ZufUqVOKi4u7qD0uLk6nTp0yXMDp06eVlpamFStW6JprrvG0l5WVadWqVVqwYIH69++vhIQErV69Wlu3btW2bdskSRs3btSnn36qF154Qd26ddOgQYP01FNPaenSpaqsrDRcCwAAMB/DYadr165asmTJRe1LlixR165dDReQnp6ulJQUJSUlebUXFBTo3LlzXu1xcXGKjY1Vfn6+JCk/P1+dO3eW3W73jElOTpbL5dK+ffsuu82Kigq5XC6vBQAAmJPh01jz589XSkqK3nnnHTkcDknfhY6ioiK9/fbbhtb10ksvadeuXdq5c+dFfU6nUyEhIQoPD/dqt9vtcjqdnjHfDzoX+i/0Xc7cuXM1c+ZMQ7UCAICGyfDMzs0336xDhw7p7rvvVmlpqUpLSzVkyBAdPHhQP/3pT694PUVFRXr88cf14osvKiwszGgZtZKZmamysjLPUlRUVK/bBwAA9cfwzI4kRUdH1+hC5O8rKChQSUmJfvzjH3vaqqqqtHnzZi1ZskQbNmxQZWWlSktLvWZ3iouLFRUVJUmKiorSjh07vNZ74W6tC2MuJTQ0VKGhobWqHwAANAyGZ3Z85dZbb9XevXv10UcfeZbExESlpaV5/g4ODlZeXp7nOwcPHlRhYaHn9JnD4dDevXtVUlLiGZObmyur1ar4+Ph63ycAABB4ajSz4wstWrTQjTfe6NXWrFkztWzZ0tM+atQoZWRkKCIiQlarVePGjZPD4VCvXr0kSQMGDFB8fLyGDx+u+fPny+l0aurUqUpPT2fmBgAASPJj2LkSCxcuVFBQkFJTU1VRUaHk5GQtW7bM09+oUSPl5ORozJgxcjgcatasmUaOHKlZs2b5sWoAABBIDIUdt9utoqIiRUZG1slFxe+//77X57CwMC1dulRLly697HfatWtn+C4wAABw9TB0zY7b7dYNN9zA3UsAAKDBMBR2goKC1LFjR3399dd1VQ8AAIBPGb4ba968eZo0aZI++eSTuqgHAADApwxfoDxixAidPXtWXbt2VUhIiJo0aeLVX5P3YwEAANQVw2Fn0aJFdVAGAABA3TAcdkaOHFkXdQAAANSJGj1B+ciRI5o6daqGDRvmeXrxunXr/uubxgEAAPzBcNjZtGmTOnfurO3bt+u1117T6dOnJUl79uzRjBkzfF4gAABAbRgOO1OmTNHs2bOVm5urkJAQT3v//v21bds2nxYHAABQW4bDzt69e3X33Xdf1B4ZGamTJ0/6pCgAAABfMRx2wsPDdfz48Yvad+/erWuvvdYnRQEAAPiK4bAzdOhQTZ48WU6nUxaLRdXV1dqyZYueeOIJjRgxoi5qBAAAqDHDYWfOnDmKi4tTTEyMTp8+rfj4eN10003q3bu3pk6dWhc1AgAA1Jjh5+yEhIRoxYoVmjZtmj755BOdPn1a3bt3V8eOHeuiPgAAgFoxHHYuiI2NVUxMjCTJYrH4rCAAAABfqtFDBVetWqUbb7xRYWFhCgsL04033qiVK1f6ujYAAIBaMzyzM336dC1YsEDjxo2Tw+GQJOXn52vChAkqLCzUrFmzfF4kAABATRkOO8uXL9eKFSs0bNgwT9vPfvYzdenSRePGjSPsAACAgGL4NNa5c+eUmJh4UXtCQoLOnz/vk6IAAAB8xXDYGT58uJYvX35R+3PPPae0tDSfFAUAAOArV3QaKyMjw/O3xWLRypUrtXHjRvXq1UuStH37dhUWFvJQQQAAEHCuKOzs3r3b63NCQoIk6ciRI5KkVq1aqVWrVtq3b5+PywMAAKidKwo77733Xl3XAQAAUCdq9JwdAACAhsLwrefl5eV69tln9d5776mkpETV1dVe/bt27fJZcQAAALVlOOyMGjVKGzdu1D333KOf/OQnvCoCAAAENMNhJycnR2+//bb69OlTF/UAAAD4lOFrdq699lq1aNGiLmoBAADwOcNh5/e//70mT56sL7/8si7qAQAA8CnDp7ESExNVXl6u6667Tk2bNlVwcLBX/6lTp3xWHAAAQG0ZDjvDhg3TP//5T82ZM0d2u50LlAEAQEAzHHa2bt2q/Px8de3atS7qAQAA8CnD1+zExcXp22+/9cnGly9fri5dushqtcpqtcrhcGjdunWe/vLycqWnp6tly5Zq3ry5UlNTVVxc7LWOwsJCpaSkqGnTpoqMjNSkSZN4+zoAAPAwHHbmzZuniRMn6v3339fXX38tl8vltRjRtm1bzZs3TwUFBfrwww/Vv39/3XnnnZ53bE2YMEF///vf9corr2jTpk06duyYhgwZ4vl+VVWVUlJSVFlZqa1btyo7O1tr1qzR9OnTje4WAAAwKYvb7XYb+UJQ0Hf56D+v1XG73bJYLKqqqqpVQREREfq///s/3XPPPWrdurXWrl2re+65R5J04MABderUSfn5+erVq5fWrVun22+/XceOHZPdbpckZWVlafLkyTpx4oRCQkKuaJsul0s2m01lZWWyWq21qh8NQ/spb/m7hFr7Yl6Kv0vwCY4FgJq60t9vw9fs1NVLQauqqvTKK6/ozJkzcjgcKigo0Llz55SUlOQZExcXp9jYWE/Yyc/PV+fOnT1BR5KSk5M1ZswY7du3T927d7/ktioqKlRRUeH5bHRGCgAANByGw87NN9/s0wL27t0rh8Oh8vJyNW/eXK+//rri4+P10UcfKSQkROHh4V7j7Xa7nE6nJMnpdHoFnQv9F/ouZ+7cuZo5c6ZP9wMAAAQmw2Fn8+bN/7X/pptuMrS+H/7wh/roo49UVlamV199VSNHjtSmTZuMlmVIZmamMjIyPJ9dLpdiYmLqdJsAAMA/DIedfv36XdT2/et3jF6zExISohtuuEGSlJCQoJ07d2rx4sW67777VFlZqdLSUq/ZneLiYkVFRUmSoqKitGPHDq/1Xbhb68KYSwkNDVVoaKihOgEAQMNk+G6sf/3rX15LSUmJ1q9frx49emjjxo21Lqi6uloVFRVKSEhQcHCw8vLyPH0HDx5UYWGhHA6HJMnhcGjv3r0qKSnxjMnNzZXValV8fHytawEAAA2f4Zkdm812Udttt92mkJAQZWRkqKCg4IrXlZmZqUGDBik2NlbffPON1q5dq/fff18bNmyQzWbTqFGjlJGRoYiICFmtVo0bN04Oh0O9evWSJA0YMEDx8fEaPny45s+fL6fTqalTpyo9PZ2ZGwAAIKkGYedy7Ha7Dh48aOg7JSUlGjFihI4fPy6bzaYuXbpow4YNuu222yRJCxcuVFBQkFJTU1VRUaHk5GQtW7bM8/1GjRopJydHY8aMkcPhULNmzTRy5EjNmjXLV7sFAAAaOMNh5+OPP/b67Ha7dfz4cc2bN0/dunUztK5Vq1b91/6wsDAtXbpUS5cuveyYdu3a6e233za0XQAAcPUwHHa6desmi8Wi/3wWYa9evfT888/7rDAAAABfMBx2jh496vU5KChIrVu3VlhYmM+KAgAA8BXDYaddu3Z1UQcAAECdqNEFynl5ecrLy1NJSYmqq6u9+jiVBQAAAonhsDNz5kzNmjVLiYmJatOmzUUvBAUAAAgkhsNOVlaW1qxZo+HDh9dFPQAAAD5l+AnKlZWV6t27d13UAgAA4HOGw87DDz+stWvX1kUtAAAAPmf4NFZ5ebmee+45vfPOO+rSpYuCg4O9+hcsWOCz4gAAAGqrRk9QvvCk5E8++cSrj4uVAQBAoDEcdt577726qAMAAKBOGL5mBwAAoCEh7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFPza9iZO3euevTooRYtWigyMlJ33XWXDh486DWmvLxc6enpatmypZo3b67U1FQVFxd7jSksLFRKSoqaNm2qyMhITZo0SefPn6/PXQEAAAHKr2Fn06ZNSk9P17Zt25Sbm6tz585pwIABOnPmjGfMhAkT9Pe//12vvPKKNm3apGPHjmnIkCGe/qqqKqWkpKiyslJbt25Vdna21qxZo+nTp/tjlwAAQICxuN1ut7+LuODEiROKjIzUpk2bdNNNN6msrEytW7fW2rVrdc8990iSDhw4oE6dOik/P1+9evXSunXrdPvtt+vYsWOy2+2SpKysLE2ePFknTpxQSEjIRdupqKhQRUWF57PL5VJMTIzKyspktVrrZ2fhV+2nvOXvEmrti3kp/i7BJzgWAGrK5XLJZrP9z9/vgLpmp6ysTJIUEREhSSooKNC5c+eUlJTkGRMXF6fY2Fjl5+dLkvLz89W5c2dP0JGk5ORkuVwu7du375LbmTt3rmw2m2eJiYmpq10CAAB+1tjfBVxQXV2t8ePHq0+fPrrxxhslSU6nUyEhIQoPD/caa7fb5XQ6PWO+H3Qu9F/ou5TMzExlZGR4Pl+Y2QEAIBCYYcZTCpxZz4AJO+np6frkk0/0j3/8o863FRoaqtDQ0DrfDgAA8L+AOI01duxY5eTk6L333lPbtm097VFRUaqsrFRpaanX+OLiYkVFRXnG/OfdWRc+XxgDAACuXn6d2XG73Ro3bpxef/11vf/+++rQoYNXf0JCgoKDg5WXl6fU1FRJ0sGDB1VYWCiHwyFJcjgcevrpp1VSUqLIyEhJUm5urqxWq+Lj4+t3hwCgAePUCczKr2EnPT1da9eu1d/+9je1aNHCc42NzWZTkyZNZLPZNGrUKGVkZCgiIkJWq1Xjxo2Tw+FQr169JEkDBgxQfHy8hg8frvnz58vpdGrq1KlKT0/nVBUAAPBv2Fm+fLkkqV+/fl7tq1ev1oMPPihJWrhwoYKCgpSamqqKigolJydr2bJlnrGNGjVSTk6OxowZI4fDoWbNmmnkyJGaNWtWfe0GAAAIYH4/jfW/hIWFaenSpVq6dOllx7Rr105vv/22L0sDAAAmERAXKAMAANQVwg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADC1xv4u4GrRfspb/i7BJ76Yl+LvEgAAMISZHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGqEHQAAYGp+DTubN2/WHXfcoejoaFksFr3xxhte/W63W9OnT1ebNm3UpEkTJSUl6fDhw15jTp06pbS0NFmtVoWHh2vUqFE6ffp0Pe4FAAAIZH4NO2fOnFHXrl21dOnSS/bPnz9fzzzzjLKysrR9+3Y1a9ZMycnJKi8v94xJS0vTvn37lJubq5ycHG3evFmjR4+ur10AAAABzq8vAh00aJAGDRp0yT63261FixZp6tSpuvPOOyVJf/zjH2W32/XGG29o6NCh2r9/v9avX6+dO3cqMTFRkvTss89q8ODB+t3vfqfo6Oh62xcAABCYAvaanaNHj8rpdCopKcnTZrPZ1LNnT+Xn50uS8vPzFR4e7gk6kpSUlKSgoCBt3779suuuqKiQy+XyWgAAgDkFbNhxOp2SJLvd7tVut9s9fU6nU5GRkV79jRs3VkREhGfMpcydO1c2m82zxMTE+Lh6AAAQKAI27NSlzMxMlZWVeZaioiJ/lwQAAOpIwIadqKgoSVJxcbFXe3FxsacvKipKJSUlXv3nz5/XqVOnPGMuJTQ0VFar1WsBAADmFLBhp0OHDoqKilJeXp6nzeVyafv27XI4HJIkh8Oh0tJSFRQUeMa8++67qq6uVs+ePeu9ZgAAEHj8ejfW6dOn9dlnn3k+Hz16VB999JEiIiIUGxur8ePHa/bs2erYsaM6dOigadOmKTo6WnfddZckqVOnTho4cKAeeeQRZWVl6dy5cxo7dqyGDh3KnVgAAECSn8POhx9+qFtuucXzOSMjQ5I0cuRIrVmzRk8++aTOnDmj0aNHq7S0VH379tX69esVFhbm+c6LL76osWPH6tZbb1VQUJBSU1P1zDPP1Pu+AACAwOTXsNOvXz+53e7L9lssFs2aNUuzZs267JiIiAitXbu2LsoDAAAmELDX7AAAAPgCYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJiaacLO0qVL1b59e4WFhalnz57asWOHv0sCAAABwBRh5y9/+YsyMjI0Y8YM7dq1S127dlVycrJKSkr8XRoAAPAzU4SdBQsW6JFHHtEvfvELxcfHKysrS02bNtXzzz/v79IAAICfNfZ3AbVVWVmpgoICZWZmetqCgoKUlJSk/Pz8S36noqJCFRUVns9lZWWSJJfLVWd1VlecrbN116e6/GdUn8xwPDgWgYNjEVjMcDw4FsbW73a7/+u4Bh92Tp48qaqqKtntdq92u92uAwcOXPI7c+fO1cyZMy9qj4mJqZMazcS2yN8V4AKOReDgWAQWjkfgqK9j8c0338hms122v8GHnZrIzMxURkaG53N1dbVOnTqlli1bymKx+LGymnO5XIqJiVFRUZGsVqu/y7mqcSwCC8cjcHAsAodZjoXb7dY333yj6Ojo/zquwYedVq1aqVGjRiouLvZqLy4uVlRU1CW/ExoaqtDQUK+28PDwuiqxXlmt1gb9L66ZcCwCC8cjcHAsAocZjsV/m9G5oMFfoBwSEqKEhATl5eV52qqrq5WXlyeHw+HHygAAQCBo8DM7kpSRkaGRI0cqMTFRP/nJT7Ro0SKdOXNGv/jFL/xdGgAA8DNThJ377rtPJ06c0PTp0+V0OtWtWzetX7/+oouWzSw0NFQzZsy46PQc6h/HIrBwPAIHxyJwXG3HwuL+X/drAQAANGAN/podAACA/4awAwAATI2wAwAATI2wAwAATI2wAwDAVehquj/JFLeeX41Onjyp559/Xvn5+XI6nZKkqKgo9e7dWw8++KBat27t5woBAIEsNDRUe/bsUadOnfxdSp3j1vMGaOfOnUpOTlbTpk2VlJTkeZ5QcXGx8vLydPbsWW3YsEGJiYl+rhSSVFRUpBkzZuj555/3dylXhW+//VYFBQWKiIhQfHy8V195eblefvlljRgxwk/VXV3279+vbdu2yeFwKC4uTgcOHNDixYtVUVGhBx54QP379/d3iVeF778L8vsWL16sBx54QC1btpQkLViwoD7LqleEnQaoV69e6tq1q7Kysi56canb7dajjz6qjz/+WPn5+X6qEN+3Z88e/fjHP1ZVVZW/SzG9Q4cOacCAASosLJTFYlHfvn310ksvqU2bNpK++x+C6OhojkU9WL9+ve688041b95cZ8+e1euvv64RI0aoa9euqq6u1qZNm7Rx40YCTz0ICgpS165dL3oH5KZNm5SYmKhmzZrJYrHo3Xff9U+B9YCw0wA1adJEu3fvVlxc3CX7Dxw4oO7du+vbb7+t58quTm+++eZ/7f/88881ceJEfmDrwd13361z585pzZo1Ki0t1fjx4/Xpp5/q/fffV2xsLGGnHvXu3Vv9+/fX7Nmz9dJLL+mxxx7TmDFj9PTTT0uSMjMzVVBQoI0bN/q5UvObN2+ennvuOa1cudIrXAYHB2vPnj0XzYCakhsNTvv27d3Z2dmX7c/Ozna3a9eu/gq6ylksFndQUJDbYrFcdgkKCvJ3mVeFyMhI98cff+z5XF1d7X700UfdsbGx7iNHjridTifHop5YrVb34cOH3W63211VVeVu3Lixe9euXZ7+vXv3uu12u7/Ku+rs2LHD/YMf/MA9ceJEd2VlpdvtdrsbN27s3rdvn58rqx/cjdUAPfHEExo9erQef/xxvfnmm9q+fbu2b9+uN998U48//rgeffRRPfnkk/4u86rRpk0bvfbaa6qurr7ksmvXLn+XeNX49ttv1bjxv++7sFgsWr58ue644w7dfPPNOnTokB+ru/pcOM0eFBSksLAw2Ww2T1+LFi1UVlbmr9KuOj169FBBQYFOnDihxMREffLJJxddBmFm3I3VAKWnp6tVq1ZauHChli1b5pmSb9SokRISErRmzRrde++9fq7y6pGQkKCCggLdeeedl+y3WCxX1S2e/hQXF6cPP/zwortLlixZIkn62c9+5o+yrkrt27fX4cOHdf3110uS8vPzFRsb6+kvLCz0XEuF+tG8eXNlZ2frpZdeUlJS0lV1Opdrdhq4c+fO6eTJk5KkVq1aKTg42M8VXX0++OADnTlzRgMHDrxk/5kzZ/Thhx/q5ptvrufKrj5z587VBx98oLfffvuS/Y899piysrJUXV1dz5VdfbKyshQTE6OUlJRL9v/qV79SSUmJVq5cWc+VQZK++uorFRQUKCkpSc2aNfN3OXWOsAMAAEyNa3YAAICpEXYAAICpEXYAAICpEXYAAICpEXYANAhut1ujR49WRESELBaLPvroI3+XBKCB4G4sAA3CunXrdOedd+r999/Xddddp1atWnk9QBAALof/UgBoEI4cOaI2bdqod+/el+yvrKxUSEhIPVcFoCHgNBaAgPfggw9q3LhxnreZt2/fXv369dPYsWM1fvx4tWrVSsnJyZK+e5PzT37yE4WGhqpNmzaaMmWKzp8/71lXv379NG7cOI0fP17XXHON7Ha7VqxYoTNnzugXv/iFWrRooRtuuEHr1q3z1+4C8DHCDoCAt3jxYs2aNUtt27bV8ePHtXPnTklSdna2QkJCtGXLFmVlZemf//ynBg8erB49emjPnj1avny5Vq1apdmzZ3utLzs7W61atdKOHTs0btw4jRkzRj//+c/Vu3dv7dq1SwMGDNDw4cN19uxZf+wuAB/jmh0ADcKiRYu0aNEiffHFF5K+m6FxuVxeL1r99a9/rb/+9a/av3+/5yWHy5Yt0+TJk1VWVqagoCD169dPVVVV+uCDDyRJVVVVstlsGjJkiP74xz9KkpxOp9q0aaP8/Hz16tWrfncUgM8xswOgwUpISPD6vH//fjkcDq+3Offp00enT5/WV1995Wnr0qWL5+9GjRqpZcuW6ty5s6fNbrdLkkpKSuqqdAD1iLADoMGq6QsM//OFuRaLxavtQljihaGAORB2AJhGp06dlJ+fr++fnd+yZYtatGihtm3b+rEyAP5E2AFgGo899piKioo0btw4HThwQH/72980Y8YMZWRkKCiI/9wBVyueswPANK699lq9/fbbmjRpkrp27aqIiAiNGjVKU6dO9XdpAPyIu7EAAICpMa8LAABMjbADAABMjbADAABMjbADAABMjbADAABMjbADAABMjbADAABMjbADAABMjbADAABMjbADAABMjbADAABM7f8Duux+zsaUDr0AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "cls_to_name = {0: \"Krystian\", 1: \"Tim\", 2: \"Alina\", 3: \"Mashiko\", 4: \"Mira\"}\n",
    "\n",
    "df.groupby(\"from\").count()[\"text\"].plot(kind=\"bar\")\n",
    "plt.ylabel(\"number of messages\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Balancing Dataset\n",
    "\n",
    "As shown in the bar plot above, the dataset is imbalanced. For the model to learn something more than the fact that half of the messages come from a single person, the dataset should be balanced.\n",
    "\n",
    "Multiple methods can be used to deal with this problem, including gathering more data, oversampling, or undersampling. Gathering more data would be complex because I would need to make people text more or find another group chat with the same people but with a better distribution of texts. Oversampling would consist of sampling from the underrepresented classes more than from the others, but this would also require data augmentation, which might lose some information pertaining to the specifics of the tone and word choices of people. Therefore, the undersampling will be used. Random samples from the overrepresented labels will be chosen and deleted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "import numpy as np\n",
    "\n",
    "# limit the number per class to n\n",
    "n = min(Counter(df[\"from\"].values).values())\n",
    "\n",
    "for cls in set(df[\"from\"]):\n",
    "    ids = df[df[\"from\"] == cls].id.values\n",
    "    n_to_drop = len(ids) - n\n",
    "    if n_to_drop > 0:\n",
    "        ids_to_drop = np.random.choice(ids, size=n_to_drop, replace=False)\n",
    "        df = df[~df[\"id\"].isin(ids_to_drop)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAGrCAYAAAAmWFaFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAuKElEQVR4nO3de1RVdeL+8ecgNy9cQoebQlI6g06KJKao3zIjr5mljWmk1Zh+TaUUU2PyUmZirlJKTUYts1bW1GTmOEkalnZBNFBTx7TMhNIDGQEBgsY5vz9anl/nqzYd2XAO2/drrb2W5/PZbB/as+qZz74ci91utwsAAMCkvNwdAAAAoD5RdgAAgKlRdgAAgKlRdgAAgKlRdgAAgKlRdgAAgKlRdgAAgKl5uzuAJ7DZbDpx4oQCAgJksVjcHQcAAPwOdrtdP/30kyIjI+XldfH1G8qOpBMnTigqKsrdMQAAwCUoLCxUmzZtLjpP2ZEUEBAg6Zd/WIGBgW5OAwAAfo/y8nJFRUU5/jt+MZQdyXHpKjAwkLIDAEAj899uQeEGZQAAYGpuLTs7duzQkCFDFBkZKYvFog0bNlx03wkTJshisSgjI8NpvKSkRMnJyQoMDFRwcLDGjh2rioqK+g0OAAAaDbeWncrKSsXFxWn58uW/ud/bb7+tnTt3KjIy8ry55ORkHTx4UFu3btWmTZu0Y8cOjR8/vr4iAwCARsat9+wMHDhQAwcO/M19vvvuO6WkpOi9997T4MGDneYOHTqkrKws7d69WwkJCZKkpUuXatCgQXr66acvWI4kqaamRjU1NY7P5eXldfxNAACAp/Loe3ZsNptGjx6t6dOn689//vN58zk5OQoODnYUHUlKSkqSl5eXcnNzL3rc9PR0BQUFOTYeOwcAwLw8uuw89dRT8vb21oMPPnjBeavVqtDQUKcxb29vhYSEyGq1XvS4aWlpKisrc2yFhYWG5gYAAJ7DYx89z8vL07PPPqv8/HzD32rs5+cnPz8/Q48JAAA8k8eu7Hz00UcqLi5WdHS0vL295e3trePHj2vatGlq27atJCk8PFzFxcVOP/fzzz+rpKRE4eHhbkgNAAA8jceu7IwePVpJSUlOY/3799fo0aN13333SZISExNVWlqqvLw8de3aVZK0bds22Ww2de/evcEzAwAAz+PWslNRUaGvvvrK8fnYsWPau3evQkJCFB0drZYtWzrt7+Pjo/DwcP3pT3+SJHXo0EEDBgzQuHHjlJmZqbNnz2ry5MkaOXLkRZ/EAgAAlxe3Xsb67LPPFB8fr/j4eElSamqq4uPjNWfOnN99jFdffVWxsbG66aabNGjQIPXu3VsrV66sr8gAAKCRsdjtdru7Q7hbeXm5goKCVFZWxndjAQDQSPze/3577A3KAAAARqDsAAAAU6PsAAAAU/PYR8/Npu0j/3Z3BEN8s3Dwf9+pETDD+eBceA7OhWcxw/ngXBiLlR0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqbi07O3bs0JAhQxQZGSmLxaINGzY45s6ePauZM2eqU6dOat68uSIjIzVmzBidOHHC6RglJSVKTk5WYGCggoODNXbsWFVUVDTwbwIAADyVW8tOZWWl4uLitHz58vPmqqqqlJ+fr9mzZys/P1/r16/X4cOHdeuttzrtl5ycrIMHD2rr1q3atGmTduzYofHjxzfUrwAAADyctzv/8oEDB2rgwIEXnAsKCtLWrVudxpYtW6brrrtOBQUFio6O1qFDh5SVlaXdu3crISFBkrR06VINGjRITz/9tCIjIy947JqaGtXU1Dg+l5eXG/QbAQAAT9Oo7tkpKyuTxWJRcHCwJCknJ0fBwcGOoiNJSUlJ8vLyUm5u7kWPk56erqCgIMcWFRVV39EBAICbNJqyU11drZkzZ2rUqFEKDAyUJFmtVoWGhjrt5+3trZCQEFmt1oseKy0tTWVlZY6tsLCwXrMDAAD3cetlrN/r7NmzGjFihOx2u1asWFHn4/n5+cnPz8+AZAAAwNN5fNk5V3SOHz+ubdu2OVZ1JCk8PFzFxcVO+//8888qKSlReHh4Q0cFAAAeyKMvY50rOl9++aXef/99tWzZ0mk+MTFRpaWlysvLc4xt27ZNNptN3bt3b+i4AADAA7l1ZaeiokJfffWV4/OxY8e0d+9ehYSEKCIiQnfccYfy8/O1adMm1dbWOu7DCQkJka+vrzp06KABAwZo3LhxyszM1NmzZzV58mSNHDnyok9iAQCAy4tby85nn32mG2+80fE5NTVVknTPPffoscce08aNGyVJXbp0cfq5Dz74QH369JEkvfrqq5o8ebJuuukmeXl5afjw4XruuecaJD8AAPB8bi07ffr0kd1uv+j8b82dExISonXr1hkZCwAAmIhH37MDAABQV5QdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgapQdAABgam4tOzt27NCQIUMUGRkpi8WiDRs2OM3b7XbNmTNHERERatq0qZKSkvTll1867VNSUqLk5GQFBgYqODhYY8eOVUVFRQP+FgAAwJO5texUVlYqLi5Oy5cvv+D8okWL9NxzzykzM1O5ublq3ry5+vfvr+rqasc+ycnJOnjwoLZu3apNmzZpx44dGj9+fEP9CgAAwMN5u/MvHzhwoAYOHHjBObvdroyMDM2aNUtDhw6VJL388ssKCwvThg0bNHLkSB06dEhZWVnavXu3EhISJElLly7VoEGD9PTTTysyMrLBfhcAAOCZPPaenWPHjslqtSopKckxFhQUpO7duysnJ0eSlJOTo+DgYEfRkaSkpCR5eXkpNzf3oseuqalReXm50wYAAMzJY8uO1WqVJIWFhTmNh4WFOeasVqtCQ0Od5r29vRUSEuLY50LS09MVFBTk2KKiogxODwAAPIXHlp36lJaWprKyMsdWWFjo7kgAAKCeeGzZCQ8PlyQVFRU5jRcVFTnmwsPDVVxc7DT/888/q6SkxLHPhfj5+SkwMNBpAwAA5mRI2SktLTXiME5iYmIUHh6u7Oxsx1h5eblyc3OVmJgoSUpMTFRpaany8vIc+2zbtk02m03du3c3PBMAAGh8XC47Tz31lP7xj384Po8YMUItW7ZU69attW/fPpeOVVFRob1792rv3r2Sfrkpee/evSooKJDFYtGUKVM0f/58bdy4Ufv379eYMWMUGRmp2267TZLUoUMHDRgwQOPGjdOuXbv0ySefaPLkyRo5ciRPYgEAAEmXUHYyMzMdN/Ru3bpVW7du1ebNmzVw4EBNnz7dpWN99tlnio+PV3x8vCQpNTVV8fHxmjNnjiRpxowZSklJ0fjx49WtWzdVVFQoKytL/v7+jmO8+uqrio2N1U033aRBgwapd+/eWrlypau/FgAAMCmX37NjtVodZWfTpk0aMWKE+vXrp7Zt27p86ahPnz6y2+0XnbdYLJo3b57mzZt30X1CQkK0bt06l/5eAABw+XB5ZeeKK65wPL2UlZXleA+O3W5XbW2tsekAAADqyOWVnWHDhumuu+5S+/bt9cMPPzjegLxnzx61a9fO8IAAAAB14XLZWbJkidq2bavCwkItWrRILVq0kCSdPHlSEydONDwgAABAXbhcdnx8fPTwww+fNz516lRDAgEAABjpkt6z88orr6h3796KjIzU8ePHJUkZGRl65513DA0HAABQVy6XnRUrVig1NVUDBw5UaWmp46bk4OBgZWRkGJ0PAACgTlwuO0uXLtWqVav06KOPqkmTJo7xhIQE7d+/39BwAAAAdeVy2Tl27JjjJYC/5ufnp8rKSkNCAQAAGMXlshMTE+P4eodfy8rKUocOHYzIBAAAYBiXn8ZKTU3VpEmTVF1dLbvdrl27dum1115Tenq6Vq9eXR8ZAQAALpnLZef+++9X06ZNNWvWLFVVVemuu+5SZGSknn32WY0cObI+MgIAAFwyl8uOJCUnJys5OVlVVVWqqKhQaGio0bkAAAAMcUll55xmzZqpWbNmRmUBAAAwnMtlJz4+XhaL5bxxi8Uif39/tWvXTvfee69uvPFGQwICAADUhctPYw0YMEBff/21mjdvrhtvvFE33nijWrRooaNHj6pbt246efKkkpKSeJsyAADwCC6v7Jw6dUrTpk3T7Nmzncbnz5+v48ePa8uWLZo7d66eeOIJDR061LCgAAAAl8LllZ033nhDo0aNOm985MiReuONNyRJo0aN0uHDh+ueDgAAoI5cLjv+/v769NNPzxv/9NNP5e/vL0my2WyOPwMAALiTy5exUlJSNGHCBOXl5albt26SpN27d2v16tX629/+Jkl677331KVLF0ODAgAAXAqXy86sWbMUExOjZcuW6ZVXXpEk/elPf9KqVat01113SZImTJigBx54wNikAAAAl6BOLxW8mKZNm15yIAAAACO5fM8OAABAY+Lyyk5tba2WLFmiN954QwUFBTpz5ozTfElJiWHhAAAA6srllZ3HH39cixcv1p133qmysjKlpqZq2LBh8vLy0mOPPVYPEQEAAC6dy2Xn1Vdf1apVqzRt2jR5e3tr1KhRWr16tebMmaOdO3fWR0YAAIBL5nLZsVqt6tSpkySpRYsWKisrkyTdcsst+ve//21sOgAAgDpyuey0adNGJ0+elCRdffXV2rJli6Rf3rXj5+dnbDoAAIA6crns3H777crOzpb0ywsGZ8+erfbt22vMmDH661//anhAAACAunD5aayFCxc6/nznnXcqOjpaOTk5at++vYYMGWJoOAAAgLq6pJcK/lpiYqISExONyAIAAGA4ly9jrV271ulG5BkzZig4OFg9e/bU8ePHDQ0HAABQVy6XnQULFji+DiInJ0fLli3TokWL1KpVK02dOtXwgAAAAHXh8mWswsJCtWvXTpK0YcMG3XHHHRo/frx69eqlPn36GJ0PAACgTlxe2WnRooV++OEHSdKWLVt08803S5L8/f11+vRpY9MBAADUkcsrOzfffLPuv/9+xcfH68iRIxo0aJAk6eDBg2rbtq3R+QAAAOrE5ZWd5cuXKzExUd9//73eeusttWzZUpKUl5enUaNGGR4QAACgLlxe2QkODtayZcvOG3/88ccNCQQAAGAkl1d2srKy9PHHHzs+L1++XF26dNFdd92lH3/80dBwAAAAdeVy2Zk+fbrKy8slSfv379e0adM0aNAgHTt2TKmpqYYHBAAAqAuXL2MdO3ZMHTt2lCS99dZbuuWWW7RgwQLl5+c7blYGAADwFC6v7Pj6+qqqqkqS9P7776tfv36SpJCQEMeKDwAAgKdwuez07t1bqampeuKJJ7Rr1y4NHjxYknTkyBG1adPG0HC1tbWaPXu2YmJi1LRpU1199dV64oknZLfbHfvY7XbNmTNHERERatq0qZKSkvTll18amgMAADReLpedZcuWydvbW//85z+1YsUKtW7dWpK0efNmDRgwwNBwTz31lFasWKFly5bp0KFDeuqpp7Ro0SItXbrUsc+iRYv03HPPKTMzU7m5uWrevLn69++v6upqQ7MAAIDGyeV7dqKjo7Vp06bzxpcsWWJIoF/79NNPNXToUMfqUdu2bfXaa69p165dkn5Z1cnIyNCsWbM0dOhQSdLLL7+ssLAwbdiwQSNHjjQ8EwAAaFxcXtmRpKNHj2rWrFkaNWqUiouLJf2ysnPw4EFDw/Xs2VPZ2dk6cuSIJGnfvn36+OOPNXDgQEm/3CxttVqVlJTk+JmgoCB1795dOTk5Fz1uTU2NysvLnTYAAGBOLped7du3q1OnTsrNzdX69etVUVEh6ZciMnfuXEPDPfLIIxo5cqRiY2Pl4+Oj+Ph4TZkyRcnJyZIkq9UqSQoLC3P6ubCwMMfchaSnpysoKMixRUVFGZobAAB4DpfLziOPPKL58+dr69at8vX1dYz37dtXO3fuNDTcG2+8oVdffVXr1q1Tfn6+1q5dq6efflpr166t03HT0tJUVlbm2AoLCw1KDAAAPI3L9+zs379f69atO288NDRUp06dMiTUOdOnT3es7khSp06ddPz4caWnp+uee+5ReHi4JKmoqEgRERGOnysqKlKXLl0uelw/Pz/5+fkZmhUAAHgml1d2goODdfLkyfPG9+zZ43gyyyhVVVXy8nKO2KRJE9lsNklSTEyMwsPDlZ2d7ZgvLy9Xbm6uEhMTDc0CAAAaJ5dXdkaOHKmZM2fqzTfflMVikc1m0yeffKKHH35YY8aMMTTckCFD9OSTTyo6Olp//vOftWfPHi1evFh//etfJUkWi0VTpkzR/Pnz1b59e8XExGj27NmKjIzUbbfdZmgWAADQOLlcdhYsWKBJkyYpKipKtbW16tixo2pra3XXXXdp1qxZhoZbunSpZs+erYkTJ6q4uFiRkZH63//9X82ZM8exz4wZM1RZWanx48ertLRUvXv3VlZWlvz9/Q3NAgAAGieXy46vr69WrVql2bNn68CBA6qoqFB8fLzat29veLiAgABlZGQoIyPjovtYLBbNmzdP8+bNM/zvBwAAjZ/LZeec6OhoRUdHG5kFAADAcC6XHbvdrn/+85/64IMPVFxc7LhZ+Jz169cbFg4AAKCuXC47U6ZM0d///nfdeOONCgsLk8ViqY9cAAAAhnC57Lzyyitav369Bg0aVB95AAAADOXye3aCgoJ01VVX1UcWAAAAw7lcdh577DE9/vjjOn36dH3kAQAAMJTLl7FGjBih1157TaGhoWrbtq18fHyc5vPz8w0LBwAAUFcul5177rlHeXl5uvvuu7lBGQAAeDyXy86///1vvffee+rdu3d95AEAADCUy/fsREVFKTAwsD6yAAAAGM7lsvPMM89oxowZ+uabb+ohDgAAgLFcvox19913q6qqSldffbWaNWt23g3KJSUlhoUDAACoK5fLzm99KScAAICnuaSnsQAAABoLl+/ZAQAAaEwoOwAAwNQoOwAAwNR+V9n5/PPPZbPZ6jsLAACA4X5X2YmPj9epU6ckSVdddZV++OGHeg0FAABglN9VdoKDg3Xs2DFJ0jfffMMqDwAAaDR+16Pnw4cP1w033KCIiAhZLBYlJCSoSZMmF9z366+/NjQgAABAXfyusrNy5UoNGzZMX331lR588EGNGzdOAQEB9Z0NAACgzn73SwUHDBggScrLy9NDDz1E2QEAAI2Cy29QXrNmjePP3377rSSpTZs2xiUCAAAwkMvv2bHZbJo3b56CgoJ05ZVX6sorr1RwcLCeeOIJblwGAAAex+WVnUcffVQvvPCCFi5cqF69ekmSPv74Yz322GOqrq7Wk08+aXhIAACAS+Vy2Vm7dq1Wr16tW2+91THWuXNntW7dWhMnTqTsAAAAj+LyZaySkhLFxsaeNx4bG6uSkhJDQgEAABjF5bITFxenZcuWnTe+bNkyxcXFGRIKAADAKC5fxlq0aJEGDx6s999/X4mJiZKknJwcFRYW6t133zU8IAAAQF24vLJzww036MiRI7r99ttVWlqq0tJSDRs2TIcPH9b//M//1EdGAACAS+byyo4kRUZGciMyAABoFFxe2QEAAGhMKDsAAMDUKDsAAMDUXCo7drtdBQUFqq6urq88AAAAhnK57LRr106FhYX1lQcAAMBQLpUdLy8vtW/fXj/88EN95QEAADCUy/fsLFy4UNOnT9eBAwfqIw8AAIChXH7PzpgxY1RVVaW4uDj5+vqqadOmTvN8PxYAAPAkLpedjIyMeogBAABQP1wuO/fcc0995Lio7777TjNnztTmzZtVVVWldu3aac2aNUpISJD0y03Tc+fO1apVq1RaWqpevXppxYoVat++fYPmBAAAnumS3rNz9OhRzZo1S6NGjVJxcbEkafPmzTp48KCh4X788Uf16tVLPj4+2rx5s/7zn//omWee0RVXXOHYZ9GiRXruueeUmZmp3NxcNW/eXP379+fxeAAAIOkSys727dvVqVMn5ebmav369aqoqJAk7du3T3PnzjU03FNPPaWoqCitWbNG1113nWJiYtSvXz9dffXVkn5Z1cnIyNCsWbM0dOhQde7cWS+//LJOnDihDRs2GJoFAAA0Ti6XnUceeUTz58/X1q1b5evr6xjv27evdu7caWi4jRs3KiEhQX/5y18UGhqq+Ph4rVq1yjF/7NgxWa1WJSUlOcaCgoLUvXt35eTkXPS4NTU1Ki8vd9oAAIA5uVx29u/fr9tvv/288dDQUJ06dcqQUOd8/fXXjvtv3nvvPT3wwAN68MEHtXbtWkmS1WqVJIWFhTn9XFhYmGPuQtLT0xUUFOTYoqKiDM0NAAA8h8tlJzg4WCdPnjxvfM+ePWrdurUhoc6x2Wy69tprtWDBAsXHx2v8+PEaN26cMjMz63TctLQ0lZWVOTbeCA0AgHm5XHZGjhypmTNnymq1ymKxyGaz6ZNPPtHDDz+sMWPGGBouIiJCHTt2dBrr0KGDCgoKJEnh4eGSpKKiIqd9ioqKHHMX4ufnp8DAQKcNAACYk8tlZ8GCBYqNjVVUVJQqKirUsWNHXX/99erZs6dmzZplaLhevXrp8OHDTmNHjhzRlVdeKUmKiYlReHi4srOzHfPl5eXKzc1VYmKioVkAAEDj5PJ7dnx9fbVq1SrNnj1bBw4cUEVFheLj4+vlvTZTp05Vz549tWDBAo0YMUK7du3SypUrtXLlSkmSxWLRlClTNH/+fLVv314xMTGaPXu2IiMjddtttxmeBwAAND4ul51zoqOjHTf2WiwWwwL9Wrdu3fT2228rLS1N8+bNU0xMjDIyMpScnOzYZ8aMGaqsrNT48eNVWlqq3r17KysrS/7+/vWSCQAANC6X9FLBF154Qddcc438/f3l7++va665RqtXrzY6myTplltu0f79+1VdXa1Dhw5p3LhxTvMWi0Xz5s2T1WpVdXW13n//ff3xj3+slywAAKDxcXllZ86cOVq8eLFSUlIc98Xk5ORo6tSpKigo0Lx58wwPCQAAcKlcLjsrVqzQqlWrNGrUKMfYrbfeqs6dOyslJYWyAwAAPIrLl7HOnj3r+BLOX+vatat+/vlnQ0IBAAAYxeWyM3r0aK1YseK88ZUrVzrdOAwAAOAJftdlrNTUVMefLRaLVq9erS1btqhHjx6SpNzcXBUUFBj+UkEAAIC6+l1lZ8+ePU6fu3btKkk6evSoJKlVq1Zq1aqVDh48aHA8AACAuvldZeeDDz6o7xwAAAD14pLeswMAANBYuPzoeXV1tZYuXaoPPvhAxcXFstlsTvP5+fmGhQMAAKgrl8vO2LFjtWXLFt1xxx267rrr6u2rIgAAAIzgctnZtGmT3n33XfXq1as+8gAAABjK5Xt2WrdurYCAgPrIAgAAYDiXy84zzzyjmTNn6vjx4/WRBwAAwFAuX8ZKSEhQdXW1rrrqKjVr1kw+Pj5O8yUlJYaFAwAAqCuXy86oUaP03XffacGCBQoLC+MGZQAA4NFcLjuffvqpcnJyFBcXVx95AAAADOXyPTuxsbE6ffp0fWQBAAAwnMtlZ+HChZo2bZo+/PBD/fDDDyovL3faAAAAPInLl7EGDBggSbrpppucxu12uywWi2pra41JBgAAYACXyw5fCgoAABoTl8vODTfcUB85AAAA6oXLZWfHjh2/OX/99ddfchgAAACjuVx2+vTpc97Yr9+1wz07AADAk7j8NNaPP/7otBUXFysrK0vdunXTli1b6iMjAADAJXN5ZScoKOi8sZtvvlm+vr5KTU1VXl6eIcEAAACM4PLKzsWEhYXp8OHDRh0OAADAEC6v7Hz++edOn+12u06ePKmFCxeqS5cuRuUCAAAwhMtlp0uXLrJYLLLb7U7jPXr00IsvvmhYMAAAACO4XHaOHTvm9NnLy0t/+MMf5O/vb1goAAAAo7hcdq688sr6yAEAAFAvXC47kpSdna3s7GwVFxfLZrM5zXEpCwAAeBKXy87jjz+uefPmKSEhQREREU4vFAQAAPA0LpedzMxMvfTSSxo9enR95AEAADCUy+/ZOXPmjHr27FkfWQAAAAznctm5//77tW7duvrIAgAAYDiXL2NVV1dr5cqVev/999W5c2f5+Pg4zS9evNiwcAAAAHV1SW9QPvem5AMHDjjNcbMyAADwNC6XnQ8++KA+cgAAANQLw74IFAAAwBNRdgAAgKlRdgAAgKk1qrKzcOFCWSwWTZkyxTFWXV2tSZMmqWXLlmrRooWGDx+uoqIi94UEAAAepdGUnd27d+vvf/+7Onfu7DQ+depU/etf/9Kbb76p7du368SJExo2bJibUgIAAE/TKMpORUWFkpOTtWrVKl1xxRWO8bKyMr3wwgtavHix+vbtq65du2rNmjX69NNPtXPnzoser6amRuXl5U4bAAAwp0ZRdiZNmqTBgwcrKSnJaTwvL09nz551Go+NjVV0dLRycnIuerz09HQFBQU5tqioqHrLDgAA3Mvjy87rr7+u/Px8paennzdntVrl6+ur4OBgp/GwsDBZrdaLHjMtLU1lZWWOrbCw0OjYAADAQ7j8UsGGVFhYqIceekhbt26Vv7+/Ycf18/OTn5+fYccDAACey6NXdvLy8lRcXKxrr71W3t7e8vb21vbt2/Xcc8/J29tbYWFhOnPmjEpLS51+rqioSOHh4e4JDQAAPIpHr+zcdNNN2r9/v9PYfffdp9jYWM2cOVNRUVHy8fFRdna2hg8fLkk6fPiwCgoKlJiY6I7IAADAw3h02QkICNA111zjNNa8eXO1bNnSMT527FilpqYqJCREgYGBSklJUWJionr06OGOyAAAwMN4dNn5PZYsWSIvLy8NHz5cNTU16t+/v55//nl3xwIAAB6i0ZWdDz/80Omzv7+/li9fruXLl7snEAAA8GgefYMyAABAXVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqVF2AACAqXl02UlPT1e3bt0UEBCg0NBQ3XbbbTp8+LDTPtXV1Zo0aZJatmypFi1aaPjw4SoqKnJTYgAA4Gk8uuxs375dkyZN0s6dO7V161adPXtW/fr1U2VlpWOfqVOn6l//+pfefPNNbd++XSdOnNCwYcPcmBoAAHgSb3cH+C1ZWVlOn1966SWFhoYqLy9P119/vcrKyvTCCy9o3bp16tu3ryRpzZo16tChg3bu3KkePXq4IzYAAPAgHr2y83+VlZVJkkJCQiRJeXl5Onv2rJKSkhz7xMbGKjo6Wjk5ORc9Tk1NjcrLy502AABgTo2m7NhsNk2ZMkW9evXSNddcI0myWq3y9fVVcHCw075hYWGyWq0XPVZ6erqCgoIcW1RUVH1GBwAAbtRoys6kSZN04MABvf7663U+VlpamsrKyhxbYWGhAQkBAIAn8uh7ds6ZPHmyNm3apB07dqhNmzaO8fDwcJ05c0alpaVOqztFRUUKDw+/6PH8/Pzk5+dXn5EBAICH8OiVHbvdrsmTJ+vtt9/Wtm3bFBMT4zTftWtX+fj4KDs72zF2+PBhFRQUKDExsaHjAgAAD+TRKzuTJk3SunXr9M477yggIMBxH05QUJCaNm2qoKAgjR07VqmpqQoJCVFgYKBSUlKUmJjIk1gAAECSh5edFStWSJL69OnjNL5mzRrde++9kqQlS5bIy8tLw4cPV01Njfr376/nn3++gZMCAABP5dFlx263/9d9/P39tXz5ci1fvrwBEgEAgMbGo+/ZAQAAqCvKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXKDgAAMDXTlJ3ly5erbdu28vf3V/fu3bVr1y53RwIAAB7AFGXnH//4h1JTUzV37lzl5+crLi5O/fv3V3FxsbujAQAANzNF2Vm8eLHGjRun++67Tx07dlRmZqaaNWumF1980d3RAACAm3m7O0BdnTlzRnl5eUpLS3OMeXl5KSkpSTk5ORf8mZqaGtXU1Dg+l5WVSZLKy8vrLaetpqrejt2Q6vOfUUMyw/ngXHgOzoVnMcP54Fy4dny73f6b+zX6snPq1CnV1tYqLCzMaTwsLExffPHFBX8mPT1djz/++HnjUVFR9ZLRTIIy3J0A53AuPAfnwrNwPjxHQ52Ln376SUFBQRedb/Rl51KkpaUpNTXV8dlms6mkpEQtW7aUxWJxY7JLV15erqioKBUWFiowMNDdcS5rnAvPwvnwHJwLz2GWc2G32/XTTz8pMjLyN/dr9GWnVatWatKkiYqKipzGi4qKFB4efsGf8fPzk5+fn9NYcHBwfUVsUIGBgY36f7hmwrnwLJwPz8G58BxmOBe/taJzTqO/QdnX11ddu3ZVdna2Y8xmsyk7O1uJiYluTAYAADxBo1/ZkaTU1FTdc889SkhI0HXXXaeMjAxVVlbqvvvuc3c0AADgZqYoO3feeae+//57zZkzR1arVV26dFFWVtZ5Ny2bmZ+fn+bOnXve5Tk0PM6FZ+F8eA7Ohee43M6Fxf7fntcCAABoxBr9PTsAAAC/hbIDAABMjbIDAABMjbIDAABMjbIDAMBl6HJ6PskUj55fjk6dOqUXX3xROTk5slqtkqTw8HD17NlT9957r/7whz+4OSEAwJP5+flp37596tChg7uj1DsePW+Edu/erf79+6tZs2ZKSkpyvE+oqKhI2dnZqqqq0nvvvaeEhAQ3J4UkFRYWau7cuXrxxRfdHeWycPr0aeXl5SkkJEQdO3Z0mquurtYbb7yhMWPGuCnd5eXQoUPauXOnEhMTFRsbqy+++ELPPvusampqdPfdd6tv377ujnhZ+PV3Qf7as88+q7vvvlstW7aUJC1evLghYzUoyk4j1KNHD8XFxSkzM/O8Ly612+2aMGGCPv/8c+Xk5LgpIX5t3759uvbaa1VbW+vuKKZ35MgR9evXTwUFBbJYLOrdu7def/11RURESPrl/xBERkZyLhpAVlaWhg4dqhYtWqiqqkpvv/22xowZo7i4ONlsNm3fvl1btmyh8DQALy8vxcXFnfcdkNu3b1dCQoKaN28ui8Wibdu2uSdgA6DsNEJNmzbVnj17FBsbe8H5L774QvHx8Tp9+nQDJ7s8bdy48Tfnv/76a02bNo3/wDaA22+/XWfPntVLL72k0tJSTZkyRf/5z3/04YcfKjo6mrLTgHr27Km+fftq/vz5ev311zVx4kQ98MADevLJJyVJaWlpysvL05YtW9yc1PwWLlyolStXavXq1U7l0sfHR/v27TtvBdSU7Gh02rZta1+7du1F59euXWu/8sorGy7QZc5isdi9vLzsFovlopuXl5e7Y14WQkND7Z9//rnjs81ms0+YMMEeHR1tP3r0qN1qtXIuGkhgYKD9yy+/tNvtdnttba3d29vbnp+f75jfv3+/PSwszF3xLju7du2y//GPf7RPmzbNfubMGbvdbrd7e3vbDx486OZkDYOnsRqhhx9+WOPHj9dDDz2kjRs3Kjc3V7m5udq4caMeeughTZgwQTNmzHB3zMtGRESE1q9fL5vNdsEtPz/f3REvG6dPn5a39/9/7sJisWjFihUaMmSIbrjhBh05csSN6S4/5y6ze3l5yd/fX0FBQY65gIAAlZWVuSvaZadbt27Ky8vT999/r4SEBB04cOC82yDMjKexGqFJkyapVatWWrJkiZ5//nnHknyTJk3UtWtXvfTSSxoxYoSbU14+unbtqry8PA0dOvSC8xaL5bJ6xNOdYmNj9dlnn533dMmyZcskSbfeeqs7Yl2W2rZtqy+//FJXX321JCknJ0fR0dGO+YKCAse9VGgYLVq00Nq1a/X6668rKSnpsrqcyz07jdzZs2d16tQpSVKrVq3k4+Pj5kSXn48++kiVlZUaMGDABecrKyv12Wef6YYbbmjgZJef9PR0ffTRR3r33XcvOD9x4kRlZmbKZrM1cLLLT2ZmpqKiojR48OALzv/tb39TcXGxVq9e3cDJIEnffvut8vLylJSUpObNm7s7Tr2j7AAAAFPjnh0AAGBqlB0AAGBqlB0AAGBqlB0AAGBqlB0AjYLdbtf48eMVEhIii8WivXv3ujsSgEaCp7EANAqbN2/W0KFD9eGHH+qqq65Sq1atnF4gCAAXw78pADQKR48eVUREhHr27HnB+TNnzsjX17eBUwFoDLiMBcDj3XvvvUpJSXF8m3nbtm3Vp08fTZ48WVOmTFGrVq3Uv39/Sb98k/N1110nPz8/RURE6JFHHtHPP//sOFafPn2UkpKiKVOm6IorrlBYWJhWrVqlyspK3XfffQoICFC7du20efNmd/26AAxG2QHg8Z599lnNmzdPbdq00cmTJ7V7925J0tq1a+Xr66tPPvlEmZmZ+u677zRo0CB169ZN+/bt04oVK/TCCy9o/vz5Tsdbu3atWrVqpV27diklJUUPPPCA/vKXv6hnz57Kz89Xv379NHr0aFVVVbnj1wVgMO7ZAdAoZGRkKCMjQ998842kX1ZoysvLnb5o9dFHH9Vbb72lQ4cOOb7k8Pnnn9fMmTNVVlYmLy8v9enTR7W1tfroo48kSbW1tQoKCtKwYcP08ssvS5KsVqsiIiKUk5OjHj16NOwvCsBwrOwAaLS6du3q9PnQoUNKTEx0+jbnXr16qaKiQt9++61jrHPnzo4/N2nSRC1btlSnTp0cY2FhYZKk4uLi+ooOoAFRdgA0Wpf6BYb/9wtzLRaL09i5ssQXhgLmQNkBYBodOnRQTk6Ofn11/pNPPlFAQIDatGnjxmQA3ImyA8A0Jk6cqMLCQqWkpOiLL77QO++8o7lz5yo1NVVeXvzrDrhc8Z4dAKbRunVrvfvuu5o+fbri4uIUEhKisWPHatasWe6OBsCNeBoLAACYGuu6AADA1Cg7AADA1Cg7AADA1Cg7AADA1Cg7AADA1Cg7AADA1Cg7AADA1Cg7AADA1Cg7AADA1Cg7AADA1Cg7AADA1P4fcdpfMXD51pAAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df.groupby(\"from\").count()[\"text\"].plot(kind=\"bar\")\n",
    "plt.ylabel(\"number of messages\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>text</th>\n",
       "      <th>from</th>\n",
       "      <th>starts_with_capital</th>\n",
       "      <th>number_count</th>\n",
       "      <th>punctuation_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>507</th>\n",
       "      <td>109993</td>\n",
       "      <td>can you an</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>707</th>\n",
       "      <td>105915</td>\n",
       "      <td>was in for a few i there</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>964</th>\n",
       "      <td>94410</td>\n",
       "      <td>was he</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>994</th>\n",
       "      <td>93996</td>\n",
       "      <td>now if</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1037</th>\n",
       "      <td>90031</td>\n",
       "      <td>but a very</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1130</th>\n",
       "      <td>67742</td>\n",
       "      <td>when will</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1258</th>\n",
       "      <td>56647</td>\n",
       "      <td>its all over again</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2230</th>\n",
       "      <td>37685</td>\n",
       "      <td>i for to be</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2765</th>\n",
       "      <td>33472</td>\n",
       "      <td>for me</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3305</th>\n",
       "      <td>25597</td>\n",
       "      <td>my to our so that i up</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          id                      text  from  starts_with_capital  \\\n",
       "507   109993                can you an     1                    1   \n",
       "707   105915  was in for a few i there     2                    0   \n",
       "964    94410                    was he     4                    1   \n",
       "994    93996                    now if     1                    1   \n",
       "1037   90031                but a very     3                    0   \n",
       "1130   67742                 when will     4                    1   \n",
       "1258   56647        its all over again     0                    0   \n",
       "2230   37685               i for to be     3                    0   \n",
       "2765   33472                    for me     1                    1   \n",
       "3305   25597    my to our so that i up     2                    0   \n",
       "\n",
       "      number_count  punctuation_count  \n",
       "507              0                  0  \n",
       "707              0                  1  \n",
       "964              0                  1  \n",
       "994              0                  0  \n",
       "1037             0                  1  \n",
       "1130             2                  2  \n",
       "1258             0                  0  \n",
       "2230             0                  0  \n",
       "2765             0                  0  \n",
       "3305             0                  2  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print 10 random samples\n",
    "df[df['id'].isin(np.random.choice(df['id'].values, 10, replace=False))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, there are more features in the dataset and the texts are formatted in a way easier to read by the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Task\n",
    "\n",
    "## Description\n",
    "\n",
    "I want to know if the model will be able to recognize the sender of a message based on the message and the feature engineered in the pre-processing step.\n",
    "\n",
    "Features: \n",
    "- text: the content of the message\n",
    "- starts_with_capital: whether or not the message starts with a capital letter\n",
    "- number_count: number of digits in the message\n",
    "- punctuation_count: number of punctuation signs in the message\n",
    "\n",
    "Dependent Variable:\n",
    "- from: number from 0 to 4 signifying who sent the message\n",
    "\n",
    "## Train-test split\n",
    "\n",
    "We are going to split data into the training and testing splits. I decided to make a 85-15 split, because the data is quite limited. We are going to shuffle data to ensure randomization to prevent any biases related to when the message was sent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_data, y_data = df[['text'] + additional_features].values, df['from'].values\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X_data, y_data, test_size=0.15, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([['are we to', 0, 0, 0],\n",
       "        ['but', 0, 0, 0],\n",
       "        ['these have the same and the same', 0, 0, 0],\n",
       "        ['do', 0, 0, 1],\n",
       "        ['it', 1, 0, 1]], dtype=object),\n",
       " array([1, 2, 2, 3, 4]))"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train[:5], y_train[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model\n",
    "\n",
    "## Motivation for the Model\n",
    "\n",
    "For the classification task, there are multiple models we can use: trees, naive Bayes, and neural networks. In this case, we do not have much data (around 140 examples per class), so using neural networks would not be effective (as they are not data efficient). Trees, on the other hand, do not perform well on data with many features as they easily overfit and are vulnerable to small data perturbations, which will often be the case in this dataset, where there can be multiple small changes to the same word/phrase. For example, \"by the way\" can be written as \"btw\" or \"btwww\" (if one wants to emphasize something). So, I decided that naive Bayes would be the most suitable for the purpose of this task.\n",
    "\n",
    "## How does Naive Bayes work?\n",
    "\n",
    "Naive Bayes is a conditional probability model that takes advantage of the Bayes theorem and works under the assumption that all the independent variables have independent probabilities.\n",
    "\n",
    "For the explanation, I will use $x = [x_1, x_2, \\dots, x_n]$ as the vector feature and $C_k$ as class k from the dependent variables.\n",
    "\n",
    "Using Bayes theorem, we can write the following:\n",
    "\n",
    "$$\n",
    "P(C_k | x) = \\frac{P(x | C_k) \\cdot P(C_k)}{P(x)}\n",
    "$$\n",
    "\n",
    "Since the denominator does not dependent on $C_k$, we can treat it as a constant, which can be calculated in the beginning.\n",
    "\n",
    "$$\n",
    "\\frac{P(x | C_k) \\cdot P(C_k)}{P(x)} \\propto P(x | C_k) \\cdot P(C_k)\n",
    "$$\n",
    "\n",
    "We expand this statement to show all individual features.\n",
    "\n",
    "$$\n",
    "P(x | C_k) \\cdot P(C_k) = \\frac{P(x \\cap C_k)}{P(C_k)} \\cdot P(C_k) \\\\\n",
    " = P(x \\cap C_k)\n",
    "$$\n",
    "\n",
    "At this point, we should remember that $x$ is a vector of features, which can be rewritten as $x = [x_1, x_2, \\dots, x_n]$. Now, using the chain rule for events, we can rewrite the above as the following.\n",
    "\n",
    "$$\n",
    "P(x \\cap C_k) = P(C_k) \\cdot P(x_1 | C_k) \\cdot P(x_2 | x_1, C_k) \\cdot \\dots \\cdot P(x_n | x_{n-1}, x_{n-2}, \\dots, x_1, C_k)\n",
    "$$\n",
    "\n",
    "At this stage, we use the assumption that all events in $x$ are indpenden. This means that $P(x_n | x_{n-1}, x_{n-2}, \\dots, x_1) = P(x_n)$. Therefore, we can rewrite the above as follows.\n",
    "\n",
    "$$\n",
    "P(x \\cap C_k) = P(C_k) \\cdot P(x_1 | C_k) \\cdot P(x_2 | x_1, C_k) \\cdot \\dots \\cdot P(x_n | x_{n-1}, x_{n-2}, \\dots, x_1, C_k)\n",
    "$$\n",
    "\n",
    "$$\n",
    "= P(C_k) \\cdot P(x_1 | C_k) \\cdot P(x_2 | C_k) \\cdot \\dots \\cdot P(x_n | C_k) \\propto P(C_k | x)\n",
    "$$\n",
    "\n",
    "When performing the inference, one should calculate the probability for each class and take the one with the highest probability."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the training step, the model has to go over all classes and calculate the number of occurrences of each class. This allows us to calculate the prior probabilities $P(x_i)$. Then, for each feature in the whole dataset, the algorithm finds the examples with this feature and calculates the conditional probability $P(x_i | C_k)$, which will be our priors for the inference. In the prediction step, the algorithm iterates over all features and classes. For each feature-class pair, the logarithmic probability of the class and conditional probability of the class given the feature is calculated to get the posterior. The logarithmic value is used instead of direct one to prevent the float percision overflow. Logs have another nice property, meaning we can use summation instead of multiplication due to the property of logs $\\log{a} + \\log{b} = \\log{a \\cdot b}$ to calculate the total conditional probabilty."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The below is an example implementation of the Naive Bayes classifier. I also fit it to the Iris dataset to check whether it works correctly. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbAAAAGwCAYAAADITjAqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAcE0lEQVR4nO3de3hUhZ3/8c9AkiGGEA25AYJiWW4/ECUiUjRpFAW1ArrVn63UgJYuEJBb5FJrEbzEVVsqgtCyCuhK1wuFIvDb/tggkAjIJQaKYjSgVTAkpNHEBBKSzNk/fEw75ZIEMjnzDe/X88wfc86ZM9/kPPDOmavHcRxHAAAY08rtAQAAOBcEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACaFuD1AIJTcmez2CGgmcevz3R4BQADUnDxS7zacgQEATCJgAACTCBgAwCQCBgAwiYABAEwiYAAAkwgYAMAkAgYAMImAAQBMImAAAJMIGADAJAIGADCJgAEATCJgAACTCBgAwCQCBgAwiYABAEwiYAAAkwgYAMAkAgYAMImAAQBMImAAAJMIGADAJAIGADCJgAEATCJgAACTCBgAwCQCBgAwiYABAEwiYAAAkwgYAMAkAgYAMImAAQBMImAAAJMIGADAJAIGADCJgAEATCJgAACTCBgAwCQCBgAwiYABAEwiYAAAkwgYAMAkAgYAMImAAQBMImAAAJMIGADAJAIGADCJgAEATCJgAACTCBgAwCQCBgAwiYABAEwiYAAAkwgYAMAkAgYAMImAGRf1u/9S9Ootp1wu+vkUt0dDgIwfl6r8j3eovOygtmW/rQHXXOX2SAgQjvXZETDjyh7+N3015s66S9mcaZKkk+9udncwBMTddw/Xc8/O0eNP/EYDBg7T3n0fasP61xQb297t0dDEONb1I2DGOWWlcr4uqbuEXTNItQWHVfNBrtujIQCmTh6r/3hppVa88oYOHPhEE9Jm6fjxExoz+l63R0MT41jXL8TNOy8uLtbLL7+s7du36+jRo5KkhIQEff/739fo0aMVGxvr5nj2hIQoLPlmVa590+1JEAChoaHq3/9KPf3MwrpljuMoc1O2rrsu0cXJ0NQ41g3j2hnYrl271L17dy1YsEBRUVFKSkpSUlKSoqKitGDBAvXs2VO7d++udz9VVVUqKyvzu1TV+prhJwg+YdfeIE9EW1Vt+n9uj4IAiImJVkhIiIoKi/2WFxUdU0I8f+y1JBzrhnHtDGzSpEm6++67tWTJEnk8Hr91juNo3LhxmjRpkrZv337W/WRkZGju3Ll+y2b06KKZvS5v6pGDnnfIbarO2Snnq7+5PQoABJxrZ2B79+7V1KlTT4mXJHk8Hk2dOlW5ubn17mf27NkqLS31u0zp3iUAEwe3VrHxCrkyUVX/s87tURAgxcUlqqmpUVx8jN/yuLhYHS085tJUCASOdcO4FrCEhATt3LnzjOt37typ+Pj4evfj9XrVrl07v4u39YX32hTvjbfKKf1a1bt3uD0KAqS6ulo5Oft0Y8r1dcs8Ho9uTLleO3bscXEyNDWOdcO49hBienq6fv7zn2vPnj266aab6mJVWFiozMxMLV26VM8995xb49ni8SjsxltVtfm/JV+t29MggOY/v1TLXpqvPTn7tGvX+3po0lhFRIRr+YrX3R4NTYxjXT/XApaWlqaYmBjNnz9fL774omprv/2Pt3Xr1kpMTNTy5ct1zz33uDWeKSFXJqp1XIKqMje4PQoC7M031yo2JlqP/SpdCQmx2rv3A93+w1EqKiqu/8YwhWNdP4/jOI7bQ1RXV6u4+NuDEhMTo9DQ0PPaX8mdyU0xFgyIW5/v9ggAAqDm5JF6t3H1fWDfCQ0NVYcOHdweAwBgyIX3agcAQItAwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACaFuD1AIMStz3d7BDSTb5Y94PYIaEbTH8lzewQEEc7AAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmnVPAsrKyNGrUKA0aNEhHjhyRJL366qvKzs5u0uEAADiTRgds1apVGjp0qMLDw/X++++rqqpKklRaWqqnnnqqyQcEAOB0Gh2wJ554QkuWLNHSpUsVGhpat3zw4MHKyclp0uEAADiTRgcsLy9PSUlJpyyPiorS119/3RQzAQBQr0YHLCEhQfn5+acsz87O1hVXXNEkQwEAUJ9GB2zs2LGaPHmy3nvvPXk8Hn355Zd67bXXlJ6ervHjxwdiRgAAThHS2BvMmjVLPp9PN910k44fP66kpCR5vV6lp6dr0qRJgZgRAIBTNDpgHo9HjzzyiB5++GHl5+ervLxcvXv3Vtu2bQMxHxpg/LhUTZ82XgkJsdq370NNnvKodu3OdXssBEBFVbUWbd6vdz46opKKKvVIuFgzhl6tPp2i3R4NTeiGUTcr6b5bFH1prCSp4JPD2rDgLX24OdfdwYJMowP2nbCwMPXu3bspZ8E5uPvu4Xru2TmakDZLO3e9r4cm/Uwb1r+m3n2SdOzY39weD01s7tu7lX+sVE+MHKjYyDZav++vGvefW7Rq/FDFt7vI7fHQRL4uKNGaf1+pos8K5PF4dN2/Jmvc72co4/YZKvjksNvjBY1GBywlJUUej+eM6zdt2nReA6Fxpk4eq/94aaVWvPKGJGlC2izddutNGjP6Xj3z7CKXp0NTqqyuUeaBw5r/fwcr8bJv/zIf/4M+2vpJgd7cfVATb+zr8oRoKn/J3ON3fe1z/6UbRt2irlf/CwH7B40O2FVXXeV3vbq6Wrm5udq/f79SU1Obai40QGhoqPr3v1JPP7OwbpnjOMrclK3rrkt0cTIEQq3PUa3jyBvS2m+5N6S13v+i2KWpEGieVh71v32QwsK9OpTzsdvjBJVGB2z+/PmnXf7YY4+pvLz8vAf6R1988YXmzJmjl19++YzbVFVV1X0ayHccxznrWWJLERMTrZCQEBUV+v/nVVR0TD17fM+lqRAoEd5QXXlpe/0+60N1jW2n9hFe/ff+L7Tv8N/UOZrnoFuajj06K/2PTyrUG6qq45X6/b89p6P5R9weK6g02Yf5jho16qyhORclJSVasWLFWbfJyMhQVFSU38XxfdOkcwDB4smRAyVHumX+27r2yVVaufMTDevTWa1a/t9rF5zCQ18q47aH9czIXyjrP/+/7v91mhK6dXJ7rKByzi/i+Gfbt29XmzZtGnWbtWvXnnX9oUOH6t3H7NmzNW3aNL9ll7Tv2ag5rCouLlFNTY3i4mP8lsfFxepo4TGXpkIgdY5uq5dGp+jEyRqVV1UrNjJcM97ark4XcwbW0tRW1+rYXwslSV/s/1SXXfk9pTxwm/7wi6UuTxY8Gh2wu+66y++64zgqKCjQ7t279eijjzZqXyNHjpTH45HjOGfcpr6HAr1er7xeb6Nu01JUV1crJ2efbky5XmvX/lnStz/7jSnX68XFy1yeDoEUHhai8LAQlZ04qW0Hj2rKkCvdHgkB5mnVSiFhofVveAFpdMCioqL8rrdq1Uo9evTQvHnzdMsttzRqXx06dNCLL76oESNGnHZ9bm6uEhN5McLZzH9+qZa9NF97cvZp16739dCksYqICNfyFa+7PRoCYFv+UTlydHn7SH1eUq75/7NPXWMiNeKqrm6PhiY0YsaP9cHmXJV8Waw2EW00YMT1+pfremvh/U+6PVpQaVTAamtrNWbMGPXt21eXXHLJed95YmKi9uzZc8aA1Xd2BunNN9cqNiZaj/0qXQkJsdq79wPd/sNRKiriVWkt0TdV1Xph0z4Vlp1QVHiYbup1qSam9FFoa76btiWJbB+l1N+kqV3sJar85riOfPRXLbz/SX2U/Re3RwsqHqeRhWjTpo0OHDigrl3P/y++rKwsVVRUaNiwYaddX1FRod27dys5OblR+w0J44nOC8U3yx5wewQ0o+mP5Lk9AprJi5+9Ue82jX4IsU+fPjp06FCTBOyGG2446/qIiIhGxwsAcGE4py+0TE9P17p161RQUKCysjK/CwAAzaHBZ2Dz5s3T9OnTddttt0mShg8f7vdqv+/ePFxbW9v0UwIA8E8aHLC5c+dq3LhxeueddwI5DwAADdLggH33Wg+ekwIABINGPQd2obxBGAAQ/Br1KsTu3bvXG7GSkpLzGggAgIZoVMDmzp17yidxAADghkYF7N5771VcXFygZgEAoMEa/BwYz38BAIJJgwPGZxICAIJJgx9C9Pl8gZwDAIBG4SOsAQAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCY5HEcx3F7iKYWEtbJ7REABMCJL7PcHgHNJDTminq34QwMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhGwFmD8uFTlf7xD5WUHtS37bQ245iq3R0IAcbxbnt25f1HajDlKGX6f+gy+VZlbt/mt37j5XY2d8gsNvvUe9Rl8qz76+KBLkwYXAmbc3XcP13PPztHjT/xGAwYO0959H2rD+tcUG9ve7dEQABzvlunEiUr16HaFHpk+4fTrKyvV/8r/o6njH2jmyYKbx3Ecx+0hmlpIWCe3R2g227Lf1q7dezV5yi8lSR6PR58d2qVFLy7TM88ucnk6NLUL/Xif+DLL7RECrs/gW/V8xqO6Ken7p6w7UlCooT8arbeWLVTP7t9zYbrmExpzRb3bcAZmWGhoqPr3v1KZm/7+j9pxHGVuytZ11yW6OBkCgeMN+HM9YCdOnFB2drY+/PDDU9ZVVlbqlVdeOevtq6qqVFZW5ndpgSeVpxUTE62QkBAVFRb7LS8qOqaE+FiXpkKgcLwBf64G7OOPP1avXr2UlJSkvn37Kjk5WQUFBXXrS0tLNWbMmLPuIyMjQ1FRUX4Xx/dNoEcHALjM1YDNnDlTffr0UVFRkfLy8hQZGanBgwfr888/b/A+Zs+erdLSUr+Lp1VkAKcOHsXFJaqpqVFcfIzf8ri4WB0tPObSVAgUjjfgz9WAbdu2TRkZGYqJiVG3bt309ttva+jQobrhhht06NChBu3D6/WqXbt2fhePxxPgyYNDdXW1cnL26caU6+uWeTwe3ZhyvXbs2OPiZAgEjjfgL8TNOz9x4oRCQv4+gsfj0eLFizVx4kQlJydr5cqVLk5nw/znl2rZS/O1J2efdu16Xw9NGquIiHAtX/G626MhADjeLdPx4yf0+eEv664f+bJQH318UFHtItUhIU6lZd+o4GiRior/Jkn69PPDkqSY9pcopn20KzMHA1cD1rNnT+3evVu9evXyW75w4UJJ0vDhw90Yy5Q331yr2JhoPfardCUkxGrv3g90+w9HqaiouP4bwxyOd8u0/6NP9MCkmXXXn3nh95KkEbcO0ZO/nK53snbol0/9pm79w3OeliSNf+A+pT04qnmHDSKuvg8sIyNDWVlZ2rBhw2nXT5gwQUuWLJHP52vUfi+k94EBF5IL4X1g+FZD3gfGG5kBmEHALhy8kRkA0GIRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJhEwAIBJBAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgksdxHMftIXD+qqqqlJGRodmzZ8vr9bo9DgKIY33h4FifHQFrIcrKyhQVFaXS0lK1a9fO7XEQQBzrCwfH+ux4CBEAYBIBAwCYRMAAACYRsBbC6/Vqzpw5PNF7AeBYXzg41mfHizgAACZxBgYAMImAAQBMImAAAJMIGADAJALWAixatEiXX3652rRpo4EDB2rnzp1uj4QA2Lp1q+644w517NhRHo9Ha9ascXskBEhGRoYGDBigyMhIxcXFaeTIkcrLy3N7rKBDwIx7/fXXNW3aNM2ZM0c5OTnq16+fhg4dqqKiIrdHQxOrqKhQv379tGjRIrdHQYBt2bJFaWlp2rFjhzZu3Kjq6mrdcsstqqiocHu0oMLL6I0bOHCgBgwYoIULF0qSfD6fOnfurEmTJmnWrFkuT4dA8Xg8Wr16tUaOHOn2KGgGx44dU1xcnLZs2aKkpCS3xwkanIEZdvLkSe3Zs0dDhgypW9aqVSsNGTJE27dvd3EyAE2ptLRUkhQdHe3yJMGFgBlWXFys2tpaxcfH+y2Pj4/X0aNHXZoKQFPy+XyaMmWKBg8erD59+rg9TlAJcXsAAMCZpaWlaf/+/crOznZ7lKBDwAyLiYlR69atVVhY6Le8sLBQCQkJLk0FoKlMnDhR69at09atW3XppZe6PU7Q4SFEw8LCwpSYmKjMzMy6ZT6fT5mZmRo0aJCLkwE4H47jaOLEiVq9erU2bdqkrl27uj1SUOIMzLhp06YpNTVV11xzja699lr99re/VUVFhcaMGeP2aGhi5eXlys/Pr7v+6aefKjc3V9HR0erSpYuLk6GppaWlaeXKlfrTn/6kyMjIuue0o6KiFB4e7vJ0wYOX0bcACxcu1LPPPqujR4/qqquu0oIFCzRw4EC3x0IT27x5s1JSUk5ZnpqaquXLlzf/QAgYj8dz2uXLli3T6NGjm3eYIEbAAAAm8RwYAMAkAgYAMImAAQBMImAAAJMIGADAJAIGADCJgAEATCJgAACTCBgQ5EaPHu33xZU/+MEPNGXKlGafY/PmzfJ4PPr666+b/b6B0yFgwDkaPXq0PB6PPB6PwsLC1K1bN82bN081NTUBvd8//vGPevzxxxu0LdFBS8aH+QLnYdiwYVq2bJmqqqq0YcMGpaWlKTQ0VLNnz/bb7uTJkwoLC2uS++RbeYFvcQYGnAev16uEhARddtllGj9+vIYMGaK1a9fWPez35JNPqmPHjurRo4ck6YsvvtA999yjiy++WNHR0RoxYoQ+++yzuv3V1tZq2rRpuvjii9W+fXvNmDFD//xxpf/8EGJVVZVmzpypzp07y+v1qlu3bnrppZf02Wef1X347yWXXCKPx1P3QbA+n08ZGRnq2rWrwsPD1a9fP7311lt+97NhwwZ1795d4eHhSklJ8ZsTCAYEDGhC4eHhOnnypCQpMzNTeXl52rhxo9atW6fq6moNHTpUkZGRysrK0rvvvqu2bdtq2LBhdbf59a9/reXLl+vll19Wdna2SkpKtHr16rPe5/33368//OEPWrBggQ4cOKDf/e53atu2rTp37qxVq1ZJkvLy8lRQUKDnn39ekpSRkaFXXnlFS5Ys0QcffKCpU6dq1KhR2rJli6RvQ3vXXXfpjjvuUG5urn72s59p1qxZgfq1AefGAXBOUlNTnREjRjiO4zg+n8/ZuHGj4/V6nfT0dCc1NdWJj493qqqq6rZ/9dVXnR49ejg+n69uWVVVlRMeHu78+c9/dhzHcTp06OA888wzdeurq6udSy+9tO5+HMdxkpOTncmTJzuO4zh5eXmOJGfjxo2nnfGdd95xJDlfffVV3bLKykrnoosucrZt2+a37YMPPuj8+Mc/dhzHcWbPnu307t3bb/3MmTNP2RfgJp4DA87DunXr1LZtW1VXV8vn8+knP/mJHnvsMaWlpalv375+z3vt3btX+fn5ioyM9NtHZWWlDh48qNLSUhUUFPh9l1tISIiuueaaUx5G/E5ubq5at26t5OTkBs+cn5+v48eP6+abb/ZbfvLkSV199dWSpAMHDpzynXJ8yzeCDQEDzkNKSooWL16ssLAwdezYUSEhf/8nFRER4bdteXm5EhMT9dprr52yn9jY2HO6/3P5dt7y8nJJ0vr169WpUye/dV6v95zmANxAwIDzEBERoW7dujVo2/79++v1119XXFyc2rVrd9ptOnTooPfee09JSUmSpJqaGu3Zs0f9+/c/7fZ9+/aVz+fTli1bNGTIkFPWf3cGWFtbW7esd+/e8nq9+vzzz8945tarVy+tXbvWb9mOHTvq/yGBZsSLOIBmct999ykmJkYjRoxQVlaWPv30U23evFkPPfSQDh8+LEmaPHmynn76aa1Zs0YfffSRJkyYcNb3cF1++eVKTU3VAw88oDVr1tTt84033pAkXXbZZfJ4PFq3bp2OHTum8vJyRUZGKj09XVOnTtWKFSt08OBB5eTk6IUXXtCKFSskSePGjdMnn3yihx9+WHl5eVq5cqWWL18e6F8R0CgEDGgmF110kbZu3aouXbrorrvuUq9evfTggw+qsrKy7oxs+vTp+ulPf6rU1FQNGjRIkZGRuvPOO8+638WLF+tHP/qRJkyYoJ49e2rs2LGqqKiQJHXq1Elz587VrFmzFB8fr4kTJ0qSHn/8cT366KPKyMhQr169NGzYMK1fv15du3aVJHXp0kWrVq3SmjVr1K9fPy1ZskRPPfVUAH87QON5nDM9OwwAQBDjDAwAYBIBAwCYRMAAACYRMACASQQMAGASAQMAmETAAAAmETAAgEkEDABgEgEDAJhEwAAAJv0vrkjvElxFkj0AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import seaborn as sns\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn import metrics\n",
    "\n",
    "class MultinomialNaiveBayes:\n",
    "    def __init__(self):\n",
    "        '''\n",
    "        Initializes the naive bayes classifier instance\n",
    "        '''\n",
    "        self.class_probabilities = {}\n",
    "        self.feature_probabilities = {}\n",
    "        self.classes = None\n",
    "        self.vocab_size = 0\n",
    "\n",
    "    def fit(self, X, y):\n",
    "        '''\n",
    "        Fits the multinomial naive bayes classifier\n",
    "        ---\n",
    "        Parameters:\n",
    "          X list[list]: vector of features\n",
    "          y list[int]: classes to be predicted\n",
    "        ---\n",
    "        Returns:\n",
    "          None\n",
    "        '''\n",
    "        # Count the number of instances and features\n",
    "        num_instances, num_features = X.shape\n",
    "        self.classes = np.unique(y)\n",
    "        \n",
    "        # Calculate class probabilities\n",
    "        class_counts = np.bincount(y)\n",
    "        self.class_probabilities = dict(zip(self.classes, class_counts / num_instances))\n",
    "\n",
    "        # Calculate feature probabilities\n",
    "        self.feature_probabilities = {c: {} for c in self.classes}\n",
    "        self.vocab_size = num_features\n",
    "\n",
    "        for c in self.classes:\n",
    "            # Select rows for class c\n",
    "            X_c = X[y == c]\n",
    "\n",
    "            # Count feature occurrences in each class\n",
    "            # Add one to prevent division by zero (Laplace smoothing)\n",
    "            feature_counts = np.sum(X_c, axis=0) + 1 \n",
    "            total_count = np.sum(feature_counts)\n",
    "\n",
    "            # Calculate probabilities\n",
    "            self.feature_probabilities[c] = feature_counts / total_count\n",
    "\n",
    "    def predict(self, X):\n",
    "        '''\n",
    "        Predicts the classes from vectors of features using Multinomial Naive Bayes \n",
    "        ---\n",
    "        Parameters:\n",
    "          X list[list]: classes transformed to model readable format\n",
    "        ---\n",
    "        Returns:\n",
    "          y list[int]: predictions made from the model and given feature vector\n",
    "        '''\n",
    "        predictions = []\n",
    "\n",
    "        for x in X:\n",
    "            class_scores = {}\n",
    "\n",
    "            for c in self.classes:\n",
    "                # Start with the log of the class probability\n",
    "                class_score = np.log(self.class_probabilities[c])\n",
    "\n",
    "                # Add log of feature probabilities\n",
    "                for feature_index in range(self.vocab_size):\n",
    "                    class_score += x[feature_index] * np.log(self.feature_probabilities[c][feature_index])\n",
    "                \n",
    "                class_scores[c] = class_score\n",
    "            \n",
    "            # Select the class with the highest score\n",
    "            predicted_class = max(class_scores, key=class_scores.get)\n",
    "            predictions.append(predicted_class)\n",
    "\n",
    "        return np.array(predictions)\n",
    "\n",
    "# Load iris dataset\n",
    "X, y = load_iris(return_X_y=True)\n",
    "# Split dataset to train and test splits\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n",
    "\n",
    "# Initialize and train the classifier\n",
    "mnb = MultinomialNaiveBayes()\n",
    "# Fit the classifier\n",
    "mnb.fit(X_train, y_train)\n",
    "# Predict the test set\n",
    "pred = mnb.predict(X_test)\n",
    "\n",
    "# Plot the confusion matrix\n",
    "confusion_mat = metrics.confusion_matrix(y_test, pred)\n",
    "sns.heatmap(confusion_mat, square=True, annot=True, fmt='d', cbar=False)\n",
    "plt.xlabel('Predicted')\n",
    "plt.ylabel('True')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As shown on the confusion matrix above, the algorithm seems to be working correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "import math\n",
    "\n",
    "\n",
    "def tf(t, d):\n",
    "    \"\"\"\n",
    "    Term frequency calculation\n",
    "    ---\n",
    "    Input:\n",
    "      t (string): term for which to calculate tf\n",
    "      d (string): document\n",
    "    ---\n",
    "    Output:\n",
    "      tf score (float): term frequency for term t in document d\n",
    "    \"\"\"\n",
    "\n",
    "    freq = Counter(d.split()).get(t, 0)\n",
    "    d_len = len(d.split())\n",
    "    \n",
    "    return freq / d_len\n",
    "\n",
    "\n",
    "def idf(t, c):\n",
    "    \"\"\"\n",
    "    Inverse document frequency\n",
    "    ---\n",
    "    Input:\n",
    "      t (string): term for which to calculate idf\n",
    "      c (list[string]): corpus\n",
    "    ---\n",
    "    Output:\n",
    "      idf score (float)\n",
    "    \"\"\"\n",
    "    # Number of documents in corpus\n",
    "    N = len(c)\n",
    "    # Number of documents with term t\n",
    "    k = sum([1 if t in d.split() else 0 for d in c])\n",
    "\n",
    "    # add 1 to prevent divisino by zero\n",
    "    return math.log((N + 1) / (k + 1), math.e) + 1\n",
    "\n",
    "\n",
    "def tfidf(c):\n",
    "    \"\"\"\n",
    "    Tf-idf vectorization\n",
    "    ---\n",
    "    Input:\n",
    "      c (list[string]): corpus with all documents for the corpus\n",
    "    Output:\n",
    "      tf-idf scores (list[float]): tf-idf score for each word in the corpus\n",
    "    \"\"\"\n",
    "    # Store scores for each document\n",
    "    _tfidf = []\n",
    "    # Get all terms in document\n",
    "    terms = sorted(list(set((\" \".join(c)).split())))\n",
    "    idfs = {t: idf(t, c) for t in terms}\n",
    "\n",
    "    # For each document calculate tf-idf\n",
    "    for d in c:\n",
    "        tfidf_d = []\n",
    "        for t in terms:\n",
    "            tfidf_d.append(tf(t, d) * idfs[t])\n",
    "        _tfidf.append(tfidf_d)\n",
    "\n",
    "    # Apply Euclidian norm\n",
    "    _tfidf = np.array(_tfidf) \n",
    "    _tfidf /= ((_tfidf**2).sum(1, keepdims=True))**(1/2)\n",
    "\n",
    "    return _tfidf "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "corpus = [\"doo be doo\", \"al do be doo\", \"all wee doo be da\"]\n",
    "\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "\n",
    "# Make sure the vectorizer is implemented correctly\n",
    "vectorizer = TfidfVectorizer()\n",
    "assert np.allclose(vectorizer.fit_transform(corpus).toarray(), tfidf(corpus))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training\n",
    "\n",
    "For training, I will use the sklearn's implementation of the Naive Bayes classifier and vectorizer, because their implementation is more efficient and robust."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.model_selection import train_test_split, GridSearchCV\n",
    "from sklearn import metrics\n",
    "\n",
    "import seaborn as sns\n",
    "\n",
    "# Define the variables for the classification\n",
    "y = df['from'].values\n",
    "# Use the texts and all additional features\n",
    "X = df[['text'] + additional_features].values\n",
    "\n",
    "# Vectorizing the texts\n",
    "vectorizer = TfidfVectorizer()\n",
    "# Combining the vectorized texts with additional features\n",
    "X = np.c_[vectorizer.fit_transform(X[:, 0]).toarray(), X[:, 1:]]\n",
    "\n",
    "# Split data into the training and testing \n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, shuffle=True)\n",
    "\n",
    "# Create the Naive Bayes classifier\n",
    "clf = MultinomialNB()\n",
    "\n",
    "# Define hyperparameter for grid search\n",
    "params = {'alpha': [1e-5, 1e-4, 1e-3, 1e-2, 1e-1, 1]}\n",
    "\n",
    "# Define the grid search for hyperparameter tuning\n",
    "grid = GridSearchCV(clf, param_grid=params, cv=6, refit=True)\n",
    "\n",
    "# Find the best parameters for the model\n",
    "clf = grid.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run the classifier with the hyperparameters on test set\n",
    "pred = clf.predict(X_test)\n",
    "# Create confusion matrix\n",
    "confusion_mat = metrics.confusion_matrix(y_test, pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbAAAAHHCAYAAADeaQ1TAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABA+UlEQVR4nO3deXhMZ+M+8HuyTSJ7ZLOGhIRYgqilliC20lJ9Ve2JJWrfGiW0FVqi+lZ4UVvtpVWU9ovXvreUWmILEjtFBAlJZJt5fn/4ZV4jkU3inCe9P9c112Wec+ace85k5p4zc87QCCEEiIiIJGOidAAiIqLCYIEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWGBERSYkFRkREUmKBqVBMTAzatm0Le3t7aDQabN68uUiXf/36dWg0GqxYsaJIl1sSVKpUCcHBwUW2vPv376Nr164oXbo0NBoNZs+eXWTL/icJDg5GpUqVlI5RpFasWAGNRoPr168bxlq0aIEWLVoU2TrCw8Oh0WiKbHlqwwJ7hStXruDjjz+Gp6cnLC0tYWdnhyZNmmDOnDl49uxZsa47KCgIZ8+exbRp07B69WrUr1+/WNdXEl24cAHh4eFGLw5KGDNmDHbs2IGwsDCsXr0a7du3fyPrTUhIgKWlJTQaDaKjo3OcJzg4GDY2Nm8kDxWflJQUhIeHY//+/UpHefMEZbNlyxZhZWUlHBwcxMiRI8XixYvFvHnzRPfu3YW5ubkICQkptnWnpKQIAGLSpEnFtg69Xi+ePXsmMjMzi20dSlu/fr0AIPbt21eg26Wmpor09PQiy+Hm5iZ69epVZMvLr8WLFwtLS0vh7u7+yr+loKAgYW1t/YaTFU5QUJDw8PBQOkaRWr58uQAgrl27ZhhLS0sTaWlpBVrOgwcPBAAxefLkbNMyMjLEs2fPXjOpepkpW5/qc+3aNXTv3h0eHh7Yu3cvypQpY5g2bNgwxMbGYuvWrcW2/gcPHgAAHBwcim0dGo0GlpaWxbZ82QghkJqaCisrK2i12iJddlxcXJE+lqmpqbCwsICJSe4fnvzwww/o0KEDPDw8sHbtWnz11VdFluGfLjMzE3q9HhYWFkW+7KJeppmZGczMSvDLvNINqjaDBw8WAMTvv/+er/kzMjLE1KlThaenp7CwsBAeHh4iLCxMpKamGs3n4eEhOnbsKA4dOiTeeustodVqReXKlcXKlSsN80yePFkAMLpkvet81TvQrNu8aOfOnaJJkybC3t5eWFtbC29vbxEWFmaYfu3aNQFALF++3Oh2e/bsEU2bNhWlSpUS9vb2olOnTuLChQs5ri8mJkYEBQUJe3t7YWdnJ4KDg0VycnKe2ysgIEDUqFFDREVFiebNmwsrKyvh5eUl1q9fL4QQYv/+/aJBgwbC0tJSeHt7i127dhnd/vr162LIkCHC29tbWFpaCicnJ9G1a1ejd7FZ72xfvmTtjWU9Ftu3bxf+/v5Cq9WKyMhIw7SgoCAhxPM91RYtWghnZ2dx//59w/LT0tJEzZo1haenp0hKSsrxfr4qQ5YrV66Irl27CkdHR2FlZSUaNmwotmzZYrSMffv2CQDixx9/FJMmTRJly5YVGo1GPH78ONdtfOPGDaHRaMTPP/8s/vzzz1f+PRdkDwyAGDZsmPj5559F9erVhaWlpWjUqJE4c+aMEEKIhQsXCi8vL6HVakVAQIDR4yGEEAcPHhRdu3YVFSpUEBYWFqJ8+fJi9OjRIiUlJdu6Nm3aJGrUqCG0Wq2oUaOG+OWXX3L8+9fpdCIyMlL4+voKrVYrXF1dxaBBg8SjR4/yvD9Z9/3KlSuibdu2olSpUqJMmTJiypQpQq/XG+bLeq588803IjIyUnh6egoTExNx6tQpIYQQ0dHR4l//+pdwdHQUWq1W+Pv7i19//TXb+s6dOydatmwpLC0tRbly5cSXX34pli5dmm0PLCAgQAQEBBjd9tmzZ2Ly5MmiatWqQqvVCnd3d9GlSxcRGxtryPfyJWtvLKfXh6J8zRJCiPT0dBEeHi6qVKkitFqtcHJyEk2aNBE7d+7M83F4XSywl5QrV054enrme/6goCABQHTt2lXMnz9f9O3bVwAQ77//vtF8Hh4ewsfHR7i5uYmJEyeKefPmiXr16gmNRiPOnTsnhBAiKipKREZGCgCiR48eYvXq1WLTpk2G9eSnwM6dOycsLCxE/fr1xZw5c8TChQtFaGioaN68uWGenAps165dwszMTHh7e4uZM2eKKVOmCGdnZ+Ho6Gj0BMtaX926dcUHH3wgvvvuOzFw4EABQHz66ad5bq+AgABRtmxZUaFCBTFu3Dgxd+5c4evrK0xNTcVPP/0k3N3dRXh4uJg9e7YoV66csLe3F0+ePDHcfv369cLPz0988cUXYvHixWLixInC0dFReHh4GAr0ypUrYuTIkQKAmDhxoli9erVYvXq1uHfvnuGxqFKlinB0dBQTJkwQCxcuNCq3rAITQoirV68KGxsb0aVLF8PYhAkThEajEQcOHHjl/bxy5YpYvXq1ACDatGljyCCEEPfu3RNubm7C1tZWTJo0ScyaNUv4+fkJExMT8csvvxiWkVVgvr6+ok6dOmLWrFkiIiIizzcKM2bMEDY2NoZy8PLyEkOHDs02X0ELrHbt2qJChQpixowZYsaMGcLe3l5UrFhRzJs3T/j6+opvv/1WfPbZZ8LCwkK0bNnS6PYjRowQHTp0ENOnTxeLFi0SAwYMEKampqJr165G8+3YsUOYmJiImjVrilmzZolJkyYJe3t7UaNGjWx//wMHDhRmZmYiJCRELFy4UIwfP15YW1uLt956K8+PgYOCgoSlpaWoWrWq6NOnj5g3b5549913BQDx+eefG+bLeq74+voKT09PMWPGDBEZGSlu3Lghzp07J+zt7YWvr6/4+uuvxbx580Tz5s2FRqMxehzv3r0rXFxchKOjowgPDxfffPONqFq1qqhdu3aeBZaZmSkCAwMFANG9e3cxb948ERERIVq1aiU2b94skpKSxIIFCwQA0aVLF8PfWVRUlBAi5wIrytcsIYSYOHGi0Gg0IiQkRCxZskR8++23okePHmLGjBm5PgZFgQX2gsTERAFAdO7cOV/znz59WgAQAwcONBoPDQ0VAMTevXsNYx4eHgKAOHjwoGEsLi5OaLVa8cknnxjGXnzH96L8FlhWAT548OCVuXMqsDp16ghXV1fx8OFDw1hUVJQwMTERffv2zba+/v37Gy2zS5cuonTp0q9cZ5aAgAABQKxdu9YwdvHiRQFAmJiYiKNHjxrGd+zYkS1nTu/Yjxw5IgCIVatWGcZy+w4s67HYvn17jtNeLDAhhFi0aJEAIH744Qdx9OhRYWpqKkaPHp3nfRXif3suLxo9erQAIA4dOmQYe/r0qahcubKoVKmS0Ol0Qoj/FZinp2eO9/tVatWqZfS928SJE4Wzs7PIyMgwmq+gBabVao1ebLO2i7u7u9GbjLCwsGwvzDnlj4iIEBqNRty4ccMwVqdOHVGmTBmRkJBgGNu5c6fRpxFCCHHo0CEBQKxZs8Zomdu3b89x/GVZL+IjRowwjOn1etGxY0dhYWFheP5kPVfs7OxEXFyc0TICAwNFrVq1jPZc9Hq9ePvtt0XVqlUNY1mP959//mkYi4uLE/b29nkW2LJlywQAMWvWrGz3IWtPMbfvwF5+fSiO1yw/Pz/RsWPHbOt+E3gU4guePHkCALC1tc3X/Nu2bQMAjB071mj8k08+AYBs35X5+vqiWbNmhusuLi7w8fHB1atXC535ZVnft/z666/Q6/X5us3du3dx+vRpBAcHw8nJyTBeu3ZttGnTxnA/XzR48GCj682aNcPDhw8N2zA3NjY26N69u+G6j48PHBwcUL16dTRs2NAwnvXvF7ePlZWV4d8ZGRl4+PAhqlSpAgcHB5w8eTIf9/a5ypUro127dvmad9CgQWjXrh1GjBiBPn36wMvLC9OnT8/3ul62bds2NGjQAE2bNjWM2djYYNCgQbh+/TouXLhgNH9QUJDR/c7NmTNncPbsWfTo0cMw1qNHD8THx2PHjh2FzgwAgYGBRoeyZz0+//rXv4yeM3k9bsnJyYiPj8fbb78NIQROnToF4H9/h0FBQbC3tzfM36ZNG/j6+hplWb9+Pezt7dGmTRvEx8cbLv7+/rCxscG+ffvydZ+GDx9u+LdGo8Hw4cORnp6O3bt3G833r3/9Cy4uLobrjx49wt69e9GtWzc8ffrUsP6HDx+iXbt2iImJwZ07dwA8f7wbNWqEBg0aGG7v4uKCXr165Zlv48aNcHZ2xogRI7JNK8zh8cXxmuXg4IDz588jJiamwHleFwvsBXZ2dgCAp0+f5mv+GzduwMTEBFWqVDEad3d3h4ODA27cuGE0XrFixWzLcHR0xOPHjwuZOLuPPvoITZo0wcCBA+Hm5obu3bvj559/zrXMsnL6+Phkm1a9enXEx8cjOTnZaPzl++Lo6AgA+bov5cuXz/bks7e3R4UKFbKNvbzMZ8+e4YsvvkCFChWg1Wrh7OwMFxcXJCQkIDExMc91Z6lcuXK+5wWApUuXIiUlBTExMVixYkW+CyUnN27ceOW2zppe2Kw//PADrK2t4enpidjYWMTGxsLS0hKVKlXCmjVrcr1tYmIi7t27Z7g8evTIaPrLj3nW45Ofx+3mzZuGN0g2NjZwcXFBQECAYb3A/+531apVs2V7eXvFxMQgMTERrq6ucHFxMbokJSUhLi4u1/sKACYmJvD09DQa8/b2BoBsp1+8/BjExsZCCIHPP/882/onT54MAIYMN27cyNd9ysmVK1fg4+NTZAdiFMdr1tSpU5GQkABvb2/UqlUL48aNw5kzZ4okb15K8OEpBWdnZ4eyZcvi3LlzBbpdft8JmZqa5jguhCj0OnQ6ndF1KysrHDx4EPv27cPWrVuxfft2rFu3Dq1atcLOnTtfmaGgXue+vOq2+VnmiBEjsHz5cowePRqNGzc2nOzdvXv3fO9xAihwAe3fvx9paWkAgLNnz6Jx48YFuv3ryG9WIQR+/PFHJCcnZ9tjAZ6/oCYlJb3y3K9Ro0Zh5cqVhusBAQFG5xYV9nHT6XRo06YNHj16hPHjx6NatWqwtrbGnTt3EBwcXKDHLYter4erq+srS/nFvaWi8PJjkJU5NDT0lXvyL5eEmhTla1bz5s1x5coV/Prrr9i5cye+//57REZGYuHChRg4cGCR5H0VFthL3n33XSxevBhHjhzJ80XKw8MDer0eMTExhnfPwPNfX0hISICHh0eR5XJ0dERCQkK28ZffMQHP31kGBgYiMDAQs2bNwvTp0zFp0iTs27cPrVu3zvF+AMClS5eyTbt48SKcnZ1hbW39+neiCGzYsAFBQUH49ttvDWOpqanZtk1R/vrA3bt3MWLECLRt2xYWFhaGF63CPr4eHh6v3NZZ0wvjwIEDuH37NqZOnWr09wg83xsaNGgQNm/ejN69e+d4+08//dRoWtZe9es6e/YsLl++jJUrV6Jv376G8V27dhnNl3W/c/oo6uXt5eXlhd27d6NJkyaF3hvW6/W4evWqYa8LAC5fvgwAef7qR9aem7m5eY7PqRd5eHjk6z7lxMvLC3/++ScyMjJgbm6e4zwF+VsvrtcsJycn9OvXD/369UNSUhKaN2+O8PDwYi8wfoT4kk8//RTW1tYYOHAg7t+/n236lStXMGfOHABAhw4dACDbzwPNmjULANCxY8ciy+Xl5YXExESjXfO7d+9i06ZNRvO9/LEPANSpUwcADHsQLytTpgzq1KmDlStXGhXBuXPnsHPnTsP9VANTU9Nse3lz587NtieaVbg5lX5BhYSEQK/XY+nSpVi8eDHMzMwwYMCAfO1t5qRDhw44duwYjhw5YhhLTk7G4sWLUalSpRz3nvIj6+PDcePGoWvXrkaXkJAQVK1aNdePEX19fdG6dWvDxd/fv1A5Xpb1Lv7F7SWEMDyPsrz4d/jix8G7du3K9r1gt27doNPp8OWXX2ZbX2ZmZr4f93nz5hllmjdvHszNzREYGJjr7VxdXdGiRQssWrQId+/ezTY963xO4PnjffToURw7dsxoel4f6QLPv3uLj483yvliXgAoVaoUgPz9rRfHa9bDhw+NrtvY2KBKlSqvfL0pStwDe4mXlxfWrl2Ljz76CNWrV0ffvn1Rs2ZNpKen448//sD69esNv5Xn5+eHoKAgLF68GAkJCQgICMCxY8ewcuVKvP/++2jZsmWR5erevTvGjx+PLl26YOTIkUhJScGCBQvg7e1tdPDC1KlTcfDgQXTs2BEeHh6Ii4vDd999h/LlyxsdNPCyb775Bu+88w4aN26MAQMG4NmzZ5g7dy7s7e0RHh5eZPfjdb377rtYvXo17O3t4evriyNHjmD37t0oXbq00Xx16tSBqakpvv76ayQmJkKr1aJVq1ZwdXUt0PqWL1+OrVu3YsWKFShfvjyA54XZu3dvLFiwAEOHDi3wfZgwYQJ+/PFHvPPOOxg5ciScnJywcuVKXLt2DRs3bszzJOWcpKWlYePGjWjTps0rT1Lv1KkT5syZg7i4uAJvh9dRrVo1eHl5ITQ0FHfu3IGdnR02btyY4/elERER6NixI5o2bYr+/fvj0aNHmDt3LmrUqIGkpCTDfAEBAfj4448RERGB06dPo23btjA3N0dMTAzWr1+POXPmoGvXrrnmsrS0xPbt2xEUFISGDRviv//9L7Zu3YqJEyfm6yPI+fPno2nTpqhVqxZCQkLg6emJ+/fv48iRI7h9+zaioqIAPH9TnPUzYqNGjYK1tTUWL14MDw+PPL8r6tu3L1atWoWxY8fi2LFjaNasGZKTk7F7924MHToUnTt3hpWVFXx9fbFu3Tp4e3vDyckJNWvWRM2aNbMtrzhes3x9fdGiRQv4+/vDyckJf/31FzZs2GB0gEyxUeLQRxlcvnxZhISEiEqVKgkLCwtha2srmjRpIubOnWt02GxGRoaYMmWKqFy5sjA3NxcVKlTI9aTAl7182OyrDqMX4vnhxDVr1hQWFhbCx8dH/PDDD9kOk92zZ4/o3LmzKFu2rLCwsBBly5YVPXr0EJcvX862jpdPZN69e7do0qSJsLKyEnZ2duK999575YnMLx+mn9PP4uQk60Tml71q++Clw9AfP34s+vXrJ5ydnYWNjY1o166duHjxYo6Hvy9ZskR4enoKU1PTHE9kzsmLy7l165awt7cX7733Xrb5unTpIqytrcXVq1dzvb8v58+SdSKzg4ODsLS0FA0aNHjlicxZJ3nnZuPGjQKAWLp06Svn2b9/vwAg5syZI4Qo3InML3rV32pOuS9cuCBat24tbGxshLOzswgJCRFRUVE5/h1u3LhRVK9eXWi1WuHr6/vKE5mFeP6TWf7+/sLKykrY2tqKWrVqiU8//VT8/fffud6fnE5kdnNzE5MnTzacxpDbfcxy5coV0bdvX+Hu7i7Mzc1FuXLlxLvvvis2bNhgNN+ZM2dEQEBAoU5kTklJEZMmTTK8xri7u4uuXbuKK1euGOb5448/hL+/v7CwsMjXicxF+Zr11VdfiQYNGggHBwdhZWUlqlWrJqZNm1akP8n2KhohCvk5CBGRpIKDg7FhwwajvTqSD78DIyIiKbHAiIhISiwwIiKSEr8DIyIiKXEPjIiIpMQCIyIiKbHAiIhISiXylzj6V8r9DHx6bt6Q/P23Mf90TSIvKx1BCnUtyygdQQoNdDn/UgoZG3zrhzzn4R4YERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWGBERSYkFRkREUmKBERGRlFhgREQkJRYYERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWGBERSYkFRkREUmKBERGRlFhgREQkJRYYERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFIyUzpASdZhaBf4t2uIMl7lkJ6ajtiTl7Bhxg+4d/VvpaOpjsbGAeYtusHUsxZgZgGREIf0bUuhv3dd6WiqUa+RH/oO7Qnf2tXg4u6MMcETsH/7IaVjqQ6fd/nj2ycQNfoEwra8CwDg0eXbODF7E27tP6NwsvxjgRUjn4a+2Lt6O65FxcLUzBQfjOuJsas+x2dtRiP9WZrS8dRDWwra3pOgvxmNtPWzIFKeQuPoBpGarHQyVbEqZYXL52Px649bMWt5hNJxVIvPu/xJvvsIf0asQ+K1e4BGA58Pm6H90rHY8M4kPL58R+l4+cICK0aRQdOMri8LnY85J5ehUi1PXD4WrVAq9TFv1AHiySOkb1tmGBOJ8QomUqff9x7F73uPKh1D9fi8y58bu08ZXT82cz18+wTCrW4VFhhlZ2VbCgCQnJCkcBJ1Ma1SB7pr52DReShMK/hAJD1Gxqm90EUdVDoalQB83uVNY6KB57sNYW6lxf2TMUrHyTdFCyw+Ph7Lli3DkSNHcO/ePQCAu7s73n77bQQHB8PFxUXJeEVKo9Ggxxf9EHM8Gncu31I6jqpoHFxhVrcVMo/vQOqRLTApUxkWgb2QrtNBd+53peORxPi8y51TtfLosjkcplpzZCSnYkfIbDyOkee7QsUK7Pjx42jXrh1KlSqF1q1bw9vbGwBw//59/Oc//8GMGTOwY8cO1K9fP9flpKWlIS3N+HNtndDBVGNabNkLo/eXA1HOpwIiun6mdBT10Wigv3cdGQc3AgB0cTeR6VwOZnVasMDotfB5l7uEK3exvv0kWNhawbNDA7SM/Bi/ffiVNCWmWIGNGDECH374IRYuXAiNRmM0TQiBwYMHY8SIEThy5Eiuy4mIiMCUKVOMxurYV0ddB98iz1xYvaYMgF8rf8zo9gUe33ukdBzVEUkJEPHGTxjx8C40Prm/eSHKDZ93edNn6PDk+n0AQPzZ63D180St/u1xMGxZHrdUB8XOA4uKisKYMWOylRfwfLd/zJgxOH36dJ7LCQsLQ2JiotGltr1PMSQunF5TBqBeuwaY2TMc8bfjlI6jSvo7sdA4uRuNaZzcIJ48VCgRyY7Pu8LRmGhgqpXn0AjFCszd3R3Hjh175fRjx47Bzc0tz+VotVrY2dkZXdTy8WHvLweicZfmWDRqDlKTU2Hn4gA7FweYay2UjqYqmcd3wqSsJ8wadYTGwRWm1RvBzK8FMk/uUTqaqliVsoJ3jarwrlEVAFCuYll416gK93J5P0/+Sfi8y58G47uhTEMf2JZ3hlO18mgwvhvKNq6OmE1/KB0t3xSr2tDQUAwaNAgnTpxAYGCgoazu37+PPXv2YMmSJfj3v/+tVLwi0apPewDAhHVTjcaXhs7D7xv2K5BInfT3riFt0zxYBHSFeZPOEIkPkL53LXQXeMj4i3zrVMP3v8wzXA+dOhIA8Nu6bZg8atqrbvaPw+dd/lg526FV5GCUcnVA+tMUPIy+ha29Z+L2oXNKR8s3jRBCKLXydevWITIyEidOnIBOpwMAmJqawt/fH2PHjkW3bt0Ktdz+lboWZcwSa94QW6UjSKFJ5GWlI0ihrmUZpSNIoYHOUukIUhh864c851H0w86PPvoIH330ETIyMhAf//zEVWdnZ5ibmysZi4iIJKCKb+vMzc1RpgzfvRERUf7x1+iJiEhKLDAiIpISC4yIiKTEAiMiIimxwIiISEosMCIikhILjIiIpMQCIyIiKbHAiIhISiwwIiKSEguMiIikxAIjIiIpscCIiEhKLDAiIpISC4yIiKTEAiMiIimxwIiISEosMCIikhILjIiIpMQCIyIiKbHAiIhISiwwIiKSEguMiIikxAIjIiIpscCIiEhKLDAiIpISC4yIiKTEAiMiIimxwIiISEosMCIikpKZ0gGKwze1HigdQQrmA2YqHUEOkX2UTiCFU6l3lY4ghXljvJWOUGJwD4yIiKTEAiMiIimxwIiISEosMCIikhILjIiIpMQCIyIiKbHAiIhISiwwIiKSEguMiIikxAIjIiIpscCIiEhKLDAiIpISC4yIiKTEAiMiIimxwIiISEosMCIikhILjIiIpMQCIyIiKbHAiIhISiwwIiKSEguMiIikxAIjIiIpscCIiEhKLDAiIpISC4yIiKTEAiMiIimxwIiISEosMCIikhILjIiIpMQCIyIiKZkpHaAks+oRjFI9+xmN6W7fQMKQvgolUiedTofvlq7Blp17Ef/wMVycnfB+hzb4OLgHNBqN0vFUo14jP/Qd2hO+tavBxd0ZY4InYP/2Q0rHUh1up/zR2DjAvEU3mHrWAswsIBLikL5tKfT3risdLd9YYMUs88ZVPPnsk/8N6HXKhVGppT+sx7rNWzHts09QpbIHzl+8jM+mRcLGxhq9P+ysdDzVsCplhcvnY/Hrj1sxa3mE0nFUi9spH7SloO09Cfqb0UhbPwsi5Sk0jm4QqclKJysQFlhx0+kgEh4pnULVTp+LRstmjRDwdgMAQLkybti26wDOXrikcDJ1+X3vUfy+96jSMVSP2ylv5o06QDx5hPRtywxjIjFewUSFwwIrZqZly8NxxUaIjHRkXjyPlFWLoX8Qp3QsValTszo2/PZfXL95G5UqlsfFmKs4eeY8Ph0RonQ0ohLJtEod6K6dg0XnoTCt4AOR9BgZp/ZCF3VQ6WgFwgIrRpmXo5E0ewZ0d27CxLE0rHoEw27GXCQMDwaePVM6nmoM7NMNySkpeK/nIJiamECn12PkoCC8266V0tGISiSNgyvM6rZC5vEdSD2yBSZlKsMisBfSdTrozv2udLx8U3WB3bp1C5MnT8ayZcteOU9aWhrS0tKMx3R6aE2VP8Ay48Sfhn/rrl9F5uVoOCxdB23TlkjbtU3BZOqyfe9BbNm5D1+Hf4oqlT1wMeYqvp6zCK7OTujcoY3S8YhKHo0G+nvXkXFwIwBAF3cTmc7lYFanhVQFpvyrfC4ePXqElStX5jpPREQE7O3tjS6zY2++oYQFI5KToP/7NkzLlFM6iqp8O38pBvbuhg6tW8DbqzI6tQ9E34+64PvVPysdjahEEkkJEPF/G489vAuNXWmFEhWOontgv/32W67Tr169mucywsLCMHbsWKOxpO4dXytXsbG0gol7Wegf86COF6WmpkFjYny4vImJCfRCKJSIqGTT34mFxsndaEzj5Abx5KFCiQpH0QJ7//33odFoIHJ5ocrrPCCtVgutVms0lqGCjw8BoFT/IUg/9gf0cfdh4lQaVj37A3o90g7sVjqaqrRo0hBLVv6EMm6uqFLZA9GXY7Fq3S/o0rGt0tFUxaqUFSpULm+4Xq5iWXjXqIonCU9w7859BZOpC7dT3jKP74S290SYNeoI3cXjMCnjCTO/FkjfsULpaAWiEbm1RzErV64cvvvuO3TunPO5PqdPn4a/vz90uoKdO/XwvYCiiPfabMZ9AfMaftDY2UGfmIDMC2eRsvp76O/9nfeN3wC75cuVjgAASE5Owdwlq7Dn4BE8epwAF2cndGjTAkP69YS5ubnS8dCgZh+lIwAA/N+ui+9/mZdt/Ld12zB51DQFEqmT2rfT72O8lY4AADDx8oNFQNfn538lPkDG8R2qOgqx1Pi8X58ULbBOnTqhTp06mDp1ao7To6KiULduXej1+gItVy0FpnZqKTC1U0uBUcmglgJTu/wUmKIfIY4bNw7Jya8+87tKlSrYt2/fG0xERESyULTAmjVrlut0a2trBARwb4qIiLJTx9EOREREBcQCIyIiKbHAiIhISiwwIiKSEguMiIikxAIjIiIpscCIiEhKLDAiIpISC4yIiKTEAiMiIimxwIiISEosMCIikhILjIiIpMQCIyIiKbHAiIhISiwwIiKSEguMiIikxAIjIiIpscCIiEhKLDAiIpISC4yIiKTEAiMiIimxwIiISEosMCIikhILjIiIpMQCIyIiKbHAiIhISiwwIiKSEguMiIikxAIjIiIpaYQQQukQRa1/pa5KR5DCqr+PKB1BCs/+PqR0BCksrfuF0hGoBBl864c85+EeGBERSYkFRkREUmKBERGRlFhgREQkJRYYERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWGBERSYkFRkREUmKBERGRlFhgREQkJRYYERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZQKVWCHDh1C79690bhxY9y5cwcAsHr1ahw+fLhIwxEREb1KgQts48aNaNeuHaysrHDq1CmkpaUBABITEzF9+vQiD0hERJSTAhfYV199hYULF2LJkiUwNzc3jDdp0gQnT54s0nBERESvUuACu3TpEpo3b55t3N7eHgkJCUWRiYiIKE8FLjB3d3fExsZmGz98+DA8PT2LJBQREVFeClxgISEhGDVqFP78809oNBr8/fffWLNmDUJDQzFkyJDiyEhERJSNWUFvMGHCBOj1egQGBiIlJQXNmzeHVqtFaGgoRowYURwZiYiIsilwgWk0GkyaNAnjxo1DbGwskpKS4OvrCxsbm+LIJ7UOQ7vAv11DlPEqh/TUdMSevIQNM37Avat/Kx1NlYYMDsInY4fA3d0FZ85cwKjRn+P4X6eVjqUaOp0O3y1dgy079yL+4WO4ODvh/Q5t8HFwD2g0GqXjqYZvn0DU6BMI2/IuAIBHl2/jxOxNuLX/jMLJ1KUkbKcCF1gWCwsL+Pr6FmWWEsenoS/2rt6Oa1GxMDUzxQfjemLsqs/xWZvRSH+WpnQ8Vfnww0749zeTMXTYBBw7fgojRwzEtq1r4FuzOR48eKh0PFVY+sN6rNu8FdM++wRVKnvg/MXL+GxaJGxsrNH7w85Kx1ON5LuP8GfEOiReuwdoNPD5sBnaLx2LDe9MwuPLd5SOpxolYTsVuMBatmyZ67u9vXv3vlagkiQyaJrR9WWh8zHn5DJUquWJy8eiFUqlTmNGheD7pWuxctXPAIChwyagwzuB6BfcHTO/ma9wOnU4fS4aLZs1QsDbDQAA5cq4YduuAzh74ZLCydTlxu5TRtePzVwP3z6BcKtbRZoX5jehJGynAh/EUadOHfj5+Rkuvr6+SE9Px8mTJ1GrVq3iyFhiWNmWAgAkJyQpnERdzM3NUa9ebezZe8gwJoTAnr2H0aiRv4LJ1KVOzer486/TuH7zNgDgYsxVnDxzHs0a1Vc4mXppTDTw6tQI5lZa3D8Zo3Qc1ZJ1OxV4DywyMjLH8fDwcCQl8YX5VTQaDXp80Q8xx6Nx5/ItpeOoirOzE8zMzBB3P95oPC7uAar5eCmUSn0G9umG5JQUvNdzEExNTKDT6zFyUBDebddK6Wiq41StPLpsDoep1hwZyanYETIbj2P43fPLZN9ORfZjvr1798ayZcsKfLtnz57h8OHDuHDhQrZpqampWLVqVa63T0tLw5MnT4wuOqErcI7i1vvLgSjnUwELR+T8BoAoL9v3HsSWnfvwdfin+Hn5XEz77BOs+HEjft22S+loqpNw5S7Wt5+EXzpNxvnVe9Ay8mM4Vi2rdCzVkX07FVmBHTlyBJaWlgW6zeXLl1G9enU0b94ctWrVQkBAAO7evWuYnpiYiH79+uW6jIiICNjb2xtdziSq6zuBXlMGwK+VP2Z2D8fje4+UjqM68fGPkJmZCVc3Z6NxV1cX3Lv/QKFU6vPt/KUY2LsbOrRuAW+vyujUPhB9P+qC71f/rHQ01dFn6PDk+n3En72OY1//jIcXbqJW//ZKx1Id2bdTgQvsgw8+MLp06dIFjRo1Qr9+/fDxxx8XaFnjx49HzZo1ERcXh0uXLsHW1hZNmjTBzZs3872MsLAwJCYmGl1q2/sU9G4Vm15TBqBeuwaY2TMc8bfjlI6jShkZGTh58gxatWxqGNNoNGjVsimOHj2hYDJ1SU1Ng8bE+AAqExMT6IVQKJE8NCYamGoLfdD1P4Zs26nASe3t7Y2um5iYwMfHB1OnTkXbtm0LtKw//vgDu3fvhrOzM5ydnfF///d/GDp0KJo1a4Z9+/bB2to6z2VotVpotVqjMVONaYFyFJfeXw5Eo87N8J+Qr5GanAo7FwcAwLMnKchIS1c2nMpEzlmC5UsjceLkGRw/fgojR4TA2toKK1auUzqaarRo0hBLVv6EMm6uqFLZA9GXY7Fq3S/o0rFgz7uSrsH4bri1PwpJdx7C3MYSVTq/jbKNq2Nr75lKR1OVkrCdClRgOp0O/fr1Q61ateDo6PjaK3/27BnMzP4XQaPRYMGCBRg+fDgCAgKwdu3a116Hklr1eb4rPmHdVKPxpaHz8PuG/QokUq/163+Di7MTwr8Ihbu7C6KizqPju70RFxef943/ISaOGYK5S1bhq3/Px6PHCXBxdsKHnTtgSL+eSkdTFStnO7SKHIxSrg5If5qCh9G3sLX3TNw+dE7paKpSEraTRoiCff5gaWmJ6OhoVK5c+bVX3qBBA4wYMQJ9+vTJNm348OFYs2bN84MydAU7KKN/pa6vne2fYNXfR5SOIIVnfx/KeybC0rpfKB2BSpDBt37Ic54CfwdWs2ZNXL16tVCBXtalSxf8+OOPOU6bN28eevTogQL2KxER/UMU6j+0DA0NxZYtW3D37t1sh7AXRFhYGLZt2/bK6d999x30en1BIxIR0T9Avr8Dmzp1Kj755BN06NABANCpUyejn5QSQkCj0RT44z4iIqLCyHeBTZkyBYMHD8a+ffuKMw8REVG+5LvAsr6LCggIKLYwRERE+VWg78D4fw4REZFaFOg8MG9v7zxL7NEj/lQSEREVvwIV2JQpU7L9EgcREZESClRg3bt3h6ura3FlISIiyrd8fwfG77+IiEhN8l1g/EUMIiJSk3x/hMhfxCAiIjUpsv/QkoiI6E1igRERkZRYYEREJCUWGBERSYkFRkREUmKBERGRlFhgREQkJRYYERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWGBERSYkFRkREUmKBERGRlFhgREQkJRYYERFJyUzpAMWhgc5S6QhSWKV0AEk0qNlH6QhSWGLmonQEKYRkPlA6ghQG52Me7oEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWGBERSYkFRkREUmKBERGRlFhgREQkJRYYERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWGBERSYkFRkREUmKBERGRlFhgREQkJRYYERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWWDHy7ROID3dOR/8LS9D/whK8v3kyKrSorXQs1RoyOAixl48i6ckV/HH4//BW/TpKR1KVeo38MHvV19h5+lecuvc7WrRvpnQk1XMf9gHq396MCuEDlI6iOiXh74kFVoyS7z7CnxHrsLHDZ9jY8XP8/ccFtF86Fo7e5ZSOpjofftgJ//5mMr78ahbeatgeUWcuYNvWNXBxKa10NNWwKmWFy+djERH2rdJRpFDKrwpcerVDyoVrSkdRpZLw92SmdICS7MbuU0bXj81cD98+gXCrWwWPL99RKJU6jRkVgu+XrsXKVT8DAIYOm4AO7wSiX3B3zPxmvsLp1OH3vUfx+96jSseQgkkpS3jOHYPrn85H2VHdlI6jSiXh74l7YG+IxkQDr06NYG6lxf2TMUrHURVzc3PUq1cbe/YeMowJIbBn72E0auSvYDKSVcVpg5C45wSeHj6jdBQqRorvgUVHR+Po0aNo3LgxqlWrhosXL2LOnDlIS0tD79690apVK6UjvhanauXRZXM4TLXmyEhOxY6Q2Xgc87fSsVTF2dkJZmZmiLsfbzQeF/cA1Xy8FEpFsnLs1BSlankhumOo0lGomClaYNu3b0fnzp1hY2ODlJQUbNq0CX379oWfnx/0ej3atm2LnTt35lpiaWlpSEtLMxrLEDqYa0yLO36+JFy5i/XtJ8HC1gqeHRqgZeTH+O3Dr1hiRMXAvIwzKk4ZiMs9J0OkZSgdh4qZoh8hTp06FePGjcPDhw+xfPly9OzZEyEhIdi1axf27NmDcePGYcaMGbkuIyIiAvb29kaXHU/Ov6F7kDd9hg5Prt9H/NnrOPb1z3h44SZq9W+vdCxViY9/hMzMTLi6ORuNu7q64N79BwqlIhlZ1/aCuYsDfP87C/7XN8L/+kbYNq4J1/4d4X99I2DCb01KEkUfzfPnzyM4OBgA0K1bNzx9+hRdu3Y1TO/VqxfOnMn9M+ywsDAkJiYaXdrZ1SjO2K9FY6KBqVbxT25VJSMjAydPnkGrlk0NYxqNBq1aNsXRoycUTEayeXI4CucCR+J8uzGGS/LpGDzadBDn240B9HqlI1IRUvyVVKPRAABMTExgaWkJe3t7wzRbW1skJibmenutVgutVms0ppaPDxuM74Zb+6OQdOchzG0sUaXz2yjbuDq29p6pdDTViZyzBMuXRuLEyTM4fvwURo4IgbW1FVasXKd0NNWwKmWFCpXLG66Xq1gW3jWq4knCE9y7c1/BZOqhT05F6qWbxmPP0pD5+Gm28X+6kvD3pGiBVapUCTExMfDyev5F/ZEjR1CxYkXD9Js3b6JMmTJKxXttVs52aBU5GKVcHZD+NAUPo29ha++ZuH3onNLRVGf9+t/g4uyE8C9C4e7ugqio8+j4bm/ExcXnfeN/CN861fD9L/MM10OnjgQA/LZuGyaPmqZULJJUSfh70gghhFIrX7hwISpUqICOHTvmOH3ixImIi4vD999/X7DlVuhdFPFKvOH39ykdQQq1nCopHUEKS8xclI4ghZBMfq+bH6fu/Z7nPIrugQ0ePDjX6dOnT39DSYiISDY8JIeIiKTEAiMiIimxwIiISEosMCIikhILjIiIpMQCIyIiKbHAiIhISiwwIiKSEguMiIikxAIjIiIpscCIiEhKLDAiIpISC4yIiKTEAiMiIimxwIiISEosMCIikhILjIiIpMQCIyIiKbHAiIhISiwwIiKSEguMiIikxAIjIiIpscCIiEhKLDAiIpISC4yIiKTEAiMiIimxwIiISEosMCIikhILjIiIpMQCIyIiKbHAiIhISmZKBygOx0xTlY4ghb5lGysdQQqr/j6idAQpNMJ1pSNI4ajrW0pHKDG4B0ZERFJigRERkZRYYEREJCUWGBERSYkFRkREUmKBERGRlFhgREQkJRYYERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWGBERSYkFRkREUmKBERGRlFhgREQkJRYYERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWGBERSYkFRkREUmKBERGRlFhgxajD0C74/NcZ+O7casz+aymGL/4U7p5llY6lOtxOBTNkcBBiLx9F0pMr+OPw/+Gt+nWUjqQ63EYF4z7sA9S/vRkVwgcoHaVAWGDFyKehL/au3o6vuoTh2z5TYWpmirGrPoeFlVbpaKrC7ZR/H37YCf/+ZjK+/GoW3mrYHlFnLmDb1jVwcSmtdDTV4DYqmFJ+VeDSqx1SLlxTOkqBaYQQQukQLxJCQKPRvNYy+lfqWkRpipatkx3mnFyGGd0+x+Vj0UrHUS21badVfx9ROoLBH4f/D8f/isKo0Z8BADQaDa5fPY753y3HzG/mK5xOHdS+jY66vqV0BAOTUpbw3f4tbkxchLKjuiHl/DXcCl+qdCwAQP3bm/OcR3V7YFqtFtHRyr9oFQcr21IAgOSEJIWTqBu3U87Mzc1Rr15t7Nl7yDAmhMCevYfRqJG/gsnUg9uoYCpOG4TEPSfw9PAZpaMUiplSKx47dmyO4zqdDjNmzEDp0s9392fNmvUmYxUbjUaDHl/0Q8zxaNy5fEvpOKrF7fRqzs5OMDMzQ9z9eKPxuLgHqObjpVAqdeE2yj/HTk1RqpYXojuGKh2l0BQrsNmzZ8PPzw8ODg5G40IIREdHw9raOl8fJaalpSEtLc1oTCd0MNWYFmXc19b7y4Eo51MBEV0/UzqKqnE7ERU/8zLOqDhlIC73nAyRlqF0nEJTrMCmT5+OxYsX49tvv0WrVq0M4+bm5lixYgV8fX3ztZyIiAhMmTLFaKyOfXXUdcjf7d+EXlMGwK+VP2Z0+wKP7z1SOo5qcTvlLj7+ETIzM+Hq5mw07urqgnv3HyiUSl24jfLHurYXzF0c4Pvf/33CpTEzhU1DX7gGd8AJzw8BvV7BhPmj2HdgEyZMwLp16zBkyBCEhoYiI6Nw7wLCwsKQmJhodKlt71PEaQuv15QBqNeuAWb2DEf87Til46gWt1PeMjIycPLkGbRq2dQwptFo0KplUxw9ekLBZOrBbZQ/Tw5H4VzgSJxvN8ZwST4dg0ebDuJ8uzFSlBeg4B4YALz11ls4ceIEhg0bhvr162PNmjUFPgJRq9VCqzU+3FotHx/2/nIgGnVuhv+EfI3U5FTYuTgAAJ49SUFGWrqy4VSE2yn/IucswfKlkThx8gyOHz+FkSNCYG1thRUr1ykdTTW4jfKmT05F6qWbxmPP0pD5+Gm2cTVTtMAAwMbGBitXrsRPP/2E1q1bQ6fTKR2pyLTq0x4AMGHdVKPxpaHz8PuG/QokUidup/xbv/43uDg7IfyLULi7uyAq6jw6vtsbcXHxed/4H4Lb6J9DVeeB3b59GydOnEDr1q1hbW1d6OWo9TwwkpOazgMj+anpPDA1y895YIrvgb2ofPnyKF++vNIxiIhIAqo7kZmIiCg/WGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWGBERSYkFRkREUmKBERGRlFhgREQkJRYYERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWGBERSYkFRkREUmKBERGRlFhgREQkJRYYERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEREJCUWGBERyUlQsUtNTRWTJ08WqampSkdRNW6n/OF2yh9up/yReTtphBBC6RIt6Z48eQJ7e3skJibCzs5O6Tiqxe2UP9xO+cPtlD8ybyd+hEhERFJigRERkZRYYEREJCUW2Bug1WoxefJkaLVapaOoGrdT/nA75Q+3U/7IvJ14EAcREUmJe2BERCQlFhgREUmJBUZERFJigRERkZRYYG/A/PnzUalSJVhaWqJhw4Y4duyY0pFU5eDBg3jvvfdQtmxZaDQabN68WelIqhQREYG33noLtra2cHV1xfvvv49Lly4pHUt1FixYgNq1a8POzg52dnZo3Lgx/vvf/yodS9VmzJgBjUaD0aNHKx2lQFhgxWzdunUYO3YsJk+ejJMnT8LPzw/t2rVDXFyc0tFUIzk5GX5+fpg/f77SUVTtwIEDGDZsGI4ePYpdu3YhIyMDbdu2RXJystLRVKV8+fKYMWMGTpw4gb/++gutWrVC586dcf78eaWjqdLx48exaNEi1K5dW+koBafsTzGWfA0aNBDDhg0zXNfpdKJs2bIiIiJCwVTqBUBs2rRJ6RhSiIuLEwDEgQMHlI6ieo6OjuL7779XOobqPH36VFStWlXs2rVLBAQEiFGjRikdqUC4B1aM0tPTceLECbRu3dowZmJigtatW+PIkSMKJqOSIDExEQDg5OSkcBL10ul0+Omnn5CcnIzGjRsrHUd1hg0bho4dOxq9RsnETOkAJVl8fDx0Oh3c3NyMxt3c3HDx4kWFUlFJoNfrMXr0aDRp0gQ1a9ZUOo7qnD17Fo0bN0ZqaipsbGywadMm+Pr6Kh1LVX766SecPHkSx48fVzpKobHAiCQ0bNgwnDt3DocPH1Y6iir5+Pjg9OnTSExMxIYNGxAUFIQDBw6wxP6/W7duYdSoUdi1axcsLS2VjlNoLLBi5OzsDFNTU9y/f99o/P79+3B3d1coFclu+PDh2LJlCw4ePIjy5csrHUeVLCwsUKVKFQCAv78/jh8/jjlz5mDRokUKJ1OHEydOIC4uDvXq1TOM6XQ6HDx4EPPmzUNaWhpMTU0VTJg//A6sGFlYWMDf3x979uwxjOn1euzZs4efx1OBCSEwfPhwbNq0CXv37kXlypWVjiQNvV6PtLQ0pWOoRmBgIM6ePYvTp08bLvXr10evXr1w+vRpKcoL4B5YsRs7diyCgoJQv359NGjQALNnz0ZycjL69eundDTVSEpKQmxsrOH6tWvXcPr0aTg5OaFixYoKJlOXYcOGYe3atfj1119ha2uLe/fuAQDs7e1hZWWlcDr1CAsLwzvvvIOKFSvi6dOnWLt2Lfbv348dO3YoHU01bG1ts313am1tjdKlS0v1nSoLrJh99NFHePDgAb744gvcu3cPderUwfbt27Md2PFP9tdff6Fly5aG62PHjgUABAUFYcWKFQqlUp8FCxYAAFq0aGE0vnz5cgQHB7/5QCoVFxeHvn374u7du7C3t0ft2rWxY8cOtGnTRuloVMT436kQEZGU+B0YERFJiQVGRERSYoEREZGUWGBERCQlFhgREUmJBUZERFJigRERkZRYYEQqFxwcjPfff99wvUWLFor8z7n79++HRqNBQkLCG183UU5YYESFFBwcDI1GA41GY/jx2KlTpyIzM7NY1/vLL7/gyy+/zNe8LB0qyfhTUkSvoX379li+fDnS0tKwbds2DBs2DObm5ggLCzOaLz09HRYWFkWyTv4HlkTPcQ+M6DVotVq4u7vDw8MDQ4YMQevWrfHbb78ZPvabNm0aypYtCx8fHwDP/x+mbt26wcHBAU5OTujcuTOuX79uWJ5Op8PYsWPh4OCA0qVL49NPP8XLv/b28keIaWlpGD9+PCpUqACtVosqVapg6dKluH79uuE3Jh0dHaHRaAy/majX6xEREYHKlSvDysoKfn5+2LBhg9F6tm3bBm9vb1hZWaFly5ZGOYnUgAVGVISsrKyQnp4OANizZw8uXbqEXbt2YcuWLcjIyEC7du1ga2uLQ4cO4ffff4eNjQ3at29vuM23336LFStWYNmyZTh8+DAePXqETZs25brOvn374scff8R//vMfREdHY9GiRbCxsUGFChWwceNGAMClS5dw9+5dzJkzBwAQERGBVatWYeHChTh//jzGjBmD3r1748CBAwCeF+0HH3yA9957D6dPn8bAgQMxYcKE4tpsRIUjiKhQgoKCROfOnYUQQuj1erFr1y6h1WpFaGioCAoKEm5ubiItLc0w/+rVq4WPj4/Q6/WGsbS0NGFlZSV27NghhBCiTJkyYubMmYbpGRkZonz58ob1CCFEQECAGDVqlBBCiEuXLgkAYteuXTlm3LdvnwAgHj9+bBhLTU0VpUqVEn/88YfRvAMGDBA9evQQQggRFhYmfH19jaaPHz8+27KIlMTvwIhew5YtW2BjY4OMjAzo9Xr07NkT4eHhGDZsGGrVqmX0vVdUVBRiY2Nha2trtIzU1FRcuXIFiYmJuHv3Lho2bGiYZmZmhvr162f7GDFL1n8+GBAQkO/MsbGxSElJyfbfi6Snp6Nu3boAgOjoaKMcAPifsJLqsMCIXkPLli2xYMECWFhYoGzZsjAz+99Tytra2mjepKQk+Pv7Y82aNdmW4+LiUqj1F+Y/skxKSgIAbN26FeXKlTOaptVqC5WDSAksMKLXYG1tjSpVquRr3nr16mHdunVwdXWFnZ1djvOUKVMGf/75J5o3bw4AyMzMxIkTJ1CvXr0c569Vqxb0ej0OHDiA1q1bZ5uetQeo0+kMY76+vtBqtbh58+Yr99yqV6+O3377zWjs6NGjed9JojeIB3EQvSG9evWCs7MzOnfujEOHDuHatWvYv38/Ro4cidu3bwMARo0ahRkzZmDz5s24ePEihg4dmus5XJUqVUJQUBD69++PzZs3G5b5888/AwA8PDyg0WiwZcsWPHjwAElJSbC1tUVoaCjGjBmDlStX4sqVKzh58iTmzp2LlStXAgAGDx6MmJgYjBs3DpcuXcLatWv5v2OT6rDAiN6QUqVK4eDBg6hYsSI++OADVK9eHQMGDEBqaqphj+yTTz5Bnz59EBQUhMaNG8PW1hZdunTJdbkLFixA165dMXToUFSrVg0hISFITk4GAJQrVw5TpkzBhAkT4ObmhuHDhwMAvvzyS3z++eeIiIhA9erV0b59e2zduhWVK1cGAFSsWBEbN27E5s2b4efnh4ULF2L69OnFuHWICk4jXvXtMBERkYpxD4yIiKTEAiMiIimxwIiISEosMCIikhILjIiIpMQCIyIiKbHAiIhISiwwIiKSEguMiIikxAIjIiIpscCIiEhKLDAiIpLS/wP51p5jYJP94QAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean accuracy = 0.38\n",
      "F1 score for the predictions = [0.14285714 0.43243243 0.59259259 0.42105263 0.32      ]\n"
     ]
    }
   ],
   "source": [
    "# Plot confusion matrix\n",
    "plt.title(\"Confusion matrix for AI-made predictions\")\n",
    "sns.heatmap(confusion_mat, square=True, annot=True, fmt='d', cbar=False)\n",
    "plt.xlabel('Predicted')\n",
    "plt.ylabel('True')\n",
    "plt.show()\n",
    "\n",
    "print(f\"Mean accuracy = {clf.score(X_test, y_test):.2f}\")\n",
    "print(f\"F1 score for the predictions = {metrics.f1_score(y_test, pred, average=None)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model seems to be doing quite good for 3 out of 5 people. It recognizes them roughly half the time. To get a better idea of how the model performs compared to a human, I built a website where the people from the group chat were guessing who the author of a text was. Below, I attached a screenshot with names taken out for privacy.\n",
    "\n",
    "![app.png](app.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbcAAAHHCAYAAAA8tRYqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABCLklEQVR4nO3dd1hT5+M28DsghD1kCIiynIhCnaUOFAdaq6jVOorirnXgHtihWCt2qLVVUKuipbW1rdW22p97j9aqddQ9wFUEF6gIYeR5/+hLvgZQQMATnt6f6+K6zJOTc+6cnJM7JzmJKiGEABERkUSMlA5ARERU1lhuREQkHZYbERFJh+VGRETSYbkREZF0WG5ERCQdlhsREUmH5UZERNJhuRERkXRYbqV08eJFdOjQAba2tlCpVNiwYUOZzj8xMREqlQqrVq0q0/nKwNPTEwMHDiyz+SUnJ6Nnz55wcHCASqXCZ599Vmbzzi/vcf3000/LbRn0bCqVCjNnzlQ6Rplq3bo1WrdurbtcHs8fZb3flRcpyu3y5ct466234O3tDTMzM9jY2KB58+ZYuHAhMjIyynXZ4eHhOHXqFD788EPEx8ejcePG5bo8GZ05cwYzZ85EYmKiojnGjx+PLVu2IDIyEvHx8ejYsaOieYiUcvDgQcycOROpqalKR3lulZQOUFqbNm1Cr169oFarMWDAAPj5+SErKwv79+/H5MmTcfr0aSxbtqxclp2RkYFDhw7hnXfewejRo8tlGR4eHsjIyICJiUm5zN8QnDlzBlFRUWjdujU8PT2Lfbvz58/DyKjsXp/t3LkToaGhmDRpUpnNk0hJz/v8cfDgQURFRWHgwIGws7PTu66s97vyUqHLLSEhAX369IGHhwd27twJV1dX3XWjRo3CpUuXsGnTpnJb/u3btwGgwINfllQqFczMzMpt/hWNEAKZmZkwNzeHWq0u03mnpKSU6WOZmZkJU1PTCvFEQMpKT0+HpaVlmc+3PJ4/ynq/KzeiAhsxYoQAIA4cOFCs6bOzs8WsWbOEt7e3MDU1FR4eHiIyMlJkZmbqTefh4SE6d+4s9u3bJ5o0aSLUarXw8vISq1ev1k0zY8YMAUDvz8PDQwghRHh4uO7fT8q7zZO2bt0qmjdvLmxtbYWlpaWoVauWiIyM1F2fkJAgAIi4uDi92+3YsUO0aNFCWFhYCFtbW9G1a1dx5syZQpd38eJFER4eLmxtbYWNjY0YOHCgSE9PL3J9BQUFiXr16okTJ06IVq1aCXNzc+Hj4yN++OEHIYQQu3fvFk2bNhVmZmaiVq1aYtu2bXq3T0xMFG+//baoVauWMDMzE5UrVxY9e/YUCQkJumni4uIKrEcAYteuXXqPxebNm0WjRo2EWq0WCxYs0F0XHh4uhBBCq9WK1q1bC0dHR5GcnKybv0ajEX5+fsLb21s8evSo0Pv5tAx5Ll++LHr27Cns7e2Fubm5aNasmdi4caPePHbt2iUAiG+//Va88847ws3NTahUKnH//v1Cl5n3uH7yySdi6dKlum2ycePG4vDhwwUeh6CgoALzyL+dPTnPRYsWCS8vL2Fubi7at28vrl27JrRarZg1a5aoWrWqMDMzE127dhV3797Vm+eGDRvEq6++KlxdXYWpqanw9vYWs2bNEjk5OQUy1atXT5w+fVq0bt1amJubCzc3N/HRRx8Ven/zy1vn+/btE2PGjBGOjo7C1tZWDB8+XGg0GnH//n3Rv39/YWdnJ+zs7MTkyZOFVqvVm8cnn3wiAgMDReXKlYWZmZlo2LChbtt8UmZmphg3bpxwdHQUVlZWokuXLuL69esCgJgxY4betDdu3BCDBg0Szs7OwtTUVPj6+ooVK1YU6z4BEKNGjRJff/21qFWrllCr1aJhw4Ziz549etPl7ZenT58Wffv2FXZ2diIgIEB3fXx8vGjYsKEwMzMT9vb2onfv3uLatWsFlpe33ZiZmYkmTZqIvXv3FthWnvb8cfbsWdGrVy/h6Oio23+nT5+uly//X95+++R+l6ck+8jatWvF7NmzRdWqVYVarRbBwcHi4sWLetNeuHBB9OjRQ1SpUkWo1WpRtWpV0bt3b5Gamlqch0IIIUSFLreqVasKb2/vYk8fHh4uAIiePXuKxYsXiwEDBggAolu3bnrTeXh4iNq1a4sqVaqI6dOni0WLFomGDRsKlUol/v77byGEECdOnBALFiwQAETfvn1FfHy8WL9+vW45xSm3v//+W/eEtnDhQrFkyRIxadIk0apVK900hW2c27ZtE5UqVRK1atUSH3/8sYiKihKOjo7C3t5erzjylvfSSy+JHj16iJiYGDF06FABQEyZMqXI9RUUFCTc3NxEtWrVxOTJk8UXX3whfH19hbGxsfjuu++Ei4uLmDlzpvjss89E1apVha2trXjw4IHu9j/88IPw9/cX77//vli2bJmYPn26sLe3Fx4eHrpyvXz5soiIiBAAxPTp00V8fLyIj48Xt27d0j0WNWrUEPb29mLatGliyZIlesX35E525coVYWVlJbp3764bmzZtmlCpVAWeYJ50+fJlER8fLwCI9u3b6zIIIcStW7dElSpVhLW1tXjnnXfE/Pnzhb+/vzAyMhI//fSTbh55O66vr68ICAgQ8+fPF9HR0U99EZH3uL700kuiRo0a4qOPPhIff/yxcHR0FO7u7iIrK0vvcShJuQUEBAhfX18xf/588e677wpTU1Px8ssvi+nTp4tXXnlFfP755yIiIkKoVCoxaNAgvXl269ZNvPHGG+KTTz4RsbGxolevXgKAmDRpkt50T24bY8eOFTExMSI4OFgAEL/99ttT13WevHILCAgQHTt2FIsXLxb9+/fXbZstWrQQ/fr1EzExMeK1114TAPReXAohhLu7uxg5cqRYtGiRmD9/vmjatKkAUOBJNSwsTAAQ/fr1E4sWLRI9evQQDRo0KFBut27dEu7u7qJatWpi1qxZIjY2VnTt2lUA0L2gehYAws/PTzg6OopZs2aJjz76SHh4eAhzc3Nx6tQp3XR5+6Wvr68IDQ0VMTExYvHixUIIIWbPni1UKpXo3bu3iImJ0e3bnp6eei+Uli9fLgDoHs9x48YJOzs74e3tXWS5nThxQtjY2AgHBwcRGRkpli5dKqZMmSLq16+vu75v3766+523P+S9OMy/35V0H3nppZdEo0aNxIIFC8TMmTOFhYWFaNq0qW46jUYjvLy8hJubm5g9e7ZYvny5iIqKEk2aNBGJiYlFPg66x6PYUxqYtLQ0AUCEhoYWa/rjx48LAGLo0KF645MmTRIAxM6dO3VjHh4eAoDYu3evbiwlJUWo1WoxceJE3diTr5SfVNxyyyvH27dvPzV3YRtnQECAcHZ21nvVfeLECWFkZCQGDBhQYHmDBw/Wm2f37t2Fg4PDU5eZJygoSAAQa9as0Y2dO3dOABBGRkbi999/141v2bKlQM7Hjx8XmOehQ4cEAPHVV1/pxn744Qe9o7Un5T0WmzdvLvS6/K8gly5dKgCIr7/+Wvz+++/C2NhYjBs3rsj7KsT/Xnk/ady4cbojjDwPHz4UXl5ewtPTU+Tm5goh/rfjent7F3q/88t7XB0cHMS9e/d04z///LMAIH799VfdWEnLzcnJSe8VbmRkpAAg/P39RXZ2tm68b9++wtTUVO+di8Kyv/XWW8LCwkJvurxt48nHUaPRCBcXF/H6668Xef/zyi0kJETviCwwMFCoVCoxYsQI3VhOTo5wd3cvsA7yZ83KyhJ+fn4iODhYN5a3348cOVJv2n79+hUotyFDhghXV1dx584dvWn79OkjbG1ti3xc845wjhw5ohu7evWqMDMz03vBlbdf9u3bV+/2iYmJwtjYWHz44Yd646dOnRKVKlXSjWdlZQlnZ2cREBAgNBqNbrply5YJAEWWW6tWrYS1tbW4evWq3nKefBw++eQTvaO1J+Xf70q6j9StW1cv98KFCwUA3QuAv/76SwAo9Ci8JCrshwEPHjwAAFhbWxdr+t9++w0AMGHCBL3xiRMnAkCBz+Z8fX3RsmVL3WUnJyfUrl0bV65cee7M+eV9vvPzzz9Dq9UW6zZJSUk4fvw4Bg4ciMqVK+vGGzRogPbt2+vu55NGjBihd7lly5a4e/eubh0+i5WVFfr06aO7XLt2bdjZ2aFu3bpo1qyZbjzv30+uH3Nzc92/s7OzcffuXdSoUQN2dnY4duxYMe7tv7y8vBASElKsaYcPH46QkBCMGTMG/fv3h4+PD+bMmVPsZeX322+/oWnTpmjRooVuzMrKCsOHD0diYiLOnDmjN314eLje/S5K7969YW9vr7uct82VZjvr1asXbG1tdZfzHpuwsDBUqlRJbzwrKws3b97UjT2Z/eHDh7hz5w5atmyJx48f49y5c3rLsbKyQlhYmO6yqakpmjZtWqLsQ4YMgUql0sskhMCQIUN0Y8bGxmjcuHGB+T6Z9f79+0hLS0PLli31tq28/SEiIkLvtuPGjdO7LITAunXr0KVLFwghcOfOHd1fSEgI0tLSirXNBgYGolGjRrrL1atXR2hoKLZs2YLc3Fy9afPvlz/99BO0Wi3eeOMNveW7uLigZs2a2LVrFwDgyJEjSElJwYgRI2Bqaqq7/cCBA/Ue98Lcvn0be/fuxeDBg1G9enW96558HEqipPvIoEGD9HLn3+bz7sOWLVvw+PHj58oEVOCvAtjY2AD4dwcsjqtXr8LIyAg1atTQG3dxcYGdnR2uXr2qN57/gQcAe3t73L9//zkTF9S7d280b94cQ4cORZUqVdCnTx98//33zyy6vJy1a9cucF3dunVx584dpKen643nvy95T6bFuS/u7u4FNnpbW1tUq1atwFj+eWZkZOD9999HtWrVoFar4ejoCCcnJ6SmpiItLa3IZefx8vIq9rQAsGLFCjx+/BgXL17EqlWrSlQ2+V29evWp6zrv+tJkLc1jU9x55j02xXnMTp8+je7du8PW1hY2NjZwcnLSFVj+x6ywbSP/PnLr1i29v/xfzSlJ1vzrZOPGjXj55ZdhZmaGypUrw8nJCbGxsXo58/Z7Hx8fvdvmf0xv376N1NRULFu2DE5OTnp/gwYNAvDvCUdFqVmzZoGxWrVq4fHjx7oT0PLk31YuXrwIIQRq1qxZIMPZs2d1y8/b5vIvy8TEBN7e3s/Ml1cgfn5+Rd6X4irpPlLUNu/l5YUJEyZg+fLlcHR0REhICBYvXlyi5wygAp8taWNjAzc3N/z9998lul1xX50YGxsXOi6EeO5l5H/lZm5ujr1792LXrl3YtGkTNm/ejLVr1yI4OBhbt259aoaSKs19edptizPPMWPGIC4uDuPGjUNgYKDui+59+vQp9pEqgBKX0+7du6HRaAAAp06dQmBgYIluXxolzVqc9ahSqQp9rPJvT0XNs6hlpaamIigoCDY2Npg1axZ8fHxgZmaGY8eOYerUqQUes+Jkf/IMZgCIi4vT+wJwSbI+Od99+/aha9euaNWqFWJiYuDq6goTExPExcVhzZo1hc7zWfLuW1hYGMLDwwudpkGDBiWe77Pk31a0Wi1UKhX+7//+r9D7b2VlVabLV0pxtpt58+Zh4MCB+Pnnn7F161ZEREQgOjoav//+O9zd3Yu1nApbbgDw2muvYdmyZTh06FCRT2AeHh7QarW4ePGi7hUF8O+vUqSmpsLDw6PMctnb2xf65cf8r2AAwMjICG3btkXbtm0xf/58zJkzB++88w527dqFdu3aFXo/gH+/a5LfuXPn4OjoWC6nFD+PH3/8EeHh4Zg3b55uLDMzs8C6ed63QwqTlJSEMWPGoEOHDjA1NcWkSZMQEhLy3I+vh4fHU9d13vXlzd7evtC3+grbnkpj9+7duHv3Ln766Se0atVKN56QkPDc89y2bZve5Xr16j33vJ60bt06mJmZYcuWLXqnpsfFxelNl7ffX758We/oIv9j6uTkBGtra+Tm5ha63xXXxYsXC4xduHABFhYWcHJyeuZtfXx8IISAl5cXatWq9dTp8ra5ixcvIjg4WDeenZ2NhIQE+Pv7P/W2eUd2RR0UlGSfLK99pH79+qhfvz7effddHDx4EM2bN8eSJUswe/bsYt2+wr4tCQBTpkyBpaUlhg4diuTk5ALXX758GQsXLgQAvPrqqwBQ4CeV5s+fDwDo3LlzmeXy8fFBWloaTp48qRtLSkrC+vXr9aa7d+9egdsGBAQAgO7IIz9XV1cEBARg9erVeiXx999/Y+vWrbr7aQiMjY0LHHF88cUXBY448sq4LH4NYdiwYdBqtVixYgWWLVuGSpUqYciQIcU6Si3Mq6++isOHD+PQoUO6sfT0dCxbtgyenp7w9fUtdeai+Pj44Ny5c3pva504cQIHDhwo0+XkvaJ+cl1lZWUhJibmuefZrl07vb/8R3LPy9jYGCqVSm9bSkxMLPDzd506dQIAfP7553rj+Z8HjI2N8frrr2PdunWFPvHnf0vxaQ4dOqT32dz169fx888/o0OHDkW+E9OjRw8YGxsjKiqqwPYqhMDdu3cBAI0bN4aTkxOWLFmCrKws3TSrVq0qch9ycnJCq1atsHLlSly7dq3AMvKUZJ8s633kwYMHyMnJ0RurX78+jIyMnvq8WJgKfeTm4+ODNWvWoHfv3qhbt67eL5QcPHgQP/zwg+4tEH9/f4SHh2PZsmW6t18OHz6M1atXo1u3bmjTpk2Z5erTpw+mTp2K7t27IyIiAo8fP0ZsbCxq1aqlt+HPmjULe/fuRefOneHh4YGUlBTExMTA3d1d78PZ/D755BN06tQJgYGBGDJkCDIyMvDFF1/A1tbWoH4r77XXXkN8fDxsbW3h6+uLQ4cOYfv27XBwcNCbLiAgAMbGxvjoo4+QlpYGtVqN4OBgODs7l2h5cXFx2LRpE1atWqV76+KLL75AWFgYYmNjMXLkyBLfh2nTpuHbb79Fp06dEBERgcqVK2P16tVISEjAunXrXsgXtAcPHoz58+cjJCQEQ4YMQUpKCpYsWYJ69eoV66Sg4nrllVdgb2+P8PBwREREQKVSIT4+/rlfGJSnzp07Y/78+ejYsSP69euHlJQULF68GDVq1NB7URkQEIC+ffsiJiYGaWlpeOWVV7Bjxw5cunSpwDznzp2LXbt2oVmzZhg2bBh8fX1x7949HDt2DNu3by/0xWh+fn5+CAkJQUREBNRqte6FQVRUVJG39fHxwezZsxEZGYnExER069YN1tbWSEhIwPr16zF8+HBMmjQJJiYmmD17Nt566y0EBwejd+/eSEhIQFxcXJGfuQH/Fn2LFi3QsGFDDB8+HF5eXkhMTMSmTZtw/PhxANCdFPPOO++gT58+MDExQZcuXQp9V6is95GdO3di9OjR6NWrF2rVqoWcnBzEx8frXoAUW6nOtTQQFy5cEMOGDROenp7C1NRUWFtbi+bNm4svvvhC7/Tl7OxsERUVJby8vISJiYmoVq3aM7/End/TviCZ/6sAQvz75Ww/Pz9hamoqateuLb7++usCXwXYsWOHCA0NFW5ubsLU1FS4ubmJvn37igsXLhRYRv4vYW7fvl00b95cmJubCxsbG9GlS5enfok7/1cN8k7DLuw03/z3t169egXGn7Z+kO9U+vv374tBgwbpvjwbEhIizp07V+gp/F9++aXw9vYWxsbGel8LeNqy8q7Lm8/169eFra2t6NKlS4HpunfvLiwtLcWVK1eeeX/z58+T9wVVOzs7YWZmJpo2bfrUL6gW9/TlZ207KOTLxV9//bXui94BAQFiy5Ytz/wSd3Gy5W0Hf/75p27swIED4uWXX9Z9KXvKlCm6r3k8+VWNp20bT/saTH6FLVuIp2+z4eHhwtLSUm9sxYoVombNmkKtVos6deqIuLi4Qn8oISMjQ0RERAgHBwdhaWn5zC9xJycni1GjRolq1aoJExMT4eLiItq2bSuWLVtW5H3K236+/vprXa6XXnqpwFdcnnYf86xbt060aNFCWFpaCktLS1GnTh0xatQocf78eb3pYmJihJeXl1Cr1aJx48Yl+hL333//Lbp3767bpmvXri3ee+89vWk++OADUbVqVWFkZKT3fPGsL3E/zz6SP+OVK1fE4MGDhY+Pj+7HH9q0aSO2b99e6Pp6GpUQBviyjIioglGpVBg1ahQWLVqkdBRCBf/MjYiIqDAsNyIikg7LjYiIpFOhz5YkIjIUPH3BsPDIjYiIpMNyIyIi6bDciIhIOlJ+5rageljRExFed05SOkKFsODOs38TkKgk3szJVDpChdD4xoZS3Z5HbkREJB2WGxERSYflRkRE0mG5ERGRdFhuREQkHZYbERFJh+VGRETSYbkREZF0WG5ERCQdlhsREUmH5UZERNJhuRERkXRYbkREJB2WGxERSYflRkRE0mG5ERGRdFhuREQkHZYbERFJh+VGRETSYbkREZF0WG5ERCQdlhsREUmH5UZERNJhuRERkXRYbkREJB2WGxERSYflRkRE0mG5ERGRdFhuREQkHZYbERFJp5LSAWRnWcUeLSP7wLNNA5iYq5GamIytk5Yh+WSC0tEMisrCHLYjBsGiTQsY2dsh+/wl3J+3GFlnzisdzWCEjOuJjuN66o0lX76JuW0nKpTIMHE9FYOREdwm9IFDjyCYONsh69Z93P1hJ5IWfq90sjLDcitHalsL9P7pfdw4dBbrB3yCjHsPYedZBZlp6UpHMziV350IEx8v3H0/Grm378Ly1XZwjvkYSb2GIPf2HaXjGYyk89cRGzZbd1mbo1UwjeHieno2l5E94DSgIxLHLUTGheuw9PeB57wI5D5MR8rKTUrHKxMst3LU5O0ueJR0D1snLdONPbh+W8FEhkmlNoVFcCvcnvgeNH+dAgCkLfsK5i0DYdWzC9Ji4xROaDi0ubl4eDtN6RgGj+vp2awa10bq1sNI23kUAJB1IwWVQ1vBMqCmwsnKDsutHHm3b4ire06ic+wYuDerg0e37uNE/Hb8/e1upaMZFmNjqCoZQ2Rl6Q1rNRqoA/wUCmWYHD1dMPOPGORospF47CI2fvwtUv+5q3Qsg8P19GyPjpyH05sdoPZygybhH5jX9YRVk7q4Pmul0tHKjKLldufOHaxcuRKHDh3CrVu3AAAuLi545ZVXMHDgQDg5OSkZr9RsqzmhQVhbHFu+GYcX/QIXf2+0iRoAbXYuzvy4T+l4BkM8zoDmxGnYDg3D3YRryL13HxYhwVDX90XOjX+Ujmcwrh6/hG8nxSLlShJsnO0QMrYnxnw/Ex+HTIYmPVPpeAaD66lotxavg7G1Ofz2LILI1UJlbISbH32De+v3Kh2tzChWbn/++SdCQkJgYWGBdu3aoVatWgCA5ORkfP7555g7dy62bNmCxo0bP3M+Go0GGo1GbyxH5KKSyrjcsheXysgIySev4MDH/35Ie/v0VTjUdkf9N4NZbvncfT8ald+fjKqbv4fIyUXW+Yt4vGUXTOvK8zZJaZ3bfVz376Rz13D1+CW8v38RAjoH4o/vdykXzMBwPRXNvktzOHQPwpXR85F54TrM63mh+szByE6+h7s/yrGOFCu3MWPGoFevXliyZAlUKpXedUIIjBgxAmPGjMGhQ4eeOZ/o6GhERUXpjXWwqY+Otg3KPHNJpaek4u5F/SOPexf/Qc1OTRRKZLhybiYh5a0JUJmZQWVpAe3de3CY8y5ybiYpHc1gZT54jNsJSXD0rKJ0FIPG9VRQtXcHImnxOtz/ZT8AIOPcVairOsFl9OvSlJti33M7ceIExo8fX6DYAEClUmH8+PE4fvx4kfOJjIxEWlqa3l87m3rlkLjk/jlyAZV9XPXG7L1d8OAGz/57GpGZCe3de1BZW8E8sAke7zmodCSDZWqhhoNHFTxISVU6ikHjeirIyNwU0Aq9MZGrhcqo4PNxRaXYkZuLiwsOHz6MOnXqFHr94cOHUaVK0a+01Go11Gq13pghvCUJAMeWb0bv9e+jyaiuuLDxD7gEeKN+vzbYPk2eD23LitnLjQGVCtlXr8OkWlXYRQxHduI1pP+yWeloBqPr9DCc3nEU927ega2zPTqO7wmRq8WxXw4oHc2gcD0VLXXbEbhG9ETWzdvIuHAdFn5eqDK8K+6s3aF0tDKjWLlNmjQJw4cPx9GjR9G2bVtdkSUnJ2PHjh348ssv8emnnyoVr0wkn7yCX4d/hhZTe+Plsd2Qdv02dkd9jXMbeDSSn5GVJWxHD0UlZ0doHzzE4537kLp4JZCbq3Q0g2HrWhn9Px8DSztrPLr3AFeOnMdn3d9D+r2HSkczKFxPRbv23jJUnfwmqs95CyaOtsi6dR+3v96CpM/k+RK3Sgghip6sfKxduxYLFizA0aNHkfv/n8SMjY3RqFEjTJgwAW+88cZzzXdB9bCyjCmt1535eVZxLLhTsc/aJcPyZg7P2CyOxjc2lOr2in4VoHfv3ujduzeys7Nx586/n0M5OjrCxMREyVhERFTBGcSXuE1MTODq6lr0hERERMXA/xWAiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumohBBC6RBlralbkNIRKoQDJ+OUjlAhmLu1VDpChdDQsYbSESqEY3cuKR2hQsjJulmq2/PIjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNzK0esDQvHN9pXYef437Dz/G1b8EoPANs2UjqW4I8dPYdSUGWjT9U34Ne+EHXsP6l3/zux58GveSe/vrQnvKpTWcLRs0Qwb1q/CtcSjyMm6ia5dQ5SOZJC43xWP7NtTJaUDyCw56TYWz1mK6wk3oFKp0LlXR3wa9yH6dxiKKxcSlY6nmIyMTNSu4Y3unTtg3PTZhU7T4uXGmD19vO6yiYnJi4pnsCwtLXDy5BnErfoO635YoXQcg8X9rnhk355YbuVo/zb9I5LYj5ajx4BQ+DXy/U/vZC0Dm6BlYJNnTmNqYgJHh8ovKFHFsHnLLmzeskvpGAaP+13xyL49sdxeECMjI7Tt0hrmFmY4deS00nEM3p9/nUSrzn1gY22Fpo38ETE8HHa2NkrHogqG+91/F8utnPnU8caKXxfDVG2KjPQMTBnyLhIuXlU6lkFr/nIjtAtqjqpuVXD9ZhIWLl2FERPfwzdL58PY2FjpeFQBcL8jgy6369evY8aMGVi5cuVTp9FoNNBoNHpjWqGFkcowzpW5evkawtoPhZW1JYJfC8KMhdMxokcEd7RneLVda92/a/l4oZaPFzq9MRh//nUSLzd+SblgVGFwvyPDaICnuHfvHlavXv3MaaKjo2Fra6v3l/To2gtKWLSc7BzcSLyJc6cuICb6S1w8cwm9h/ZUOlaFUq2qK+ztbHDtRpLSUaiC4H5Hih65/fLLL8+8/sqVK0XOIzIyEhMmTNAbC67duVS5ypORygimpjzzryRupdxGatpDOPEEE3pO3O/+exQtt27dukGlUkEI8dRpVCrVM+ehVquhVqv1xgzlLcmRkcNwaOcfuHUzBRZWFgjp3hYNXwlARL/JSkdT1OPHGbh24x/d5Zv/JOPchcuwtbGGrY01YlZ+g/atm8PRoTKu3/wH82NWorq7G5o3a6hgauVZWlqgRg0v3WUvz+rw96+He/fu4/r1f55xy/8W7nfFI/v2pGi5ubq6IiYmBqGhoYVef/z4cTRq1OgFpyo7lR3tMePz6XB0dsCjh+m4dPYyIvpNxuG9R5SOpqi/z13E4DFTdZc//mIZACC0Uzu8N3k0LlxOwC//tx0PHqXD2bEyXmnaEKOHDYCpqalSkQ1C40b+2LH9R93leZ/OBACs/up7DBk6/im3+u/hflc8sm9PKvGsw6Zy1rVrVwQEBGDWrFmFXn/ixAm89NJL0Gq1JZpvU7egsognvQMn45SOUCGYu7VUOkKF0NCxhtIRKoRjdy4pHaFCyMm6WarbK3rkNnnyZKSnpz/1+ho1amDXLnm/ZEhEROVD0XJr2fLZr4gtLS0RFMSjMCIiKhnDOPOCiIioDLHciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6KiGEUDpEWWvqFqR0hArB3cRW6QgkkRvZaUpHqBCmaasqHaFC6HFrTaluzyM3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6TxXue3btw9hYWEIDAzEzZs3AQDx8fHYv39/mYYjIiJ6HiUut3Xr1iEkJATm5ub466+/oNFoAABpaWmYM2dOmQckIiIqqRKX2+zZs7FkyRJ8+eWXMDEx0Y03b94cx44dK9NwREREz6PE5Xb+/Hm0atWqwLitrS1SU1PLIhMREVGplLjcXFxccOnSpQLj+/fvh7e3d5mEIiIiKo0Sl9uwYcMwduxY/PHHH1CpVPjnn3/wzTffYNKkSXj77bfLIyMREVGJVCrpDaZNmwatVou2bdvi8ePHaNWqFdRqNSZNmoQxY8aUR0YiIqISUQkhxPPcMCsrC5cuXcKjR4/g6+sLKyurss723Jq6BSkdAQDw+oBQ9BgQCtdqLgCAhPOJWL5gNQ7t+kPhZP9yN7FVOkIB3d9+Hf2nhWPjil+wctZypeMYLENcTzey05SOAMDw97tp2qpKRwAAhPy5EJbVnAqMX47bihORq158oHx63FpTqtuX+Mgtj6mpKXx9fUu1cNklJ93G4jlLcT3hBlQqFTr36ohP4z5E/w5DceVCotLxDE6NBjXQ4c2OSDyToHQUg8b19Gzc74pnV8d3oTL63ydTNnWqoeUP03HzV8N4EVBaJS63Nm3aQKVSPfX6nTt3liqQTPZvO6h3Ofaj5egxIBR+jXy5k+VjZmGGcQsnInbqIvQc84bScQwW11PRuN8VT9bdh3qXXcd0xaOEW7hz8KxCicpWiU8oCQgIgL+/v+7P19cXWVlZOHbsGOrXr18eGaVgZGSE9qHBMLcww6kjp5WOY3CGfTACR3cewckDJ5SOYtC4nkqG+13xqEyMUe31Frj67R6lo5SZEh+5LViwoNDxmTNn4tGjR6UOJBufOt5Y8etimKpNkZGegSlD3kXCxatKxzIozbu0hLefN6Z0nah0FIPG9VR83O9Kxq1TY5jYWuDqWnnKrcx+ODksLAwrV64s8e0yMjKwf/9+nDlzpsB1mZmZ+Oqrr555e41GgwcPHuj9aYW2xDnKy9XL1xDWfigGd34b6776GTMWTodXTQ+lYxkMB1dHDJkxDJ+NnY9sTbbScQwW11PJcL8rGc++bZC88wQyk1OVjlJmnvtsyfzi4+MxdepU/PPPP8W+zYULF9ChQwdcu3YNKpUKLVq0wHfffQdXV1cAQHJyMtzc3JCbm/vUecycORNRUVF6Y25W1VHV2vO57kd5W7R2Hm4k/oO5U+cpHcUgzpZs2qEZpn35DnJz/vcYG1cyhlarhdAK9K75OrRaw3mxopSKsJ4M5WzJwhjSfmcoZ0vmMXd3RMc/PsPvgxcgactRpePovPCzJXv06KF3WQiBpKQkHDlyBO+9916J5jV16lT4+fnhyJEjSE1Nxbhx49C8eXPs3r0b1atXL9Y8IiMjMWHCBL2x4NqdS5TjRTJSGcHU1KToCf8jTh44iXHtR+uNjf50LG5cvoENsesUf8I2FFxPpcP97uk8+wRBcycNt7b/pXSUMlXicrO11X+1b2RkhNq1a2PWrFno0KFDieZ18OBBbN++HY6OjnB0dMSvv/6KkSNHomXLlti1axcsLS2LnIdarYZardbPpDKM/6ZuZOQwHNr5B27dTIGFlQVCurdFw1cCENFvstLRDEZmegauXbimP/Y4E4/uPyww/l/G9VR83O9KQKWCR59WuPr9PohcuV4glajccnNzMWjQINSvXx/29valXnhGRgYqVfpfBJVKhdjYWIwePRpBQUFYs6Z0h6VKq+xojxmfT4ejswMePUzHpbOXEdFvMg7vPaJ0NCJpcb8rPudWfrBwd8LVb3crHaXMlfgzNzMzM5w9exZeXl6lXnjTpk0xZswY9O/fv8B1o0ePxjfffIMHDx488zO3QudrIL9QYugM4TM3kochf+ZmSAztMzdDVdrP3Er8/p2fnx+uXLlSqoXm6d69O7799ttCr1u0aBH69u2LMjrfhYiI/kNKfOS2efNmREZG4oMPPkCjRo0KfC5mY2NTpgGfB4/ciodHblSWeORWPDxyK54XdrbkrFmzMHHiRLz66qsAgK5du+r9DJcQAiqVqsRvIRIREZW1YpdbVFQURowYgV27dpVnHiIiolIrdrnlvXsZFMS3/IiIyLCV6ISSZ/1vAERERIaiRN9zq1WrVpEFd+/evVIFIiIiKq0SlVtUVFSBXyghIiIyNCUqtz59+sDZ2bm8shAREZWJYn/mxs/biIiooih2ufGXQoiIqKIo9tuS/C81iIioojCM/xuGiIioDLHciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKRTSekA5eHYnUtKR6gYHGsonaBC4PZUPKnjmykdoUIIjr+pdIQKoUcpb88jNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IrRy1bNMOG9atwLfEocrJuomvXEKUjGaTXB4Tim+0rsfP8b9h5/jes+CUGgW2aKR3L4HB7KpyRly/MwiNh8c5yWH30E4x9m+pdb1yvGcyGvA/L91fD6qOfYOTqqUxQAyP7fsdyK0eWlhY4efIMxox9R+koBi056TYWz1mK8I7DMLDTcBw5cAyfxn0I71qeSkczKNyeCqcyVUOblAjNhi+fcr0ZchPPQvN/8S84mWGTfb+rpHQAmW3esgubt+xSOobB27/toN7l2I+Wo8eAUPg18sWVC4nKhDJA3J4Kl3v+L+Se/+up1+f8tQcAoLJ3elGRKgTZ9zuWGxkUIyMjtO3SGuYWZjh15LTScYj+E2Tc7xQvt7Nnz+L3339HYGAg6tSpg3PnzmHhwoXQaDQICwtDcHCw0hHpBfCp440Vvy6GqdoUGekZmDLkXSRcvKp0LCKpybzfKVpumzdvRmhoKKysrPD48WOsX78eAwYMgL+/P7RaLTp06ICtW7c+s+A0Gg00Go3emBACKpWqvONTGbp6+RrC2g+FlbUlgl8LwoyF0zGiR4Q0OxqRIZJ5v1P0hJJZs2Zh8uTJuHv3LuLi4tCvXz8MGzYM27Ztw44dOzB58mTMnTv3mfOIjo6Gra2t3p/QPnxB94DKSk52Dm4k3sS5UxcQE/0lLp65hN5Deyodi0hqMu93ipbb6dOnMXDgQADAG2+8gYcPH6Jnz/+t2DfffBMnT5585jwiIyORlpam96cysi7P2PQCGKmMYGpqonQMov8UmfY7xT9zy3v70MjICGZmZrC1tdVdZ21tjbS0tGfeXq1WQ61WFzpPpVlaWqBGDS/dZS/P6vD3r4d79+7j+vV/FExmWEZGDsOhnX/g1s0UWFhZIKR7WzR8JQAR/SYrHc2gcHt6ClMzGDm46C4aVXaGcPWEyHgEkXoHMLeCkZ0jVDaV/73eqSoAQDxMhXiUqkRigyD7fqdouXl6euLixYvw8fEBABw6dAjVq1fXXX/t2jW4uroqFa/UGjfyx47tP+ouz/t0JgBg9VffY8jQ8QqlMjyVHe0x4/PpcHR2wKOH6bh09jIi+k3G4b1HlI5mULg9Fc7Y3Qfmb32gu6zuMhgAkH1kJzQ/LEIl3yYwe2OM7nqzNycCALK2rUXW9rUvNqwBkX2/UwkhhFILX7JkCapVq4bOnTsXev306dORkpKC5cuXl2i+lUyrlkU86TV0rKF0hArh2J1LSkeoEFLHy/PrFuUpOP6u0hEqhMP/7CnV7RU9chsxYsQzr58zZ84LSkJERDLhz28REZF0WG5ERCQdlhsREUmH5UZERNJhuRERkXRYbkREJB2WGxERSYflRkRE0mG5ERGRdFhuREQkHZYbERFJh+VGRETSYbkREZF0WG5ERCQdlhsREUmH5UZERNJhuRERkXRYbkREJB2WGxERSYflRkRE0mG5ERGRdFhuREQkHZYbERFJh+VGRETSYbkREZF0WG5ERCQdlhsREUmH5UZERNJhuRERkXRYbkREJB2WGxERSaeS0gHKQ1fXRkpHqBA8VBZKR6gQYpztlY5QIQTH31U6QoUwTVtV6Qj/CTxyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6LDciIpIOy42IiKTDciMiIumw3IiISDosNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIiEg6lZQO8F/S/e3X0X9aODau+AUrZy1XOo7BCBnXEx3H9dQbS758E3PbTlQokYEyMoLbhD5w6BEEE2c7ZN26j7s/7ETSwu+VTmZQXh8Qih4DQuFazQUAkHA+EcsXrMahXX8onMywhPy5EJbVnAqMX47bihORq158oDLGcntBajSogQ5vdkTimQSloxikpPPXERs2W3dZm6NVMI1hchnZA04DOiJx3EJkXLgOS38feM6LQO7DdKSs3KR0PIORnHQbi+csxfWEG1CpVOjcqyM+jfsQ/TsMxZULiUrHMxi7Or4LldH/3ryzqVMNLX+Yjpu/yvEiwODKTQgBlUqldIwyZWZhhnELJyJ26iL0HPOG0nEMkjY3Fw9vpykdw6BZNa6N1K2HkbbzKAAg60YKKoe2gmVATYWTGZb92w7qXY79aDl6DAiFXyNfltsTsu4+1LvsOqYrHiXcwp2DZxVKVLYM7jM3tVqNs2flWLl5hn0wAkd3HsHJAyeUjmKwHD1dMPOPGLy7dyHCPhsNOzcHpSMZnEdHzsOmeQOovdwAAOZ1PWHVpC7Sdh1TOJnhMjIyQvvQYJhbmOHUkdNKxzFYKhNjVHu9Ba5+u0fpKGVGsSO3CRMmFDqem5uLuXPnwsHh3ye3+fPnv8hYZa55l5bw9vPGlK78/Ohprh6/hG8nxSLlShJsnO0QMrYnxnw/Ex+HTIYmPVPpeAbj1uJ1MLY2h9+eRRC5WqiMjXDzo29wb/1epaMZHJ863ljx62KYqk2RkZ6BKUPeRcLFq0rHMlhunRrDxNYCV9ey3Erts88+g7+/P+zs7PTGhRA4e/YsLC0ti/X2pEajgUaj0RvLFbkwVhmXZdzn4uDqiCEzhiEq7H1ka7KVjmOwzu0+rvt30rlruHr8Et7fvwgBnQPxx/e7lAtmYOy7NIdD9yBcGT0fmReuw7yeF6rPHIzs5Hu4+yPX05OuXr6GsPZDYWVtieDXgjBj4XSM6BHBgnsKz75tkLzzBDKTU5WOUmYUK7c5c+Zg2bJlmDdvHoKDg3XjJiYmWLVqFXx9fYs1n+joaERFRemN1bGphbp2tcs07/Pwqe8DOyc7fLppgW7MuJIxfJvVQ6fwzuhd83VotTxxIr/MB49xOyEJjp5VlI5iUKq9OxBJi9fh/i/7AQAZ565CXdUJLqNfZ7nlk5OdgxuJNwEA505dgG9AHfQe2hNzp85TOJnhMXd3hHMrP/w+eEHRE1cgipXbtGnT0LZtW4SFhaFLly6Ijo6GiYlJiecTGRlZ4C3O/n59yypmqZw8cBLj2o/WGxv96VjcuHwDG2LXsdiewtRCDQePKniwfp/SUQyKkbkpoBV6YyJXC5WRXCdglQcjlRFMTUv+/PJf4NknCJo7abi1/S+lo5QpRc+WbNKkCY4ePYpRo0ahcePG+Oabb0p8pqRarYZardYbM4S3JAEgMz0D1y5c0x97nIlH9x8WGP8v6zo9DKd3HMW9m3dg62yPjuN7QuRqceyXA0pHMyip247ANaInsm7eRsaF67Dw80KV4V1xZ+0OpaMZlJGRw3Bo5x+4dTMFFlYWCOneFg1fCUBEv8lKRzM8KhU8+rTC1e/3QeTK9WJb8a8CWFlZYfXq1fjuu+/Qrl075ObmKh2JXjBb18ro//kYWNpZ49G9B7hy5Dw+6/4e0u89LPrG/yHX3luGqpPfRPU5b8HE0RZZt+7j9tdbkPQZv8T9pMqO9pjx+XQ4Ojvg0cN0XDp7GRH9JuPw3iNKRzM4zq38YOHuhKvf7lY6SplTCSFE0ZO9GDdu3MDRo0fRrl07WFpaPvd8enh0LcNU8vJQWSgdoUJ4M4dnbBbHSO19pSNUCNO0VZWOUCH0uLWmVLdX/MjtSe7u7nB3d1c6BhERVXAG9yVuIiKi0mK5ERGRdFhuREQkHZYbERFJh+VGRETSYbkREZF0WG5ERCQdlhsREUmH5UZERNJhuRERkXRYbkREJB2WGxERSYflRkRE0mG5ERGRdFhuREQkHZYbERFJh+VGRETSYbkREZF0WG5ERCQdlhsREUmH5UZERNJhuRERkXRYbkREJB2WGxERSYflRkRE0mG5ERGRdFhuREQkHZYbERFJh+VGRETSYbkREZF0WG5ERCQdlRBCKB1CdhqNBtHR0YiMjIRarVY6jsHieioerqfi4XoqHlnXE8vtBXjw4AFsbW2RlpYGGxsbpeMYLK6n4uF6Kh6up+KRdT3xbUkiIpIOy42IiKTDciMiIumw3F4AtVqNGTNmSPVhbXngeioerqfi4XoqHlnXE08oISIi6fDIjYiIpMNyIyIi6bDciIhIOiw3IiKSDsvtBVi8eDE8PT1hZmaGZs2a4fDhw0pHMih79+5Fly5d4ObmBpVKhQ0bNigdySBFR0ejSZMmsLa2hrOzM7p164bz588rHcvgxMbGokGDBrCxsYGNjQ0CAwPxf//3f0rHMmhz586FSqXCuHHjlI5SZlhu5Wzt2rWYMGECZsyYgWPHjsHf3x8hISFISUlROprBSE9Ph7+/PxYvXqx0FIO2Z88ejBo1Cr///ju2bduG7OxsdOjQAenp6UpHMyju7u6YO3cujh49iiNHjiA4OBihoaE4ffq00tEM0p9//omlS5eiQYMGSkcpW4LKVdOmTcWoUaN0l3Nzc4Wbm5uIjo5WMJXhAiDWr1+vdIwKISUlRQAQe/bsUTqKwbO3txfLly9XOobBefjwoahZs6bYtm2bCAoKEmPHjlU6UpnhkVs5ysrKwtGjR9GuXTvdmJGREdq1a4dDhw4pmIxkkJaWBgCoXLmywkkMV25uLr777jukp6cjMDBQ6TgGZ9SoUejcubPec5QsKikdQGZ37txBbm4uqlSpojdepUoVnDt3TqFUJAOtVotx48ahefPm8PPzUzqOwTl16hQCAwORmZkJKysrrF+/Hr6+vkrHMijfffcdjh07hj///FPpKOWC5UZUAY0aNQp///039u/fr3QUg1S7dm0cP34caWlp+PHHHxEeHo49e/aw4P6/69evY+zYsdi2bRvMzMyUjlMuWG7lyNHREcbGxkhOTtYbT05OhouLi0KpqKIbPXo0Nm7ciL1798Ld3V3pOAbJ1NQUNWrUAAA0atQIf/75JxYuXIilS5cqnMwwHD16FCkpKWjYsKFuLDc3F3v37sWiRYug0WhgbGysYMLS42du5cjU1BSNGjXCjh07dGNarRY7duzg+/9UYkIIjB49GuvXr8fOnTvh5eWldKQKQ6vVQqPRKB3DYLRt2xanTp3C8ePHdX+NGzfGm2++iePHj1f4YgN45FbuJkyYgPDwcDRu3BhNmzbFZ599hvT0dAwaNEjpaAbj0aNHuHTpku5yQkICjh8/jsqVK6N69eoKJjMso0aNwpo1a/Dzzz/D2toat27dAgDY2trC3Nxc4XSGIzIyEp06dUL16tXx8OFDrFmzBrt378aWLVuUjmYwrK2tC3xWa2lpCQcHB2k+w2W5lbPevXvj9u3beP/993Hr1i0EBARg8+bNBU4y+S87cuQI2rRpo7s8YcIEAEB4eDhWrVqlUCrDExsbCwBo3bq13nhcXBwGDhz44gMZqJSUFAwYMABJSUmwtbVFgwYNsGXLFrRv317paPQC8b+8ISIi6fAzNyIikg7LjYiIpMNyIyIi6bDciIhIOiw3IiKSDsuNiIikw3IjIiLpsNyIDNzAgQPRrVs33eXWrVsr8j8m7969GyqVCqmpqS982UQlxXIjek4DBw6ESqWCSqXS/VDvrFmzkJOTU67L/emnn/DBBx8Ua1oWEv1X8ee3iEqhY8eOiIuLg0ajwW+//YZRo0bBxMQEkZGRetNlZWXB1NS0TJbJ/5yUqGg8ciMqBbVaDRcXF3h4eODtt99Gu3bt8Msvv+jeSvzwww/h5uaG2rVrA/j3/9F64403YGdnh8qVKyM0NBSJiYm6+eXm5mLChAmws7ODg4MDpkyZgvy/kJf/bUmNRoOpU6eiWrVqUKvVqFGjBlasWIHExETdb3ba29tDpVLpfoNSq9UiOjoaXl5eMDc3h7+/P3788Ue95fz222+oVasWzM3N0aZNG72cRIaO5UZUhszNzZGVlQUA2LFjB86fP49t27Zh48aNyM7ORkhICKytrbFv3z4cOHAAVlZW6Nixo+428+bNw6pVq7By5Urs378f9+7dw/r165+5zAEDBuDbb7/F559/jrNnz2Lp0qWwsrJCtWrVsG7dOgDA+fPnkZSUhIULFwIAoqOj8dVXX2HJkiU4ffo0xo8fj7CwMOzZswfAvyXco0cPdOnSBcePH8fQoUMxbdq08lptRGVPENFzCQ8PF6GhoUIIIbRardi2bZtQq9Vi0qRJIjw8XFSpUkVoNBrd9PHx8aJ27dpCq9XqxjQajTA3NxdbtmwRQgjh6uoqPv74Y9312dnZwt3dXbccIYQICgoSY8eOFUIIcf78eQFAbNu2rdCMu3btEgDE/fv3dWOZmZnCwsJCHDx4UG/aIUOGiL59+wohhIiMjBS+vr5610+dOrXAvIgMFT9zIyqFjRs3wsrKCtnZ2dBqtejXrx9mzpyJUaNGoX79+nqfs504cQKXLl2CtbW13jwyMzNx+fJlpKWlISkpCc2aNdNdV6lSJTRu3LjAW5N58v5jyaCgoGJnvnTpEh4/flzgv4DJysrCSy+9BAA4e/asXg4A/A92qUJhuRGVQps2bRAbGwtTU1O4ubmhUqX/7VKWlpZ60z569AiNGjXCN998U2A+Tk5Oz7X85/lPSh89egQA2LRpE6pWrap3nVqtfq4cRIaG5UZUCpaWlqhRo0axpm3YsCHWrl0LZ2dn2NjYFDqNq6sr/vjjD7Rq1QoAkJOTg6NHj6Jhw4aFTl+/fn1otVrs2bMH7dq1K3B93pFjbm6ubszX1xdqtRrXrl176hFf3bp18csvv+iN/f7770XfSSIDwRNKiF6QN998E46OjggNDcW+ffuQkJCA3bt3IyIiAjdu3AAAjB07FnPnzsWGDRtw7tw5jBw58pnfUfP09ER4eDgGDx6MDRs26Ob5/fffAwA8PDygUqmwceNG3L59G48ePYK1tTUmTZqE8ePHY/Xq1bh8+TKOHTuGL774AqtXrwYAjBgxAhcvXsTkyZNx/vx5rFmzhv8rOlUoLDeiF8TCwgJ79+5F9erV0aNHD9StWxdDhgxBZmam7khu4sSJ6N+/P8LDwxEYGAhra2t07979mfONjY1Fz549MXLkSNSpUwfDhg1Deno6AKBq1aqIiorCtGnTUKVKFYwePRoA8MEHH+C9995DdHQ06tati44dO2LTpk3w8vICAFSvXh3r1q3Dhg0b4O/vjyVLlmDOnDnluHaIypZKPO2TaiIiogqKR25ERCQdlhsREUmH5UZERNJhuRERkXRYbkREJB2WGxERSYflRkRE0mG5ERGRdFhuREQkHZYbERFJh+VGRETSYbkREZF0/h9PBdyKOyVsPQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mean accuracy = 0.35\n",
      "F1 score for the predictions = [0.24       0.50847458 0.2        0.5        0.26415094]\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "import pandas as pd\n",
    "\n",
    "# Open file with data\n",
    "with open('human.json', 'r') as f:\n",
    "    # Read data with json module\n",
    "    data = json.load(f)\n",
    "\n",
    "# Create a dataframe based on the data\n",
    "human_df = pd.DataFrame(data)\n",
    "\n",
    "# Print the first 5 examples\n",
    "human_df.head()\n",
    "\n",
    "y_pred = human_df['predicted']\n",
    "y = human_df[\"actual\"]\n",
    "\n",
    "# Create confusion matrix for the human predictions\n",
    "confusion_mat = metrics.confusion_matrix(y, y_pred)\n",
    "# Plot confusion matrix\n",
    "sns.heatmap(confusion_mat, square=True, annot=True, fmt='d', cbar=False)\n",
    "plt.xlabel('Predicted')\n",
    "plt.ylabel('True')\n",
    "plt.title(\"Confusion matrix for human-made predictions\")\n",
    "plt.show()\n",
    "\n",
    "print(f\"Mean accuracy = {(y - y_pred == 0).sum()/len(y):.2f}\")\n",
    "print(f\"F1 score for the predictions = {metrics.f1_score(y, y_pred, average=None)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the AI does a pretty good job at predicting the sender, even compared to humans. The mean accuracy can be even better at around 38%, while the human performance is at around 35%. However, they are both quite low—slightly above a random choice, which would lead to 20% accuracy. Therefore, it might suggest that the task is quite hard. To get a better performance, I would probably need to gather more data and use more sophisticated techniques, such as transformer-based encoder neural networks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Executive Summary\n",
    "\n",
    "In this work, I wanted to find out how well the ML model can predict who the sender of a message is. I gathered data from a group chat using the `telethon` module. I filtered it out of any links, empty messages, and stop words. I used the lemmatization technique to normalize words so that their semantical meaning is better understood by the model. I created features using the data, for example, an indication of whether the text starts with a capital letter and the number of digits and punctuation signs in the text. I trained the Naive Bayes classifier to classifier and achieved 38% accuracy, which is comparable to human-level performance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# References\n",
    "\n",
    "6.2. Feature extraction. (n.d.). Scikit-Learn. https://scikit-learn.org/stable/modules/feature_extraction.html#feature-extraction\n",
    "\n",
    "Harder, H. de. (2022, November 5). An easy example explaining Naive Bayes. Medium. https://hennie-de-harder.medium.com/an-easy-example-explaining-naive-bayes-3f706d683625\n",
    "\n",
    "Scikit learn. (2018). sklearn.feature_extraction.text.TfidfVectorizer — scikit-learn 0.20.3 documentation. Scikit-Learn.org. https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html\n",
    "\n",
    "scikit-learn. (2019a). sklearn.metrics.f1_score — scikit-learn 0.21.2 documentation. Scikit-Learn.org. https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html\n",
    "\n",
    "scikit-learn. (2019b). sklearn.model_selection.GridSearchCV — scikit-learn 0.22 documentation. Scikit-Learn.org. https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html\n",
    "\n",
    "what is the “learning” that takes place in Naive Bayes? (n.d.). Cross Validated. Retrieved February 18, 2024, from https://stats.stackexchange.com/questions/272493/what-is-the-learning-that-takes-place-in-naive-bayes\n",
    "\n",
    "Wikipedia Contributors. (2019, June 17). Naive Bayes classifier. Wikipedia; Wikimedia Foundation. https://en.wikipedia.org/wiki/Naive_Bayes_classifier\n",
    "Wikipedia Contributors. (2023, December 31). tf–idf. Wikipedia; Wikimedia Foundation. https://en.wikipedia.org/wiki/Tf%E2%80%93idf#Definition\n",
    "\n",
    "WordNet | A Lexical Database for English. (2019). Princeton.edu. https://wordnet.princeton.edu"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}