Practical_9_The_Role_Of_Machine_Learning_In_NLP.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00001-20a5efd9-f7be-426c-aa3d-ae979135afef",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "# What is machine learning?\n",
    "\n",
    "\n",
    "Giving the machine the ability to learn a pattern and use the pattern to identify them later on unseen data.\n",
    "\n",
    "**NLP is not only machine learning and vice versa!.**\n",
    "![](https://lh3.googleusercontent.com/RQ1bMI_oo1K1ikB45ufBU72BCLhvCoDKww0xgDi45A93Ad9iT39m48qf9_7WlWDnrUrzF0wh_SQp_vBlnTzRYqgGpJXiL4YOY5ZmGyiTQfA_hWTtqIWxjFGrkGywz8cAPNbVxO1J)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00001-b00b1d2c-cbc1-40a1-89a3-05e424124617",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "## Machine learning algorithms:\n",
    "\n",
    "- Rule based machine learning\n",
    "- Statistical machine learning\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00002-ddb2f479-0785-479f-a342-67556936a0a0",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "From the perspective of data:\n",
    "\n",
    "- Supervised machine learning algorithms\n",
    "\n",
    "- Unsupervised machine learning algorithms\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00003-40e5666d-1948-4861-8a70-d66f42fca0f4",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "NLP tasks which benefit from machine learning algorithms:\n",
    "\n",
    "- Sentiment Analysis\n",
    "- Machine Translation\n",
    "- Text Classification\n",
    "- Information retrieval\n",
    "- ...\n",
    "\n",
    "Most NLP tasks nowadays benefit from machine learning approches to improve the quality of their results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00002-333a9a54-e54b-46c3-80c4-f3e9c6ce0c0a",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00004-6549ca2c-69e4-400e-971b-0b2ead0f00de",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "## Supervised machine learning algorithms:\n",
    "\n",
    "A supervised machine learning algorithm learning patterns from a data which we already know the output for.\n",
    "\n",
    "One of the general supervised machine learning algorithms is called classification, in which an already labeled dataset is used to learn a generalized pattern , and then using this pattern the algorithm assigns new items that it hasn't seen before to the classes.\n",
    "\n",
    "**Classification** is when the output of the algorithm is discrete.\n",
    "\n",
    "\n",
    "- Example:  Spam detection   (input : an email , output : spam - not spam)\n",
    "\n",
    "**Regression** is another general type of supervised machine learning algorithm, unlike classification in regression the range of the output data is continuous.\n",
    "\n",
    "So the output is not limited to a number of labels.\n",
    "\n",
    "- Example: Price estimation (input: product, output : price: 0\\$ - n\\$)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00006-d58563ac-2098-491a-af99-3f9b015d66d0",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "### Supervised machine learning algorithms:\n",
    "    - Naive Bayes classifier\n",
    "    - Support vector machines\n",
    "    - Linear regression\n",
    "    - Neural networks \n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00006-331e0d38-ef6b-4a26-9a66-3600e1827bf0",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "## Unsupervised machine learning algorithms:\n",
    "\n",
    "There is no labeled dataset. The algorihtms learns patterns based on the similarity of the data. One of the general algorithms of unsupervised machine learning is called clustring.\n",
    "\n",
    "Example: Topic Modeling (we will see this later)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00007-dbc01c90-03f6-4459-92e0-ace721cd85f5",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00008-353eeb5c-22bb-41ba-9a21-451944a5b679",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "In this session we focus on the general framework of a supervised machine learning algorithm and use one of the mentioned algorithms for a text-classification task. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00010-7d8b4761-a638-4072-8f2f-bf12ea912276",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "## Train - Test - Validation sets:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00011-d433adf9-e6e8-4956-ae41-562a5a046b0b",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "In order to evaluate the machine learning algorithm, a framework using supervised machine learning algorithm usually splits a labeled-data set to 3 sets.\n",
    "\n",
    "\n",
    "The **train** set is used by the algorithm to learn and generalize patterns from data.\n",
    "\n",
    "The **validation** set is used by the algorithm to evaluate the algorithm while being trained. (may or may not be used in a framework)\n",
    "\n",
    "The **test** set is used for evaludating the algorithm after the training is done and the model is not changing anymore.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00012-3a69dac4-7df1-40f7-a1a0-1a547449272b",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "## Preprocessing\n",
    "\n",
    "\n",
    "Data is messy!\n",
    "\n",
    "We usually need to apply some preprocessing to the data before learning patterns from it.\n",
    "\n",
    "![](https://i.pinimg.com/564x/db/4f/88/db4f88f155d22599f59765e14f4c5497.jpg)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00013-3eaa1555-6b04-4f20-8ec1-3cb521853777",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "**Research and Reflect:** Do you remember what preprocessing methods we already talked about?  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00014-82a45134-e188-4d00-b4b1-ec79b8ebd620",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "### write your answer here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00015-46ee91e0-1289-46c6-ba89-593cf5c6a218",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00016-dbe38c93-c172-4b7b-8a5d-f9e574076ad2",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "## Feature extraction\n",
    "\n",
    "Feature extraction step means to extract and produce feature representations that are appropriate for the type of NLP task you are trying to accomplish and the type of model you are planning to use.\n",
    "\n",
    "Remeber the one-hot coding representation? \n",
    "\n",
    "\n",
    "\n",
    "A simple but effective and prominently used feature representation method in NLP is called **Bag of Words**\n",
    "\n",
    "In this method the input text is looked at without considering the order in which the tokens occur. and only counts the frequency of the occurence of a token.\n",
    "\n",
    "\n",
    "![](https://miro.medium.com/max/764/1*MeSYCKGDOdwkJKVZKxJuvg.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00017-b8b42a30-3849-41d4-a034-81b181b53511",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "**Code it** We have already seen an nltk function which gets a text and returns the bag of words representation of the text, Use it to extract the bag of word representation of the [this story by Edgar Allen Poe](https://www.gutenberg.org/files/1064/1064-0.txt).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cell_id": "00017-a0bc9123-3cd4-48ea-b6e9-9df8c4f0fd2d",
    "deepnote_cell_type": "code",
    "tags": []
   },
   "outputs": [],
   "source": [
    "#insert your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00019-1833a78a-9f3d-4610-9502-229f253ab1ba",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00020-f2aa341a-0f93-42a0-934d-77d05ea146e0",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "Lets gets our hands on training and testing a simple supervised machine learning text classification algorithm based using bag-of-words.\n",
    "\n",
    "Download the data set of consumer complaints from the following link.\n",
    "https://www.kaggle.com/dushyantv/consumer_complaints"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00021-250f23c2-657a-471e-864a-48f4447fa203",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "Load the dataset into memory using the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cell_id": "00022-53d3e853-ceed-498a-9f46-5d990632de08",
    "deepnote_cell_type": "code",
    "execution_millis": 5956,
    "execution_start": 1605013614160,
    "output_cleared": false,
    "source_hash": "d8854dc0",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# if not yet installed install libraries:\n",
    "\n",
    "!pip install numpy\n",
    "!pip install pandas\n",
    "!pip install matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00023-5da5aa82-90f2-4987-bd1b-c961350bc421",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "We reduce the data to contain products and the text of the consumer complaint.\n",
    "\n",
    "Then we add a column which maps a number to each of the product categories calling it category_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cell_id": "00021-62c33ed2-a018-4185-acd4-c6c91f676a40",
    "deepnote_cell_type": "code",
    "execution_millis": 12570,
    "execution_start": 1605023679157,
    "output_cleared": false,
    "source_hash": "6e6accba",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# This Python 3 environment comes with many helpful analytics libraries installed\n",
    "# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n",
    "# For example, here's several helpful packages to load in \n",
    "\n",
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
    "\n",
    "# Input data files are available in the \"../input/\" directory.\n",
    "# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n",
    "\n",
    "\n",
    "import pandas as pd\n",
    "df = pd.read_csv('Consumer_Complaints.csv')\n",
    "\n",
    "# Remove rows at random to avoid memory error \n",
    "np.random.seed(10)\n",
    "remove_n = 1000000\n",
    "drop_indices = np.random.choice(df.index, remove_n, replace=True)\n",
    "df = df.drop(drop_indices)\n",
    "df.shape\n",
    "# Remove above code after mem optimization\n",
    "\n",
    "# We need “Product”(OUT) and “Consumer complaint narrative”(IN) columns.\n",
    "\n",
    "\n",
    "from io import StringIO\n",
    "col = ['Product', 'Consumer Complaint']\n",
    "df = df[col]\n",
    "df = df[pd.notnull(df['Consumer Complaint'])]\n",
    "df.columns = ['Product', 'Consumer_Complaint']\n",
    "df['category_id'] = df['Product'].factorize()[0]\n",
    "category_id_df = df[['Product', 'category_id']].drop_duplicates().sort_values('category_id')\n",
    "category_to_id = dict(category_id_df.values)\n",
    "id_to_category = dict(category_id_df[['category_id', 'Product']].values)\n",
    "\n",
    "df = df[0:5000]\n",
    "df.to_csv(\"Complaints.csv\")\n",
    "display(df)\n",
    "df.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cell_id": "00025-c0d916d8-4d4c-428f-be8b-493ab3657f36",
    "deepnote_cell_type": "code",
    "execution_millis": 84,
    "execution_start": 1605022202580,
    "output_cleared": false,
    "source_hash": "702a4652",
    "tags": []
   },
   "outputs": [],
   "source": [
    "list_text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00025-99aa3b41-f769-4d34-8a6d-aa41bdac7ff7",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "Running the following code you can see the number of consumer complaints on each of the products.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cell_id": "00025-a62f4b04-b152-4326-b04e-0ac9eaaf520b",
    "deepnote_cell_type": "code",
    "execution_millis": 588,
    "execution_start": 1605023696676,
    "output_cleared": false,
    "source_hash": "c102480e",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "df = pd.read_csv(\"Complaints.csv\")\n",
    "fig = plt.figure(figsize=(8,6))\n",
    "df.groupby('Product').Consumer_Complaint.count().plot.bar(ylim=0)\n",
    "plt.show()\n",
    "\n",
    "# We see here imbalance of classes\n",
    "# We want a classifier that gives high prediction accuracy over the majority class,\n",
    "# while maintaining reasonable accuracy for the minority classes as the majority classes might be of use"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00027-bfed7cbd-16da-4be6-bea8-cd07c3a2c0ad",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "### TF-IDF\n",
    "\n",
    "Instead of using the counts of a word in a document we use another feature which is based on the frequency of tokens in a document called Tf-IDF.\n",
    "The Term-Frequency Inverse Document Frequency measures the frequency of a token with regard to the document. It is intended to measure how important a token or word is in a document and it is computed by:\n",
    "\n",
    "![](http://www.digitalmarketingchef.org/wp-content/uploads/2020/02/TFIDF_FORMULA.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00028-8aff853a-92b8-41d3-ad4a-bdd57a246e2a",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "**Reflect and Reply** Considering the formula above for tf-idf measure, Do you expect the tf-idf of stopwords to be relatively high or low compared to other words in a document?\n",
    "\n",
    "Why?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00029-8df83591-b002-40de-b0e9-cb6b98cac9f4",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "### write your answer here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00030-1fb06d8b-7df0-4c05-854a-1793e75fc294",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00031-7b39cae8-0757-490f-b029-34ee364e0d4f",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "The following code uses a library called sklearn to extract and numerize the tf-idf of words from text.\n",
    "\n",
    "The code prints the dimention of the feature vector.\n",
    "\n",
    "** Observe and Reflect **   Can you explain the dimentions of the output? what does each dimention of the output vector represent?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00032-ac3c616e-cdb1-4e28-9aa5-c062ed0eed91",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "### write your answer here\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cell_id": "00028-b023d9f7-35f5-4eb9-a805-6e54d2d4d854",
    "deepnote_cell_type": "code",
    "execution_millis": 3595,
    "execution_start": 1605023703089,
    "output_cleared": false,
    "source_hash": "f0042aa5",
    "tags": []
   },
   "outputs": [],
   "source": [
    "\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "tfidf = TfidfVectorizer(sublinear_tf=True, min_df=5, norm='l2', encoding='latin-1', ngram_range=(1, 2), stop_words='english')\n",
    "features = tfidf.fit_transform(df.Consumer_Complaint).toarray()\n",
    "labels = df.category_id\n",
    "features.shape\n",
    "\n",
    "features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cell_id": "00034-4c515388-d9c4-4d04-bfa4-d0dac3ad9c33",
    "deepnote_cell_type": "code",
    "execution_millis": 3323,
    "execution_start": 1605014371951,
    "output_cleared": false,
    "source_hash": "876c3c82",
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install seaborn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00035-3a06d28e-ecd8-4eb4-bac2-4aa5ec5fe3f4",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "THe following code uses a supervised machine learning algorithm called [SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)(support vector machine) to train a model which is able to identify from the consumer complaint as input the category of the product.\n",
    "\n",
    "We don't go into further detail of the algorithm.\n",
    "\n",
    "1- The code uses the tf-idf features extracted in the previous step from the data.\n",
    "\n",
    "2- Then splits the data to train and test set.\n",
    "\n",
    "3- Trains a model based on train set \n",
    "\n",
    "4- uses the train model to identify the category of the test samples. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cell_id": "00034-fa8c18d1-6f23-4521-8d9d-a4d4fb2f6b1b",
    "deepnote_cell_type": "code",
    "execution_millis": 4656,
    "execution_start": 1605023711487,
    "output_cleared": false,
    "source_hash": "7fabdc27",
    "tags": []
   },
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.feature_extraction.text import TfidfTransformer\n",
    "from sklearn.svm import LinearSVC\n",
    "model = LinearSVC()\n",
    "X_train, X_test, y_train, y_test, indices_train, indices_test = train_test_split(features, labels, df.index, test_size=0.33, random_state=0)\n",
    "model.fit(X_train, y_train)\n",
    "y_pred = model.predict(X_test)\n",
    "from sklearn.metrics import confusion_matrix\n",
    "conf_mat = confusion_matrix(y_test, y_pred)\n",
    "fig, ax = plt.subplots(figsize=(10,10))\n",
    "sns.heatmap(conf_mat, annot=True, fmt='d',\n",
    "            xticklabels=category_id_df.Product.values, yticklabels=category_id_df.Product.values)\n",
    "plt.ylabel('Actual')\n",
    "plt.xlabel('Predicted')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00037-291b097c-060c-42d3-a6ea-6668f7f08126",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "**Reflect and Reply:**  Can you map parts of the code with each of the steps 1,2,3,4 of the supervised machine learning framework in the above?( for instance Which line of code does task 1? The code uses the tf-idf features extracted in the previous step from the data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00038-f379fdcf-d4c7-4e31-aa57-f0100b32b05a",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "## write your answer here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00039-d027222a-99de-4bdc-8151-fb6d4213dfbf",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "**Reflect and Reply:** according to the heap map which categories where better identified using the above machine learning algorithm?\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00040-e91667bd-c2ba-414f-a8b1-c678db8df930",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "## write your answer here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00038-1141d743-87dc-488a-bcbe-ee7148955feb",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "-----\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00039-37e3d6be-e561-4738-acc8-57f13a021fb7",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "**Homework exercise:**\n",
    " How do we evaluate supervised machine learning algorithms? Do a research on measures such as : **Precision**, **Recall** and **Accuracy** \n",
    "\n",
    "Explain the difference between them (you may refer to the sklearn documentation): \n",
    "\n",
    "https://scikit-learn.org/stable/auto_examples/model_selection/plot_precision_recall.html\n",
    "\n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html\n",
    "\n",
    "**CODE IT**\n",
    "evaluate the above method for the product categorization based on consumer complaints, with precision, recall and accuracy measures from sklearn library.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00040-d8dc7fcc-e361-4cc5-b07f-a6e8906abce9",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "## write your answer here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "cell_id": "00041-d0407d32-4662-4955-aa79-217601ee11a9",
    "deepnote_cell_type": "code",
    "tags": []
   },
   "outputs": [],
   "source": [
    "#insert your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "cell_id": "00046-fa59c86b-bb53-4f1f-bed6-dbe39fa4508c",
    "deepnote_cell_type": "markdown",
    "tags": []
   },
   "source": [
    "### References\n",
    "\n",
    "https://www.kaggle.com/anucool007/multi-class-text-classification-bag-of-words"
   ]
  }
 ],
 "metadata": {
  "deepnote_execution_queue": [],
  "deepnote_notebook_id": "28780a53-c8a3-4adb-8e9d-35bebb687e10",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}