notebook/10-03-sentiment-classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "***\n",
    "***\n",
    "# 基于机器学习的情感分析\n",
    "***\n",
    "***\n",
    "\n",
    "![image.png](img/chengjun.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "\n",
    "## LIWC & TextMind \n",
    "\n",
    "http://ccpl.psych.ac.cn/textmind/\n",
    "\n",
    "“文心(TextMind)”中文心理分析系统是由中科院心理所计算网络心理实验室研发的，针对中文文本进行语言分析的软件系统，通过“文心”，您可以便捷地分析文本中使用的不同类别语言的程度、偏好等特点。针对中国大陆地区简体环境下的语言特点，参照LIWC2007和正體中文C-LIWC词库，我们开发了“文心（TextMind）”中文心理分析系统。“文心”为用户提供从简体中文自动分词，到语言心理分析的一揽子分析解决方案，其词库、文字和符号等处理方法专门针对简体中文语境，词库分类体系也与LIWC兼容一致。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<!--NAVIGATION-->\n",
    "< [基于字典的情感分析](08.02-sentiment-analysis-with-dict.ipynb)  >"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "情感分析(sentiment analysis)和意见挖掘(opinion mining)虽然相关，但是从社会科学的角度而言，二者截然不同。这里主要是讲情感分析(sentiment or emotion)，而非意见挖掘（opinion, 后者通过机器学习效果更可信）。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<div><img src=\"img/txt2.png\" align=\"right\"></div>\n",
    "\n",
    "### Emotion\n",
    "Different types of emotion: anger, disgust, fear, joy, sadness, and surprise. The classification can be performed using different algorithms: e.g., naive Bayes classiﬁer trained on Carlo Strapparava and Alessandro Valitutti’s emotions lexicon.\n",
    "\n",
    "\n",
    "### Polarity\n",
    "\n",
    "To classify some text as positive or negative. In this case, the classification can be done by using a naive Bayes algorithm trained on Janyce Wiebe’s subjectivity lexicon."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![image.png](img/txt3.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Sentiment Analysis with Sklearn\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:43:21.307812Z",
     "start_time": "2022-07-21T07:43:19.579147Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:43:21.957396Z",
     "start_time": "2022-07-21T07:43:21.953052Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "pos_tweets = [('I love this car', 'positive'),\n",
    "    ('This view is amazing', 'positive'),\n",
    "    ('I feel great this morning', 'positive'),\n",
    "    ('I am so excited about the concert', 'positive'),\n",
    "    ('He is my best friend', 'positive')]\n",
    "\n",
    "neg_tweets = [('I do not like this car', 'negative'),\n",
    "    ('This view is horrible', 'negative'),\n",
    "    ('I feel tired this morning', 'negative'),\n",
    "    ('I am not looking forward to the concert', 'negative'),\n",
    "    ('He is my enemy', 'negative')]\n",
    "\n",
    "test_tweets = [\n",
    "    ('feel happy this morning', 'positive'),\n",
    "    ('larry is my friend', 'positive'),\n",
    "    ('I do not like that man', 'negative'),\n",
    "    ('house is not great', 'negative'),\n",
    "    ('your song is annoying', 'negative')]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:43:22.754666Z",
     "start_time": "2022-07-21T07:43:22.751652Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "dat = []\n",
    "for i in pos_tweets+neg_tweets+test_tweets:\n",
    "    dat.append(i)\n",
    "    \n",
    "X = np.array(dat).T[0]\n",
    "y = np.array(dat).T[1]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:43:23.517409Z",
     "start_time": "2022-07-21T07:43:23.464884Z"
    }
   },
   "outputs": [],
   "source": [
    "TfidfVectorizer?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:43:29.362468Z",
     "start_time": "2022-07-21T07:43:29.354119Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "vec = TfidfVectorizer(stop_words='english', ngram_range = (1, 1), lowercase = True)\n",
    "X = vec.fit_transform(X)\n",
    "Xtrain = X[:10]\n",
    "Xtest = X[10:]\n",
    "ytrain = y[:10]\n",
    "ytest= y[10:] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:43:30.202696Z",
     "start_time": "2022-07-21T07:43:30.192802Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['amazing',\n",
       " 'annoying',\n",
       " 'best',\n",
       " 'car',\n",
       " 'concert',\n",
       " 'enemy',\n",
       " 'excited',\n",
       " 'feel',\n",
       " 'forward',\n",
       " 'friend',\n",
       " 'great',\n",
       " 'happy',\n",
       " 'horrible',\n",
       " 'house',\n",
       " 'larry',\n",
       " 'like',\n",
       " 'looking',\n",
       " 'love',\n",
       " 'man',\n",
       " 'morning',\n",
       " 'song',\n",
       " 'tired',\n",
       " 'view']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vec.get_feature_names()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:43:31.047014Z",
     "start_time": "2022-07-21T07:43:31.016634Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>amazing</th>\n",
       "      <th>annoying</th>\n",
       "      <th>best</th>\n",
       "      <th>car</th>\n",
       "      <th>concert</th>\n",
       "      <th>enemy</th>\n",
       "      <th>excited</th>\n",
       "      <th>feel</th>\n",
       "      <th>forward</th>\n",
       "      <th>friend</th>\n",
       "      <th>...</th>\n",
       "      <th>house</th>\n",
       "      <th>larry</th>\n",
       "      <th>like</th>\n",
       "      <th>looking</th>\n",
       "      <th>love</th>\n",
       "      <th>man</th>\n",
       "      <th>morning</th>\n",
       "      <th>song</th>\n",
       "      <th>tired</th>\n",
       "      <th>view</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.655648</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.755067</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.755067</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.655648</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.554219</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.554219</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.655648</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.755067</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.755067</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.655648</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.707107</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.707107</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.655648</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.522329</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.522329</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.67405</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.523243</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.602585</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.602585</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.522329</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.522329</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.655648</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.755067</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.655648</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.755067</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.755067</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.707107</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.707107</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>15 rows × 23 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     amazing  annoying      best       car   concert  enemy   excited  \\\n",
       "0   0.000000  0.000000  0.000000  0.655648  0.000000    0.0  0.000000   \n",
       "1   0.755067  0.000000  0.000000  0.000000  0.000000    0.0  0.000000   \n",
       "2   0.000000  0.000000  0.000000  0.000000  0.000000    0.0  0.000000   \n",
       "3   0.000000  0.000000  0.000000  0.000000  0.655648    0.0  0.755067   \n",
       "4   0.000000  0.000000  0.755067  0.000000  0.000000    0.0  0.000000   \n",
       "5   0.000000  0.000000  0.000000  0.707107  0.000000    0.0  0.000000   \n",
       "6   0.000000  0.000000  0.000000  0.000000  0.000000    0.0  0.000000   \n",
       "7   0.000000  0.000000  0.000000  0.000000  0.000000    0.0  0.000000   \n",
       "8   0.000000  0.000000  0.000000  0.000000  0.523243    0.0  0.000000   \n",
       "9   0.000000  0.000000  0.000000  0.000000  0.000000    1.0  0.000000   \n",
       "10  0.000000  0.000000  0.000000  0.000000  0.000000    0.0  0.000000   \n",
       "11  0.000000  0.000000  0.000000  0.000000  0.000000    0.0  0.000000   \n",
       "12  0.000000  0.000000  0.000000  0.000000  0.000000    0.0  0.000000   \n",
       "13  0.000000  0.000000  0.000000  0.000000  0.000000    0.0  0.000000   \n",
       "14  0.000000  0.707107  0.000000  0.000000  0.000000    0.0  0.000000   \n",
       "\n",
       "        feel   forward    friend  ...     house     larry      like   looking  \\\n",
       "0   0.000000  0.000000  0.000000  ...  0.000000  0.000000  0.000000  0.000000   \n",
       "1   0.000000  0.000000  0.000000  ...  0.000000  0.000000  0.000000  0.000000   \n",
       "2   0.554219  0.000000  0.000000  ...  0.000000  0.000000  0.000000  0.000000   \n",
       "3   0.000000  0.000000  0.000000  ...  0.000000  0.000000  0.000000  0.000000   \n",
       "4   0.000000  0.000000  0.655648  ...  0.000000  0.000000  0.000000  0.000000   \n",
       "5   0.000000  0.000000  0.000000  ...  0.000000  0.000000  0.707107  0.000000   \n",
       "6   0.000000  0.000000  0.000000  ...  0.000000  0.000000  0.000000  0.000000   \n",
       "7   0.522329  0.000000  0.000000  ...  0.000000  0.000000  0.000000  0.000000   \n",
       "8   0.000000  0.602585  0.000000  ...  0.000000  0.000000  0.000000  0.602585   \n",
       "9   0.000000  0.000000  0.000000  ...  0.000000  0.000000  0.000000  0.000000   \n",
       "10  0.522329  0.000000  0.000000  ...  0.000000  0.000000  0.000000  0.000000   \n",
       "11  0.000000  0.000000  0.655648  ...  0.000000  0.755067  0.000000  0.000000   \n",
       "12  0.000000  0.000000  0.000000  ...  0.000000  0.000000  0.655648  0.000000   \n",
       "13  0.000000  0.000000  0.000000  ...  0.755067  0.000000  0.000000  0.000000   \n",
       "14  0.000000  0.000000  0.000000  ...  0.000000  0.000000  0.000000  0.000000   \n",
       "\n",
       "        love       man   morning      song    tired      view  \n",
       "0   0.755067  0.000000  0.000000  0.000000  0.00000  0.000000  \n",
       "1   0.000000  0.000000  0.000000  0.000000  0.00000  0.655648  \n",
       "2   0.000000  0.000000  0.554219  0.000000  0.00000  0.000000  \n",
       "3   0.000000  0.000000  0.000000  0.000000  0.00000  0.000000  \n",
       "4   0.000000  0.000000  0.000000  0.000000  0.00000  0.000000  \n",
       "5   0.000000  0.000000  0.000000  0.000000  0.00000  0.000000  \n",
       "6   0.000000  0.000000  0.000000  0.000000  0.00000  0.655648  \n",
       "7   0.000000  0.000000  0.522329  0.000000  0.67405  0.000000  \n",
       "8   0.000000  0.000000  0.000000  0.000000  0.00000  0.000000  \n",
       "9   0.000000  0.000000  0.000000  0.000000  0.00000  0.000000  \n",
       "10  0.000000  0.000000  0.522329  0.000000  0.00000  0.000000  \n",
       "11  0.000000  0.000000  0.000000  0.000000  0.00000  0.000000  \n",
       "12  0.000000  0.755067  0.000000  0.000000  0.00000  0.000000  \n",
       "13  0.000000  0.000000  0.000000  0.000000  0.00000  0.000000  \n",
       "14  0.000000  0.000000  0.000000  0.707107  0.00000  0.000000  \n",
       "\n",
       "[15 rows x 23 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(X.toarray(), columns=vec.get_feature_names())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:43:31.696807Z",
     "start_time": "2022-07-21T07:43:31.688901Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['positive', 'positive', 'negative', 'positive', 'negative'],\n",
       "      dtype='<U8')"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.naive_bayes import GaussianNB # 1. choose model class\n",
    "model = GaussianNB()                       # 2. instantiate model\n",
    "model.fit(Xtrain.toarray(), ytrain)                  # 3. fit model to data\n",
    "y_model = model.predict(Xtest.toarray())              # 4. predict on new data\n",
    "y_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:43:32.548837Z",
     "start_time": "2022-07-21T07:43:32.521145Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "accuracy_score(ytest, y_model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:43:33.170798Z",
     "start_time": "2022-07-21T07:43:33.111379Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "svc=SVC(kernel='rbf', gamma=1)\n",
    "svc.fit(Xtrain.toarray(), ytrain)  # 3. fit model to data\n",
    "y_model=svc.predict(Xtest.toarray())\n",
    "accuracy_score(ytest, y_model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:43:33.827377Z",
     "start_time": "2022-07-21T07:43:33.823741Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['positive', 'positive', 'negative', 'positive', 'negative'],\n",
       "      dtype='<U39')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:43:34.530918Z",
     "start_time": "2022-07-21T07:43:34.527505Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "y_model=svc.predict(Xtest.toarray())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:44:04.201123Z",
     "start_time": "2022-07-21T07:44:04.193970Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['negative', 'negative'], dtype='<U8')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Don’t be too positive, let’s try another example:\n",
    "vocabulary = vec.get_feature_names()\n",
    "\n",
    "def classify_sentiment(str_list, model, vocabulary):\n",
    "    # str_list = ['a str']\n",
    "    vec_pred = TfidfVectorizer(stop_words='english', ngram_range = (1, 1), lowercase = True, vocabulary = vocabulary)\n",
    "    return model.predict(vec_pred.fit_transform(str_list).toarray())\n",
    "\n",
    "classify_sentiment(['Your song is annoying','larry is horrible'], model, vocabulary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2022-07-21T07:44:15.862066Z",
     "start_time": "2022-07-21T07:44:15.856169Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['negative', 'positive'], dtype='<U39')"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classify_sentiment(['I do not like larry', 'larry is my friend'], svc, vocabulary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 推荐阅读：\n",
    "\n",
    "movies reviews情感分析  http://nbviewer.jupyter.org/github/rasbt/python-machine-learning-book/blob/master/code/ch08/ch08.ipynb\n",
    "\n",
    "Sentiment analysis with machine learning in R http://chengjun.github.io/en/2014/04/sentiment-analysis-with-machine-learning-in-R/\n",
    "\n",
    "使用R包sentiment进行情感分析 https://site.douban.com/146782/widget/notes/15462869/note/344846192/\n",
    "\n",
    "中文的手机评论的情感分析 https://github.com/computational-class/Review-Helpfulness-Prediction\n",
    "\n",
    "基于词典的中文情感倾向分析 https://site.douban.com/146782/widget/notes/15462869/note/355625387/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-04T06:59:04.666933Z",
     "start_time": "2020-06-04T06:57:34.302439Z"
    },
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting textblob\n",
      "  Downloading textblob-0.15.3-py2.py3-none-any.whl (636 kB)\n",
      "\u001b[K     |████████████████████████████████| 636 kB 6.9 kB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: nltk>=3.1 in /opt/anaconda3/lib/python3.7/site-packages (from textblob) (3.4.5)\n",
      "Requirement already satisfied: six in /opt/anaconda3/lib/python3.7/site-packages (from nltk>=3.1->textblob) (1.14.0)\n",
      "Installing collected packages: textblob\n",
      "Successfully installed textblob-0.15.3\n",
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "#  TextBlob\n",
    "\n",
    "https://github.com/sloria/TextBlob\n",
    "    \n",
    "> pip install -U textblob\n",
    "\n",
    "> python -m textblob.download_corpora\n",
    "\n",
    "pip install textblob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-12T08:47:19.379231Z",
     "start_time": "2020-06-12T08:47:19.328752Z"
    },
    "code_folding": [],
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.06000000000000001\n",
      "-0.34166666666666673\n"
     ]
    }
   ],
   "source": [
    "# textblob\n",
    "from textblob import TextBlob\n",
    "\n",
    "text = '''\n",
    "The titular threat of The Blob has always struck me as the ultimate movie\n",
    "monster: an insatiably hungry, amoeba-like mass able to penetrate\n",
    "virtually any safeguard, capable of--as a doomed doctor chillingly\n",
    "describes it--\"assimilating flesh on contact.\n",
    "Snide comparisons to gelatin be damned, it's a concept with the most\n",
    "devastating of potential consequences, not unlike the grey goo scenario\n",
    "proposed by technological theorists fearful of\n",
    "artificial intelligence run rampant.\n",
    "'''\n",
    "\n",
    "blob = TextBlob(text)\n",
    "blob.tags           # [('The', 'DT'), ('titular', 'JJ'),\n",
    "                    #  ('threat', 'NN'), ('of', 'IN'), ...]\n",
    "\n",
    "blob.noun_phrases   # WordList(['titular threat', 'blob',\n",
    "                    #            'ultimate movie monster',\n",
    "                    #            'amoeba-like mass', ...])\n",
    "\n",
    "for sentence in blob.sentences:\n",
    "    print(sentence.sentiment.polarity)\n",
    "# 0.060\n",
    "# -0.341 \n",
    "\n",
    "#blob.translate(to=\"es\")  # 'La amenaza titular de The Blob...'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-12T08:47:24.380804Z",
     "start_time": "2020-06-12T08:47:24.347664Z"
    },
    "code_folding": [
     2
    ],
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I feel happy this morning ： neg\n",
      "Oh I love my friend ： pos\n",
      "not like that man ： neg\n",
      "this house is not great ： neg\n",
      "your song is annoying ： neg\n"
     ]
    }
   ],
   "source": [
    "from textblob.classifiers import NaiveBayesClassifier\n",
    "\n",
    "train=[\n",
    "        ('I love this car','pos'),\n",
    "        ('This view is amazing','pos'),\n",
    "        ('I feel great','pos'),\n",
    "        ('I am so excited about the concert','pos'),\n",
    "        (\"He is my best friend\",'pos'),\n",
    "        ('I do not like this car','neg'),\n",
    "        ('This view is horrible','neg'),\n",
    "        (\"I feel tired this morning\",'neg'),\n",
    "        ('I am not looking forward to the concert','neg'),\n",
    "        ('He is an annoying enemy','neg') \n",
    "]\n",
    "\n",
    "test=[\n",
    "        ('I feel happy this morning','pos'),\n",
    "        ('Oh I love my friend','pos'),\n",
    "        ('not like that man','neg'),\n",
    "        (\"this house is not great\",'neg'),\n",
    "        ('your song is annoying','neg')\n",
    "]\n",
    "\n",
    "cl=NaiveBayesClassifier(train)\n",
    "\n",
    "for sentence in test:\n",
    "    print(sentence[0],'：',cl.classify(sentence[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## SnowNLP\n",
    "\n",
    "https://pypi.org/project/snownlp/\n",
    "\n",
    "https://github.com/isnowfy/snownlp\n",
    "\n",
    "SnowNLP介绍：是一个python写的类库，可以方便的处理中文文本内容，是受到了TextBlob的启发而写的，由于现在大部分的自然语言处理库基本都是针对英文的，于是写了一个方便处理中文的类库，并且和TextBlob不同的是，这里没有用NLTK，所有的算法都是自己实现的，并且自带了一些训练好的字典。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Turicreate\n",
    "\n",
    "https://github.com/apple/turicreate\n",
    "\n",
    "<div><img src=\"img/txt4.png\" align = \"right\"></div>\n",
    "Turi Create simplifies the development of custom machine learning models. You don't have to be a machine learning expert to add recommendations, object detection, image classification, image similarity or activity classification to your app.\n",
    "\n",
    "\n",
    "https://apple.github.io/turicreate/docs/userguide/text_classifier/\n",
    "\n",
    "https://www.kaggle.com/prakharrathi25/updated-turicreate-sentiment-analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## PaddlePaddle\n",
    " \n",
    "<div><img src=\"img/txt5.png\" align=\"right\"></div>  飞桨（PaddlePaddle）以百度多年的深度学习技术研究和业务应用为基础，集深度学习核心框架、基础模型库、端到端开发套件、工具组件和服务平台于一体，2016 年正式开源，是全面开源开放、技术领先、功能完备的产业级深度学习平台。 \n",
    "\n",
    "http://paddlepaddle.org\n",
    "\n",
    "https://github.com/PaddlePaddle/book/tree/develop/06.understand_sentiment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![image.png](img/chengjun2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Creating Sentiment Classifier with Turicreate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "In this notebook, I will explain how to develop sentiment analysis classifiers that are based on a bag-of-words model. \n",
    "Then, I will demonstrate how these classifiers can be utilized to solve Kaggle's \"When Bag of Words Meets Bags of Popcorn\" challenge."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Using <del>GraphLab</del> Turicreate it is very easy and straight foward to create a sentiment classifier based on bag-of-words model. Given a dataset stored as a CSV file, you can construct your sentiment classifier using the following code: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# toy code, do not run it\n",
    "import turicreate as tc\n",
    "train_data = tc.SFrame.read_csv(traindata_path,header=True, \n",
    "                                delimiter='\\t',quote_char='\"', \n",
    "                                column_type_hints = {'id':str, \n",
    "                                                     'sentiment' : int, \n",
    "                                                     'review':str } )\n",
    "train_data['1grams features'] = tc.text_analytics.count_ngrams(\n",
    "    train_data['review'],1)\n",
    "train_data['2grams features'] = tc.text_analytics.count_ngrams(\n",
    "    train_data['review'],2)\n",
    "cls = tc.classifier.create(train_data, target='sentiment', \n",
    "                           features=['1grams features',\n",
    "                                     '2grams features'])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "In the rest of this notebook, we will explain this code recipe in details, by demonstrating how this recipe can used to create IMDB movie reviews sentiment classifier."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Before we begin constructing the classifiers, we need to import some Python libraries: turicreate (tc), and IPython display utilities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:50:16.291727Z",
     "start_time": "2019-06-14T16:50:15.367322Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "import turicreate as tc\n",
    "from IPython.display import display\n",
    "from IPython.display import Image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### IMDB movies reviews Dataset \n",
    "\n",
    "> Bag of Words Meets Bags of Popcorn\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Throughout this notebook, I will use Kaggle's IMDB movies reviews datasets that is available to download from the following link: https://www.kaggle.com/c/word2vec-nlp-tutorial/data. I downloaded labeledTrainData.tsv and testData.tsv files, and unzipped them to the following local files.\n",
    "\n",
    "###  DeepLearningMovies\n",
    "\n",
    "Kaggle's competition for using Google's word2vec package for sentiment analysis\n",
    "\n",
    "https://github.com/wendykan/DeepLearningMovies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:50:47.697332Z",
     "start_time": "2019-06-14T16:50:47.694256Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": [
    "traindata_path = \"/Users/datalab/bigdata/cjc/kaggle_popcorn_data/labeledTrainData.tsv\"\n",
    "testdata_path = \"/Users/datalab/bigdata/cjc/kaggle_popcorn_data/testData.tsv\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "## Loading Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We will load the data with IMDB movie reviews to an SFrame using SFrame.read_csv function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:50:50.443338Z",
     "start_time": "2019-06-14T16:50:49.571767Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre>Finished parsing file /Users/datalab/bigdata/cjc/kaggle_popcorn_data/labeledTrainData.tsv</pre>"
      ],
      "text/plain": [
       "Finished parsing file /Users/datalab/bigdata/cjc/kaggle_popcorn_data/labeledTrainData.tsv"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Parsing completed. Parsed 100 lines in 0.318532 secs.</pre>"
      ],
      "text/plain": [
       "Parsing completed. Parsed 100 lines in 0.318532 secs."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Finished parsing file /Users/datalab/bigdata/cjc/kaggle_popcorn_data/labeledTrainData.tsv</pre>"
      ],
      "text/plain": [
       "Finished parsing file /Users/datalab/bigdata/cjc/kaggle_popcorn_data/labeledTrainData.tsv"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Parsing completed. Parsed 25000 lines in 0.499892 secs.</pre>"
      ],
      "text/plain": [
       "Parsing completed. Parsed 25000 lines in 0.499892 secs."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "movies_reviews_data = tc.SFrame.read_csv(traindata_path,header=True, \n",
    "                                         delimiter='\\t',quote_char='\"', \n",
    "                                         column_type_hints = {'id':str, \n",
    "                                                              'sentiment' : str, \n",
    "                                                              'review':str } )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "By using the SFrame show function, we can visualize the data and notice that the train dataset consists of 12,500 positive and 12,500 negative, and overall 24,932 unique reviews."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:50:55.267343Z",
     "start_time": "2019-06-14T16:50:55.212701Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">id</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sentiment</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">review</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5814_8</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">With all this stuff going<br>down at the moment with ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2381_9</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">&quot;The Classic War of the<br>Worlds&quot; by Timothy Hines ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7759_3</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">The film starts with a<br>manager (Nicholas Bell) ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3630_4</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">It must be assumed that<br>those who praised this ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">9495_8</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Superbly trashy and<br>wondrously unpretentious ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">8196_8</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">I dont know why people<br>think this is such a bad ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7166_2</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">This movie could have<br>been very good, but c ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">10633_1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">I watched this video at a<br>friend&#x27;s house. I&#x27;m glad ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">319_1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">A friend of mine bought<br>this film for £1, and ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">8713_10</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">&lt;br /&gt;&lt;br /&gt;This movie is<br>full of references. Like ...</td>\n",
       "    </tr>\n",
       "</table>\n",
       "[25000 rows x 3 columns]<br/>Note: Only the head of the SFrame is printed.<br/>You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.\n",
       "</div>"
      ],
      "text/plain": [
       "Columns:\n",
       "\tid\tstr\n",
       "\tsentiment\tstr\n",
       "\treview\tstr\n",
       "\n",
       "Rows: 25000\n",
       "\n",
       "Data:\n",
       "+---------+-----------+-------------------------------+\n",
       "|    id   | sentiment |             review            |\n",
       "+---------+-----------+-------------------------------+\n",
       "|  5814_8 |     1     | With all this stuff going ... |\n",
       "|  2381_9 |     1     | \"The Classic War of the Wo... |\n",
       "|  7759_3 |     0     | The film starts with a man... |\n",
       "|  3630_4 |     0     | It must be assumed that th... |\n",
       "|  9495_8 |     1     | Superbly trashy and wondro... |\n",
       "|  8196_8 |     1     | I dont know why people thi... |\n",
       "|  7166_2 |     0     | This movie could have been... |\n",
       "| 10633_1 |     0     | I watched this video at a ... |\n",
       "|  319_1  |     0     | A friend of mine bought th... |\n",
       "| 8713_10 |     1     | <br /><br />This movie is ... |\n",
       "+---------+-----------+-------------------------------+\n",
       "[25000 rows x 3 columns]\n",
       "Note: Only the head of the SFrame is printed.\n",
       "You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns."
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_reviews_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Constructing Bag-of-Words Classifier "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "One of the common techniques to perform document classification (and reviews classification) is using Bag-of-Words model, in which the frequency of each word in the document is used as a feature for training a classifier. GraphLab's text analytics toolkit makes it easy to calculate the frequency of each word in each review. Namely, by using the count_ngrams function with n=1, we can calculate the frequency of each word in each review. By running the following command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:51:02.529329Z",
     "start_time": "2019-06-14T16:51:02.517257Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "movies_reviews_data['1grams features'] = tc.text_analytics.count_ngrams(movies_reviews_data ['review'],1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "By running the last command, we created a new column in movies_reviews_data SFrame object. In this column each value is a dictionary object, where each dictionary's keys are the different words which appear in the corresponding review, and the dictionary's values are the frequency of each word.\n",
    "We can view the values of this new column using the following command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:51:05.248555Z",
     "start_time": "2019-06-14T16:51:05.116080Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">id</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sentiment</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">review</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">1grams features</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5814_8</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">With all this stuff going<br>down at the moment with ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;just&#x27;: 3, &#x27;sickest&#x27;: 1,<br>&#x27;smooth&#x27;: 1, &#x27;this&#x27;: 11, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2381_9</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">&quot;The Classic War of the<br>Worlds&quot; by Timothy Hines ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;year&#x27;: 1, &#x27;others&#x27;: 1,<br>&#x27;those&#x27;: 2, &#x27;this&#x27;: 1, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7759_3</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">The film starts with a<br>manager (Nicholas Bell) ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;hair&#x27;: 1, &#x27;bound&#x27;: 1,<br>&#x27;this&#x27;: 1, &#x27;when&#x27;: 2, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3630_4</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">It must be assumed that<br>those who praised this ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;crocuses&#x27;: 1, &#x27;that&#x27;:<br>7, &#x27;batonzilla&#x27;: 1, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">9495_8</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Superbly trashy and<br>wondrously unpretentious ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;unshaven&#x27;: 1, &#x27;just&#x27;:<br>1, &#x27;in&#x27;: 5, &#x27;when&#x27;: 2, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">8196_8</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">I dont know why people<br>think this is such a bad ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;harry&#x27;: 3, &#x27;this&#x27;: 4,<br>&#x27;of&#x27;: 2, &#x27;hurt&#x27;: 1, &#x27; ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7166_2</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">This movie could have<br>been very good, but c ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;acting&#x27;: 1,<br>&#x27;background&#x27;: 1, &#x27;just&#x27;: ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">10633_1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">I watched this video at a<br>friend&#x27;s house. I&#x27;m glad ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;photography&#x27;: 1,<br>&#x27;others&#x27;: 1, &#x27;zapruder&#x27;: ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">319_1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">A friend of mine bought<br>this film for £1, and ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;just&#x27;: 1, &#x27;this&#x27;: 2,<br>&#x27;when&#x27;: 1, &#x27;as&#x27;: 5, &#x27;s&#x27;: ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">8713_10</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">&lt;br /&gt;&lt;br /&gt;This movie is<br>full of references. Like ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;peter&#x27;: 1, &#x27;ii&#x27;: 1,<br>&#x27;full&#x27;: 1, &#x27;others&#x27;: 1, ...</td>\n",
       "    </tr>\n",
       "</table>\n",
       "[25000 rows x 4 columns]<br/>Note: Only the head of the SFrame is printed.<br/>You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.\n",
       "</div>"
      ],
      "text/plain": [
       "Columns:\n",
       "\tid\tstr\n",
       "\tsentiment\tstr\n",
       "\treview\tstr\n",
       "\t1grams features\tdict\n",
       "\n",
       "Rows: 25000\n",
       "\n",
       "Data:\n",
       "+---------+-----------+-------------------------------+\n",
       "|    id   | sentiment |             review            |\n",
       "+---------+-----------+-------------------------------+\n",
       "|  5814_8 |     1     | With all this stuff going ... |\n",
       "|  2381_9 |     1     | \"The Classic War of the Wo... |\n",
       "|  7759_3 |     0     | The film starts with a man... |\n",
       "|  3630_4 |     0     | It must be assumed that th... |\n",
       "|  9495_8 |     1     | Superbly trashy and wondro... |\n",
       "|  8196_8 |     1     | I dont know why people thi... |\n",
       "|  7166_2 |     0     | This movie could have been... |\n",
       "| 10633_1 |     0     | I watched this video at a ... |\n",
       "|  319_1  |     0     | A friend of mine bought th... |\n",
       "| 8713_10 |     1     | <br /><br />This movie is ... |\n",
       "+---------+-----------+-------------------------------+\n",
       "+-------------------------------+\n",
       "|        1grams features        |\n",
       "+-------------------------------+\n",
       "| {'just': 3, 'sickest': 1, ... |\n",
       "| {'year': 1, 'others': 1, '... |\n",
       "| {'hair': 1, 'bound': 1, 't... |\n",
       "| {'crocuses': 1, 'that': 7,... |\n",
       "| {'unshaven': 1, 'just': 1,... |\n",
       "| {'harry': 3, 'this': 4, 'o... |\n",
       "| {'acting': 1, 'background'... |\n",
       "| {'photography': 1, 'others... |\n",
       "| {'just': 1, 'this': 2, 'wh... |\n",
       "| {'peter': 1, 'ii': 1, 'ful... |\n",
       "+-------------------------------+\n",
       "[25000 rows x 4 columns]\n",
       "Note: Only the head of the SFrame is printed.\n",
       "You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns."
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_reviews_data#[['review','1grams features']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We are now ready to construct and evaluate the movie reviews sentiment classifier using the calculated above features. But first, to be able to perform a quick evaluation of the constructed classifier, we need to create labeled train and test datasets. We will create train and test datasets by randomly splitting the train dataset into two parts. The first part will contain 80% of the labeled train dataset and will be used as the training dataset, while the second part will contain 20% of the labeled train dataset and will be used as the testing dataset. We will create these two dataset by using the following command:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:51:09.958230Z",
     "start_time": "2019-06-14T16:51:09.954844Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "train_set, test_set = movies_reviews_data.random_split(0.8, seed=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We are now ready to create a classifier using the following command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:51:30.848861Z",
     "start_time": "2019-06-14T16:51:16.305109Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
      "          You can set ``validation_set=None`` to disable validation tracking.\n",
      "\n",
      "PROGRESS: The following methods are available for this type of problem.\n",
      "PROGRESS: LogisticClassifier, SVMClassifier\n",
      "PROGRESS: The returned model will be chosen according to validation accuracy.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre>Logistic regression:</pre>"
      ],
      "text/plain": [
       "Logistic regression:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 19077</pre>"
      ],
      "text/plain": [
       "Number of examples          : 19077"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of classes           : 2</pre>"
      ],
      "text/plain": [
       "Number of classes           : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of feature columns   : 1</pre>"
      ],
      "text/plain": [
       "Number of feature columns   : 1"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 68246</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 68246"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients      : 68247</pre>"
      ],
      "text/plain": [
       "Number of coefficients      : 68247"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting L-BFGS</pre>"
      ],
      "text/plain": [
       "Starting L-BFGS"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Step size | Elapsed Time | Training Accuracy | Validation Accuracy |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Step size | Elapsed Time | Training Accuracy | Validation Accuracy |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 0         | 2        | 1.000000  | 1.111660     | 0.942182          | 0.860697            |</pre>"
      ],
      "text/plain": [
       "| 0         | 2        | 1.000000  | 1.111660     | 0.942182          | 0.860697            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 4        | 1.000000  | 1.253890     | 0.968444          | 0.865672            |</pre>"
      ],
      "text/plain": [
       "| 1         | 4        | 1.000000  | 1.253890     | 0.968444          | 0.865672            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 2         | 6        | 1.000000  | 1.390344     | 0.990040          | 0.897512            |</pre>"
      ],
      "text/plain": [
       "| 2         | 6        | 1.000000  | 1.390344     | 0.990040          | 0.897512            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 3         | 7        | 1.000000  | 1.474481     | 0.992923          | 0.899502            |</pre>"
      ],
      "text/plain": [
       "| 3         | 7        | 1.000000  | 1.474481     | 0.992923          | 0.899502            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 4         | 8        | 1.000000  | 1.563669     | 0.997379          | 0.891542            |</pre>"
      ],
      "text/plain": [
       "| 4         | 8        | 1.000000  | 1.563669     | 0.997379          | 0.891542            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 9         | 13       | 1.000000  | 2.052863     | 1.000000          | 0.867662            |</pre>"
      ],
      "text/plain": [
       "| 9         | 13       | 1.000000  | 2.052863     | 1.000000          | 0.867662            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>SVM:</pre>"
      ],
      "text/plain": [
       "SVM:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 19077</pre>"
      ],
      "text/plain": [
       "Number of examples          : 19077"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of classes           : 2</pre>"
      ],
      "text/plain": [
       "Number of classes           : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of feature columns   : 1</pre>"
      ],
      "text/plain": [
       "Number of feature columns   : 1"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 68246</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 68246"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients    : 68247</pre>"
      ],
      "text/plain": [
       "Number of coefficients    : 68247"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting L-BFGS</pre>"
      ],
      "text/plain": [
       "Starting L-BFGS"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Step size | Elapsed Time | Training Accuracy | Validation Accuracy |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Step size | Elapsed Time | Training Accuracy | Validation Accuracy |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 0         | 2        | 1.000000  | 0.125585     | 0.942182          | 0.860697            |</pre>"
      ],
      "text/plain": [
       "| 0         | 2        | 1.000000  | 0.125585     | 0.942182          | 0.860697            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 4        | 1.000000  | 0.268260     | 0.973738          | 0.875622            |</pre>"
      ],
      "text/plain": [
       "| 1         | 4        | 1.000000  | 0.268260     | 0.973738          | 0.875622            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 2         | 5        | 1.000000  | 0.348993     | 0.989411          | 0.881592            |</pre>"
      ],
      "text/plain": [
       "| 2         | 5        | 1.000000  | 0.348993     | 0.989411          | 0.881592            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 3         | 6        | 1.000000  | 0.433099     | 0.992976          | 0.884577            |</pre>"
      ],
      "text/plain": [
       "| 3         | 6        | 1.000000  | 0.433099     | 0.992976          | 0.884577            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 4         | 7        | 1.000000  | 0.519594     | 0.996016          | 0.881592            |</pre>"
      ],
      "text/plain": [
       "| 4         | 7        | 1.000000  | 0.519594     | 0.996016          | 0.881592            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 9         | 12       | 1.000000  | 0.923521     | 0.999685          | 0.886567            |</pre>"
      ],
      "text/plain": [
       "| 9         | 12       | 1.000000  | 0.923521     | 0.999685          | 0.886567            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PROGRESS: Model selection based on validation accuracy:\n",
      "PROGRESS: ---------------------------------------------\n",
      "PROGRESS: LogisticClassifier              : 0.8676616915422886\n",
      "PROGRESS: SVMClassifier                   : 0.8865671641791045\n",
      "PROGRESS: ---------------------------------------------\n",
      "PROGRESS: Selecting SVMClassifier based on validation set performance.\n"
     ]
    }
   ],
   "source": [
    "model_1 = tc.classifier.create(train_set, target='sentiment', \\\n",
    "                               features=['1grams features'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We can evaluate the performence of the classifier by evaluating it on the test dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:51:55.289534Z",
     "start_time": "2019-06-14T16:51:54.504129Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "result1 = model_1.evaluate(test_set)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "In order to get an easy view of the classifier's prediction result, we define and use the following function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:51:56.666544Z",
     "start_time": "2019-06-14T16:51:56.656636Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "******************************\n",
      "Accuracy        :  0.8710858072387149\n",
      "Confusion Matrix: \n",
      " +--------------+-----------------+-------+\n",
      "| target_label | predicted_label | count |\n",
      "+--------------+-----------------+-------+\n",
      "|      0       |        1        |  374  |\n",
      "|      1       |        0        |  260  |\n",
      "|      1       |        1        |  2133 |\n",
      "|      0       |        0        |  2151 |\n",
      "+--------------+-----------------+-------+\n",
      "[4 rows x 3 columns]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "def print_statistics(result):\n",
    "    print( \"*\" * 30)\n",
    "    print( \"Accuracy        : \", result[\"accuracy\"])\n",
    "    print( \"Confusion Matrix: \\n\", result[\"confusion_matrix\"])\n",
    "print_statistics(result1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "As can be seen in the results above, in just a few relatively straight foward lines of code, we have developed a sentiment classifier that has accuracy of about ~0.88. Next, we demonstrate how we can improve the classifier accuracy even more."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Improving The Classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "One way to improve the movie reviews sentiment classifier is to extract more meaningful features from the reviews. One method to add additional features, which might be meaningful, is to calculate the frequency of every two consecutive words in each review. To calculate the frequency of each two consecutive words in each review, as before, we will use turicreate's count_ngrams function only this time we will set n to be equal 2 (n=2) to create new column named '2grams features'.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:52:19.472533Z",
     "start_time": "2019-06-14T16:52:19.463443Z"
    },
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "movies_reviews_data['2grams features'] = tc.text_analytics.count_ngrams(movies_reviews_data['review'],2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:52:20.971123Z",
     "start_time": "2019-06-14T16:52:20.734798Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div style=\"max-height:1000px;max-width:1500px;overflow:auto;\"><table frame=\"box\" rules=\"cols\">\n",
       "    <tr>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">id</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">sentiment</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">review</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">1grams features</th>\n",
       "        <th style=\"padding-left: 1em; padding-right: 1em; text-align: center\">2grams features</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">5814_8</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">With all this stuff going<br>down at the moment with ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;just&#x27;: 3, &#x27;sickest&#x27;: 1,<br>&#x27;smooth&#x27;: 1, &#x27;this&#x27;: 11, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;alone a&#x27;: 1, &#x27;most<br>people&#x27;: 1, &#x27;hope he&#x27; ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">2381_9</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">&quot;The Classic War of the<br>Worlds&quot; by Timothy Hines ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;year&#x27;: 1, &#x27;others&#x27;: 1,<br>&#x27;those&#x27;: 2, &#x27;this&#x27;: 1, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;slightest resemblance&#x27;:<br>1, &#x27;which is&#x27;: 1, &#x27;very ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7759_3</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">The film starts with a<br>manager (Nicholas Bell) ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;hair&#x27;: 1, &#x27;bound&#x27;: 1,<br>&#x27;this&#x27;: 1, &#x27;when&#x27;: 2, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;quite boring&#x27;: 1,<br>&#x27;packs a&#x27;: 1, &#x27;small ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">3630_4</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">It must be assumed that<br>those who praised this ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;crocuses&#x27;: 1, &#x27;that&#x27;:<br>7, &#x27;batonzilla&#x27;: 1, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;but i&#x27;: 1, &#x27;is<br>represented&#x27;: 1, &#x27;opera ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">9495_8</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">Superbly trashy and<br>wondrously unpretentious ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;unshaven&#x27;: 1, &#x27;just&#x27;:<br>1, &#x27;in&#x27;: 5, &#x27;when&#x27;: 2, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;unpretentious 80&#x27;: 1,<br>&#x27;sleazy black&#x27;: 1, &#x27;d ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">8196_8</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">I dont know why people<br>think this is such a bad ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;harry&#x27;: 3, &#x27;this&#x27;: 4,<br>&#x27;of&#x27;: 2, &#x27;hurt&#x27;: 1, &#x27; ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;like that&#x27;: 1, &#x27;see<br>this&#x27;: 1, &#x27;is such&#x27;: 1, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">7166_2</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">This movie could have<br>been very good, but c ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;acting&#x27;: 1,<br>&#x27;background&#x27;: 1, &#x27;just&#x27;: ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;linked to&#x27;: 1, &#x27;way<br>short&#x27;: 1, &#x27;good but&#x27; ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">10633_1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">I watched this video at a<br>friend&#x27;s house. I&#x27;m glad ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;photography&#x27;: 1,<br>&#x27;others&#x27;: 1, &#x27;zapruder&#x27;: ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;curiously ends&#x27;: 1,<br>&#x27;several clips&#x27;: 1, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">319_1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">0</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">A friend of mine bought<br>this film for £1, and ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;just&#x27;: 1, &#x27;this&#x27;: 2,<br>&#x27;when&#x27;: 1, &#x27;as&#x27;: 5, &#x27;s&#x27;: ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;bob thornton&#x27;: 1, &#x27;in<br>the&#x27;: 1, &#x27;taking a&#x27;: 1, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">8713_10</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">1</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">&lt;br /&gt;&lt;br /&gt;This movie is<br>full of references. Like ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;peter&#x27;: 1, &#x27;ii&#x27;: 1,<br>&#x27;full&#x27;: 1, &#x27;others&#x27;: 1, ...</td>\n",
       "        <td style=\"padding-left: 1em; padding-right: 1em; text-align: center; vertical-align: top\">{&#x27;in the&#x27;: 1, &#x27;is a&#x27;: 1,<br>&#x27;lorre this&#x27;: 1, &#x27;much ...</td>\n",
       "    </tr>\n",
       "</table>\n",
       "[25000 rows x 5 columns]<br/>Note: Only the head of the SFrame is printed.<br/>You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns.\n",
       "</div>"
      ],
      "text/plain": [
       "Columns:\n",
       "\tid\tstr\n",
       "\tsentiment\tstr\n",
       "\treview\tstr\n",
       "\t1grams features\tdict\n",
       "\t2grams features\tdict\n",
       "\n",
       "Rows: 25000\n",
       "\n",
       "Data:\n",
       "+---------+-----------+-------------------------------+\n",
       "|    id   | sentiment |             review            |\n",
       "+---------+-----------+-------------------------------+\n",
       "|  5814_8 |     1     | With all this stuff going ... |\n",
       "|  2381_9 |     1     | \"The Classic War of the Wo... |\n",
       "|  7759_3 |     0     | The film starts with a man... |\n",
       "|  3630_4 |     0     | It must be assumed that th... |\n",
       "|  9495_8 |     1     | Superbly trashy and wondro... |\n",
       "|  8196_8 |     1     | I dont know why people thi... |\n",
       "|  7166_2 |     0     | This movie could have been... |\n",
       "| 10633_1 |     0     | I watched this video at a ... |\n",
       "|  319_1  |     0     | A friend of mine bought th... |\n",
       "| 8713_10 |     1     | <br /><br />This movie is ... |\n",
       "+---------+-----------+-------------------------------+\n",
       "+-------------------------------+-------------------------------+\n",
       "|        1grams features        |        2grams features        |\n",
       "+-------------------------------+-------------------------------+\n",
       "| {'just': 3, 'sickest': 1, ... | {'alone a': 1, 'most peopl... |\n",
       "| {'year': 1, 'others': 1, '... | {'slightest resemblance': ... |\n",
       "| {'hair': 1, 'bound': 1, 't... | {'quite boring': 1, 'packs... |\n",
       "| {'crocuses': 1, 'that': 7,... | {'but i': 1, 'is represent... |\n",
       "| {'unshaven': 1, 'just': 1,... | {'unpretentious 80': 1, 's... |\n",
       "| {'harry': 3, 'this': 4, 'o... | {'like that': 1, 'see this... |\n",
       "| {'acting': 1, 'background'... | {'linked to': 1, 'way shor... |\n",
       "| {'photography': 1, 'others... | {'curiously ends': 1, 'sev... |\n",
       "| {'just': 1, 'this': 2, 'wh... | {'bob thornton': 1, 'in th... |\n",
       "| {'peter': 1, 'ii': 1, 'ful... | {'in the': 1, 'is a': 1, '... |\n",
       "+-------------------------------+-------------------------------+\n",
       "[25000 rows x 5 columns]\n",
       "Note: Only the head of the SFrame is printed.\n",
       "You can use print_rows(num_rows=m, num_columns=n) to print more rows and columns."
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies_reviews_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "As before, we will construct and evaluate a movie reviews sentiment classifier. However, this time we will use both the '1grams features' and the '2grams features' features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:53:14.076698Z",
     "start_time": "2019-06-14T16:52:28.001732Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
      "          You can set ``validation_set=None`` to disable validation tracking.\n",
      "\n",
      "PROGRESS: The following methods are available for this type of problem.\n",
      "PROGRESS: LogisticClassifier, SVMClassifier\n",
      "PROGRESS: The returned model will be chosen according to validation accuracy.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre>Logistic regression:</pre>"
      ],
      "text/plain": [
       "Logistic regression:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 19077</pre>"
      ],
      "text/plain": [
       "Number of examples          : 19077"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of classes           : 2</pre>"
      ],
      "text/plain": [
       "Number of classes           : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of feature columns   : 2</pre>"
      ],
      "text/plain": [
       "Number of feature columns   : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 1206694</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 1206694"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients      : 1206695</pre>"
      ],
      "text/plain": [
       "Number of coefficients      : 1206695"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting L-BFGS</pre>"
      ],
      "text/plain": [
       "Starting L-BFGS"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Step size | Elapsed Time | Training Accuracy | Validation Accuracy |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Step size | Elapsed Time | Training Accuracy | Validation Accuracy |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 0         | 3        | 0.500000  | 0.884358     | 0.999266          | 0.866667            |</pre>"
      ],
      "text/plain": [
       "| 0         | 3        | 0.500000  | 0.884358     | 0.999266          | 0.866667            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 5        | 0.500000  | 1.542838     | 0.999948          | 0.866667            |</pre>"
      ],
      "text/plain": [
       "| 1         | 5        | 0.500000  | 1.542838     | 0.999948          | 0.866667            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 2         | 6        | 0.625000  | 1.909261     | 1.000000          | 0.865672            |</pre>"
      ],
      "text/plain": [
       "| 2         | 6        | 0.625000  | 1.909261     | 1.000000          | 0.865672            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 3         | 8        | 0.625000  | 2.436618     | 1.000000          | 0.864677            |</pre>"
      ],
      "text/plain": [
       "| 3         | 8        | 0.625000  | 2.436618     | 1.000000          | 0.864677            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 4         | 10       | 0.625000  | 2.971373     | 1.000000          | 0.863682            |</pre>"
      ],
      "text/plain": [
       "| 4         | 10       | 0.625000  | 2.971373     | 1.000000          | 0.863682            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 9         | 18       | 0.976563  | 5.228981     | 1.000000          | 0.862687            |</pre>"
      ],
      "text/plain": [
       "| 9         | 18       | 0.976563  | 5.228981     | 1.000000          | 0.862687            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>SVM:</pre>"
      ],
      "text/plain": [
       "SVM:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 19077</pre>"
      ],
      "text/plain": [
       "Number of examples          : 19077"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of classes           : 2</pre>"
      ],
      "text/plain": [
       "Number of classes           : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of feature columns   : 2</pre>"
      ],
      "text/plain": [
       "Number of feature columns   : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 1206694</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 1206694"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients    : 1206695</pre>"
      ],
      "text/plain": [
       "Number of coefficients    : 1206695"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting L-BFGS</pre>"
      ],
      "text/plain": [
       "Starting L-BFGS"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Step size | Elapsed Time | Training Accuracy | Validation Accuracy |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Step size | Elapsed Time | Training Accuracy | Validation Accuracy |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 0         | 2        | 1.000000  | 0.710178     | 0.999266          | 0.866667            |</pre>"
      ],
      "text/plain": [
       "| 0         | 2        | 1.000000  | 0.710178     | 0.999266          | 0.866667            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 4        | 1.000000  | 1.227603     | 1.000000          | 0.865672            |</pre>"
      ],
      "text/plain": [
       "| 1         | 4        | 1.000000  | 1.227603     | 1.000000          | 0.865672            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 2         | 5        | 1.000000  | 1.524246     | 1.000000          | 0.865672            |</pre>"
      ],
      "text/plain": [
       "| 2         | 5        | 1.000000  | 1.524246     | 1.000000          | 0.865672            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 3         | 6        | 1.000000  | 1.824261     | 1.000000          | 0.865672            |</pre>"
      ],
      "text/plain": [
       "| 3         | 6        | 1.000000  | 1.824261     | 1.000000          | 0.865672            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 4         | 13       | 0.001263  | 3.080125     | 1.000000          | 0.865672            |</pre>"
      ],
      "text/plain": [
       "| 4         | 13       | 0.001263  | 3.080125     | 1.000000          | 0.865672            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 9         | 26       | 0.262737  | 6.006328     | 1.000000          | 0.865672            |</pre>"
      ],
      "text/plain": [
       "| 9         | 26       | 0.262737  | 6.006328     | 1.000000          | 0.865672            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PROGRESS: Model selection based on validation accuracy:\n",
      "PROGRESS: ---------------------------------------------\n",
      "PROGRESS: LogisticClassifier              : 0.8626865671641791\n",
      "PROGRESS: SVMClassifier                   : 0.8656716417910447\n",
      "PROGRESS: ---------------------------------------------\n",
      "PROGRESS: Selecting SVMClassifier based on validation set performance.\n"
     ]
    }
   ],
   "source": [
    "train_set, test_set = movies_reviews_data.random_split(0.8, seed=5)\n",
    "model_2 = tc.classifier.create(train_set, target='sentiment', features=['1grams features','2grams features'])\n",
    "result2 = model_2.evaluate(test_set)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:53:48.981670Z",
     "start_time": "2019-06-14T16:53:48.974028Z"
    },
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "******************************\n",
      "Accuracy        :  0.8816592110614071\n",
      "Confusion Matrix: \n",
      " +--------------+-----------------+-------+\n",
      "| target_label | predicted_label | count |\n",
      "+--------------+-----------------+-------+\n",
      "|      0       |        1        |  343  |\n",
      "|      1       |        0        |  239  |\n",
      "|      1       |        1        |  2154 |\n",
      "|      0       |        0        |  2182 |\n",
      "+--------------+-----------------+-------+\n",
      "[4 rows x 3 columns]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print_statistics(result2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Indeed, the new constructed classifier seems to be more accurate with an accuracy of about ~0.9."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "## Unlabeled Test File"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "To test how well the presented method works, we will use all the 25,000 labeled IMDB movie reviews in the train dataset to construct a classifier. Afterwards, we will utilize the constructed classifier to predict sentiment for each review in the unlabeled dataset. Lastly, we will create a submission file according to Kaggle's guidelines and submit it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2019-06-14T16:55:06.281150Z",
     "start_time": "2019-06-14T16:54:02.968826Z"
    },
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre>Finished parsing file /Users/datalab/bigdata/cjc/kaggle_popcorn_data/labeledTrainData.tsv</pre>"
      ],
      "text/plain": [
       "Finished parsing file /Users/datalab/bigdata/cjc/kaggle_popcorn_data/labeledTrainData.tsv"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Parsing completed. Parsed 100 lines in 0.282738 secs.</pre>"
      ],
      "text/plain": [
       "Parsing completed. Parsed 100 lines in 0.282738 secs."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Finished parsing file /Users/datalab/bigdata/cjc/kaggle_popcorn_data/labeledTrainData.tsv</pre>"
      ],
      "text/plain": [
       "Finished parsing file /Users/datalab/bigdata/cjc/kaggle_popcorn_data/labeledTrainData.tsv"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Parsing completed. Parsed 25000 lines in 0.507212 secs.</pre>"
      ],
      "text/plain": [
       "Parsing completed. Parsed 25000 lines in 0.507212 secs."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PROGRESS: Creating a validation set from 5 percent of training data. This may take a while.\n",
      "          You can set ``validation_set=None`` to disable validation tracking.\n",
      "\n",
      "PROGRESS: The following methods are available for this type of problem.\n",
      "PROGRESS: LogisticClassifier, SVMClassifier\n",
      "PROGRESS: The returned model will be chosen according to validation accuracy.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre>Logistic regression:</pre>"
      ],
      "text/plain": [
       "Logistic regression:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 23750</pre>"
      ],
      "text/plain": [
       "Number of examples          : 23750"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of classes           : 2</pre>"
      ],
      "text/plain": [
       "Number of classes           : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of feature columns   : 2</pre>"
      ],
      "text/plain": [
       "Number of feature columns   : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 1407914</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 1407914"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients      : 1407915</pre>"
      ],
      "text/plain": [
       "Number of coefficients      : 1407915"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting L-BFGS</pre>"
      ],
      "text/plain": [
       "Starting L-BFGS"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Step size | Elapsed Time | Training Accuracy | Validation Accuracy |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Step size | Elapsed Time | Training Accuracy | Validation Accuracy |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 0         | 2        | 1.000000  | 0.772874     | 0.998821          | 0.896000            |</pre>"
      ],
      "text/plain": [
       "| 0         | 2        | 1.000000  | 0.772874     | 0.998821          | 0.896000            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 4        | 1.000000  | 1.443709     | 0.999916          | 0.894400            |</pre>"
      ],
      "text/plain": [
       "| 1         | 4        | 1.000000  | 1.443709     | 0.999916          | 0.894400            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 2         | 6        | 0.648072  | 2.077022     | 0.999958          | 0.895200            |</pre>"
      ],
      "text/plain": [
       "| 2         | 6        | 0.648072  | 2.077022     | 0.999958          | 0.895200            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 3         | 8        | 0.648072  | 2.769055     | 0.999958          | 0.894400            |</pre>"
      ],
      "text/plain": [
       "| 3         | 8        | 0.648072  | 2.769055     | 0.999958          | 0.894400            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 4         | 10       | 0.648072  | 3.420360     | 0.999958          | 0.894400            |</pre>"
      ],
      "text/plain": [
       "| 4         | 10       | 0.648072  | 3.420360     | 0.999958          | 0.894400            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 9         | 22       | 0.486054  | 6.816458     | 1.000000          | 0.892800            |</pre>"
      ],
      "text/plain": [
       "| 9         | 22       | 0.486054  | 6.816458     | 1.000000          | 0.892800            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>SVM:</pre>"
      ],
      "text/plain": [
       "SVM:"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of examples          : 23750</pre>"
      ],
      "text/plain": [
       "Number of examples          : 23750"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of classes           : 2</pre>"
      ],
      "text/plain": [
       "Number of classes           : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of feature columns   : 2</pre>"
      ],
      "text/plain": [
       "Number of feature columns   : 2"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of unpacked features : 1407914</pre>"
      ],
      "text/plain": [
       "Number of unpacked features : 1407914"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Number of coefficients    : 1407915</pre>"
      ],
      "text/plain": [
       "Number of coefficients    : 1407915"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Starting L-BFGS</pre>"
      ],
      "text/plain": [
       "Starting L-BFGS"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>--------------------------------------------------------</pre>"
      ],
      "text/plain": [
       "--------------------------------------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| Iteration | Passes   | Step size | Elapsed Time | Training Accuracy | Validation Accuracy |</pre>"
      ],
      "text/plain": [
       "| Iteration | Passes   | Step size | Elapsed Time | Training Accuracy | Validation Accuracy |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 0         | 2        | 1.000000  | 0.724382     | 0.998821          | 0.896000            |</pre>"
      ],
      "text/plain": [
       "| 0         | 2        | 1.000000  | 0.724382     | 0.998821          | 0.896000            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 1         | 4        | 1.000000  | 1.284643     | 0.999916          | 0.896000            |</pre>"
      ],
      "text/plain": [
       "| 1         | 4        | 1.000000  | 1.284643     | 0.999916          | 0.896000            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 2         | 5        | 1.000000  | 1.634216     | 0.999958          | 0.896000            |</pre>"
      ],
      "text/plain": [
       "| 2         | 5        | 1.000000  | 1.634216     | 0.999958          | 0.896000            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 3         | 6        | 1.000000  | 2.002875     | 0.999958          | 0.896000            |</pre>"
      ],
      "text/plain": [
       "| 3         | 6        | 1.000000  | 2.002875     | 0.999958          | 0.896000            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 4         | 13       | 0.000338  | 3.462338     | 1.000000          | 0.895200            |</pre>"
      ],
      "text/plain": [
       "| 4         | 13       | 0.000338  | 3.462338     | 1.000000          | 0.895200            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>| 9         | 38       | 4.080042  | 9.019969     | 1.000000          | 0.895200            |</pre>"
      ],
      "text/plain": [
       "| 9         | 38       | 4.080042  | 9.019969     | 1.000000          | 0.895200            |"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>+-----------+----------+-----------+--------------+-------------------+---------------------+</pre>"
      ],
      "text/plain": [
       "+-----------+----------+-----------+--------------+-------------------+---------------------+"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PROGRESS: Model selection based on validation accuracy:\n",
      "PROGRESS: ---------------------------------------------\n",
      "PROGRESS: LogisticClassifier              : 0.8928\n",
      "PROGRESS: SVMClassifier                   : 0.8952\n",
      "PROGRESS: ---------------------------------------------\n",
      "PROGRESS: Selecting SVMClassifier based on validation set performance.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre>Finished parsing file /Users/datalab/bigdata/cjc/kaggle_popcorn_data/testData.tsv</pre>"
      ],
      "text/plain": [
       "Finished parsing file /Users/datalab/bigdata/cjc/kaggle_popcorn_data/testData.tsv"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Parsing completed. Parsed 100 lines in 0.313905 secs.</pre>"
      ],
      "text/plain": [
       "Parsing completed. Parsed 100 lines in 0.313905 secs."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Finished parsing file /Users/datalab/bigdata/cjc/kaggle_popcorn_data/testData.tsv</pre>"
      ],
      "text/plain": [
       "Finished parsing file /Users/datalab/bigdata/cjc/kaggle_popcorn_data/testData.tsv"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre>Parsing completed. Parsed 25000 lines in 0.560208 secs.</pre>"
      ],
      "text/plain": [
       "Parsing completed. Parsed 25000 lines in 0.560208 secs."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "traindata_path = \"/Users/datalab/bigdata/cjc/kaggle_popcorn_data/labeledTrainData.tsv\"\n",
    "testdata_path = \"/Users/datalab/bigdata/cjc/kaggle_popcorn_data/testData.tsv\"\n",
    "#creating classifier using all 25,000 reviews\n",
    "train_data = tc.SFrame.read_csv(traindata_path,header=True, delimiter='\\t',quote_char='\"', \n",
    "                                column_type_hints = {'id':str, 'sentiment' : int, 'review':str } )\n",
    "train_data['1grams features'] = tc.text_analytics.count_ngrams(train_data['review'],1)\n",
    "train_data['2grams features'] = tc.text_analytics.count_ngrams(train_data['review'],2)\n",
    "\n",
    "cls = tc.classifier.create(train_data, target='sentiment', features=['1grams features','2grams features'])\n",
    "#creating the test dataset\n",
    "test_data = tc.SFrame.read_csv(testdata_path,header=True, delimiter='\\t',quote_char='\"', \n",
    "                               column_type_hints = {'id':str, 'review':str } )\n",
    "test_data['1grams features'] = tc.text_analytics.count_ngrams(test_data['review'],1)\n",
    "test_data['2grams features'] = tc.text_analytics.count_ngrams(test_data['review'],2)\n",
    "\n",
    "#predicting the sentiment of each review in the test dataset\n",
    "test_data['sentiment'] = cls.classify(test_data)['class'].astype(int)\n",
    "\n",
    "#saving the prediction to a CSV for submission\n",
    "test_data[['id','sentiment']].save(\"/Users/datalab/bigdata/cjc/kaggle_popcorn_data/predictions.csv\", format=\"csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We then submitted the predictions.csv file to the Kaggle challange website and scored AUC of about 0.88."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "## Further Readings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Further reading materials can be found in the following links:\n",
    "\n",
    "http://en.wikipedia.org/wiki/Bag-of-words_model\n",
    "\n",
    "https://dato.com/products/create/docs/generated/graphlab.SFrame.html\n",
    "\n",
    "https://dato.com/products/create/docs/graphlab.toolkits.classifier.html\n",
    "\n",
    "https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-1-for-beginners-bag-of-words\n",
    "\n",
    "Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. (2011). \"Learning Word Vectors for Sentiment Analysis.\" The 49th Annual Meeting of the Association for Computational Linguistics (ACL 2011).\n"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "幻灯片",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "312px",
    "left": "998px",
    "top": "111px",
    "width": "260px"
   },
   "toc_section_display": false,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}