Working_with_Numerai_data_and_SKLearn.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Working with Numerai data and SKLearn.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyM0DpphODJoj+R7SmGopKF5",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/numerai/example-scripts/blob/master/Working_with_Numerai_data_and_SKLearn.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8JryWtLgXWfb",
        "colab_type": "text"
      },
      "source": [
        "ATTENTION!!! YOU MUST RUN EACH CODE BLOCK FROM TOP TO BOTTOM IN ORDER TO EXECUTE THE CODE!!!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T2iFGd6vYP0f",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#import statements always go first\n",
        "\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import sklearn #this is the Scikit-Learn package (sklearn for short)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rnSADrG0Xw3G",
        "colab_type": "text"
      },
      "source": [
        "Scikit-Learn has conventions for code which must be followed. For instance, an algorithm requires training data. We already know that the Numerai data is labeled as features and targets, and observations exist in eras.\n",
        "So, our rows are the observations, features are the independent variables, and the target is the dependent variable.\n",
        "We will have a matrix of n observations, X columns, and the y target. Your data should conform to this convention in every step of your process!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ORt4dBQdV9d0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#define variables used inside functions above those functions\n",
        "\n",
        "train_datalink = 'https://numerai-public-datasets.s3-us-west-2.amazonaws.com/latest_numerai_training_data.csv.xz'\n",
        "\n",
        "df_train = pd.read_csv(train_datalink, nrows=50000) #download the training data and keep only the first 50,000 rows\n",
        "#df_train = pd.read_csv('https://numerai-public-datasets.s3-us-west-2.amazonaws.com/latest_numerai_training_data.csv.xz', nrows=50000) #this line of code would also work"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YaTKDCYUYrms",
        "colab_type": "text"
      },
      "source": [
        "Let's break the training data into the parts we need. First, let's create our dependent variable as a new object, y."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qixyoNnIWV7J",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "y = df_train.target.values  #we use the .values command to extract only the numerical values from the column 'target'"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hxj0Qe1QY-3-",
        "colab_type": "text"
      },
      "source": [
        "Let's also create a shortcut to access the feature columns"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "P3Icmk7dW808",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#create your training matrix of the 310 features\n",
        "#there are several ways to do this! This is a way to maintain the dataframe's structure but to isolate the feature columns.\n",
        "features = [c for c in df_train if c.startswith(\"feature\")]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-HdtaUu1ZGE6",
        "colab_type": "text"
      },
      "source": [
        "Have we accomplished our goal? Let's check!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "80M31rtgZHmL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df_train[features].head(10) #first 10 rows"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iSQdV1afZQ9Y",
        "colab_type": "text"
      },
      "source": [
        "Above, you'll notice that 10 rows x 310 columns are printed for your review. We have specified that features represents every column in the dataframe that contains the string \"feature\".\n",
        "What do you notice about the values in each column?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J_XO_rJsZOVz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "df_train.feature_intelligence1.describe()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5TNu6ETsZdOG",
        "colab_type": "text"
      },
      "source": [
        "Let's look at just one Era"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zBVZpo5JZd12",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "era1 = df_train[df_train.era == \"era1\"].copy()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d5eVwqQhZf8n",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "era1.describe()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZjsQsYc_ZqBT",
        "colab_type": "text"
      },
      "source": [
        "You do it! Follow the two code blocks above, and print the descriptive statistics for the second era."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7s79DGzPZjO3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#hint: You have to change some things in order to select the second era, but the code above works and can be modified for your use.\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "S2ReE9pHZsJl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kwvxqKDpZu8B",
        "colab_type": "text"
      },
      "source": [
        "Now that you've explored the data a little bit, let's begin to use Scikit-Learn with our subsample.\n",
        "Begin with Generalized Linear Models: https://scikit-learn.org/stable/modules/linear_model.html\n",
        "Scikit-learn is extremely well documented. Most algorithms come with code examples. Let's first take a look at Ordinary Least Squares.\n",
        "The documentation shows that LinearRegression is part of the linear_model module. To use any algorithm in linear_model, we must first import the module."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "E7e3VNHDZwrV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn import linear_model #This is how you import a specific module to your python environment. In order to use LinearRegression, we must access LinearRegression within the linear_model module."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_fHsJnP-Z5JL",
        "colab_type": "text"
      },
      "source": [
        "LinearRegression takes several parameters.\n",
        "\n",
        "https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html#sklearn.linear_model.LinearRegression\n",
        "\n",
        "Visiting the above link shows you the source code for LinearRegression.\n",
        "\n",
        "You can see that:\n",
        "    \n",
        "    sklearn.linear_model.LinearRegression(fit_intercept=True, normalize=False, copy_X=True, n_jobs=None)\n",
        "\n",
        "represents the default settings for the algorithm. This is a standard presentation format throughout Scikit-Learn.\n",
        "\n",
        "We can ignore the default parameters and focus on using the algorithm, since LinearRegression is such a basic algorithm.\n",
        "\n",
        "Below the explanation of the parameters, you'll see an \"Examples\" section.\n",
        "\n",
        "We need to define our explanatory variables (independent variables, \"features\") and our dependent variable, y. We can define X as:\n",
        "\n",
        "`X = df_train[features].values`\n",
        "\n",
        "You need to copy the line of code above and paste it in the cell below in order to define X. Run the cell."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4I1j6jjsaPVi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Use this code cell to define X\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I0jomGXRaQLE",
        "colab_type": "text"
      },
      "source": [
        "Also, we can define the algorithm as its own python variable. The example names LinearRegression as reg.\n",
        "\n",
        "You can do this as well, or call the algorithm whatever you want. Personally, I like to use a systematic naming method, because I often evaluate several versions of the algorithm at once.\n",
        "\n",
        "`REG1 = linear_model.LinearRegression()`\n",
        "\n",
        "You need to copy the line of code above and paste it in the cell below in order to define REG1. Run the cell."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Gj-6vZ1wZ7-o",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Use this code cell to define REG1\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N-CeU4cpaTiz",
        "colab_type": "text"
      },
      "source": [
        "Now, REG1 is my algorithm, and I can access commands using \".\" (Dot Notation)\n",
        "\n",
        "We have to fit the algorithm to the training data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KBrCt7C1Z0We",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#We have to fit the model to the training data using the following convention:\n",
        "#REG1.fit(X, y)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cwh3K2HJbUwA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#YOU DO IT! Fit your algorithm to the training data.\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FjQeIrB9bYb0",
        "colab_type": "text"
      },
      "source": [
        "Let's see how the model performed by using the .score() function.\n",
        "After you've fit an algorithm in scikit-learn, you can evaluate the in-sample performance by using .score()\n",
        "\n",
        "`REG1.score(X, y)`\n",
        "\n",
        "Running the code snippet above will give you the R-squared value for the model. A score close to 1 is very good. A score close to zero is very bad. But remember! This is only an in-sample estimate!\n",
        "\n",
        "You do it! Run your model's .score() and evaluate the R-squared value."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TzCA5ADsbh2s",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#Copy the code from the cell above to produce the model's score\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T2F-ByFpbo8D",
        "colab_type": "text"
      },
      "source": [
        "You've now fit a model to a subset of the training data. If you can do this using LinearRegression, then what other algorithms can you use within the linear_model module?\n",
        "\n",
        "https://scikit-learn.org/stable/modules/classes.html#module-sklearn.linear_model\n",
        "\n",
        "ASSIGNMENT:\n",
        "\n",
        "```\n",
        "PICK ANY ALGORITHM FROM THE LINK ABOVE OTHER THAN LINEARREGRESSION AND FIT THE ALGORITHM TO THE TRAINING DATA.\n",
        "\n",
        "SCORE YOUR MODEL USING THE .SCORE() FUNCTION.\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "btpnovVub-9j",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g5GKVRVxcM0k",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I0DmNofQcNIk",
        "colab_type": "text"
      },
      "source": [
        "Congratulations! You have successfully demonstrated how to: define your training data, fit a machine learning model to the data, and evaluate the model's in-sample performance.\n",
        "\n",
        "Don't stop now! Read other sections of scikit-learn and see which models you can use for a regression task.\n",
        "\n",
        "ASSIGNMENT:\n",
        "\n",
        "```\n",
        "NAME 5 ALGORITHMS WHICH WILL WORK WITH A REGRESSION TASK.\n",
        "IMPORT THEM TO THIS SHEET. AT LEAST ONE MUST COME FROM A MODULE OTHER THAN LINEAR_MODEL.\n",
        "\n",
        "REMEMBER, SCIKIT-LEARN IS VERY WELL DOCUMENTED! GOOGLE IS YOUR FRIEND! DO SOME WORK!\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RxtcQQW7cU8p",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hu9u2qNGcVFB",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oSTh56cxcVNl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F5Se17a5cVVd",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AcFAgi0PcVdf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}