Project_SLC_DSBA_INNHotels_LowCode_%281%29_(1).ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "dense-medicaid",
      "metadata": {
        "id": "dense-medicaid"
      },
      "source": [
        "# INN Hotels Project\n",
        "###### Marks : 60\n",
        "\n",
        "## Context\n",
        "\n",
        "A significant number of hotel bookings are called off due to cancellations or no-shows. The typical reasons for cancellations include change of plans, scheduling conflicts, etc. This is often made easier by the option to do so free of charge or preferably at a low cost which is beneficial to hotel guests but it is a less desirable and possibly revenue-diminishing factor for hotels to deal with. Such losses are particularly high on last-minute cancellations.\n",
        "\n",
        "The new technologies involving online booking channels have dramatically changed customers’ booking possibilities and behavior. This adds a further dimension to the challenge of how hotels handle cancellations, which are no longer limited to traditional booking and guest characteristics.\n",
        "\n",
        "The cancellation of bookings impact a hotel on various fronts:\n",
        "1. Loss of resources (revenue) when the hotel cannot resell the room.\n",
        "2. Additional costs of distribution channels by increasing commissions or paying for publicity to help sell these rooms.\n",
        "3. Lowering prices last minute, so the hotel can resell a room, resulting in reducing the profit margin.\n",
        "4. Human resources to make arrangements for the guests.\n",
        "\n",
        "## Objective\n",
        "\n",
        "The increasing number of cancellations calls for a Machine Learning based solution that can help in predicting which booking is likely to be canceled. INN Hotels Group has a chain of hotels in Portugal, they are facing problems with the high number of booking cancellations and have reached out to your firm for data-driven solutions. You as a data scientist have to analyze the data provided to find which factors have a high influence on booking cancellations, build a predictive model that can predict which booking is going to be canceled in advance, and help in formulating profitable policies for cancellations and refunds.\n",
        "\n",
        "\n",
        "## Data Description\n",
        "\n",
        "The data contains the different attributes of customers' booking details. The detailed data dictionary is given below.\n",
        "\n",
        "\n",
        "**Data Dictionary**\n",
        "\n",
        "* Booking_ID: unique identifier of each booking\n",
        "* no_of_adults: Number of adults\n",
        "* no_of_children: Number of Children\n",
        "* no_of_weekend_nights: Number of weekend nights (Saturday or Sunday) the guest stayed or booked to stay at the hotel\n",
        "* no_of_week_nights: Number of week nights (Monday to Friday) the guest stayed or booked to stay at the hotel\n",
        "* type_of_meal_plan: Type of meal plan booked by the customer:\n",
        "    * Not Selected – No meal plan selected\n",
        "    * Meal Plan 1 – Breakfast\n",
        "    * Meal Plan 2 – Half board (breakfast and one other meal)\n",
        "    * Meal Plan 3 – Full board (breakfast, lunch, and dinner)\n",
        "* required_car_parking_space: Does the customer require a car parking space? (0 - No, 1- Yes)\n",
        "* room_type_reserved: Type of room reserved by the customer. The values are ciphered (encoded) by INN Hotels.\n",
        "* lead_time: Number of days between the date of booking and the arrival date\n",
        "* arrival_year: Year of arrival date\n",
        "* arrival_month: Month of arrival date\n",
        "* arrival_date: Date of the month\n",
        "* market_segment_type: Market segment designation.\n",
        "* repeated_guest: Is the customer a repeated guest? (0 - No, 1- Yes)\n",
        "* no_of_previous_cancellations: Number of previous bookings that were canceled by the customer prior to the current booking\n",
        "* no_of_previous_bookings_not_canceled: Number of previous bookings not canceled by the customer prior to the current booking\n",
        "* avg_price_per_room: Average price per day of the reservation; prices of the rooms are dynamic. (in euros)\n",
        "* no_of_special_requests: Total number of special requests made by the customer (e.g. high floor, view from the room, etc)\n",
        "* booking_status: Flag indicating if the booking was canceled or not."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "286ede49",
      "metadata": {
        "id": "286ede49"
      },
      "source": [
        "### **Please read the instructions carefully before starting the project.** \n",
        "This is a commented Jupyter IPython Notebook file in which all the instructions and tasks to be performed are mentioned. \n",
        "* Blanks '_______' are provided in the notebook that \n",
        "needs to be filled with an appropriate code to get the correct result. With every '_______' blank, there is a comment that briefly describes what needs to be filled in the blank space. \n",
        "* Identify the task to be performed correctly, and only then proceed to write the required code.\n",
        "* Fill the code wherever asked by the commented lines like \"# write your code here\" or \"# complete the code\". Running incomplete code may throw error.\n",
        "* Please run the codes in a sequential manner from the beginning to avoid any unnecessary errors.\n",
        "* Add the results/observations (wherever mentioned) derived from the analysis in the presentation and submit the same.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "proper-calgary",
      "metadata": {
        "id": "proper-calgary"
      },
      "outputs": [],
      "source": [
        "# this will help in making the Python code more structured automatically (help adhere to good coding practices)\n",
        "%load_ext nb_black\n",
        "\n",
        "import warnings\n",
        "\n",
        "warnings.filterwarnings(\"ignore\")\n",
        "from statsmodels.tools.sm_exceptions import ConvergenceWarning\n",
        "\n",
        "warnings.simplefilter(\"ignore\", ConvergenceWarning)\n",
        "\n",
        "# Libraries to help with reading and manipulating data\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "# libaries to help with data visualization\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "\n",
        "# Removes the limit for the number of displayed columns\n",
        "pd.set_option(\"display.max_columns\", None)\n",
        "# Sets the limit for the number of displayed rows\n",
        "pd.set_option(\"display.max_rows\", 200)\n",
        "# setting the precision of floating numbers to 5 decimal points\n",
        "pd.set_option(\"display.float_format\", lambda x: \"%.5f\" % x)\n",
        "\n",
        "# Library to split data\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "# To build model for prediction\n",
        "import statsmodels.stats.api as sms\n",
        "from statsmodels.stats.outliers_influence import variance_inflation_factor\n",
        "import statsmodels.api as sm\n",
        "from statsmodels.tools.tools import add_constant\n",
        "from sklearn.tree import DecisionTreeClassifier\n",
        "from sklearn import tree\n",
        "\n",
        "# To tune different models\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "\n",
        "\n",
        "# To get diferent metric scores\n",
        "from sklearn.metrics import (\n",
        "    f1_score,\n",
        "    accuracy_score,\n",
        "    recall_score,\n",
        "    precision_score,\n",
        "    confusion_matrix,\n",
        "    roc_auc_score,\n",
        "    plot_confusion_matrix,\n",
        "    precision_recall_curve,\n",
        "    roc_curve,\n",
        "    make_scorer,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fantastic-rebel",
      "metadata": {
        "id": "fantastic-rebel"
      },
      "source": [
        "## Import Dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "precious-leonard",
      "metadata": {
        "id": "precious-leonard"
      },
      "outputs": [],
      "source": [
        "hotel = pd.read_csv('_______') ##  Fill the blank to read the data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "geographic-gender",
      "metadata": {
        "id": "geographic-gender"
      },
      "outputs": [],
      "source": [
        "# copying data to another variable to avoid any changes to original data\n",
        "data = hotel.copy()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "convinced-blackberry",
      "metadata": {
        "id": "convinced-blackberry"
      },
      "source": [
        "### View the first and last 5 rows of the dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "tested-adjustment",
      "metadata": {
        "id": "tested-adjustment"
      },
      "outputs": [],
      "source": [
        "data.'_______' ##  Complete the code to view top 5 rows of the data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "demonstrated-charger",
      "metadata": {
        "id": "demonstrated-charger"
      },
      "outputs": [],
      "source": [
        "data.'_______' ##  Complete the code to view last 5 rows of the data "
      ]
    },
    {
      "cell_type": "markdown",
      "id": "prepared-clause",
      "metadata": {
        "id": "prepared-clause"
      },
      "source": [
        "### Understand the shape of the dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "likely-scene",
      "metadata": {
        "id": "likely-scene"
      },
      "outputs": [],
      "source": [
        "data.'_______' ##  Complete the code to view dimensions of the data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "creative-warner",
      "metadata": {
        "id": "creative-warner"
      },
      "source": [
        "### Check the data types of the columns for the dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "expanded-technique",
      "metadata": {
        "id": "expanded-technique"
      },
      "outputs": [],
      "source": [
        "data.info()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "greenhouse-vertical",
      "metadata": {
        "id": "greenhouse-vertical"
      },
      "outputs": [],
      "source": [
        "# checking for duplicate values\n",
        "data.'_______' ##  Complete the code to check duplicate entries in the data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "928c50b6",
      "metadata": {
        "id": "928c50b6"
      },
      "source": [
        "**Let's drop the Booking_ID column first before we proceed forward**."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "referenced-architect",
      "metadata": {
        "id": "referenced-architect"
      },
      "outputs": [],
      "source": [
        "data = data.'_________' ## Complete the code to drop the Booking_ID column from the dataframe "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "40b248aa",
      "metadata": {
        "id": "40b248aa"
      },
      "outputs": [],
      "source": [
        "data.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "realistic-mortgage",
      "metadata": {
        "id": "realistic-mortgage"
      },
      "source": [
        "## Exploratory Data Analysis"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "seeing-newman",
      "metadata": {
        "id": "seeing-newman"
      },
      "source": [
        "**Let's check the statistical summary of the data.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "backed-solution",
      "metadata": {
        "id": "backed-solution"
      },
      "outputs": [],
      "source": [
        "data.'_______' ##  Complete the code to print the statistical summary of the data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "arbitrary-intelligence",
      "metadata": {
        "id": "arbitrary-intelligence"
      },
      "source": [
        "### Univariate Analysis"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "several-cheese",
      "metadata": {
        "id": "several-cheese"
      },
      "outputs": [],
      "source": [
        "def histogram_boxplot(data, feature, figsize=(15, 10), kde=False, bins=None):\n",
        "    \"\"\"\n",
        "    Boxplot and histogram combined\n",
        "\n",
        "    data: dataframe\n",
        "    feature: dataframe column\n",
        "    figsize: size of figure (default (15,10))\n",
        "    kde: whether to show the density curve (default False)\n",
        "    bins: number of bins for histogram (default None)\n",
        "    \"\"\"\n",
        "    f2, (ax_box2, ax_hist2) = plt.subplots(\n",
        "        nrows=2,  # Number of rows of the subplot grid= 2\n",
        "        sharex=True,  # x-axis will be shared among all subplots\n",
        "        gridspec_kw={\"height_ratios\": (0.25, 0.75)},\n",
        "        figsize=figsize,\n",
        "    )  # creating the 2 subplots\n",
        "    sns.boxplot(\n",
        "        data=data, x=feature, ax=ax_box2, showmeans=True, color=\"violet\"\n",
        "    )  # boxplot will be created and a triangle will indicate the mean value of the column\n",
        "    sns.histplot(\n",
        "        data=data, x=feature, kde=kde, ax=ax_hist2, bins=bins\n",
        "    ) if bins else sns.histplot(\n",
        "        data=data, x=feature, kde=kde, ax=ax_hist2\n",
        "    )  # For histogram\n",
        "    ax_hist2.axvline(\n",
        "        data[feature].mean(), color=\"green\", linestyle=\"--\"\n",
        "    )  # Add mean to the histogram\n",
        "    ax_hist2.axvline(\n",
        "        data[feature].median(), color=\"black\", linestyle=\"-\"\n",
        "    )  # Add median to the histogram"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "western-elevation",
      "metadata": {
        "id": "western-elevation"
      },
      "source": [
        "### Observations on lead time"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "italian-imagination",
      "metadata": {
        "id": "italian-imagination"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot(data, \"lead_time\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "intimate-hearing",
      "metadata": {
        "id": "intimate-hearing"
      },
      "source": [
        "### Observations on average price per room"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "liable-guess",
      "metadata": {
        "id": "liable-guess"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot('_______')  ## Complete the code to create histogram_boxplot for average price per room "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "southern-organic",
      "metadata": {
        "id": "southern-organic"
      },
      "outputs": [],
      "source": [
        "data[data[\"avg_price_per_room\"] == 0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "written-stone",
      "metadata": {
        "id": "written-stone"
      },
      "outputs": [],
      "source": [
        "data.loc[data[\"avg_price_per_room\"] == 0, \"market_segment_type\"].value_counts()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "potential-klein",
      "metadata": {
        "id": "potential-klein"
      },
      "outputs": [],
      "source": [
        "# Calculating the 25th quantile\n",
        "Q1 = data[\"avg_price_per_room\"].quantile(0.25)\n",
        "\n",
        "# Calculating the 75th quantile\n",
        "Q3 = ('_______')  ## Complete the code to calculate 75th quantile for average price per room\n",
        "\n",
        "# Calculating IQR\n",
        "IQR = Q3 - Q1\n",
        "\n",
        "# Calculating value of upper whisker\n",
        "Upper_Whisker = Q3 + 1.5 * IQR\n",
        "Upper_Whisker"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "assured-holder",
      "metadata": {
        "id": "assured-holder"
      },
      "outputs": [],
      "source": [
        "# assigning the outliers the value of upper whisker\n",
        "data.loc[data[\"avg_price_per_room\"] >= 500, \"avg_price_per_room\"] = Upper_Whisker"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "studied-arrangement",
      "metadata": {
        "id": "studied-arrangement"
      },
      "source": [
        "### Observations on number of previous booking cancellations"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "molecular-opposition",
      "metadata": {
        "id": "molecular-opposition"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot('_______')  ## Complete the code to create histogram_boxplot for number of previous booking cancellations"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "rough-contributor",
      "metadata": {
        "id": "rough-contributor"
      },
      "source": [
        "### Observations on number of previous booking not canceled"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "laughing-bridge",
      "metadata": {
        "id": "laughing-bridge"
      },
      "outputs": [],
      "source": [
        "histogram_boxplot('_______')  ## Complete the code to create histogram_boxplot for number of previous booking not canceled"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "attended-grounds",
      "metadata": {
        "id": "attended-grounds"
      },
      "outputs": [],
      "source": [
        "# function to create labeled barplots\n",
        "\n",
        "\n",
        "def labeled_barplot(data, feature, perc=False, n=None):\n",
        "    \"\"\"\n",
        "    Barplot with percentage at the top\n",
        "\n",
        "    data: dataframe\n",
        "    feature: dataframe column\n",
        "    perc: whether to display percentages instead of count (default is False)\n",
        "    n: displays the top n category levels (default is None, i.e., display all levels)\n",
        "    \"\"\"\n",
        "\n",
        "    total = len(data[feature])  # length of the column\n",
        "    count = data[feature].nunique()\n",
        "    if n is None:\n",
        "        plt.figure(figsize=(count + 2, 6))\n",
        "    else:\n",
        "        plt.figure(figsize=(n + 2, 6))\n",
        "\n",
        "    plt.xticks(rotation=90, fontsize=15)\n",
        "    ax = sns.countplot(\n",
        "        data=data,\n",
        "        x=feature,\n",
        "        palette=\"Paired\",\n",
        "        order=data[feature].value_counts().index[:n],\n",
        "    )\n",
        "\n",
        "    for p in ax.patches:\n",
        "        if perc == True:\n",
        "            label = \"{:.1f}%\".format(\n",
        "                100 * p.get_height() / total\n",
        "            )  # percentage of each class of the category\n",
        "        else:\n",
        "            label = p.get_height()  # count of each level of the category\n",
        "\n",
        "        x = p.get_x() + p.get_width() / 2  # width of the plot\n",
        "        y = p.get_height()  # height of the plot\n",
        "\n",
        "        ax.annotate(\n",
        "            label,\n",
        "            (x, y),\n",
        "            ha=\"center\",\n",
        "            va=\"center\",\n",
        "            size=12,\n",
        "            xytext=(0, 5),\n",
        "            textcoords=\"offset points\",\n",
        "        )  # annotate the percentage\n",
        "\n",
        "    plt.show()  # show the plot"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "industrial-implementation",
      "metadata": {
        "id": "industrial-implementation"
      },
      "source": [
        "### Observations on number of adults"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "israeli-sympathy",
      "metadata": {
        "id": "israeli-sympathy"
      },
      "outputs": [],
      "source": [
        "labeled_barplot(data, \"no_of_adults\", perc=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "celtic-florist",
      "metadata": {
        "id": "celtic-florist"
      },
      "source": [
        "### Observations on number of children"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "finite-kingston",
      "metadata": {
        "id": "finite-kingston"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______')  ## Complete the code to create labeled_barplot for number of children "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "necessary-employee",
      "metadata": {
        "id": "necessary-employee"
      },
      "outputs": [],
      "source": [
        "# replacing 9, and 10 children with 3\n",
        "data[\"no_of_children\"] = data[\"no_of_children\"].replace([9, 10], 3)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "affiliated-accreditation",
      "metadata": {
        "id": "affiliated-accreditation"
      },
      "source": [
        "### Observations on number of week nights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "great-kitchen",
      "metadata": {
        "id": "great-kitchen"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______')  ## Complete the code to create labeled_barplot for number of week nights"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "thermal-resource",
      "metadata": {
        "id": "thermal-resource"
      },
      "source": [
        "### Observations on number of weekend nights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bizarre-serbia",
      "metadata": {
        "id": "bizarre-serbia",
        "scrolled": true
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______')  ## Complete the code to create labeled_barplot for number of weekend nights"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "greenhouse-regression",
      "metadata": {
        "id": "greenhouse-regression"
      },
      "source": [
        "### Observations on required car parking space"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "handy-talent",
      "metadata": {
        "id": "handy-talent"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______')  ## Complete the code to create labeled_barplot for car parking space"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "southeast-avenue",
      "metadata": {
        "id": "southeast-avenue"
      },
      "source": [
        "### Observations on type of meal plan"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "retired-preliminary",
      "metadata": {
        "id": "retired-preliminary"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______')  ## Complete the code to create labeled_barplot for type of mean plan"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "competitive-brass",
      "metadata": {
        "id": "competitive-brass"
      },
      "source": [
        "### Observations on room type reserved"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "cordless-assurance",
      "metadata": {
        "id": "cordless-assurance"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______')  ## Complete the code to create labeled_barplot for room type reserved"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "thick-coordination",
      "metadata": {
        "id": "thick-coordination"
      },
      "source": [
        "### Observations on arrival month"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "expensive-deposit",
      "metadata": {
        "id": "expensive-deposit"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______')  ## Complete the code to create labeled_barplot for arrival month"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bulgarian-paint",
      "metadata": {
        "id": "bulgarian-paint"
      },
      "source": [
        "### Observations on market segment type"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ordinary-foster",
      "metadata": {
        "id": "ordinary-foster"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______')  ## Complete the code to create labeled_barplot for market segment type "
      ]
    },
    {
      "cell_type": "markdown",
      "id": "behavioral-portfolio",
      "metadata": {
        "id": "behavioral-portfolio"
      },
      "source": [
        "### Observations on number of special requests"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "abroad-moldova",
      "metadata": {
        "id": "abroad-moldova"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______')  ## Complete the code to create labeled_barplot for number of special requests"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "dramatic-karaoke",
      "metadata": {
        "id": "dramatic-karaoke"
      },
      "source": [
        "### Observations on booking status"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "happy-collins",
      "metadata": {
        "id": "happy-collins"
      },
      "outputs": [],
      "source": [
        "labeled_barplot('_______')  ## Complete the code to create labeled_barplot for booking status"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "pharmaceutical-grill",
      "metadata": {
        "id": "pharmaceutical-grill"
      },
      "source": [
        "**Let's encode Canceled bookings to 1 and Not_Canceled as 0 for further analysis**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "voluntary-irish",
      "metadata": {
        "id": "voluntary-irish"
      },
      "outputs": [],
      "source": [
        "data[\"booking_status\"] = data[\"booking_status\"].apply(\n",
        "    lambda x: 1 if x == \"Canceled\" else 0\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "arranged-courtesy",
      "metadata": {
        "id": "arranged-courtesy"
      },
      "source": [
        "### Bivariate Analysis"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "official-wyoming",
      "metadata": {
        "id": "official-wyoming"
      },
      "outputs": [],
      "source": [
        "cols_list = data.select_dtypes(include=np.number).columns.tolist()\n",
        "\n",
        "plt.figure(figsize=(12, 7))\n",
        "sns.heatmap(\n",
        "    data[cols_list].corr(), annot=True, vmin=-1, vmax=1, fmt=\".2f\", cmap=\"Spectral\"\n",
        ")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "upper-glass",
      "metadata": {
        "id": "upper-glass"
      },
      "source": [
        "**Creating functions that will help us with further analysis.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "sought-bunny",
      "metadata": {
        "id": "sought-bunny"
      },
      "outputs": [],
      "source": [
        "### function to plot distributions wrt target\n",
        "\n",
        "\n",
        "def distribution_plot_wrt_target(data, predictor, target):\n",
        "\n",
        "    fig, axs = plt.subplots(2, 2, figsize=(12, 10))\n",
        "\n",
        "    target_uniq = data[target].unique()\n",
        "\n",
        "    axs[0, 0].set_title(\"Distribution of target for target=\" + str(target_uniq[0]))\n",
        "    sns.histplot(\n",
        "        data=data[data[target] == target_uniq[0]],\n",
        "        x=predictor,\n",
        "        kde=True,\n",
        "        ax=axs[0, 0],\n",
        "        color=\"teal\",\n",
        "        stat=\"density\",\n",
        "    )\n",
        "\n",
        "    axs[0, 1].set_title(\"Distribution of target for target=\" + str(target_uniq[1]))\n",
        "    sns.histplot(\n",
        "        data=data[data[target] == target_uniq[1]],\n",
        "        x=predictor,\n",
        "        kde=True,\n",
        "        ax=axs[0, 1],\n",
        "        color=\"orange\",\n",
        "        stat=\"density\",\n",
        "    )\n",
        "\n",
        "    axs[1, 0].set_title(\"Boxplot w.r.t target\")\n",
        "    sns.boxplot(data=data, x=target, y=predictor, ax=axs[1, 0], palette=\"gist_rainbow\")\n",
        "\n",
        "    axs[1, 1].set_title(\"Boxplot (without outliers) w.r.t target\")\n",
        "    sns.boxplot(\n",
        "        data=data,\n",
        "        x=target,\n",
        "        y=predictor,\n",
        "        ax=axs[1, 1],\n",
        "        showfliers=False,\n",
        "        palette=\"gist_rainbow\",\n",
        "    )\n",
        "\n",
        "    plt.tight_layout()\n",
        "    plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "quick-progress",
      "metadata": {
        "id": "quick-progress"
      },
      "outputs": [],
      "source": [
        "def stacked_barplot(data, predictor, target):\n",
        "    \"\"\"\n",
        "    Print the category counts and plot a stacked bar chart\n",
        "\n",
        "    data: dataframe\n",
        "    predictor: independent variable\n",
        "    target: target variable\n",
        "    \"\"\"\n",
        "    count = data[predictor].nunique()\n",
        "    sorter = data[target].value_counts().index[-1]\n",
        "    tab1 = pd.crosstab(data[predictor], data[target], margins=True).sort_values(\n",
        "        by=sorter, ascending=False\n",
        "    )\n",
        "    print(tab1)\n",
        "    print(\"-\" * 120)\n",
        "    tab = pd.crosstab(data[predictor], data[target], normalize=\"index\").sort_values(\n",
        "        by=sorter, ascending=False\n",
        "    )\n",
        "    tab.plot(kind=\"bar\", stacked=True, figsize=(count + 5, 5))\n",
        "    plt.legend(\n",
        "        loc=\"lower left\", frameon=False,\n",
        "    )\n",
        "    plt.legend(loc=\"upper left\", bbox_to_anchor=(1, 1))\n",
        "    plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "quick-works",
      "metadata": {
        "id": "quick-works"
      },
      "source": [
        "**Hotel rates are dynamic and change according to demand and customer demographics. Let's see how prices vary across different market segments**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "animal-depth",
      "metadata": {
        "id": "animal-depth"
      },
      "outputs": [],
      "source": [
        "plt.figure(figsize=(10, 6))\n",
        "sns.boxplot(\n",
        "    data=data, x=\"market_segment_type\", y=\"avg_price_per_room\", palette=\"gist_rainbow\"\n",
        ")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "adjacent-voice",
      "metadata": {
        "id": "adjacent-voice"
      },
      "source": [
        "**Let's see how booking status varies across different market segments. Also, how average price per room impacts booking status**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "specified-novel",
      "metadata": {
        "id": "specified-novel"
      },
      "outputs": [],
      "source": [
        "stacked_barplot(data, \"market_segment_type\", \"booking_status\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "welcome-shore",
      "metadata": {
        "id": "welcome-shore"
      },
      "source": [
        "**Many guests have special requirements when booking a hotel room. Let's see how it impacts cancellations**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "nominated-burlington",
      "metadata": {
        "id": "nominated-burlington"
      },
      "outputs": [],
      "source": [
        "stacked_barplot('_______') ## Complete the code to plot stacked barplot for no of special requests and booking status"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "female-yemen",
      "metadata": {
        "id": "female-yemen"
      },
      "source": [
        "**Let's see if the special requests made by the customers impacts the prices of a room**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "spoken-shoulder",
      "metadata": {
        "id": "spoken-shoulder"
      },
      "outputs": [],
      "source": [
        "plt.figure(figsize=(10, 5))\n",
        "sns.boxplot('_______')  ## Complete the code to create boxplot for no of special requests and average price per room (excluding the outliers)\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "informed-charger",
      "metadata": {
        "id": "informed-charger"
      },
      "source": [
        "**We saw earlier that there is a positive correlation between booking status and average price per room. Let's analyze it**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "green-yellow",
      "metadata": {
        "id": "green-yellow"
      },
      "outputs": [],
      "source": [
        "distribution_plot_wrt_target(data, \"avg_price_per_room\", \"booking_status\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "attempted-velvet",
      "metadata": {
        "id": "attempted-velvet"
      },
      "source": [
        "**There is a positive correlation between booking status and lead time also. Let's analyze it further**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "planned-stylus",
      "metadata": {
        "id": "planned-stylus",
        "scrolled": false
      },
      "outputs": [],
      "source": [
        "distribution_plot_wrt_target('_______') ## Complete the code to find distribution of lead time wrt booking status"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "english-attraction",
      "metadata": {
        "id": "english-attraction"
      },
      "source": [
        "**Generally people travel with their spouse and children for vacations or other activities. Let's create a new dataframe of the customers who traveled with their families and analyze the impact on booking status.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "absent-diabetes",
      "metadata": {
        "id": "absent-diabetes"
      },
      "outputs": [],
      "source": [
        "family_data = data[(data[\"no_of_children\"] >= 0) & (data[\"no_of_adults\"] > 1)]\n",
        "family_data.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "animal-oxide",
      "metadata": {
        "id": "animal-oxide"
      },
      "outputs": [],
      "source": [
        "family_data[\"no_of_family_members\"] = (\n",
        "    family_data[\"no_of_adults\"] + family_data[\"no_of_children\"]\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bored-supplier",
      "metadata": {
        "id": "bored-supplier"
      },
      "outputs": [],
      "source": [
        "stacked_barplot('_______') ## Complete the code to plot stacked barplot for no of family members and booking status"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "formed-arthur",
      "metadata": {
        "id": "formed-arthur"
      },
      "source": [
        "**Let's do a similar analysis for the customer who stay for at least a day at the hotel.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "typical-insured",
      "metadata": {
        "id": "typical-insured"
      },
      "outputs": [],
      "source": [
        "stay_data = data[(data[\"no_of_week_nights\"] > 0) & (data[\"no_of_weekend_nights\"] > 0)]\n",
        "stay_data.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "african-foundation",
      "metadata": {
        "id": "african-foundation"
      },
      "outputs": [],
      "source": [
        "stay_data[\"total_days\"] = (\n",
        "    stay_data[\"no_of_week_nights\"] + stay_data[\"no_of_weekend_nights\"]\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "certified-vacation",
      "metadata": {
        "id": "certified-vacation"
      },
      "outputs": [],
      "source": [
        "stacked_barplot('_______') ## Complete the code to plot stacked barplot for total days and booking status"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "wrong-solomon",
      "metadata": {
        "id": "wrong-solomon"
      },
      "source": [
        "**Repeating guests are the guests who stay in the hotel often and are important to brand equity. Let's see what percentage of repeating guests cancel?**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "greatest-sunglasses",
      "metadata": {
        "id": "greatest-sunglasses"
      },
      "outputs": [],
      "source": [
        "stacked_barplot('_______') ## Complete the code to plot stacked barplot for repeated guests and booking status"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "attached-speech",
      "metadata": {
        "id": "attached-speech"
      },
      "source": [
        "**Let's find out what are the busiest months in the hotel.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "configured-japan",
      "metadata": {
        "id": "configured-japan"
      },
      "outputs": [],
      "source": [
        "# grouping the data on arrival months and extracting the count of bookings\n",
        "monthly_data = data.groupby([\"arrival_month\"])[\"booking_status\"].count()\n",
        "\n",
        "# creating a dataframe with months and count of customers in each month\n",
        "monthly_data = pd.DataFrame(\n",
        "    {\"Month\": list(monthly_data.index), \"Guests\": list(monthly_data.values)}\n",
        ")\n",
        "\n",
        "# plotting the trend over different months\n",
        "plt.figure(figsize=(10, 5))\n",
        "sns.lineplot(data=monthly_data, x=\"Month\", y=\"Guests\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "signed-isaac",
      "metadata": {
        "id": "signed-isaac"
      },
      "source": [
        "**Let's check the percentage of bookings canceled in each month.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "civic-print",
      "metadata": {
        "id": "civic-print"
      },
      "outputs": [],
      "source": [
        "stacked_barplot('_______') ## Complete the code to plot stacked barplot for arrival month and booking status"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "tired-bridges",
      "metadata": {
        "id": "tired-bridges"
      },
      "source": [
        "**As hotel room prices are dynamic, Let's see how the prices vary across different months**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "atomic-locator",
      "metadata": {
        "id": "atomic-locator"
      },
      "outputs": [],
      "source": [
        "plt.figure(figsize=(10, 5))\n",
        "sns.lineplot('_______') ## Complete the code to create lineplot between average price per room and arrival month\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "powerful-couple",
      "metadata": {
        "id": "powerful-couple"
      },
      "source": [
        "### Outlier Check\n",
        "\n",
        "- Let's check for outliers in the data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "imported-uganda",
      "metadata": {
        "id": "imported-uganda"
      },
      "outputs": [],
      "source": [
        "# outlier detection using boxplot\n",
        "numeric_columns = data.select_dtypes(include=np.number).columns.tolist()\n",
        "# dropping booking_status\n",
        "numeric_columns.remove(\"booking_status\")\n",
        "\n",
        "plt.figure(figsize=(15, 12))\n",
        "\n",
        "for i, variable in enumerate(numeric_columns):\n",
        "    plt.subplot(4, 4, i + 1)\n",
        "    plt.boxplot(data[variable], whis=1.5)\n",
        "    plt.tight_layout()\n",
        "    plt.title(variable)\n",
        "\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0-mbGLo1dkaC",
      "metadata": {
        "id": "0-mbGLo1dkaC"
      },
      "source": [
        "## Model Building"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cultural-engagement",
      "metadata": {
        "id": "cultural-engagement"
      },
      "source": [
        "### Model evaluation criterion\n",
        "\n",
        "### Model can make wrong predictions as:\n",
        "\n",
        "1. Predicting a customer will not cancel their booking but in reality, the customer will cancel their booking.\n",
        "2. Predicting a customer will cancel their booking but in reality, the customer will not cancel their booking. \n",
        "\n",
        "### Which case is more important? \n",
        "* Both the cases are important as:\n",
        "\n",
        "* If we predict that a booking will not be canceled and the booking gets canceled then the hotel will lose resources and will have to bear additional costs of distribution channels.\n",
        "\n",
        "* If we predict that a booking will get canceled and the booking doesn't get canceled the hotel might not be able to provide satisfactory services to the customer by assuming that this booking will be canceled. This might damage the brand equity. \n",
        "\n",
        "\n",
        "\n",
        "### How to reduce the losses?\n",
        "\n",
        "* Hotel would want `F1 Score` to be maximized, greater the F1  score higher are the chances of minimizing False Negatives and False Positives. "
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ruled-appointment",
      "metadata": {
        "id": "ruled-appointment"
      },
      "source": [
        "#### First, let's create functions to calculate different metrics and confusion matrix so that we don't have to use the same code repeatedly for each model.\n",
        "* The model_performance_classification_statsmodels function will be used to check the model performance of models. \n",
        "* The confusion_matrix_statsmodels function will be used to plot the confusion matrix."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "prime-front",
      "metadata": {
        "id": "prime-front"
      },
      "outputs": [],
      "source": [
        "# defining a function to compute different metrics to check performance of a classification model built using statsmodels\n",
        "def model_performance_classification_statsmodels(\n",
        "    model, predictors, target, threshold=0.5\n",
        "):\n",
        "    \"\"\"\n",
        "    Function to compute different metrics to check classification model performance\n",
        "\n",
        "    model: classifier\n",
        "    predictors: independent variables\n",
        "    target: dependent variable\n",
        "    threshold: threshold for classifying the observation as class 1\n",
        "    \"\"\"\n",
        "\n",
        "    # checking which probabilities are greater than threshold\n",
        "    pred_temp = model.predict(predictors) > threshold\n",
        "    # rounding off the above values to get classes\n",
        "    pred = np.round(pred_temp)\n",
        "\n",
        "    acc = accuracy_score(target, pred)  # to compute Accuracy\n",
        "    recall = recall_score(target, pred)  # to compute Recall\n",
        "    precision = precision_score(target, pred)  # to compute Precision\n",
        "    f1 = f1_score(target, pred)  # to compute F1-score\n",
        "\n",
        "    # creating a dataframe of metrics\n",
        "    df_perf = pd.DataFrame(\n",
        "        {\"Accuracy\": acc, \"Recall\": recall, \"Precision\": precision, \"F1\": f1,},\n",
        "        index=[0],\n",
        "    )\n",
        "\n",
        "    return df_perf"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "amateur-scotland",
      "metadata": {
        "id": "amateur-scotland"
      },
      "outputs": [],
      "source": [
        "# defining a function to plot the confusion_matrix of a classification model\n",
        "\n",
        "\n",
        "def confusion_matrix_statsmodels(model, predictors, target, threshold=0.5):\n",
        "    \"\"\"\n",
        "    To plot the confusion_matrix with percentages\n",
        "\n",
        "    model: classifier\n",
        "    predictors: independent variables\n",
        "    target: dependent variable\n",
        "    threshold: threshold for classifying the observation as class 1\n",
        "    \"\"\"\n",
        "    y_pred = model.predict(predictors) > threshold\n",
        "    cm = confusion_matrix(target, y_pred)\n",
        "    labels = np.asarray(\n",
        "        [\n",
        "            [\"{0:0.0f}\".format(item) + \"\\n{0:.2%}\".format(item / cm.flatten().sum())]\n",
        "            for item in cm.flatten()\n",
        "        ]\n",
        "    ).reshape(2, 2)\n",
        "\n",
        "    plt.figure(figsize=(6, 4))\n",
        "    sns.heatmap(cm, annot=labels, fmt=\"\")\n",
        "    plt.ylabel(\"True label\")\n",
        "    plt.xlabel(\"Predicted label\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "opposed-glance",
      "metadata": {
        "id": "opposed-glance"
      },
      "source": [
        "### Logistic Regression (with statsmodels library)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "dWLU261ecU8E",
      "metadata": {
        "id": "dWLU261ecU8E"
      },
      "source": [
        "#### Data Preparation for modeling (Logistic Regression)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "tdyBE3l6Xunp",
      "metadata": {
        "id": "tdyBE3l6Xunp"
      },
      "source": [
        "- We want to predict which bookings will be canceled.\n",
        "- Before we proceed to build a model, we'll have to encode categorical features.\n",
        "- We'll split the data into train and test to be able to evaluate the model that we build on the train data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "nominated-tumor",
      "metadata": {
        "id": "nominated-tumor"
      },
      "outputs": [],
      "source": [
        "X = data.drop([\"booking_status\"], axis=1)\n",
        "Y = data[\"booking_status\"]\n",
        "\n",
        "# adding constant\n",
        "X = sm.'_______' ## Complete the code to add constant to X \n",
        "\n",
        "X = pd.'_______' ## Complete the code to create dummies for X \n",
        "\n",
        "# Splitting data in train and test sets\n",
        "X_train, X_test, y_train, y_test = train_test_split('_______') ## Complete the code to split the data into train test in the ratio 70:30 with random_state = 1"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "PNQL4GAuchRx",
      "metadata": {
        "id": "PNQL4GAuchRx"
      },
      "outputs": [],
      "source": [
        "print(\"Shape of Training set : \", X_train.shape)\n",
        "print(\"Shape of test set : \", X_test.shape)\n",
        "print(\"Percentage of classes in training set:\")\n",
        "print(y_train.value_counts(normalize=True))\n",
        "print(\"Percentage of classes in test set:\")\n",
        "print(y_test.value_counts(normalize=True))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "dMaSwLNvcrUU",
      "metadata": {
        "id": "dMaSwLNvcrUU"
      },
      "source": [
        "#### Building Logistic Regression Model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "generous-investing",
      "metadata": {
        "id": "generous-investing"
      },
      "outputs": [],
      "source": [
        "# fitting logistic regression model\n",
        "logit = sm.Logit(y_train, X_train.astype(float))\n",
        "lg = logit.'_______' ## Complete the code to fit logistic regression\n",
        "\n",
        "print(lg.'_______') ## Complete the code to print summary of the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "raising-ultimate",
      "metadata": {
        "id": "raising-ultimate"
      },
      "outputs": [],
      "source": [
        "print(\"Training performance:\")\n",
        "model_performance_classification_statsmodels(lg, X_train, y_train)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "humanitarian-macedonia",
      "metadata": {
        "id": "humanitarian-macedonia"
      },
      "source": [
        "#### Multicollinearity"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "greenhouse-tattoo",
      "metadata": {
        "id": "greenhouse-tattoo"
      },
      "outputs": [],
      "source": [
        "# we will define a function to check VIF\n",
        "def checking_vif(predictors):\n",
        "    vif = pd.DataFrame()\n",
        "    vif[\"feature\"] = predictors.columns\n",
        "\n",
        "    # calculating VIF for each feature\n",
        "    vif[\"VIF\"] = [\n",
        "        variance_inflation_factor(predictors.values, i)\n",
        "        for i in range(len(predictors.columns))\n",
        "    ]\n",
        "    return vif"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "restricted-negative",
      "metadata": {
        "id": "restricted-negative"
      },
      "outputs": [],
      "source": [
        "checking_vif(X_train)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "expensive-folks",
      "metadata": {
        "id": "expensive-folks"
      },
      "source": [
        "#### Dropping high p-value variables\n",
        "\n",
        "- We will drop the predictor variables having a p-value greater than 0.05 as they do not significantly impact the target variable.\n",
        "- But sometimes p-values change after dropping a variable. So, we'll not drop all variables at once.\n",
        "- Instead, we will do the following:\n",
        "    - Build a model, check the p-values of the variables, and drop the column with the highest p-value.\n",
        "    - Create a new model without the dropped feature, check the p-values of the variables, and drop the column with the highest p-value.\n",
        "    - Repeat the above two steps till there are no columns with p-value > 0.05.\n",
        "\n",
        "The above process can also be done manually by picking one variable at a time that has a high p-value, dropping it, and building a model again. But that might be a little tedious and using a loop will be more efficient."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "transsexual-presence",
      "metadata": {
        "id": "transsexual-presence"
      },
      "outputs": [],
      "source": [
        "# initial list of columns\n",
        "cols = X_train.columns.tolist()\n",
        "\n",
        "# setting an initial max p-value\n",
        "max_p_value = 1\n",
        "\n",
        "while len(cols) > 0:\n",
        "    # defining the train set\n",
        "    x_train_aux = X_train[cols]\n",
        "\n",
        "    # fitting the model\n",
        "    model = sm.Logit(y_train, x_train_aux).fit(disp=False)\n",
        "\n",
        "    # getting the p-values and the maximum p-value\n",
        "    p_values = model.pvalues\n",
        "    max_p_value = max(p_values)\n",
        "\n",
        "    # name of the variable with maximum p-value\n",
        "    feature_with_p_max = p_values.idxmax()\n",
        "\n",
        "    if max_p_value > 0.05:\n",
        "        cols.remove(feature_with_p_max)\n",
        "    else:\n",
        "        break\n",
        "\n",
        "selected_features = cols\n",
        "print(selected_features)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "vertical-diploma",
      "metadata": {
        "id": "vertical-diploma"
      },
      "outputs": [],
      "source": [
        "X_train1 = X_train[selected_features]\n",
        "X_test1 = X_test[selected_features]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "composed-athens",
      "metadata": {
        "id": "composed-athens"
      },
      "outputs": [],
      "source": [
        "logit1 = sm.'_______' ## Complete the code to train logistic regression on X_train1 and y_train\n",
        "lg1 = logit1.'_______' ## Complete the code to fit logistic regression\n",
        "print(lg1.'_______') ## Complete the code to print summary of the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "floral-daily",
      "metadata": {
        "id": "floral-daily"
      },
      "outputs": [],
      "source": [
        "print(\"Training performance:\")\n",
        "model_performance_classification_statsmodels(lg1,'_______') ## Complete the code to check performance on X_train1 and y_train"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "external-magnitude",
      "metadata": {
        "id": "external-magnitude"
      },
      "source": [
        "####  Converting coefficients to odds\n",
        "* The coefficients of the logistic regression model are in terms of log(odd), to find the odds we have to take the exponential of the coefficients. \n",
        "* Therefore, **odds =  exp(b)**\n",
        "* The percentage change in odds is given as **odds = (exp(b) - 1) * 100**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "armed-omega",
      "metadata": {
        "id": "armed-omega"
      },
      "outputs": [],
      "source": [
        "# converting coefficients to odds\n",
        "odds = np.exp(lg1.params)\n",
        "\n",
        "# finding the percentage change\n",
        "perc_change_odds = (np.exp(lg1.params) - 1) * 100\n",
        "\n",
        "# removing limit from number of columns to display\n",
        "pd.set_option(\"display.max_columns\", None)\n",
        "\n",
        "# adding the odds to a dataframe\n",
        "pd.DataFrame({\"Odds\": odds, \"Change_odd%\": perc_change_odds}, index=X_train1.columns).T"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "waiting-deputy",
      "metadata": {
        "id": "waiting-deputy"
      },
      "source": [
        "#### Checking model performance on the training set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "auburn-lambda",
      "metadata": {
        "id": "auburn-lambda"
      },
      "outputs": [],
      "source": [
        "# creating confusion matrix\n",
        "confusion_matrix_statsmodels(lg1, X_train1, y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "stable-fabric",
      "metadata": {
        "id": "stable-fabric"
      },
      "outputs": [],
      "source": [
        "print(\"Training performance:\")\n",
        "log_reg_model_train_perf = model_performance_classification_statsmodels(lg1, '_______') ## Complete the code to check performance on X_train1 and y_train\n",
        "log_reg_model_train_perf"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "polished-stewart",
      "metadata": {
        "id": "polished-stewart"
      },
      "source": [
        "#### ROC-AUC\n",
        "* ROC-AUC on training set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "quarterly-arnold",
      "metadata": {
        "id": "quarterly-arnold"
      },
      "outputs": [],
      "source": [
        "logit_roc_auc_train = roc_auc_score(y_train, lg1.predict(X_train1))\n",
        "fpr, tpr, thresholds = roc_curve(y_train, lg1.predict(X_train1))\n",
        "plt.figure(figsize=(7, 5))\n",
        "plt.plot(fpr, tpr, label=\"Logistic Regression (area = %0.2f)\" % logit_roc_auc_train)\n",
        "plt.plot([0, 1], [0, 1], \"r--\")\n",
        "plt.xlim([0.0, 1.0])\n",
        "plt.ylim([0.0, 1.01])\n",
        "plt.xlabel(\"False Positive Rate\")\n",
        "plt.ylabel(\"True Positive Rate\")\n",
        "plt.title(\"Receiver operating characteristic\")\n",
        "plt.legend(loc=\"lower right\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "monthly-segment",
      "metadata": {
        "id": "monthly-segment"
      },
      "source": [
        "#### Model Performance Improvement"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "rapid-conflict",
      "metadata": {
        "id": "rapid-conflict"
      },
      "source": [
        "* Let's see if the recall score can be improved further, by changing the model threshold using AUC-ROC Curve."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "turkish-valuation",
      "metadata": {
        "id": "turkish-valuation"
      },
      "source": [
        "#### Optimal threshold using AUC-ROC curve"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "demonstrated-raise",
      "metadata": {
        "id": "demonstrated-raise"
      },
      "outputs": [],
      "source": [
        "# Optimal threshold as per AUC-ROC curve\n",
        "# The optimal cut off would be where tpr is high and fpr is low\n",
        "fpr, tpr, thresholds = roc_curve(y_train, lg1.predict(X_train1))\n",
        "\n",
        "optimal_idx = np.argmax(tpr - fpr)\n",
        "optimal_threshold_auc_roc = thresholds[optimal_idx]\n",
        "print(optimal_threshold_auc_roc)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "saved-plastic",
      "metadata": {
        "id": "saved-plastic"
      },
      "outputs": [],
      "source": [
        "# creating confusion matrix\n",
        "confusion_matrix_statsmodels(\n",
        "    lg1, '______________', \n",
        ") ## Complete the code to create the confusion matrix for X_train1 and y_train with optimal_threshold_auc_roc as threshold "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "geological-strengthening",
      "metadata": {
        "id": "geological-strengthening"
      },
      "outputs": [],
      "source": [
        "# checking model performance for this model\n",
        "log_reg_model_train_perf_threshold_auc_roc = model_performance_classification_statsmodels(\n",
        "    lg1, X_train1, y_train, threshold=optimal_threshold_auc_roc\n",
        ")\n",
        "print(\"Training performance:\")\n",
        "log_reg_model_train_perf_threshold_auc_roc"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "heated-intersection",
      "metadata": {
        "id": "heated-intersection"
      },
      "source": [
        "#### Let's use Precision-Recall curve and see if we can find a better threshold"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "acute-newark",
      "metadata": {
        "id": "acute-newark"
      },
      "outputs": [],
      "source": [
        "y_scores = lg1.predict(X_train1)\n",
        "prec, rec, tre = precision_recall_curve(y_train, y_scores,)\n",
        "\n",
        "\n",
        "def plot_prec_recall_vs_tresh(precisions, recalls, thresholds):\n",
        "    plt.plot(thresholds, precisions[:-1], \"b--\", label=\"precision\")\n",
        "    plt.plot(thresholds, recalls[:-1], \"g--\", label=\"recall\")\n",
        "    plt.xlabel(\"Threshold\")\n",
        "    plt.legend(loc=\"upper left\")\n",
        "    plt.ylim([0, 1])\n",
        "\n",
        "\n",
        "plt.figure(figsize=(10, 7))\n",
        "plot_prec_recall_vs_tresh(prec, rec, tre)\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "brown-advertiser",
      "metadata": {
        "id": "brown-advertiser"
      },
      "outputs": [],
      "source": [
        "# setting the threshold\n",
        "optimal_threshold_curve = 0.42"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "environmental-surface",
      "metadata": {
        "id": "environmental-surface"
      },
      "source": [
        "#### Checking model performance on training set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "mental-albany",
      "metadata": {
        "id": "mental-albany"
      },
      "outputs": [],
      "source": [
        "# creating confusion matrix\n",
        "confusion_matrix_statsmodels(\n",
        "    lg1, '______________', \n",
        ") ## Complete the code to create the confusion matrix for X_train1 and y_train with optimal_threshold_curve as threshold "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "fifteen-bundle",
      "metadata": {
        "id": "fifteen-bundle"
      },
      "outputs": [],
      "source": [
        "log_reg_model_train_perf_threshold_curve = model_performance_classification_statsmodels(\n",
        "    lg1, X_train1, y_train, threshold=optimal_threshold_curve\n",
        ")\n",
        "print(\"Training performance:\")\n",
        "log_reg_model_train_perf_threshold_curve"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "organic-pocket",
      "metadata": {
        "id": "organic-pocket"
      },
      "source": [
        "#### Let's check the performance on the test set"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "first-governor",
      "metadata": {
        "id": "first-governor"
      },
      "source": [
        "**Using model with default threshold**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "visible-voluntary",
      "metadata": {
        "id": "visible-voluntary"
      },
      "outputs": [],
      "source": [
        "# creating confusion matrix\n",
        "confusion_matrix_statsmodels('_______') ## Complete the code to create confusion matrix for X_test1 and y_test"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "biological-single",
      "metadata": {
        "id": "biological-single"
      },
      "outputs": [],
      "source": [
        "log_reg_model_test_perf = model_performance_classification_statsmodels('_______') ## Complete the code to check performance on X_test1 and y_test\n",
        "\n",
        "print(\"Test performance:\")\n",
        "log_reg_model_test_perf "
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fitted-marsh",
      "metadata": {
        "id": "fitted-marsh"
      },
      "source": [
        "* ROC curve on test set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "included-package",
      "metadata": {
        "id": "included-package"
      },
      "outputs": [],
      "source": [
        "logit_roc_auc_train = roc_auc_score(y_test, lg1.predict(X_test1))\n",
        "fpr, tpr, thresholds = roc_curve(y_test, lg1.predict(X_test1))\n",
        "plt.figure(figsize=(7, 5))\n",
        "plt.plot(fpr, tpr, label=\"Logistic Regression (area = %0.2f)\" % logit_roc_auc_train)\n",
        "plt.plot([0, 1], [0, 1], \"r--\")\n",
        "plt.xlim([0.0, 1.0])\n",
        "plt.ylim([0.0, 1.01])\n",
        "plt.xlabel(\"False Positive Rate\")\n",
        "plt.ylabel(\"True Positive Rate\")\n",
        "plt.title(\"Receiver operating characteristic\")\n",
        "plt.legend(loc=\"lower right\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "computational-concord",
      "metadata": {
        "id": "computational-concord"
      },
      "source": [
        "**Using model with threshold=0.37** "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "stone-wireless",
      "metadata": {
        "id": "stone-wireless"
      },
      "outputs": [],
      "source": [
        "# creating confusion matrix\n",
        "confusion_matrix_statsmodels(lg1, '_______') ## Complete the code to create confusion matrix for X_test1 and y_test using optimal_threshold_auc_roc as threshold"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "southern-inquiry",
      "metadata": {
        "id": "southern-inquiry"
      },
      "outputs": [],
      "source": [
        "# checking model performance for this model\n",
        "log_reg_model_test_perf_threshold_auc_roc = model_performance_classification_statsmodels(\n",
        "    lg1, X_test1, y_test, threshold=optimal_threshold_auc_roc\n",
        ")\n",
        "print(\"Test performance:\")\n",
        "log_reg_model_test_perf_threshold_auc_roc"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "graphic-penetration",
      "metadata": {
        "id": "graphic-penetration"
      },
      "source": [
        "**Using model with threshold = 0.42**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "needed-trial",
      "metadata": {
        "id": "needed-trial"
      },
      "outputs": [],
      "source": [
        "# creating confusion matrix\n",
        "confusion_matrix_statsmodels(lg1, '_______') ## Complete the code to create confusion matrix for X_test1 and y_test using optimal_threshold_curve as threshold"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "major-sucking",
      "metadata": {
        "id": "major-sucking"
      },
      "outputs": [],
      "source": [
        "log_reg_model_test_perf_threshold_curve = model_performance_classification_statsmodels(\n",
        "    lg1, X_test1, y_test, threshold=optimal_threshold_curve\n",
        ")\n",
        "print(\"Test performance:\")\n",
        "log_reg_model_test_perf_threshold_curve"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "polish-moldova",
      "metadata": {
        "id": "polish-moldova"
      },
      "source": [
        "#### Model performance summary"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "domestic-election",
      "metadata": {
        "id": "domestic-election"
      },
      "outputs": [],
      "source": [
        "# training performance comparison\n",
        "\n",
        "models_train_comp_df = pd.concat(\n",
        "    [\n",
        "        log_reg_model_train_perf.T,\n",
        "        log_reg_model_train_perf_threshold_auc_roc.T,\n",
        "        log_reg_model_train_perf_threshold_curve.T,\n",
        "    ],\n",
        "    axis=1,\n",
        ")\n",
        "models_train_comp_df.columns = [\n",
        "    \"Logistic Regression-default Threshold\",\n",
        "    \"Logistic Regression-0.37 Threshold\",\n",
        "    \"Logistic Regression-0.42 Threshold\",\n",
        "]\n",
        "\n",
        "print(\"Training performance comparison:\")\n",
        "models_train_comp_df"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6bf04b12",
      "metadata": {
        "id": "6bf04b12"
      },
      "outputs": [],
      "source": [
        "# test performance comparison\n",
        "\n",
        "'_______' ## Complete the code to compare test performance "
      ]
    },
    {
      "cell_type": "markdown",
      "id": "separated-prague",
      "metadata": {
        "id": "separated-prague"
      },
      "source": [
        "### Decision Tree"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "jkIkXHk6c1iu",
      "metadata": {
        "id": "jkIkXHk6c1iu"
      },
      "source": [
        "#### Data Preparation for modeling (Decision Tree)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "moXH_XnNc_T9",
      "metadata": {
        "id": "moXH_XnNc_T9"
      },
      "source": [
        "- We want to predict which bookings will be canceled.\n",
        "- Before we proceed to build a model, we'll have to encode categorical features.\n",
        "- We'll split the data into train and test to be able to evaluate the model that we build on the train data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "eight-class",
      "metadata": {
        "id": "eight-class"
      },
      "outputs": [],
      "source": [
        "X = data.drop([\"booking_status\"], axis=1)\n",
        "Y = data[\"booking_status\"]\n",
        "\n",
        "X = pd.'_______' ## Complete the code to create dummies for X\n",
        "\n",
        "# Splitting data in train and test sets\n",
        "X_train, X_test, y_train, y_test = train_test_split('_______') ## Complete the code to split the data into train test in the ratio 70:30 with random_state = 1"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "Dkqqg1f1c7ya",
      "metadata": {
        "id": "Dkqqg1f1c7ya"
      },
      "outputs": [],
      "source": [
        "print(\"Shape of Training set : \", X_train.shape)\n",
        "print(\"Shape of test set : \", X_test.shape)\n",
        "print(\"Percentage of classes in training set:\")\n",
        "print(y_train.value_counts(normalize=True))\n",
        "print(\"Percentage of classes in test set:\")\n",
        "print(y_test.value_counts(normalize=True))"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "employed-charger",
      "metadata": {
        "id": "employed-charger"
      },
      "source": [
        "#### First, let's create functions to calculate different metrics and confusion matrix so that we don't have to use the same code repeatedly for each model.\n",
        "* The model_performance_classification_sklearn function will be used to check the model performance of models. \n",
        "* The confusion_matrix_sklearnfunction will be used to plot the confusion matrix."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "industrial-parent",
      "metadata": {
        "id": "industrial-parent"
      },
      "outputs": [],
      "source": [
        "# defining a function to compute different metrics to check performance of a classification model built using sklearn\n",
        "def model_performance_classification_sklearn(model, predictors, target):\n",
        "    \"\"\"\n",
        "    Function to compute different metrics to check classification model performance\n",
        "\n",
        "    model: classifier\n",
        "    predictors: independent variables\n",
        "    target: dependent variable\n",
        "    \"\"\"\n",
        "\n",
        "    # predicting using the independent variables\n",
        "    pred = model.predict(predictors)\n",
        "\n",
        "    acc = accuracy_score(target, pred)  # to compute Accuracy\n",
        "    recall = recall_score(target, pred)  # to compute Recall\n",
        "    precision = precision_score(target, pred)  # to compute Precision\n",
        "    f1 = f1_score(target, pred)  # to compute F1-score\n",
        "\n",
        "    # creating a dataframe of metrics\n",
        "    df_perf = pd.DataFrame(\n",
        "        {\"Accuracy\": acc, \"Recall\": recall, \"Precision\": precision, \"F1\": f1,},\n",
        "        index=[0],\n",
        "    )\n",
        "\n",
        "    return df_perf"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "thrown-protein",
      "metadata": {
        "id": "thrown-protein"
      },
      "outputs": [],
      "source": [
        "def confusion_matrix_sklearn(model, predictors, target):\n",
        "    \"\"\"\n",
        "    To plot the confusion_matrix with percentages\n",
        "\n",
        "    model: classifier\n",
        "    predictors: independent variables\n",
        "    target: dependent variable\n",
        "    \"\"\"\n",
        "    y_pred = model.predict(predictors)\n",
        "    cm = confusion_matrix(target, y_pred)\n",
        "    labels = np.asarray(\n",
        "        [\n",
        "            [\"{0:0.0f}\".format(item) + \"\\n{0:.2%}\".format(item / cm.flatten().sum())]\n",
        "            for item in cm.flatten()\n",
        "        ]\n",
        "    ).reshape(2, 2)\n",
        "\n",
        "    plt.figure(figsize=(6, 4))\n",
        "    sns.heatmap(cm, annot=labels, fmt=\"\")\n",
        "    plt.ylabel(\"True label\")\n",
        "    plt.xlabel(\"Predicted label\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "immune-malta",
      "metadata": {
        "id": "immune-malta"
      },
      "source": [
        "#### Building Decision Tree Model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "recognized-nurse",
      "metadata": {
        "id": "recognized-nurse"
      },
      "outputs": [],
      "source": [
        "model = DecisionTreeClassifier(random_state=1)\n",
        "model.('_______') ## Complete the code to fit decision tree on train data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "identified-upper",
      "metadata": {
        "id": "identified-upper"
      },
      "source": [
        "#### Checking model performance on training set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "female-kennedy",
      "metadata": {
        "id": "female-kennedy"
      },
      "outputs": [],
      "source": [
        "confusion_matrix_sklearn('_______') ## Complete the code to create confusion matrix for train data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "falling-squad",
      "metadata": {
        "id": "falling-squad",
        "scrolled": true
      },
      "outputs": [],
      "source": [
        "decision_tree_perf_train = model_performance_classification_sklearn(\n",
        "    model, X_train, y_train\n",
        ")\n",
        "decision_tree_perf_train"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "neither-omaha",
      "metadata": {
        "id": "neither-omaha"
      },
      "source": [
        "#### Checking model performance on test set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "884d0575",
      "metadata": {
        "id": "884d0575"
      },
      "outputs": [],
      "source": [
        "confusion_matrix_sklearn('_______') ## Complete the code to create confusion matrix for test data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "applied-magazine",
      "metadata": {
        "id": "applied-magazine",
        "scrolled": true
      },
      "outputs": [],
      "source": [
        "decision_tree_perf_test = model_performance_classification_sklearn('_______') ## Complete the code to check performance on test set\n",
        "decision_tree_perf_test"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "rational-details",
      "metadata": {
        "id": "rational-details"
      },
      "source": [
        "**Before pruning the tree let's check the important features.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "genetic-channels",
      "metadata": {
        "id": "genetic-channels"
      },
      "outputs": [],
      "source": [
        "feature_names = list(X_train.columns)\n",
        "importances = model.feature_importances_\n",
        "indices = np.argsort(importances)\n",
        "\n",
        "plt.figure(figsize=(8, 8))\n",
        "plt.title(\"Feature Importances\")\n",
        "plt.barh(range(len(indices)), importances[indices], color=\"violet\", align=\"center\")\n",
        "plt.yticks(range(len(indices)), [feature_names[i] for i in indices])\n",
        "plt.xlabel(\"Relative Importance\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "detailed-possible",
      "metadata": {
        "id": "detailed-possible"
      },
      "source": [
        "#### Pruning the tree"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ancient-composer",
      "metadata": {
        "id": "ancient-composer"
      },
      "source": [
        "**Pre-Pruning**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "romantic-stationery",
      "metadata": {
        "id": "romantic-stationery"
      },
      "outputs": [],
      "source": [
        "# Choose the type of classifier.\n",
        "estimator = DecisionTreeClassifier(random_state=1, class_weight=\"balanced\")\n",
        "\n",
        "# Grid of parameters to choose from\n",
        "parameters = {\n",
        "    \"max_depth\": np.arange(2, 7, 2),\n",
        "    \"max_leaf_nodes\": [50, 75, 150, 250],\n",
        "    \"min_samples_split\": [10, 30, 50, 70],\n",
        "}\n",
        "\n",
        "# Type of scoring used to compare parameter combinations\n",
        "acc_scorer = make_scorer(f1_score)\n",
        "\n",
        "# Run the grid search\n",
        "grid_obj = GridSearchCV(estimator, parameters, scoring=acc_scorer, cv=5)\n",
        "grid_obj = grid_obj.fit(X_train, y_train)\n",
        "\n",
        "# Set the clf to the best combination of parameters\n",
        "estimator = grid_obj.best_estimator_\n",
        "\n",
        "# Fit the best algorithm to the data.\n",
        "estimator.fit(X_train, y_train)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "endangered-image",
      "metadata": {
        "id": "endangered-image"
      },
      "source": [
        "#### Checking performance on training set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "skilled-poster",
      "metadata": {
        "id": "skilled-poster"
      },
      "outputs": [],
      "source": [
        "confusion_matrix_sklearn('_______') ## Complete the code to create confusion matrix for train data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "detected-folks",
      "metadata": {
        "id": "detected-folks"
      },
      "outputs": [],
      "source": [
        "decision_tree_tune_perf_train = model_performance_classification_sklearn('_______') ## Complete the code to check performance on train set\n",
        "decision_tree_tune_perf_train"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "concrete-season",
      "metadata": {
        "id": "concrete-season"
      },
      "source": [
        "#### Checking performance on test set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "banner-comparative",
      "metadata": {
        "id": "banner-comparative"
      },
      "outputs": [],
      "source": [
        "confusion_matrix_sklearn('_______') ## Complete the code to create confusion matrix for test data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "killing-magnet",
      "metadata": {
        "id": "killing-magnet"
      },
      "outputs": [],
      "source": [
        "decision_tree_tune_perf_test = model_performance_classification_sklearn('_______') ## Complete the code to check performance on test set\n",
        "decision_tree_tune_perf_test"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "frequent-grenada",
      "metadata": {
        "id": "frequent-grenada"
      },
      "source": [
        "#### Visualizing the Decision Tree"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "driving-state",
      "metadata": {
        "id": "driving-state"
      },
      "outputs": [],
      "source": [
        "plt.figure(figsize=(20, 10))\n",
        "out = tree.plot_tree(\n",
        "    estimator,\n",
        "    feature_names=feature_names,\n",
        "    filled=True,\n",
        "    fontsize=9,\n",
        "    node_ids=False,\n",
        "    class_names=None,\n",
        ")\n",
        "# below code will add arrows to the decision tree split if they are missing\n",
        "for o in out:\n",
        "    arrow = o.arrow_patch\n",
        "    if arrow is not None:\n",
        "        arrow.set_edgecolor(\"black\")\n",
        "        arrow.set_linewidth(1)\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "narrow-johns",
      "metadata": {
        "id": "narrow-johns",
        "scrolled": true
      },
      "outputs": [],
      "source": [
        "# Text report showing the rules of a decision tree -\n",
        "print(tree.export_text(estimator, feature_names=feature_names, show_weights=True))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "secure-killing",
      "metadata": {
        "id": "secure-killing"
      },
      "outputs": [],
      "source": [
        "# importance of features in the tree building\n",
        "\n",
        "importances = estimator.feature_importances_\n",
        "indices = np.argsort(importances)\n",
        "\n",
        "plt.figure(figsize=(8, 8))\n",
        "plt.title(\"Feature Importances\")\n",
        "plt.barh(range(len(indices)), importances[indices], color=\"violet\", align=\"center\")\n",
        "plt.yticks(range(len(indices)), [feature_names[i] for i in indices])\n",
        "plt.xlabel(\"Relative Importance\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "based-recruitment",
      "metadata": {
        "id": "based-recruitment"
      },
      "source": [
        "**Cost Complexity Pruning**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "white-advocacy",
      "metadata": {
        "id": "white-advocacy"
      },
      "outputs": [],
      "source": [
        "clf = DecisionTreeClassifier(random_state=1, class_weight=\"balanced\")\n",
        "path = clf.cost_complexity_pruning_path(X_train, y_train)\n",
        "ccp_alphas, impurities = abs(path.ccp_alphas), path.impurities"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "associate-audit",
      "metadata": {
        "id": "associate-audit"
      },
      "outputs": [],
      "source": [
        "pd.DataFrame(path)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "numeric-internet",
      "metadata": {
        "id": "numeric-internet"
      },
      "outputs": [],
      "source": [
        "fig, ax = plt.subplots(figsize=(10, 5))\n",
        "ax.plot(ccp_alphas[:-1], impurities[:-1], marker=\"o\", drawstyle=\"steps-post\")\n",
        "ax.set_xlabel(\"effective alpha\")\n",
        "ax.set_ylabel(\"total impurity of leaves\")\n",
        "ax.set_title(\"Total Impurity vs effective alpha for training set\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "reasonable-coral",
      "metadata": {
        "id": "reasonable-coral"
      },
      "source": [
        "Next, we train a decision tree using effective alphas. The last value\n",
        "in ``ccp_alphas`` is the alpha value that prunes the whole tree,\n",
        "leaving the tree, ``clfs[-1]``, with one node."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "stone-causing",
      "metadata": {
        "id": "stone-causing"
      },
      "outputs": [],
      "source": [
        "clfs = []\n",
        "for ccp_alpha in ccp_alphas:\n",
        "    clf = DecisionTreeClassifier(\n",
        "        random_state=1, ccp_alpha=ccp_alpha, class_weight=\"balanced\"\n",
        "    )\n",
        "    clf.'_______' ## Complete the code to fit decision tree on training data\n",
        "    clfs.append(clf)\n",
        "print(\n",
        "    \"Number of nodes in the last tree is: {} with ccp_alpha: {}\".format(\n",
        "        clfs[-1].tree_.node_count, ccp_alphas[-1]\n",
        "    )\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "statistical-palace",
      "metadata": {
        "id": "statistical-palace"
      },
      "outputs": [],
      "source": [
        "clfs = clfs[:-1]\n",
        "ccp_alphas = ccp_alphas[:-1]\n",
        "\n",
        "node_counts = [clf.tree_.node_count for clf in clfs]\n",
        "depth = [clf.tree_.max_depth for clf in clfs]\n",
        "fig, ax = plt.subplots(2, 1, figsize=(10, 7))\n",
        "ax[0].plot(ccp_alphas, node_counts, marker=\"o\", drawstyle=\"steps-post\")\n",
        "ax[0].set_xlabel(\"alpha\")\n",
        "ax[0].set_ylabel(\"number of nodes\")\n",
        "ax[0].set_title(\"Number of nodes vs alpha\")\n",
        "ax[1].plot(ccp_alphas, depth, marker=\"o\", drawstyle=\"steps-post\")\n",
        "ax[1].set_xlabel(\"alpha\")\n",
        "ax[1].set_ylabel(\"depth of tree\")\n",
        "ax[1].set_title(\"Depth vs alpha\")\n",
        "fig.tight_layout()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "wooden-subsection",
      "metadata": {
        "id": "wooden-subsection"
      },
      "source": [
        "#### F1 Score vs alpha for training and testing sets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "abandoned-danger",
      "metadata": {
        "id": "abandoned-danger"
      },
      "outputs": [],
      "source": [
        "f1_train = []\n",
        "for clf in clfs:\n",
        "    pred_train = clf.predict(X_train)\n",
        "    values_train = f1_score(y_train, pred_train)\n",
        "    f1_train.append(values_train)\n",
        "\n",
        "f1_test = []\n",
        "for clf in clfs:\n",
        "    pred_test = clf.predict(X_test)\n",
        "    values_test = f1_score(y_test, pred_test)\n",
        "    f1_test.append(values_test)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "golden-grenada",
      "metadata": {
        "id": "golden-grenada"
      },
      "outputs": [],
      "source": [
        "fig, ax = plt.subplots(figsize=(15, 5))\n",
        "ax.set_xlabel(\"alpha\")\n",
        "ax.set_ylabel(\"F1 Score\")\n",
        "ax.set_title(\"F1 Score vs alpha for training and testing sets\")\n",
        "ax.plot(ccp_alphas, f1_train, marker=\"o\", label=\"train\", drawstyle=\"steps-post\")\n",
        "ax.plot(ccp_alphas, f1_test, marker=\"o\", label=\"test\", drawstyle=\"steps-post\")\n",
        "ax.legend()\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "extreme-inspiration",
      "metadata": {
        "id": "extreme-inspiration"
      },
      "outputs": [],
      "source": [
        "index_best_model = np.argmax(f1_test)\n",
        "best_model = clfs[index_best_model]\n",
        "print(best_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bearing-illinois",
      "metadata": {
        "id": "bearing-illinois"
      },
      "source": [
        "#### Checking performance on training set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "closing-natural",
      "metadata": {
        "id": "closing-natural"
      },
      "outputs": [],
      "source": [
        "confusion_matrix_sklearn(best_model, X_train, y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "hired-beauty",
      "metadata": {
        "id": "hired-beauty"
      },
      "outputs": [],
      "source": [
        "decision_tree_post_perf_train = model_performance_classification_sklearn(\n",
        "    best_model, X_train, y_train\n",
        ")\n",
        "decision_tree_post_perf_train"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "smoking-invitation",
      "metadata": {
        "id": "smoking-invitation"
      },
      "source": [
        "#### Checking performance on test set"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "blocked-bracket",
      "metadata": {
        "id": "blocked-bracket"
      },
      "outputs": [],
      "source": [
        "'_______' ## Complete the code to create confusion matrix for test data on best model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "absent-transsexual",
      "metadata": {
        "id": "absent-transsexual"
      },
      "outputs": [],
      "source": [
        "decision_tree_post_test = '_______' ## Complete the code to check performance of test set on best model\n",
        "decision_tree_post_test"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "classified-banana",
      "metadata": {
        "id": "classified-banana"
      },
      "outputs": [],
      "source": [
        "plt.figure(figsize=(20, 10))\n",
        "\n",
        "out = tree.plot_tree(\n",
        "    best_model,\n",
        "    feature_names=feature_names,\n",
        "    filled=True,\n",
        "    fontsize=9,\n",
        "    node_ids=False,\n",
        "    class_names=None,\n",
        ")\n",
        "for o in out:\n",
        "    arrow = o.arrow_patch\n",
        "    if arrow is not None:\n",
        "        arrow.set_edgecolor(\"black\")\n",
        "        arrow.set_linewidth(1)\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "offshore-colors",
      "metadata": {
        "id": "offshore-colors"
      },
      "outputs": [],
      "source": [
        "# Text report showing the rules of a decision tree -\n",
        "\n",
        "print(tree.export_text(best_model, feature_names=feature_names, show_weights=True))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "crucial-disclaimer",
      "metadata": {
        "id": "crucial-disclaimer"
      },
      "outputs": [],
      "source": [
        "importances = best_model.feature_importances_\n",
        "indices = np.argsort(importances)\n",
        "\n",
        "plt.figure(figsize=(12, 12))\n",
        "plt.title(\"Feature Importances\")\n",
        "plt.barh(range(len(indices)), importances[indices], color=\"violet\", align=\"center\")\n",
        "plt.yticks(range(len(indices)), [feature_names[i] for i in indices])\n",
        "plt.xlabel(\"Relative Importance\")\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "specific-columbus",
      "metadata": {
        "id": "specific-columbus"
      },
      "source": [
        "#### Comparing Decision Tree models"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "superior-reality",
      "metadata": {
        "id": "superior-reality"
      },
      "outputs": [],
      "source": [
        "# training performance comparison\n",
        "\n",
        "models_train_comp_df = pd.concat(\n",
        "    [\n",
        "        decision_tree_perf_train.T,\n",
        "        decision_tree_tune_perf_train.T,\n",
        "        decision_tree_post_perf_train.T,\n",
        "    ],\n",
        "    axis=1,\n",
        ")\n",
        "models_train_comp_df.columns = [\n",
        "    \"Decision Tree sklearn\",\n",
        "    \"Decision Tree (Pre-Pruning)\",\n",
        "    \"Decision Tree (Post-Pruning)\",\n",
        "]\n",
        "print(\"Training performance comparison:\")\n",
        "models_train_comp_df"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "working-employment",
      "metadata": {
        "id": "working-employment"
      },
      "outputs": [],
      "source": [
        "# testing performance comparison\n",
        "\n",
        "'_______' ## Complete the code to compare performance of test set"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c178b0e2",
      "metadata": {
        "id": "c178b0e2"
      },
      "source": [
        "### Business Recommendations"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [
        "dense-medicaid",
        "286ede49",
        "fantastic-rebel",
        "convinced-blackberry",
        "prepared-clause",
        "creative-warner",
        "realistic-mortgage",
        "arbitrary-intelligence",
        "western-elevation",
        "intimate-hearing",
        "studied-arrangement",
        "rough-contributor",
        "industrial-implementation",
        "celtic-florist",
        "affiliated-accreditation",
        "thermal-resource",
        "greenhouse-regression",
        "southeast-avenue",
        "competitive-brass",
        "thick-coordination",
        "bulgarian-paint",
        "behavioral-portfolio",
        "dramatic-karaoke",
        "arranged-courtesy",
        "powerful-couple",
        "0-mbGLo1dkaC",
        "cultural-engagement",
        "ruled-appointment",
        "opposed-glance",
        "dWLU261ecU8E",
        "dMaSwLNvcrUU",
        "humanitarian-macedonia",
        "expensive-folks",
        "external-magnitude",
        "waiting-deputy",
        "polished-stewart",
        "monthly-segment",
        "turkish-valuation",
        "heated-intersection",
        "environmental-surface",
        "organic-pocket",
        "polish-moldova",
        "separated-prague",
        "jkIkXHk6c1iu",
        "employed-charger",
        "immune-malta",
        "identified-upper",
        "neither-omaha",
        "detailed-possible",
        "endangered-image",
        "concrete-season",
        "frequent-grenada",
        "wooden-subsection",
        "bearing-illinois",
        "smoking-invitation",
        "specific-columbus",
        "c178b0e2"
      ],
      "name": "Project_SLC_DSBA_INNHotels_LowCode_(1) (4).ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": ".venv",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.10"
    },
    "vscode": {
      "interpreter": {
        "hash": "9d3efd2f2773b376bcbfafd2cf5748c36ebd086818d5a09f3b333a5e4a10f0ed"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}