2023_08_15_spam_email.qmd

---
title: "Spam E-mail"
date: 2023-08-15
format: html
execute: 
  echo: true
---

# TidyTuesday

Join the R4DS Online Learning Community in the weekly #TidyTuesday event!
Every week we post a raw dataset, a chart or article related to that dataset, and ask you to explore the data.
While the dataset will be “tamed”, it will not always be tidy! As such you might need to apply various R for Data Science techniques to wrangle the data into a true tidy format.
The goal of TidyTuesday is to apply your R skills, get feedback, explore other’s work, and connect with the greater #RStats community!
As such we encourage everyone of all skills to participate!

```{r}
#| label: setup
#| include: false

library(tidyverse)
library(tidytuesdayR)

theme_set(theme_light())
```

# Load the weekly Data

Download the weekly data and make available in the `tt` object.

```{r}
#| label: Load

tt <- tt_load("2023-08-15")

```


# Readme

Take a look at the readme for the weekly data to get insight on the dataset.
This includes a data dictionary, source, and a link to an article on the data.

```{r}
#| label: Readme
#| eval: interactive()

tt

```


# Glimpse Data

Take an initial look at the format of the data available.

```{r}
#| label: Glimpse

tt %>% 
  map(glimpse)

```

# Wrangle

Explore the data and process it into a nice format for plotting! Access each dataset by name by using a dollarsign after the `tt` object and then the name of the data set.

```{r}
#| label: Wrangle

spam <- tt$spam |> 
  mutate(yesno = factor(yesno, levels = c("y", "n")))
```

```{r}
spam |> 
  count(yesno)
```

# Visualize

Using your processed dataset, create your unique visualization.

```{r}
#| label: Visualize

spam |> 
  ggplot(aes(crl.tot)) +
  geom_freqpoly(aes(color = yesno)) +
  scale_x_log10()

# No crl.tot == 0
spam |> 
  filter(crl.tot < 1)
```

```{r}
spam |> 
  ggplot(aes(dollar)) +
  geom_density(aes(color = yesno), adjust = 1/2)

spam |> 
  ggplot(aes(bang)) +
  geom_density(aes(color = yesno), adjust = 1/2)

spam |> 
  ggplot(aes(money)) +
  geom_density(aes(color = yesno), adjust = 1/2)

spam |> 
  ggplot(aes(n000)) +
  geom_density(aes(color = yesno), adjust = 1/2)

spam |> 
  ggplot(aes(make)) +
  geom_density(aes(color = yesno), adjust = 1/2)

```

```{r}
library(tidymodels)
tidymodels_prefer()
```

```{r}
set.seed(815)

spam_split <- initial_split(spam, strata = yesno)
spam_train <- training(spam_split)
spam_test <- testing(spam_split)
```

Recipe

```{r}
spam_recipe <- recipe(yesno ~ ., data = spam_train) |> 
  step_log(crl.tot, base = 10)
```

Logistic regression

```{r}
lr_model <- logistic_reg(
  mode = "classification",
  engine = "glm",
  penalty = NULL,
  mixture = NULL
)

lr_wflow <- workflow(spam_recipe, lr_model)

lr_fit <- fit(lr_wflow, data = spam_train)

spam_train |> 
  select(yesno) |> 
  bind_cols(predict(lr_fit, new_data = spam_train)) |> 
  table()

spam_test |> 
  select(yesno) |> 
  bind_cols(predict(lr_fit, new_data = spam_test)) |> 
  table()
```

Draw ROC curve of both train and test data

```{r}
bind_rows(
  spam_train |> 
    select(yesno) |> 
    bind_cols(predict(lr_fit, new_data = spam_train, type = "prob")) |> 
    roc_curve(yesno, .pred_y) |> 
    mutate(split = "train"),
  spam_test |> 
    select(yesno) |> 
    bind_cols(predict(lr_fit, new_data = spam_test, type = "prob")) |> 
    roc_curve(yesno, .pred_y) |> 
    mutate(split = "test")
) |> 
  ggplot(aes(x = 1 - specificity, y = sensitivity)) +
  geom_path(aes(color = split)) +
  geom_abline(lty = 3) +
  coord_equal() +
  theme_bw()
```

Xgboost without tuning

```{r}
xgb_model <- boost_tree(
  mode = "classification",
  engine = "xgboost",
  mtry = NULL,
  trees = NULL,
  min_n = NULL,
  tree_depth = NULL,
  learn_rate = NULL,
  loss_reduction = NULL,
  sample_size = NULL,
  stop_iter = NULL
)

xgb_wflow <- workflow(spam_recipe, xgb_model)

xgb_fit <- fit(xgb_wflow, data = spam_train)

spam_train |> 
  select(yesno) |> 
  bind_cols(predict(xgb_fit, new_data = spam_train)) |> 
  table()

spam_test |> 
  select(yesno) |> 
  bind_cols(predict(xgb_fit, new_data = spam_test)) |> 
  table()
```

ROC curve: Xgboost is superior to linear regression?

```{r}
p <- bind_rows(
  spam_test |> 
    select(yesno) |> 
    bind_cols(predict(lr_fit, new_data = spam_test, type = "prob")) |> 
    roc_curve(yesno, .pred_y) |> 
    mutate(model = "Logistic regression"),
  spam_test |> 
    select(yesno) |> 
    bind_cols(predict(xgb_fit, new_data = spam_test, type = "prob")) |> 
    roc_curve(yesno, .pred_y) |> 
    mutate(model = "Xgboost")
) |> 
  ggplot(aes(x = 1 - specificity, y = sensitivity)) +
  geom_path(aes(color = model)) +
  geom_abline(lty = 3) +
  geom_hline(yintercept = 0.8, color = "gray70") +
  scale_x_continuous(breaks = seq(0, 1, by = 0.1)) +
  scale_y_continuous(breaks = seq(0, 1, by = 0.1)) +
  labs(title = "To detect 80 percent of scam, Xgboost requires 10 percent\nof non-scam to be classified as scam,\nwhile Logistic regression requires 20 percent",
       caption = "Note: Results on test data (20%) by the model trained on 80% of the data\nSource: spam email dataset in Vincent Arel-Bundock's Rdatasets package",
       color = "Model") +
  coord_equal() +
  theme_bw() +
  theme(panel.grid.minor = element_blank(),
        plot.caption = element_text(hjust = 0),
        plot.margin = margin(1, 0, 1, -20, "mm"))
```

Xgboost with tuning
Refer to https://juliasilge.com/blog/xgboost-tune-volleyball/
However, I failed.

# Save Image

Save your image for sharing. Be sure to use the `#TidyTuesday` hashtag in your post on twitter! 

```{r}

# This will save your most recent plot
ggsave("image/spam_email.png", p, width = 7, height = 5)

```