Skip to content

Latest commit

 

History

History
211 lines (91 loc) · 11 KB

session_3.md

File metadata and controls

211 lines (91 loc) · 11 KB

Session 3

### Linear Regression Recap:

Linear regression is a kind of statistical analysis that attempts to show a relationship between two variables.

The core idea is to obtain a line that best fits the data. The best fit line is the one for which total prediction error (all data points) are as small as possible. Error is the distance between the points to the regression line.

Important Formulas and Parameters:

Hypothesis:

###### Parameters:

###### Cost Function:

Goal:

Gradient Descent:

Gradient descent is a first-order iterative optimization algorithm for finding the minimum of a function. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or approximate gradient) of the function at the current point.

Graphical Representation of Gradient Descent Algorithm

Graph of y = tan(x)

Here we can see how the tan(x) value, that is nothing but our gradient which is varying based on the angle.

Deriving the Gradient Descent Formula:

Therefore,

Understanding Correlation Matrix

The Logistic Regression

Logistic regression is a statistical model that in its basic form uses a logistic function to model a binary dependent variable.

Problem Statement:

1> Given a list of passengers who survived and did not survive the sinking of the Titanic, predict if someone might survive the disaster (from Kaggle)

2> Given a set of images of cats and dogs, identify if the next image contains a dog or a cat (from Kaggle)

Why don't we use Linear Regression to solve these classification problems?

Let's say we have a data set that contains a list of customers and a label to determine if the customer had purchased a certain product. In the data set, there are 20 customers. 10 customers age between 10 to 19 who purchased, and 10 customers age between 20 to 29 who didn't purchase. "Purchased" is a binary label denoted by 0 and 1, where 0 denote "customer didn't make a purchase" and 1 denote "customer made a purchase".

If we plot the graph for linear regression we will get the red line that best fits the model.

Given any age, we will predict the value along Y-axis. If Y is greater than 0.5(above the green line), the customer had purchased the product.

Now the problem arises when the data is imbalanced. Let's add 10 more customers, whose age is between 60 to 70. Let's train the network and try to find the best fit line.

Do you see the problem here?

For a few of the customers whose age is in the range of 20-30, the prediction will be wrong.

If we use logistic regression, that's how it tries to solve the above problem.

Now the question is, how does it work??

The first requirement we can see here is that the output here is not continuous like linear regression. The output here is binary that is either 0 or 1.

For that, we will take the help of Sigmoid Function

Sigmoid Function:

If t goes to infinity, sig(t) will become 1 and if t goes to negative infinity, sig(t) will become 0.

So, if we put Sigmoid Function to the hypothesis of Linear Regression the output will be in the range of 0 to 1.

Hypothesis Function:

Cost Function:

Now for the cost function we could have simply used the cost function of linear regression which is

But this cost function can't be used with Logistic Regression. It will result in a non-convex cost function. Which results in cost function with local minimas which is a very big problem for Gradient Descent to compute the global optima.

So, for Logistic Regression the cost function is

##### Cost Function Explanation:
for y = 1,

for y = 0,

Simplified Cost Function:

Derivation of Gradient Descent Formula:

Assignment_3:
  1. Download the diabetes dataset from the given link. in the dataset the target varible is nothing but "a quantitative measure of disease progression one year after baseline". You have to predict this value based on the other two parameters "BMI" and "Blood serum measurement".
  2. Open this link linear regression and make a copy in your google colab. Implement the linear regression on the diabetes dataset you have downloaded.
  3. Try to understand each and every line of the code and put comments for every lines about your understanding.
  4. Submit the link to LMS Assignment_3 submit link.
Session 3 video:

Session_3