Lesson 2.3 · 23 min · video lesson
Predicting Numbers vs Categories: Regression Compared
Despite the shared word "regression", one of these models predicts numbers and the other answers yes-or-no questions. How can one small change, a sigmoid, turn the first into the second?
In short: Linear regression predicts a continuous number by fitting a straight line (or plane) that minimises squared error. Logistic regression predicts the probability of a yes/no outcome by passing the same kind of linear score through a sigmoid that squashes it into 0 to 1, and it is trained with log loss. Use linear regression for "how much?" and logistic regression for "which class?".
What you will learn
- Linear Regression
- Logistic Regression
- Differences Between Linear Regression and Logistic Regression
In this lesson
- Two kinds of questions
- Linear regression
- Logistic regression
- Code: both models on the same students
- Differences between linear and logistic regression
- Real-world use
- Common mistakes and limits
- Worked example, step by step
- Practice: try it yourself
Key terms in this lesson
- Linear regression
- A model that predicts a continuous value as a weighted sum of features plus a bias.
- Logistic regression
- A classification model that outputs a probability by applying the sigmoid to a linear score.
- Sigmoid
- The function σ(z) = 1 / (1 + e⁻ᶻ), which maps any real number into the range 0 to 1.
- Log-odds (logit)
- ln(p / (1 − p)); logistic regression models this as a linear function of the features.
- Log loss
- Binary cross-entropy: the loss that heavily penalises confident wrong probability predictions.
- Decision boundary
- The set of inputs where the model switches between predicted classes.
This lesson is free. Sign in or create a free account to open it with its video and quiz.
← 2.2 Labeled vs Unlabeled: Two Ways Machines Learn · 2.4 Feature Engineering: Turning Raw Data into Signal →