Lesson 2.5 · 22 min · video lesson
Precision and Recall: Picking the Right Metric
A spam filter that marks nothing as spam can still score 99% accuracy. So how do we actually tell whether a classifier is good?
In short: Precision asks: of everything the model flagged, how much was really positive? Recall asks: of everything that was really positive, how much did the model catch? Raising the decision threshold usually raises precision and lowers recall, so we choose based on which mistake costs more: false alarms (favour precision) or misses (favour recall). F1 combines both into one number.
What you will learn
- The problem we are trying to solve
- The four possible outcomes
- What is Precision?
- What is Recall?
- Precision vs Recall
- When to use which one?
- A quick recap of the formulas
- Summary
In this lesson
- The problem we are trying to solve
- The four possible outcomes
- What is precision?
- What is recall?
- Precision vs recall: the trade-off
- Code: computing everything from scratch
- When to use which one?
- Going one level deeper
- Practice: try it yourself
- A quick recap of the formulas, and summary
Key terms in this lesson
- Confusion matrix
- A table of counts of true positives, false positives, false negatives and true negatives.
- Precision
- The fraction of predicted positives that are actually positive: TP / (TP + FP).
- Recall
- The fraction of actual positives that the model found: TP / (TP + FN).
- F1 score
- The harmonic mean of precision and recall.
- Threshold
- The score above which a classifier predicts the positive class.
- Class imbalance
- When one class is much rarer than the other, making accuracy misleading.
This lesson is free. Sign in or create a free account to open it with its video and quiz.
← 2.4 Feature Engineering: Turning Raw Data into Signal · 2.6 L1 vs L2 Loss: Choosing Your Error Penalty →