Modern AI Engineering

Lesson 8.5 · 23 min · video lesson

Continual Learning: Training Without Forgetting the Past

If we teach a trained model one new thing, why might it suddenly get worse at the things it already knew?

In short: Continual learning is training a model on a stream of new data or tasks over time without losing what it learned before. Neural networks tend to suffer catastrophic forgetting: new training overwrites shared weights that old skills depended on. Practical fixes include replaying old data, penalising changes to important weights, giving new tasks their own parameters, and keeping fast-changing facts outside the model.

What you will learn

  • What is Continual Learning?
  • Why do we need Continual Learning in LLMs?
  • The big problem: Catastrophic Forgetting
  • Approaches to Continual Learning in LLMs
  • Challenges in Continual Learning
  • Real-world use cases

In this lesson

  1. What is continual learning?
  2. Why we need continual learning in LLMs
  3. The big problem: catastrophic forgetting
  4. Approaches to continual learning in LLMs
  5. Challenges in continual learning
  6. Real-world use cases
  7. Worked example, step by step
  8. Practice: try it yourself

Key terms in this lesson

Continual learning
Training a model on data or tasks that arrive over time while keeping earlier knowledge.
Catastrophic forgetting
A sharp loss of old abilities after training on new data.
Stability–plasticity dilemma
The trade-off between remembering old knowledge and being able to learn new knowledge.
Replay
Mixing stored or generated old examples into new training.
EWC
Elastic Weight Consolidation: a penalty that protects weights important to earlier tasks.
Continual pretraining
Further pretraining an existing model on new raw text, such as a new domain or time period.

This lesson is part of the Generative AI Engineering and Complete AI Engineer tracks. See plans and pricing to open the full lesson, its video, labs and quiz, or try the free lesson first (free account needed).

← 8.4 Knowledge Distillation: Compressing Large Models into Small Ones · 8.6 Deep RL from Human Preferences: The Foundational Paper →