Modern AI Engineering

Lesson 16.4 · 27 min · video lesson

Diffusion Models: Iterative Denoising to Generate Images

How can a model that only ever learned to remove a little noise end up painting a brand-new picture out of pure static?

In short: A diffusion model learns to generate data by reversing a simple destruction process. During training we gradually add Gaussian noise to real images until nothing is left, and teach a network to predict the noise that was added at each level. To generate, we start from pure noise and repeatedly subtract the predicted noise, step by step, until a clean image appears; text conditioning steers which image we get.

What you will learn

  • What is a Diffusion Model?
  • Why do we need Diffusion Models?
  • The two processes: Forward and Reverse
  • The Forward Process (adding noise)
  • The Reverse Process (removing noise)
  • A step-by-step example walk-through
  • How the model is trained
  • A simple code example
  • Conditional Diffusion (text to image)
  • Advantages of Diffusion Models
  • Where Diffusion Models are used

In this lesson

  1. What is a diffusion model?
  2. Why do we need diffusion models?
  3. The two processes: forward and reverse
  4. The forward process (adding noise)
  5. The reverse process (removing noise)
  6. A step-by-step example walk-through
  7. How the model is trained
  8. A simple code example
  9. Conditional diffusion (text to image)
  10. Advantages and where diffusion models are used
  11. Going one level deeper
  12. Practice: try it yourself

Key terms in this lesson

Generative model
A model that learns a data distribution so it can create new samples from it.
Forward process
The fixed procedure that adds Gaussian noise to data over T steps.
Reverse process
The learned procedure that removes noise step by step to generate data.
Noise schedule
The sequence β₁…β_T controlling how much noise is added at each step.
U-Net
A convolutional encoder-decoder with skip connections, the classic denoising network in diffusion models.
Latent diffusion
Running diffusion in a compressed latent space produced by a VAE, then decoding to pixels.
Classifier-free guidance
Mixing conditional and unconditional noise predictions to make samples follow the prompt more strongly.

This lesson is part of the Generative AI Engineering and Complete AI Engineer tracks. See plans and pricing to open the full lesson, its video, labs and quiz, or try the free lesson first (free account needed).

← 16.3 Image Embeddings: Encoding Visual Content as Vectors · 16.5 GANs: A Generator and Discriminator in Constant Competition →