Modern AI Engineering

Lesson 16.3 · 23 min · video lesson

Image Embeddings: Encoding Visual Content as Vectors

Two photos of the same red mug can differ in almost every pixel — so how does a computer still know they show the same thing?

In short: An image embedding is a short list of numbers (a vector) that captures what an image shows, produced by a trained neural network called an encoder. Similar-looking or similar-meaning images get vectors that point in similar directions, so we can compare images with simple math such as cosine similarity. Embeddings power image search, duplicate detection, recommendations, clustering and text-to-image search.

What you will learn

  • What is an embedding?
  • What is an image embedding?
  • Why do we need image embeddings?
  • How does a computer see an image?
  • How are image embeddings created?
  • A simple numeric walkthrough
  • How do we measure similarity between two embeddings?
  • A code example
  • Where are image embeddings used?
  • Summary

In this lesson

  1. What is an embedding?
  2. What is an image embedding?
  3. Why do we need image embeddings?
  4. How does a computer see an image?
  5. How are image embeddings created?
  6. A simple numeric walkthrough
  7. How do we measure similarity between two embeddings?
  8. A code example
  9. Where are image embeddings used?
  10. Common mistakes and how to spot them
  11. Practice: try it yourself
  12. Summary

Key terms in this lesson

Embedding
A fixed-length vector representing an item so that similar items are close together.
Image encoder
A trained network (CNN or ViT) that turns an image into features or an embedding.
Cosine similarity
The cosine of the angle between two vectors: dot product divided by both lengths.
L2 normalisation
Dividing a vector by its length so it has length 1.
Contrastive learning
Training that pulls matching pairs together and pushes non-matching pairs apart.
Nearest-neighbour search
Finding the stored vectors most similar to a query vector.

This lesson is part of the Generative AI Engineering and Complete AI Engineer tracks. See plans and pricing to open the full lesson, its video, labs and quiz, or try the free lesson first (free account needed).

← 16.2 Vision Transformers: Applying Self-Attention to Image Patches · 16.4 Diffusion Models: Iterative Denoising to Generate Images →