Modern AI Engineering

Lesson 4.1 · 27 min · video lesson

Generative AI: Creating Instead of Classifying

How can a program that was only ever shown existing text, images and sound produce a sentence, picture or song that never existed before?

In short: Generative AI is AI that creates new content (text, images, audio, video, code) instead of only labelling or scoring existing content. It learns the patterns of a huge amount of example data, stores them as numbers called parameters, and then creates something new by sampling from what it learned, one small piece at a time. It is powerful and useful every day, but it can be confidently wrong, biased and out of date, so we must use it with care.

What you will learn

  • What is Generative AI?
  • Generative AI = Generative + AI
  • What does "generate" mean here?
  • How is Generative AI different from the old AI?
  • How does Generative AI learn?
  • How does Generative AI actually create something new?
  • What can Generative AI create?
  • What is a model in Generative AI?
  • The complete flow of Generative AI
  • Where do we use Generative AI every day?
  • The limitations we must know
  • Summary

In this lesson

  1. What is Generative AI?
  2. How is Generative AI different from the old AI?
  3. How does Generative AI learn?
  4. How does Generative AI actually create something new?
  5. What can Generative AI create?
  6. What is a model in Generative AI?
  7. The complete flow of Generative AI
  8. Code: a tiny generative model you can run
  9. Where do we use Generative AI every day?
  10. The limitations we must know
  11. Worked example, step by step
  12. Practice: try it yourself
  13. Summary

Key terms in this lesson

Generative AI
AI that produces new content such as text, images, audio, video or code.
Discriminative model
A model that maps an input to a label or number, such as spam or not spam.
Token
A small piece of text, a word or part of a word, that a language model reads and writes.
Parameters
The learned numbers inside a model that determine its behaviour.
Sampling
Picking the next piece of output at random, weighted by the model's probabilities.
Temperature
A setting that sharpens (low) or flattens (high) the probabilities before sampling.
Hallucination
Fluent, confident output that is false or not supported by any source.

This lesson is free. Sign in or create a free account to open it with its video and quiz.

← 3.10 TensorFlow Explained: Static Graphs and Production ML · 4.2 Autoregressive Models: Predicting One Token at a Time →