Modern AI Engineering

Lesson 7.2 · 27 min · video lesson

Large Reasoning Models: Chain-of-Thought at Inference Time

Why would a model that answers more slowly, and costs more per question, be the right choice for your hardest problems?

In short: A Large Reasoning Model (LRM) is an LLM trained, mostly with reinforcement learning on problems with checkable answers, to produce a long chain of reasoning before its final answer. Spending more tokens on thinking at answer time (test-time compute) makes it markedly better at maths, coding, logic and planning. The cost is latency and tokens, so we use LRMs for hard, multi-step problems and regular LLMs for simple, fast tasks.

What you will learn

  • The Big Picture
  • What is a Large Reasoning Model (LRM)?
  • LLM vs LRM
  • How does an LRM actually think?
  • Test-time compute: thinking longer makes them smarter
  • How are LRMs trained?
  • Input and Output: training phase vs prediction phase
  • When to use an LRM, and when to use a regular LLM
  • Popular LRMs we should know
  • Common Mistakes when using LRMs
  • Quick Summary

In this lesson

  1. The big picture
  2. What is a Large Reasoning Model (LRM)?
  3. LLM vs LRM
  4. How does an LRM actually think?
  5. Test-time compute: thinking longer makes them smarter
  6. How are LRMs trained?
  7. Input and output: training phase vs prediction phase
  8. When to use an LRM, and when to use a regular LLM
  9. Popular LRMs we should know
  10. Common mistakes when using LRMs
  11. Going one level deeper
  12. Practice: try it yourself
  13. Quick summary

Key terms in this lesson

Large Reasoning Model (LRM)
An LLM trained to produce a long reasoning trace before its final answer.
Chain of thought
Intermediate reasoning steps written out before the final answer.
Test-time compute
Extra computation spent while answering, such as longer reasoning or more samples.
Self-consistency
Sampling several answers and choosing the most common one.
RLVR
Reinforcement learning with verifiable rewards: rewards come from automatic correctness checks.
GRPO
Group Relative Policy Optimization: an RL method that scores each sample against its group's average reward.
Reasoning effort
An API setting that controls how much a model thinks before answering.

This lesson is part of the Generative AI Engineering and Complete AI Engineer tracks. See plans and pricing to open the full lesson, its video, labs and quiz, or try the free lesson first (free account needed).

← 7.1 Small Language Models: Big Capability in Compact Form · 7.3 Recursive Language Models: Self-Referential Generation →