Modern AI Engineering

Lesson 10.11 · 27 min · video lesson

Agentic RAG: Dynamic Retrieval with Multi-Step Reasoning

Standard RAG searches once and hopes for the best. What if the system could notice the first search was not enough, and decide what to look up next?

In short: Agentic RAG puts an LLM agent in charge of retrieval. Instead of one fixed "retrieve then answer" step, the agent reasons about the question, chooses which tool or source to query, reads the results, judges whether they are enough, and loops (rewriting queries, searching again, calling other tools) until it can answer. It handles multi-step and multi-source questions far better than standard RAG, at the cost of more latency, more tokens and less predictability.

What you will learn

  • The Big Picture
  • A Quick Recap of RAG
  • A Quick Recap of AI Agent
  • Why Standard RAG Falls Short
  • What is Agentic RAG
  • The Agentic RAG Loop
  • The Three Building Blocks
  • A Walkthrough with a Real Example
  • Common Patterns of Agentic RAG
  • Standard RAG vs Agentic RAG
  • When to Use Agentic RAG
  • Limitations of Agentic RAG
  • Quick Summary

In this lesson

  1. The big picture
  2. A quick recap of RAG and of AI agents
  3. Why standard RAG falls short
  4. What is agentic RAG, and the agentic RAG loop
  5. The three building blocks
  6. A walkthrough with a real example, in code
  7. Common patterns of agentic RAG
  8. Standard RAG vs agentic RAG, and when to use it
  9. Common mistakes and how to spot them
  10. Practice: try it yourself
  11. Limitations of agentic RAG, and quick summary

Key terms in this lesson

Agentic RAG
RAG in which an LLM agent decides when, where and what to retrieve, in a loop.
AI agent
An LLM that repeatedly reasons, calls tools and observes results until a goal is met.
Tool
A function with a name, description and parameters that the agent can ask the application to run.
Multi-hop question
A question that needs several lookups, where each depends on the previous answer.
Query decomposition
Splitting a complex question into simpler sub-questions answered in order.
ReAct
A prompting pattern that interleaves reasoning steps with tool actions and observations.

This lesson is part of the Generative AI Engineering and Complete AI Engineer tracks. See plans and pricing to open the full lesson, its video, labs and quiz, or try the free lesson first (free account needed).

← 10.10 Semantic Caching: Skipping the LLM for Similar Queries · 10.12 GraphRAG: Combining Knowledge Graphs with Retrieval →