Modern AI Engineering

Lesson 10.13 · 25 min · video lesson

Vectorless RAG: Retrieval Without Embeddings or a Vector Store

When you look something up in a 300-page manual, you don't compute embeddings: you open the table of contents, pick a chapter, then a section. Can an LLM retrieve the same way?

In short: Vectorless RAG retrieves context for an LLM without embeddings or a vector database. The best-known form turns a document into a tree, like a table of contents with section summaries, and lets an LLM reason its way down the tree to the right section; other forms use keyword search, agentic file search, SQL or simply long context. It avoids chunking and similarity problems and gives explainable retrieval paths, but costs more LLM calls per query and does not scale to huge unstructured collections as easily as vector search.

What you will learn

  • What is an LLM
  • What is RAG
  • How the normal Vector RAG works
  • Problems with Vector RAG
  • What is Vectorless RAG
  • How Vectorless RAG works
  • An example of Vectorless RAG
  • Other Vectorless approaches
  • Advantages of Vectorless RAG
  • Disadvantages of Vectorless RAG
  • Vector RAG vs Vectorless RAG
  • When to use which one
  • [AI Engineering Explained: LLM, RAG, MCP, Agent, Fine-Tuning, Quantization](https://www.youtube.com/watch?v=lnfWvX66FUk) (Video)
  • [Agentic RAG Explained](https://www.youtube.com/watch?v=6nSegpuWJVw) (Video)

In this lesson

  1. What is an LLM, and what is RAG?
  2. How normal vector RAG works, and its problems
  3. What is vectorless RAG?
  4. How vectorless (tree-based) RAG works
  5. An example of vectorless RAG, in code
  6. Other vectorless approaches
  7. Advantages, disadvantages and the comparison
  8. When to use which one
  9. Worked example, step by step
  10. Practice: try it yourself
  11. How this lesson connects

Key terms in this lesson

Vectorless RAG
RAG that retrieves context without embeddings or a vector database.
Tree index
A hierarchy of document sections, each with a title and summary, used for navigation.
Reasoning-based retrieval
Retrieval where an LLM decides step by step which part of the content to read.
Long-context stuffing
Putting a whole document directly into the LLM prompt instead of retrieving parts.
Text-to-SQL
An LLM writes a database query from a natural-language question and reads the result.

This lesson is part of the Generative AI Engineering and Complete AI Engineer tracks. See plans and pricing to open the full lesson, its video, labs and quiz, or try the free lesson first (free account needed).

← 10.12 GraphRAG: Combining Knowledge Graphs with Retrieval · 11.1 AI Agents: Autonomous Decision-Making Systems →