Modern AI Engineering

Lesson 7.3 · 26 min · video lesson

Recursive Language Models: Self-Referential Generation

What if, instead of stuffing a 10-million-token document into a model's prompt, we let the model write code to explore it and call itself on the pieces?

In short: A Recursive Language Model (RLM) is an inference strategy, not a new network: the long input is stored as a variable in a programming environment (a Python REPL), and the model writes code to peek at it, search it, split it and call language models, including itself, on chosen pieces. Only small results ever enter the model's own context. This sidesteps context limits and "context rot", and it handles inputs far beyond any context window, at the cost of more calls, latency and reliance on the model's coding skill.

What you will learn

  • What is a Recursive Language Model (RLM)?
  • Why do we need RLMs?
  • How an RLM works
  • How the model writes and runs code
  • Why RLMs work better
  • Recursion inside RLMs
  • How RLMs differ from simple chunking
  • Advantages of RLMs
  • Limitations of RLMs
  • When to use RLMs
  • RLM vs RAG
  • A real use case

In this lesson

  1. What is a Recursive Language Model (RLM)?
  2. Why do we need RLMs?
  3. How an RLM works
  4. How the model writes and runs code
  5. Why RLMs work better
  6. Recursion inside RLMs
  7. How RLMs differ from simple chunking
  8. Advantages and limitations of RLMs
  9. When to use RLMs
  10. RLM vs RAG
  11. A real use case
  12. Common mistakes and how to spot them
  13. Practice: try it yourself

Key terms in this lesson

Recursive Language Model (RLM)
An inference strategy where a model explores a long input stored in a REPL and calls models on pieces of it.
REPL
Read-Eval-Print Loop: an interactive programming session that runs code and returns output.
Root model
The top-level model call that plans, writes code and returns the final answer.
Sub-call
A language-model call made from within the REPL on a selected piece of the input.
Recursion depth
How many levels of nested model calls are allowed below the root.
Context rot
The drop in model accuracy as the amount of text in its context grows.

This lesson is part of the Generative AI Engineering and Complete AI Engineer tracks. See plans and pricing to open the full lesson, its video, labs and quiz, or try the free lesson first (free account needed).

← 7.2 Large Reasoning Models: Chain-of-Thought at Inference Time · 7.4 Diffusion Language Models: Text Generation Beyond Autoregression →