Modern AI Engineering

Lesson 7.1 · 25 min · video lesson

Small Language Models: Big Capability in Compact Form

When would a 3-billion-parameter model on a phone beat a frontier model in a data centre?

In short: Small Language Models (SLMs) are language models with roughly a few hundred million to around ten billion parameters, small enough to run cheaply, quickly and often on a single device. They stay capable thanks to high-quality and synthetic training data, distillation from larger models, long training, pruning and quantization. They shine for focused, high-volume, private or offline tasks; large models remain better for broad knowledge and hard multi-step reasoning.

What you will learn

  • SLM = Small + Language Model
  • What is a Language Model?
  • What Counts as "Small"?
  • Popular SLMs we should know
  • How SLMs Stay Capable Despite Being Small
  • Why SLMs Matter
  • SLM vs LLM
  • The Size Spectrum
  • Where SLMs Shine - Use Cases
  • Trade-offs of SLMs
  • When to Pick an SLM
  • Quick Summary

In this lesson

  1. SLM = Small + Language Model
  2. What is a language model?
  3. What counts as "small"?
  4. Popular SLMs we should know
  5. How SLMs stay capable despite being small
  6. Why SLMs matter
  7. SLM vs LLM
  8. Where SLMs shine: use cases
  9. Trade-offs of SLMs
  10. When to pick an SLM
  11. Worked example, step by step
  12. Practice: try it yourself
  13. Quick summary

Key terms in this lesson

Small Language Model (SLM)
A language model small enough (about 0.1–10B parameters) to run cheaply on one device or modest GPU.
Parameter
A learned number inside a neural network.
Knowledge distillation
Training a small student model to imitate a larger teacher's output distributions.
Pruning
Removing less important parts of a trained model, then retraining briefly.
Quantization
Storing weights with fewer bits (e.g. 8 or 4) to save memory and speed up inference.
On-device inference
Running the model directly on a phone, laptop or edge device instead of a server.

This lesson is part of the Generative AI Engineering and Complete AI Engineer tracks. See plans and pricing to open the full lesson, its video, labs and quiz, or try the free lesson first (free account needed).

← 6.7 DeepSeek-V4: Anatomy of an Open-Source Frontier Model · 7.2 Large Reasoning Models: Chain-of-Thought at Inference Time →