Modern AI Engineering

Lesson 15.2 · 27 min · video lesson

Prompt Injection: Attacks Against LLM-Powered Systems

What if a web page your AI assistant reads could quietly tell it to email your inbox to a stranger, and the assistant obeyed?

In short: Prompt injection is an attack in which text written by an attacker is treated by an LLM as instructions, overriding what the developer or user intended. It works because an LLM reads instructions and data as one stream of tokens, with no hard boundary between them. Direct injection comes from the user; indirect injection hides in content the model reads, such as web pages, emails or documents. There is no complete fix yet, so we combine detection, careful prompt design, least privilege, human confirmation and system designs that limit what an injected instruction can do.

What you will learn

  • What is a Large Language Model
  • What is a prompt
  • The system prompt and the user prompt
  • What is Prompt Injection
  • The root cause of Prompt Injection
  • A simple example of Prompt Injection
  • Direct Prompt Injection
  • Indirect Prompt Injection
  • A step-by-step walkthrough of a real attack
  • A code example of how the attack sneaks in
  • Prompt Injection vs Jailbreaking
  • Why Prompt Injection is not like SQL Injection
  • What an attacker can achieve
  • The defenses, one approach at a time
  • A defense checklist
  • How to test our own application
  • Why this problem is still not solved

In this lesson

  1. LLMs, prompts, system prompts and user prompts
  2. What is prompt injection, and what is its root cause?
  3. Direct and indirect prompt injection
  4. A step-by-step walkthrough of a real-world style attack
  5. A code example: how the attack sneaks in, and three defences
  6. Prompt injection vs jailbreaking vs SQL injection
  7. What an attacker can achieve
  8. The defences, one approach at a time
  9. Testing our own application, and why this is still not solved
  10. Going one level deeper
  11. Practice: try it yourself

Key terms in this lesson

System prompt
Developer-written instructions that set the model's role and rules for an application.
Prompt injection
An attack where untrusted text in the input is followed by the model as an instruction.
Indirect prompt injection
Injection planted in content the model reads, such as web pages, emails or documents.
Jailbreaking
Crafting prompts that make a model ignore its own safety training.
Exfiltration
Moving private data out of a system to an attacker.
Least privilege
Giving a system only the permissions it needs for the current task.
Lethal trifecta
The risky combination of private data access, untrusted content and an outbound channel in one agent.

This lesson is part of the Generative AI Engineering and Complete AI Engineer tracks. See plans and pricing to open the full lesson, its video, labs and quiz, or try the free lesson first (free account needed).

← 15.1 LLM Guardrails: Filtering Inputs and Outputs for Safety · 15.3 LLM Watermarking: Embedding Invisible Signatures in AI Text →