Modern AI Engineering

Lesson 17.11 · 29 min · video lesson

How do Voice And Video Call Work?

When you video-call a friend, your faces often travel directly between your two phones, so how do two devices hidden behind home routers even find each other?

In short: A call has two phases. First, signaling: the two apps exchange messages through a server to agree on codecs and share possible network addresses (session descriptions and ICE candidates). Then media flows, ideally peer-to-peer over UDP using WebRTC. Because most devices sit behind NAT routers, a STUN server tells each device its public address so a direct path can be punched through, and when that fails a TURN server relays the media. Codecs such as Opus, VP8, H.264 and AV1 compress the audio and video so it fits the network.

What you will learn

  • Signaling
  • Peer-to-Peer Connection
  • STUN Server
  • TURN Server

In this lesson

  1. The big picture: two phases of a call
  2. Signaling: agreeing how to talk
  3. Peer-to-peer connection and the NAT problem
  4. STUN server and TURN server
  5. Carrying the media: RTP, codecs and bad networks
  6. Real-world architecture and timeline
  7. Common mistakes and when P2P is not enough
  8. Worked example, step by step
  9. Practice: try it yourself

Key terms in this lesson

Signaling
Exchanging setup messages (session descriptions and network candidates) between call participants through a server.
SDP
Session Description Protocol: a text format describing media streams, codecs and connection details in an offer or answer.
NAT
Network Address Translation: a router maps many private addresses to one public address and port.
ICE
Interactive Connectivity Establishment: gathers candidate addresses and tests pairs to find a working path.
STUN server
A server that tells a device which public IP address and port its packets appear to come from.
TURN server
A server that relays media between peers when they cannot connect directly.
SFU
Selective Forwarding Unit: a media server that receives each participant's stream once and forwards it to others.
Codec
Software that compresses and decompresses audio or video, such as Opus or VP8.

This lesson is part of the Generative AI Engineering and Complete AI Engineer tracks. See plans and pricing to open the full lesson, its video, labs and quiz, or try the free lesson first (free account needed).

← 17.10 Transport Protocols: HTTP, WebSockets, and SSE Compared · 18.1 JEPA: LeCun's Vision for World Model AI →