Lesson 3.1 · 20 min · video lesson
Neural Network Bias: What It Is and Why It Matters
If a neuron can already multiply every input by a learned weight, why does it also need one extra number that ignores the input completely?
In short: A bias is a learned constant added to a neuron's weighted sum: z = w·x + b. It shifts the neuron's output up or down, so the neuron can fit lines that do not pass through the origin and can choose where its activation switches on. Without bias, many simple patterns become impossible or much harder to learn.
In this lesson
- The problem: a line stuck at the origin
- What a bias is, in plain words
- Two jobs a bias does
- How a bias is learned, step by step
- Code you can run: with and without bias
- With bias vs without bias
- Bias in real networks
- Common mistakes and when not to use bias
- Worked example, step by step
- Practice: try it yourself
Key terms in this lesson
- Bias
- A learnable constant added to a neuron's weighted sum that shifts its output independently of the inputs.
- Weight
- A learnable number that scales one input, controlling how strongly that input influences the neuron.
- Pre-activation
- The value z = w·x + b computed before the activation function is applied.
- Activation function
- A non-linear function such as ReLU or sigmoid applied to the pre-activation to produce the neuron's output.
- Decision boundary
- The set of inputs where w·x + b = 0; the bias moves it away from the origin.
- Dead ReLU
- A ReLU neuron whose pre-activation is negative for all inputs, so it always outputs zero and receives no gradient.
This lesson is free. Sign in or create a free account to open it with its video and quiz.
← 2.9 Contrastive Learning: Training by Comparison · 3.2 Gradient Descent: Rolling Downhill to the Optimum →