MODULE 01 · LESSON 04
Gradients
A gradient collects one local slope for every adjustable input.
PLAIN-LANGUAGE INTRODUCTION
The idea
A gradient collects one local slope for every adjustable input.
A worked example
For
L=(w−1)²+(b+2)² at (3,−1), the gradient is [4,2].The input
A loss function and current values. Here, w=3 and b=−1.
The result
One slope per value. Here, ∂L/∂w=4 and ∂L/∂b=2.
Why you use it
Moving opposite the gradient can lower the loss.
A limit to remember
A gradient is not the next parameter value. A learning rate sets the move size.
Check your understanding
- The symbol
∂means we change one input while holding the other fixed. - At
(w,b)=(3,−1), the loss is2² + 1² = 5. - The two slopes are
2(w−1)=4and2(b+2)=2. The gradient is[4,2]. - With learning rate
0.1, descent gives(2.6,−1.2). The new loss is3.2. - Common mistake: adding the gradient increases this loss. Descent subtracts it.