02.08 · UNIT 02 · Supervised learning and reliable experiments · Lesson

Loss functions and the full training loop

A loss measures prediction error. The training loop repeatedly lowers that loss.

PLAIN-LANGUAGE INTRODUCTION

What is this?

A loss measures prediction error. The training loop repeatedly lowers that loss.

One simple example

Logits [2,1,0] become probabilities [0.665,0.245,0.090]. Correct class 0 has loss 0.408.

Model scores and the correct target index.

One loss value, then gradients for an optimizer step.

The loop connects predictions, error, gradients, and parameter updates.

Calling the model once does not train it.

WORK THROUGH THE IDEA

Logits are raw class scores. Here, they are [2,1,0]. The correct class index is 0.
Softmax turns the scores into [0.665,0.245,0.090]. These values add to 1.
Cross-entropy uses −log(0.665) ≈ 0.408. More probability on the correct class gives lower loss.
A training step clears gradients, predicts, calculates loss, backpropagates, then updates parameters.
Common mistake: applying softmax before a loss that already expects logits can harm numerical stability.