02.07 · UNIT 02 · PyTorch in One Hour: models and training · Lesson
The training loop
A training loop repeats five steps: forward, loss, zero_grad, backward, and step.
PLAIN-LANGUAGE INTRODUCTION
The idea
A training loop repeats five steps: forward, loss, zero_grad, backward, and step.
A worked example
On the toy data, the loss falls from
0.75 to 0.00 in three epochs.The input
A model, a data loader, a loss function, and an optimizer.
The result
Updated parameters and a falling loss.
Why you use it
The loop connects predictions, error, gradients, and parameter updates.
A limit to remember
A falling training loss does not prove good generalization.
Check your understanding
- A training step changes parameters once. Compute predictions and loss. Clear old gradients before the backward pass. Update parameters after you compute new gradients.
model.train()andmodel.eval()set the mode for layers such as dropout and batch normalization.- Pass logits to
cross_entropy. The function applies softmax internally. - After training,
argmaxturns logits into labels. An accuracy function measures them. - Common mistake: without
zero_grad(), new gradients add to the old gradients.