02.33 · UNIT 10 · Train, evaluate, and serve your transformer · Lab
Lab: train complete transformer models
A complete transformer lab joins data, masks, loss, updates, and held-out generation.
PLAIN-LANGUAGE INTRODUCTION
The idea
A complete transformer lab joins data, masks, loss, updates, and held-out generation.
A worked example
For reversal, source
[2,5,7] has target [7,5,2]. Evaluation must generate all three target tokens.The input
Training batches, shifted targets, masks, and a fixed model configuration.
The result
Updated parameters and free-running held-out predictions.
Why you use it
The lab proves that separate components form one learning system.
A limit to remember
Low teacher-forced loss does not prove correct free generation.
Check your understanding
- This task maps source
[2,5,7]to target[7,5,2]. Keep some sequences held out. - Training uses shifted targets. The loss compares each predicted next token with its correct label.
- An optimizer updates parameters after backpropagation. Record the configuration and random seed.
- During evaluation, generate
[7,5,2]without supplying correct earlier target tokens. - Common mistake: a separately sampled test tensor can still repeat a generated training sequence. Check overlap.