02.37 · UNIT 10 · Train, evaluate, and serve your transformer · Project
Capstone: build and defend your own transformer
The capstone turns a written specification into a tested, reproducible transformer.
PLAIN-LANGUAGE INTRODUCTION
The idea
The capstone turns a written specification into a tested, reproducible transformer.
A worked example
A reversal model receives
[2,5,7]. It must generate [7,5,2] without correct target history.The input
A task, split rule, tensor shapes, model settings, and required tests.
The result
Runnable source, checkpoint, measurements, and documented failure cases.
Why you use it
Evidence shows whether your implementation learned the intended task.
A limit to remember
Copied code and training loss alone do not complete the project.
Check your understanding
- Write the contract first. For this example, input
[2,5,7]maps to[7,5,2]. - State vocabulary, padding, target shift, tensor shapes, baseline, and success measure.
- Test masks, gradients, reload behavior, and cached score agreement before a long run.
- Generate held-out answers freely. Record settings, seed, measurements, and observed failures.
- Common mistake: reporting only the best output hides variation and failed cases.