02.32 · UNIT 08 · Train, evaluate, and serve your transformer · Project
Capstone: build and defend your own transformer
The capstone turns a written specification into a tested, reproducible transformer.
PLAIN-LANGUAGE INTRODUCTION
What is this?
The capstone turns a written specification into a tested, reproducible transformer.
One simple example
A reversal model receives [2,5,7]. It must generate [7,5,2] without correct target history.
What goes in?
A task, split rule, tensor shapes, model settings, and required tests.
What comes out?
Runnable source, checkpoint, measurements, and documented failure cases.
Why does it matter?
Evidence shows whether your implementation learned the intended task.
What is it not?
Copied code and training loss alone do not complete the project.
WORK THROUGH THE IDEA
See the idea in more detail
- Write the contract first. For this example, input
[2,5,7]maps to[7,5,2]. - State vocabulary, padding, target shift, tensor shapes, baseline, and success measure.
- Test masks, gradients, reload behavior, and cached score agreement before a long run.
- Generate held-out answers freely. Record settings, seed, measurements, and observed failures.
- Common mistake: reporting only the best output hides variation and failed cases.