Tensor by Tensor

02.37 · UNIT 10 · Train, evaluate, and serve your transformer · Project

Capstone: build and defend your own transformer

The capstone turns a written specification into a tested, reproducible transformer.

PLAIN-LANGUAGE INTRODUCTION

The idea

The capstone turns a written specification into a tested, reproducible transformer.

A worked example

A reversal model receives [2,5,7]. It must generate [7,5,2] without correct target history.

The input

A task, split rule, tensor shapes, model settings, and required tests.

The result

Runnable source, checkpoint, measurements, and documented failure cases.

Why you use it

Evidence shows whether your implementation learned the intended task.

A limit to remember

Copied code and training loss alone do not complete the project.

Check your understanding

  1. Write the contract first. For this example, input [2,5,7] maps to [7,5,2].
  2. State vocabulary, padding, target shift, tensor shapes, baseline, and success measure.
  3. Test masks, gradients, reload behavior, and cached score agreement before a long run.
  4. Generate held-out answers freely. Record settings, seed, measurements, and observed failures.
  5. Common mistake: reporting only the best output hides variation and failed cases.
Open the detailed notes ↗