02.32 · UNIT 10 · Train, evaluate, and serve your transformer · Lesson
Decoder-only language modeling and generation
A decoder-only language model predicts each next token from earlier tokens only.
PLAIN-LANGUAGE INTRODUCTION
The idea
A decoder-only language model predicts each next token from earlier tokens only.
A worked example
For
hello, training input is hell. Targets are ello. After prefix he, the next target is l.The input
A prefix of token IDs and a causal attention mask.
The result
Next-token scores at each position, then generated tokens.
Why you use it
One shifted objective trains all prefix positions in parallel.
A limit to remember
A small next-token model is not automatically a helpful chat system.
Check your understanding
- Shift the sequence by one position. Input
hellaligns with targetello. - At input position one, the visible prefix is
he. Its correct next token isl. - A causal mask blocks later input positions. Training still evaluates all allowed positions together.
- Generation chooses one next token, appends it, then repeats with the longer prefix.
- Common mistake: forgetting the shift trains the model to copy its current token.