02.17 · UNIT 05 · Text representations and neural language baselines · Lab
Lab: train and evaluate embeddings
An embedding gives each token a short learned vector.
PLAIN-LANGUAGE INTRODUCTION
The idea
An embedding gives each token a short learned vector.
A worked example
Use
red→[1,0] and blue→[0,1]. Looking up [red,blue] returns a tensor with shape (2,2).The input
Token IDs. Here, the sequence contains red then blue.
The result
One vector per token position.
Why you use it
Learned vectors let nearby numeric directions represent useful token patterns.
A limit to remember
Embedding coordinates do not have fixed human meanings by default.
Check your understanding
- An embedding table has one row per vocabulary item. Each row is trainable.
- This illustration assigns
red→[1,0]andblue→[0,1]. - Looking up two IDs returns two rows. The result has shape
(sequence=2, width=2). - Training changes the selected rows when their gradients are nonzero. Evaluate them on a real task.
- Common mistake: a visually close pair of vectors does not prove equal meaning in every task.