02.12 · UNIT 04 · Supervised learning and reliable experiments · Lab
Bag of words, TF-IDF, and classical text baselines
Bag of words counts tokens. TF-IDF lowers the influence of common tokens.
PLAIN-LANGUAGE INTRODUCTION
The idea
Bag of words counts tokens. TF-IDF lowers the influence of common tokens.
A worked example
With vocabulary
[bad,good,movie], “good movie” becomes [0,1,1].The input
Tokenized documents and one fixed vocabulary order.
The result
One numeric feature vector per document.
Why you use it
These fast baselines show whether a neural model adds useful value.
A limit to remember
Bag of words does not preserve token order.
Check your understanding
- A vocabulary assigns one column to each known token. Here, the order is
[bad,good,movie]. - Count each token in “good movie”. The counts are
[0,1,1]. - The document “bad movie” becomes
[1,0,1]. The shared movie column cannot separate their labels alone. - Term frequency–inverse document frequency (TF-IDF) reduces weight for tokens seen in many documents.
- Common mistake: changing vocabulary order between training and testing changes every feature meaning.