02.06 · UNIT 02 · Supervised learning and reliable experiments · Lab

Bag of words, TF-IDF, and classical text baselines

Bag of words counts tokens. TF-IDF lowers the influence of common tokens.

PLAIN-LANGUAGE INTRODUCTION

What is this?

Bag of words counts tokens. TF-IDF lowers the influence of common tokens.

One simple example

With vocabulary [bad,good,movie], “good movie” becomes [0,1,1].

Tokenized documents and one fixed vocabulary order.

One numeric feature vector per document.

These fast baselines show whether a neural model adds useful value.

Bag of words does not preserve token order.

WORK THROUGH THE IDEA

A vocabulary assigns one column to each known token. Here, the order is [bad,good,movie].
Count each token in “good movie”. The counts are [0,1,1].
The document “bad movie” becomes [1,0,1]. The shared movie column cannot separate their labels alone.
Term frequency–inverse document frequency (TF-IDF) reduces weight for tokens seen in many documents.
Common mistake: changing vocabulary order between training and testing changes every feature meaning.