Tensor by Tensor

02.12 · UNIT 04 · Supervised learning and reliable experiments · Lab

Bag of words, TF-IDF, and classical text baselines

Bag of words counts tokens. TF-IDF lowers the influence of common tokens.

PLAIN-LANGUAGE INTRODUCTION

The idea

Bag of words counts tokens. TF-IDF lowers the influence of common tokens.

A worked example

With vocabulary [bad,good,movie], “good movie” becomes [0,1,1].

The input

Tokenized documents and one fixed vocabulary order.

The result

One numeric feature vector per document.

Why you use it

These fast baselines show whether a neural model adds useful value.

A limit to remember

Bag of words does not preserve token order.

Check your understanding

  1. A vocabulary assigns one column to each known token. Here, the order is [bad,good,movie].
  2. Count each token in “good movie”. The counts are [0,1,1].
  3. The document “bad movie” becomes [1,0,1]. The shared movie column cannot separate their labels alone.
  4. Term frequency–inverse document frequency (TF-IDF) reduces weight for tokens seen in many documents.
  5. Common mistake: changing vocabulary order between training and testing changes every feature meaning.
Open the detailed notes ↗