Tensor by Tensor

02.36 · UNIT 10 · Train, evaluate, and serve your transformer · Lab

NLP applications: classification, tagging, retrieval, and reranking

An NLP task starts by defining the exact output and how success will be measured.

PLAIN-LANGUAGE INTRODUCTION

The idea

An NLP task starts by defining the exact output and how success will be measured.

A worked example

For query “card charged twice”, a retriever gives duplicate-charge help score 0.82 and reset-PIN score 0.15.

The input

A query plus a fixed collection of candidate documents.

The result

A ranked document list with one relevance score per candidate.

Why you use it

A clear output selects the right labels, baseline, and metric.

A limit to remember

A retrieval score is not automatically a calibrated probability.

Check your understanding

  1. This example uses retrieval. The required output is a ranked list of help articles.
  2. The illustrative scores are 0.82 for duplicate charges and 0.15 for resetting a PIN.
  3. Sort larger scores first. The duplicate-charge article ranks above the reset-PIN article.
  4. Compare with a count-based baseline on the same queries and candidate collection.
  5. Common mistake: accuracy for one label per document does not measure ranking quality well.
Open the detailed notes ↗