02.36 · UNIT 10 · Train, evaluate, and serve your transformer · Lab
NLP applications: classification, tagging, retrieval, and reranking
An NLP task starts by defining the exact output and how success will be measured.
PLAIN-LANGUAGE INTRODUCTION
The idea
An NLP task starts by defining the exact output and how success will be measured.
A worked example
For query “card charged twice”, a retriever gives duplicate-charge help score
0.82 and reset-PIN score 0.15.The input
A query plus a fixed collection of candidate documents.
The result
A ranked document list with one relevance score per candidate.
Why you use it
A clear output selects the right labels, baseline, and metric.
A limit to remember
A retrieval score is not automatically a calibrated probability.
Check your understanding
- This example uses retrieval. The required output is a ranked list of help articles.
- The illustrative scores are
0.82for duplicate charges and0.15for resetting a PIN. - Sort larger scores first. The duplicate-charge article ranks above the reset-PIN article.
- Compare with a count-based baseline on the same queries and candidate collection.
- Common mistake: accuracy for one label per document does not measure ranking quality well.