02.11 · UNIT 04 · Supervised learning and reliable experiments · Lesson
Datasets, batching, splits, and trustworthy metrics
Separate data gives parameter learning, model choices, and final testing different jobs.
PLAIN-LANGUAGE INTRODUCTION
The idea
Separate data gives parameter learning, model choices, and final testing different jobs.
A worked example
Split
100 rows into 70 training, 15 validation, and 15 test rows.The input
Examples, targets, and a split rule chosen before training.
The result
Separate batches and honest measurements on untouched rows.
Why you use it
A final test must not guide training or model choices.
A limit to remember
A low training error does not show performance on new data.
Check your understanding
- Training rows change model parameters. Validation rows help choose settings. Test rows give the final check.
- For
100rows, this example assigns70,15, and15rows. - Suppose training values have mean
10and standard deviation2. Transform validation value14to(14−10)/2 = 2. - Fit preprocessing only on training rows. Then reuse those fitted numbers everywhere else.
- Common mistake: fitting a scaler on all rows leaks information from validation and test data.