Tensor by Tensor

02.11 · UNIT 04 · Supervised learning and reliable experiments · Lesson

Datasets, batching, splits, and trustworthy metrics

Separate data gives parameter learning, model choices, and final testing different jobs.

PLAIN-LANGUAGE INTRODUCTION

The idea

Separate data gives parameter learning, model choices, and final testing different jobs.

A worked example

Split 100 rows into 70 training, 15 validation, and 15 test rows.

The input

Examples, targets, and a split rule chosen before training.

The result

Separate batches and honest measurements on untouched rows.

Why you use it

A final test must not guide training or model choices.

A limit to remember

A low training error does not show performance on new data.

Check your understanding

  1. Training rows change model parameters. Validation rows help choose settings. Test rows give the final check.
  2. For 100 rows, this example assigns 70, 15, and 15 rows.
  3. Suppose training values have mean 10 and standard deviation 2. Transform validation value 14 to (14−10)/2 = 2.
  4. Fit preprocessing only on training rows. Then reuse those fitted numbers everywhere else.
  5. Common mistake: fitting a scaler on all rows leaks information from validation and test data.
Open the detailed notes ↗