02.17 · UNIT 04 · Sequence models and memory · Lab

Lab: train a gated sequence predictor

A gated sequence lab tests whether controlled memory helps a prediction task.

PLAIN-LANGUAGE INTRODUCTION

What is this?

A gated sequence lab tests whether controlled memory helps a prediction task.

One simple example

For inputs [1,0,1], an illustrative gated model keeps hidden states [0.2,0.15,0.4] and predicts target 1 with score 0.8.

Training sequences, targets, and initial hidden states.

Predictions, losses, and held-out measurements.

The lab connects gate equations to training and evaluation.

One correct sequence does not prove that the model learned the rule.

WORK THROUGH THE IDEA

Use sequence [1,0,1] with target 1. Keep its order unchanged.
The shown hidden states [0.2,0.15,0.4] are plausible illustrative outputs. They are separate from the earlier plain RNN rule.
Suppose the output score is 0.8. Compare it with the target using the chosen loss.
Train on training sequences. Select settings with validation data. Report final held-out results separately.
Common mistake: checking only training sequences can hide poor performance on unseen sequences.