02.16 · UNIT 04 · Sequence models and memory · Lesson

LSTM and GRU gates: controlling memory

LSTM and GRU gates control how much old and new information enters memory.

PLAIN-LANGUAGE INTRODUCTION

What is this?

LSTM and GRU gates control how much old and new information enters memory.

One simple example

Old memory 2 with forget gate 0.75 keeps 1.5. New candidate 0.5 with input gate 0.4 adds 0.2. New memory is 1.7.

The current token, previous state, and learned gate values.

An updated memory and hidden state.

Gates give gradients and information a controlled path through time.

A gate does not make a hard yes-or-no choice unless its value reaches 0 or 1.

WORK THROUGH THE IDEA

A gate is a number between 0 and 1. It scales information.
The forget gate keeps 0.75×2 = 1.5 from the old memory.
The input gate adds 0.4×0.5 = 0.2 from the new candidate.
The new long short-term memory (LSTM) cell value is 1.5+0.2 = 1.7.
Common mistake: gate values are learned for each step. They are not fixed constants in a trained model.