02.21 · UNIT 06 · Sequence models and memory · Lesson
LSTM and GRU gates: controlling memory
LSTM and GRU gates control how much old and new information enters memory.
PLAIN-LANGUAGE INTRODUCTION
The idea
LSTM and GRU gates control how much old and new information enters memory.
A worked example
Old memory
2 with forget gate 0.75 keeps 1.5. New candidate 0.5 with input gate 0.4 adds 0.2. New memory is 1.7.The input
The current token, previous state, and learned gate values.
The result
An updated memory and hidden state.
Why you use it
Gates give gradients and information a controlled path through time.
A limit to remember
A gate does not make a hard yes-or-no choice unless its value reaches 0 or 1.
Check your understanding
- A gate is a number between
0and1. It scales information. - The forget gate keeps
0.75×2 = 1.5from the old memory. - The input gate adds
0.4×0.5 = 0.2from the new candidate. - The new long short-term memory (LSTM) cell value is
1.5+0.2 = 1.7. - Common mistake: gate values are learned for each step. They are not fixed constants in a trained model.