02.10 · UNIT 03 · Text representations and neural language baselines · Lab
Convolutional networks: locality and shared weights
A convolution reuses one small weight pattern at many nearby positions.
PLAIN-LANGUAGE INTRODUCTION
What is this?
A convolution reuses one small weight pattern at many nearby positions.
One simple example
Apply kernel [[1,0],[0,−1]] to the top-left of [[1,1,0],[1,0,1],[0,1,1]]. The result is 1.
What goes in?
A numeric grid and a smaller kernel grid.
What comes out?
A feature grid with one response per valid position.
Why does it matter?
Shared weights detect the same local pattern in different places.
What is it not?
A convolution does not compare every position with every other position.
WORK THROUGH THE IDEA
See the idea in more detail
- A kernel is a small table of shared weights. Here, its shape is
(2,2). - The first input patch is
[[1,1],[1,0]]. Multiply matching entries. - Add the products:
1×1 + 1×0 + 1×0 + 0×(−1) = 1. - Slide the same kernel to each valid position. Reusing weights keeps the parameter count small.
- Common mistake: output size changes with kernel size, padding, and stride.