Tensor by Tensor

02.15 · UNIT 05 · Text representations and neural language baselines · Lab

Convolutional networks: locality and shared weights

A convolution reuses one small weight pattern at many nearby positions.

PLAIN-LANGUAGE INTRODUCTION

The idea

A convolution reuses one small weight pattern at many nearby positions.

A worked example

Apply kernel [[1,0],[0,−1]] to the top-left of [[1,1,0],[1,0,1],[0,1,1]]. The result is 1.

The input

A numeric grid and a smaller kernel grid.

The result

A feature grid with one response per valid position.

Why you use it

Shared weights detect the same local pattern in different places.

A limit to remember

A convolution does not compare every position with every other position.

Check your understanding

  1. A kernel is a small table of shared weights. Here, its shape is (2,2).
  2. The first input patch is [[1,1],[1,0]]. Multiply matching entries.
  3. Add the products: 1×1 + 1×0 + 1×0 + 0×(−1) = 1.
  4. Slide the same kernel to each valid position. Reusing weights keeps the parameter count small.
  5. Common mistake: output size changes with kernel size, padding, and stride.
Open the detailed notes ↗