Tensor by Tensor

02.23 · UNIT 07 · Optional bridge: probabilistic latent models · Lab

Lab: build a variational autoencoder

A VAE learns a probability distribution for hidden codes, then reconstructs inputs from samples.

PLAIN-LANGUAGE INTRODUCTION

The idea

A VAE learns a probability distribution for hidden codes, then reconstructs inputs from samples.

A worked example

Input [1,0] becomes mean [0.5,0.5]. With unit spread and sample noise [0,0], code z=[0.5,0.5] reconstructs [0.75,0.25].

The input

An input vector and random sample noise.

The result

A sampled hidden code, a reconstruction, and two loss terms.

Why you use it

The probability constraint supports sampling and a smoother hidden space.

A limit to remember

A VAE does not store each training example unchanged.

Check your understanding

  1. Variational autoencoder (VAE) means the encoder predicts a distribution, not one fixed code.
  2. This illustration uses mean [0.5,0.5], unit spread, and chosen noise [0,0]. The sampled code is [0.5,0.5].
  3. The decoder returns [0.75,0.25]. Squared reconstruction error is 0.25²+0.25² = 0.125.
  4. A KL term also compares the code distribution with a chosen prior distribution. Training balances both terms.
  5. Common mistake: the shown zero noise is for clear arithmetic. Real training samples changing noise.
Open the detailed notes ↗