02.18 · UNIT 05 · Optional bridge: probabilistic latent models · Lab
Lab: build a variational autoencoder
A VAE learns a probability distribution for hidden codes, then reconstructs inputs from samples.
PLAIN-LANGUAGE INTRODUCTION
What is this?
A VAE learns a probability distribution for hidden codes, then reconstructs inputs from samples.
One simple example
Input [1,0] becomes mean [0.5,0.5]. With unit spread and sample noise [0,0], code z=[0.5,0.5] reconstructs [0.75,0.25].
What goes in?
An input vector and random sample noise.
What comes out?
A sampled hidden code, a reconstruction, and two loss terms.
Why does it matter?
The probability constraint supports sampling and a smoother hidden space.
What is it not?
A VAE does not store each training example unchanged.
WORK THROUGH THE IDEA
See the idea in more detail
- Variational autoencoder (VAE) means the encoder predicts a distribution, not one fixed code.
- This illustration uses mean
[0.5,0.5], unit spread, and chosen noise[0,0]. The sampled code is[0.5,0.5]. - The decoder returns
[0.75,0.25]. Squared reconstruction error is0.25²+0.25² = 0.125. - A KL term also compares the code distribution with a chosen prior distribution. Training balances both terms.
- Common mistake: the shown zero noise is for clear arithmetic. Real training samples changing noise.