Block 1 of 7~2 min
What the encoder outputs
A plain autoencoder maps an input to one point and back. It reconstructs well, but it is not a generator: pick a random point in its latent space and the decoder has no reason to produce anything sensible.
The variational autoencoder (Kingma and Welling, 2014) organizes the latent space. Each input is encoded into a Gaussian, not a point:
and every such Gaussian is pulled toward one fixed prior . In code, the encoder ends with two heads: one for and one for .
Question
What does a VAE encoder output for a single input ?