VAEs and GANs: Two Classic Generative AI Ideas That Still Feel Like Magic

How hidden maps and creative competition taught machines to generate.Before diffusion models and multimodal AI took over the headlines, VAEs and GANs taught machines two powerful ways to imagine: by mapping hidden structure and by learning through competition.A VAE is a cartographer. A GAN is a…

How hidden maps and creative competition taught machines to generate.Before diffusion models and multimodal AI took over the headlines, VAEs and GANs taught machines two powerful ways to imagine: by mapping hidden structure and by learning through competition.A VAE is a cartographer. A GAN is a rival artist. One learns the map of imagination; the other learns to fool the critic.Why I am writing about VAEs and GANs nowGenerative AI feels new because the tools around us are new: text-to-image apps, AI video, voice cloning, design copilots, synthetic data platforms, and multimodal assistants. But many of the ideas behind modern generative AI were shaped by two older, beautifully simple architectures: Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs).While experimenting with a VAE on MNIST handwritten digits, I had a small but important learning moment. At 5 epochs, the latent space looked messy. At 15 epochs, digit regions began to form. At 30 epochs, clusters became more meaningful and generated digits became more recognisable. That is when “latent space” stopped being just a term from a lecture and started feeling like a map I could explore.That is why VAEs and GANs are still worth learning. They are not just historical models. They teach the grammar of generative AI.A short history: where did these ideas come from?The Variational Autoencoder was introduced by Diederik P. Kingma and Max Welling in the 2013 paper *Auto-Encoding Variational Bayes*. The big idea was to combine neural networks with probabilistic inference, using the reparameterization trick so that latent-variable models could be trained efficiently with gradient descent.The Generative Adversarial Network arrived shortly after in the 2014 paper *Generative Adversarial Nets* by Ian Goodfellow and collaborators. GANs introduced a new training game: a generator creates fake samples, while a discriminator learns to tell real from fake. The generator improves by trying to fool the discriminator.Two models. Two philosophies.- VAE: learn a smooth, structured latent space.- GAN: learn realism through adversarial pressure.The simplest intuition.### VAE: learn the hidden mapA VAE compresses data into a latent distribution and then decodes a sample from that distribution back into data.Instead of saying, “this image maps to exactly this point,” a VAE says, “this image probably lives around this area.” That small probabilistic shift is powerful. It makes the latent space smoother, more continuous, and more useful for generation.A typical VAE has:An encoder that maps input data to mean and variance.A sampling step that draws a latent vector.A decoder that reconstructs or generates data.A loss made of reconstruction error + KL divergence.In MNIST, this means the model learns where digits live in a two-dimensional or higher-dimensional hidden space.GAN: learn through competitionA GAN does not reconstruct an input. It starts from random noise.The generator tries to create realistic samples. The discriminator tries to detect whether each sample is real or fake. Over time, the generator becomes better because the discriminator keeps raising the standard.This is why GANs became famous for sharp, realistic images. The model is not directly rewarded for “average correctness.” It is rewarded for realism.VAE vs GAN: the core differencesMy personal shortcut:Use a VAE when you want to understand and navigate the hidden structure of data. Use a GAN when you want outputs that look impressively real.Why should an AI learner still learn these models?Because VAEs and GANs teach skills that transfer far beyond MNIST.A VAE teaches you:What a latent space is.How probabilistic generation works.Why sampling matters.How reconstruction and regularisation balance each other.Why smooth representations are useful.A GAN teaches you:How two networks can train each other.Why loss design is not always straightforward.Why generation is different from classification.How realism can be learned without pixel-by-pixel reconstruction.Why training stability matters in deep learning.Even if you later study diffusion models, transformers, or multimodal models, these ideas keep coming back.A small realistic VAE code example: visualising MNIST latent spaceThe most satisfying part of a 2D VAE is plotting the latent space. Here is a clean version of the plotting logic you can use after training a VAE encoder on MNIST.import matplotlib.pyplot as pltimport numpy as np# x_test: MNIST images, y_test: digit labels 0–9# encoder should return z_mean, z_log_var, z_sampleddef plot_latent_space(encoder, x_test, y_test, batch_size=128):z_mean, z_log_var, z = encoder.predict(x_test, batch_size=batch_size)plt.figure(figsize=(8, 6))scatter = plt.scatter(z_mean[:, 0],z_mean[:, 1],c=y_test,cmap="tab10",alpha=0.7,s=3)plt.colorbar(scatter, ticks=range(10), label="Digit label")plt.xlabel("Latent dimension 1")plt.ylabel("Latent dimension 2")plt.title("2D VAE Latent Space on MNIST")plt.grid(True, alpha=0.2)plt.show()When you train for more epochs, this plot usually becomes more organised. Digits that look similar may live near each other. For example, 4 and 9 may overlap in some regions, while 0 often forms a more distinct area because its shape is visually different.Generating digits by walking through latent spaceOnce the decoder is trained, we can sample points from latent space and ask the decoder to turn them into digits.def plot_generated_digits(decoder, n=15, digit_size=28):figure = np.zeros((digit_size * n, digit_size * n))grid_x = np.linspace(-3, 3, n)grid_y = np.linspace(-3, 3, n)[::-1]for i, yi in enumerate(grid_y):for j, xi in enumerate(grid_x):z_sample = np.array([[xi, yi]])generated = decoder.predict(z_sample, verbose=0)digit = generated[0].reshape(digit_size, digit_size)figure[i * digit_size: (i + 1) * digit_size,j * digit_size: (j + 1) * digit_size] = digitplt.figure(figsize=(10, 10))plt.imshow(figure, cmap="gray")plt.axis("off")plt.title("Digits Generated from VAE Latent Space")plt.show()This is the moment where a VAE becomes more than a reconstruction model. You are no longer only testing whether it remembers digits. You are asking it to imagine new ones.A tiny GAN training loop for intuitionA GAN training loop looks different because we train two models: the discriminator and the generator.import torchimport torch.nn as nncriterion = nn.BCELoss()latent_dim = 100for real_images, _ in dataloader:real_images = real_images.to(device)batch_size = real_images.size(0)real_labels = torch.ones(batch_size, 1, device=device)fake_labels = torch.zeros(batch_size, 1, device=device)# 1. Train Discriminatornoise = torch.randn(batch_size, latent_dim, device=device)fake_images = generator(noise)real_loss = criterion(discriminator(real_images), real_labels)fake_loss = criterion(discriminator(fake_images.detach()), fake_labels)d_loss = real_loss + fake_lossd_optimizer.zero_grad()d_loss.backward()d_optimizer.step()# 2. Train Generatornoise = torch.randn(batch_size, latent_dim, device=device)fake_images = generator(noise)# Generator wants discriminator to classify fake images as realg_loss = criterion(discriminator(fake_images), real_labels)g_optimizer.zero_grad()g_loss.backward()g_optimizer.step()Notice the difference in mindset:- In a VAE, the model asks: “Can I reconstruct and sample smoothly?”- In a GAN, the model asks: “Can I fool the critic?”What is being built on these ideas today?1. Latent diffusion modelsModern image generation systems often use a latent representation rather than generating directly in pixel space. Latent diffusion models compress images into a lower-dimensional latent space, perform generation there, and decode the result back into images. This is very close in spirit to what VAEs teach us: the power of a meaningful latent space.2. VQ-VAE and discrete representationsVQ-VAE extended the VAE family by learning discrete latent codes. This idea influenced later work in image, audio, video, and token-like representations. It is one reason VAEs still matter even in the era of transformer-style thinking.3. StyleGAN and controllable image generationGANs evolved from simple image generators into highly controllable systems like StyleGAN. These models made latent space editing popular: changing age, pose, expression, style, and other semantic attributes by moving in latent space.4. Super-resolution and enhancementGAN-based models such as SRGAN and ESRGAN became important for image super-resolution because adversarial loss helps create visually realistic texture. This is useful in media, design, restoration, gaming, and medical imaging research.5. Hybrid systemsThe future is not “VAE or GAN.” It is increasingly hybrid. Modern systems borrow from VAEs for compression and representation, GANs for realism and adversarial feedback, diffusion for iterative generation, and transformers for scaling and conditioning.How these models evolvedThe evolution looks something like this:Autoencoders taught models to compress and reconstruct.VAEs made autoencoders probabilistic and generative.GANs made generation sharper through competition.StyleGAN, BigGAN, and ESRGAN improved quality, control, and resolution.VQ-VAE and VQGAN-like ideas connected latent representations with token-like generation.Diffusion models improved stability and diversity.Latent diffusion brought the VAE-style latent space back into the centre of high-quality generation.Multimodal generative AI now combines text, image, audio, video, and 3D.So, learning VAEs and GANs is not learning outdated history. It is learning the ancestry of the tools we use today.My takeaway as an AI learnerWhen I first saw MNIST digits generated by a VAE, they looked simple. Almost too simple. But the deeper lesson was not the digit itself. It was the space between digits.A good latent space is like a geography of meaning. Move a little, and a 3 becomes an 8. Move somewhere else, and a 1 starts looking like a 7. The model is not just memorising images; it is learning a compressed world where variation has direction.GANs gave me a different lesson. Intelligence can emerge from pressure. The generator improves because the discriminator refuses to be easily fooled. That adversarial setup feels almost social: one model creates, the other critiques, and both become better.This is also a useful mindset for learning AI itself. Build, evaluate, improve. Generate, criticise, repeat.The future: where do VAEs and GANs go next?I do not think the future belongs to one architecture. The future belongs to useful combinations.VAEs will continue to matter wherever we need compact, structured, controllable representations. GANs will continue to matter wherever realism, speed, and perceptual quality are important. Diffusion and consistency-style models are pushing generation toward better quality and faster sampling. Transformers are making generative systems more scalable and multimodal.The next wave will likely focus on faster generation, better controllability, smaller and more efficient models, after synthetic data, better evaluation of generated content, and multimodal generation across text, image, audio, video, and 3D.But under all of that, the core questions remain familiar:What is the hidden structure of the data?How do we sample from it?How do we make outputs realistic?How do we control what gets generated?VAEs and GANs are two of the best starting points for answering those questions.Final thoughtIf you are learning generative AI, do not skip VAEs and GANs just because newer models are trending.A VAE will teach you how machines organise imagination.A GAN will teach you how machines improve through competition.Together, they make the jump from “AI can classify” to “AI can create.”And that is still one of the most exciting jumps in machine learning.ReferencesKingma, D. P., & Welling, M. (2013). Auto-Encoding Variational Bayes.https://arxiv.org/abs/1312.6114Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative Adversarial Nets.https://proceedings.neurips.cc/paper/2014/hash/f033ed80deb0234979a61f95710dbe25-Abstract.htmlTensorFlow. Convolutional Variational Autoencoder.https://www.tensorflow.org/tutorials/generative/cvaeRombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-Resolution Image Synthesis with Latent Diffusion Models.https://openaccess.thecvf.com/content/CVPR2022/html/Rombach_High-Resolution_Image_Synthesis_With_Latent_Diffusion_Models_CVPR_2022_paper.htmlvan den Oord, A., Vinyals, O., & Kavukcuoglu, K. (2017). Neural Discrete Representation Learning.https://arxiv.org/abs/1711.00937Karras, T., Aittala, M., Laine, S., Härkönen, E., Hellsten, J., Lehtinen, J., & Aila, T. (2021). Alias-Free Generative Adversarial Networks.https://proceedings.neurips.cc/paper/2021/hash/076ccd93ad68be51f23707988e934906-Abstract.htmlWang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Loy, C. C., Qiao, Y., & Tang, X. (2018). ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks.https://arxiv.org/abs/1809.00219This story is published on Generative AI. Connect with us on LinkedIn and follow Zeniteq to stay in the loop with the latest AI stories.Subscribe to our newsletter and YouTube channel to stay updated with the latest news and updates on generative AI. Let’s shape the future of AI together!VAEs and GANs: Two Classic Generative AI Ideas That Still Feel Like Magic was originally published in Generative AI on Medium, where people are continuing the conversation by highlighting and responding to this story.

Source: Generative AI Pub — Published August 10, 2026 at 05:32 UTC — Category: Image AI