Beyond the Noise: Mastering GAN Stability and Image Translation with WGAN-GP, Pix2Pix, and CycleGAN

They say a picture is worth a thousand words, but in the world of Generative Adversarial Networks (GANs), a picture is usually worth a thousand failed training runs. We’ve all seen it: the dreaded Mode Collapse, where your Generator gets “lazy” and spits out the same blurry blob repeatedly, or the “Loss: NaN” error that haunts your logs.

But what if you could stabilize that chaos? What if you could turn a rough pencil sketch into a photorealistic face, or transform a doodle into a masterpiece without ever showing the model a paired example?

In this project, I went under the hood of three major GAN architectures to solve the stability crisis and master image-to-image translation. Here’s how I did it using PyTorch and Kaggle’s Dual T4 GPUs.

1. The Stability Battle: DCGAN vs. WGAN-GP

The first challenge was tackling the infamous Mode Collapse.

• The Baseline (DCGAN): Using standard Binary Cross Entropy (BCE) loss, the DCGAN is a classic, but it’s finicky. It often struggles to learn the full diversity of a dataset (like Pokemon sprites or Anime faces).
• The Upgrade (WGAN-GP): To fix this, I implemented Wasserstein GAN with Gradient Penalty. By replacing the Discriminator with a “Critic” and using the Earth Mover’s distance instead of BCE, the training becomes remarkably stable.
• Result: No more mode collapse. The WGAN-GP produces a diverse range of high-quality sprites, even when the DCGAN would have folded.

2. Controlled Creativity: Pix2Pix (Paired Translation)

Next, I moved into Conditional GANs. Unlike standard GANs that generate images from random noise, Pix2Pix takes an input image (like a sketch) and “translates” it into a target domain (like a colored face).

• The Architecture: A U-Net Generator with skip connections. These connections are vital — they allow low-level structural information from the sketch to bypass the bottleneck, ensuring the final output perfectly aligns with the original drawing.
• The Critic: A PatchGAN discriminator that looks at local 16x16 patches rather than the whole image, forcing the model to focus on high-frequency details and textures.

3. The Holy Grail: CycleGAN (Unpaired Translation)

The hardest part of AI art is finding paired data. What if you have a folder of sketches and a folder of photos, but they aren’t of the same things?

Enter CycleGAN. Using Cycle Consistency Loss, the model learns that if you translate a Sketch to a Photo and then back to a Sketch, you should end up with exactly what you started with.

• Tech Highlight: I used a ResNet-based Generator (optimized to 6 ResNet blocks for the Kaggle T4 environment) to handle the complex domain mapping between the TU-Berlin Sketch dataset and real-world photos.

The Tech Stack & Optimization Strategy

Training three different GAN architectures is a memory nightmare. To make this work on Kaggle’s Dual T4 GPUs, I employed several “Pro” techniques:

• Mixed Precision (torch.cuda.amp): Drastically reduced VRAM usage and sped up training.
• Gradient Penalty ($\lambda=10$): Used in WGAN-GP to enforce the 1-Lipschitz constraint.
• Gradio Deployment: Wrapped the entire system into an interactive app where you can upload a sketch and see the GANs work their magic in real-time.

Conclusion

Generative AI is more than just prompting a black box; it’s about understanding the mathematical constraints that keep models from collapsing. By moving from DCGAN to CycleGAN, this project demonstrates the transition from “generating noise” to “understanding structure.”