🚀 Tackling Mode Collapse in GANs using DCGAN and WGAN-GP

GANs

Introduction

Generative Adversarial Networks (GANs) are one of the most powerful deep learning frameworks for generating realistic synthetic data such as images, audio, and videos. However, despite their success, GANs suffer from a major problem called mode collapse, where the generator produces limited varieties of outputs instead of capturing the full diversity of the dataset.

In this project, we explore this problem and compare two architectures:

• DCGAN (Deep Convolutional GAN) — baseline model
• WGAN-GP (Wasserstein GAN with Gradient Penalty) — improved stable version

We also analyze how WGAN-GP improves training stability and image diversity.

🎯 Objective of the Project

The main goals of this project are:

• Build a baseline DCGAN model
• Implement WGAN-GP for improved performance
• Compare both models in terms of:
• Training stability
• Image quality
• Diversity of generated samples
• Reduce mode collapse problem
• Visualize and evaluate generated outputs

📊 Dataset Used

We used two datasets:

• Pokémon Sprites Dataset
• Anime Faces Dataset (64×64 images)

Both datasets were resized to 64×64 resolution and normalized to [-1, 1] range for stable GAN training.

⚙️ Data Preprocessing

Before training the model, the following preprocessing steps were applied:

• Resize images to 64×64
• Center crop for uniformity
• Convert images to tensor format
• Normalize pixel values to [-1, 1]

Normalization is important because:

It helps stabilize GAN training by ensuring pixel values are centered around zero, which improves gradient flow.

🧠 DCGAN Architecture (Baseline Model)

DCGAN consists of two networks:

🔹 Generator

• Takes random noise vector (100-dimensional)
• Uses transposed convolution layers
• Batch normalization applied
• ReLU activation used
• Output activation: Tanh

🔹 Discriminator

• Uses convolutional layers
• LeakyReLU activation
• Batch normalization
• Output layer predicts real/fake probability using BCE loss

Loss Function:

Binary Cross Entropy (BCEWithLogitsLoss)

DCGAN works well but suffers from:

• Training instability
• Mode collapse
• Vanishing gradients

⚡ WGAN-GP (Improved Model)

To solve DCGAN limitations, we use WGAN with Gradient Penalty.

🔹 Key Improvements

1. Critic instead of Discriminator

Instead of classifying real/fake, the model assigns a real-valued score.

2. Wasserstein Loss

Instead of BCE, we use:

• Real images → high score
• Fake images → low score

This helps measure distribution distance directly.

3. Gradient Penalty

We enforce Lipschitz constraint by ensuring:

Gradient norm ≈ 1

This stabilizes training and avoids weight clipping issues used in older WGANs.

🔥 Gradient Penalty Concept

Gradient penalty is computed as:

• Interpolate real and fake images
• Compute gradients of critic output
• Penalize deviation from norm = 1

This ensures smooth learning and stable training.

⚙️ Training Strategy

We used the following setup:

• Optimizer: Adam
• Learning rate: 0.0002
• Batch size: 128
• Beta values: (0.5, 0.999)

Training Details:

DCGAN:

• Discriminator and Generator trained alternately

WGAN-GP:

• Critic trained 5 times per generator update
• No sigmoid activation in critic
• No batch normalization in critic

📈 Evaluation Metrics

To evaluate model performance, we used:

1. Diversity Score

We compute pairwise L2 distances between generated images.

• Higher distance = more diversity
• Lower distance = mode collapse

2. Visual Comparison

We compare generated samples from both models.

🔍 Key Observations

DCGAN:

• Faster convergence
• But suffers from mode collapse
• Less diverse outputs

WGAN-GP:

• Slower but stable training
• Higher diversity
• Better image quality
• No mode collapse

📌 Why WGAN-GP Works Better

WGAN-GP improves GAN training because:

• It uses Wasserstein distance instead of probability loss
• Provides smooth gradients
• Enforces Lipschitz constraint via gradient penalty
• Reduces training instability

🧪 Conclusion

This project demonstrates that:

WGAN-GP significantly improves GAN performance compared to DCGAN by reducing mode collapse and improving diversity of generated images.

While DCGAN is simpler and faster, WGAN-GP is more stable and reliable for high-quality image generation.

🚀 Future Work

• Use StyleGAN for higher resolution images
• Apply FID and Inception Score for better evaluation
• Train on larger datasets (CelebA, ImageNet)
• Deploy model using Streamlit or Gradio app

📌 Final Note

This project shows how changing only the loss function and training strategy can drastically improve generative model performance.

DCGAN#WGAN#WGAN_GP#ModeCollapse#ImageGeneration#AIResearch