GANs- Generative Adverserial Networks

Deep Convolutional Generative Adversarial Networks (DCGANs) are an extension of GANs (Generative Adversarial Networks) that use deep convolutional neural networks for both the generator and the discriminator models.

Deep Convolutional GAN (DCGAN)

DCGANs were proposed in a paper titled “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks” by Alec Radford, Luke Metz, and Soumith Chintala in 2015.

Good Description

DCGANs aim to generate realistic images by leveraging convolutional layers to learn spatial hierarchies and patterns in data. Unlike fully connected layers used in traditional GANs, DCGANs use convolutional layers for better feature extraction and representation. This architecture makes DCGANs highly effective for image synthesis tasks, such as generating faces, animals, and other objects.

A DCGAN consists of two components:

1. Generator:

• Creates realistic images from random noise vectors sampled from a latent space (e.g., Gaussian or uniform distribution).
• Uses transposed convolutional layers (also known as deconvolutions) to upsample the noise into a high-dimensional image.

2. Discriminator:

• Acts as a binary classifier to distinguish between real and generated images.
• Utilizes convolutional layers to downsample input images and extract discriminative features.

These two networks are trained together in an adversarial manner, where the generator tries to fool the discriminator, and the discriminator tries to correctly identify real vs. fake images.

Key Points

Architecture Highlights:

• Generator: Uses transposed convolutions, batch normalization, and ReLU activations (except for the output layer, which uses Tanh).
• Discriminator: Employs convolutional layers, batch normalization, and Leaky ReLU activations.

Loss Function:

• Both the generator and discriminator optimize a

minimax loss: min⁡ Gmax⁡DE[log⁡D(x)]+E[log⁡(1−D(G(z)))]Gmin​Dmax​E[logD(x)]+E[log(1−D(G(z)))]

Key Design Principles:

• Replace pooling layers with strided convolutions in the discriminator and fractional-strided convolutions in the generator.
• Use batch normalization to stabilize training and avoid mode collapse.
• Avoid fully connected layers to ensure a deeper convolutional architecture.
• Use Tanh activation for the generator’s output to scale values between -1 and 1.

Tools and Frameworks:

• PyTorch or TensorFlow/Keras for model implementation.
• Libraries such as Matplotlib and Pillow for visualizing generated images.
• Datasets like CIFAR-10, CelebA, or custom datasets for training.

Language + Tech Stack:

• Languages: Python is most commonly used.
• Frameworks: PyTorch, TensorFlow/Keras.
• Libraries: NumPy, OpenCV, Matplotlib for preprocessing and visualization.

Common Problems

• Mode Collapse: The generator produces limited variations of data, resulting in repetitive outputs.
• Training Instability: Balancing the generator and discriminator can be challenging.
• Vanishing Gradients: Particularly when the discriminator becomes too strong early in training.

Applications

• Image synthesis.
• Super-resolution.
• Image-to-image translation.
• Domain adaptation.

Let’s Code

Generator

The Generator is responsible for creating realistic images from random noise.

class Generator(nn.Module):
    '''
    Generator Class
    Values:
        z_dim: the dimension of the noise vector, a scalar
        im_chan: the number of channels in the images, fitted for the dataset used, a scalar
              (MNIST is black-and-white, so 1 channel is your default)
        hidden_dim: the inner dimension, a scalar
    '''
    def __init__(self, z_dim=10, im_chan=1, hidden_dim=64):
        super(Generator, self).__init__()
        self.z_dim = z_dim
        # Build the neural network
        self.gen = nn.Sequential(
            self.make_gen_block(z_dim, hidden_dim * 4),
            self.make_gen_block(hidden_dim * 4, hidden_dim * 2, kernel_size=4, stride=1),
            self.make_gen_block(hidden_dim * 2, hidden_dim),
            self.make_gen_block(hidden_dim, im_chan, kernel_size=4, final_layer=True),
        )

# This method builds individual generator blocks based on the parameters.
    def make_gen_block(self, input_channels, output_channels, kernel_size=3, stride=2, final_layer=False):
        '''
        Function to return a sequence of operations corresponding to a generator block of DCGAN, 
        corresponding to a transposed convolution, a batchnorm (except for in the last layer), and an activation.
        Parameters:
            input_channels: how many channels the input feature representation has
            output_channels: how many channels the output feature representation should have
            kernel_size: the size of each convolutional filter, equivalent to (kernel_size, kernel_size)
            stride: the stride of the convolution
            final_layer: a boolean, true if it is the final layer and false otherwise 
                      (affects activation and batchnorm)
        '''

        #     Steps:
        #       1) Do a transposed convolution using the given parameters.
        #       2) Do a batchnorm, except for the last layer.
        #       3) Follow each batchnorm with a ReLU activation.
        #       4) If its the final layer, use a Tanh activation after the deconvolution.

        # Build the neural block
        if not final_layer:
            return nn.Sequential(

               # Upsamples the feature map
                nn.ConvTranspose2d(input_channels, output_channels, kernel_size, stride),

                # Normalizes the output for stability
                nn.BatchNorm2d(output_channels),

                # ntroduces non-linearity to learn complex patterns.
                nn.ReLU(inplace=True)
            )
        else: # Final Layer
            return nn.Sequential(
                nn.ConvTranspose2d(input_channels, output_channels, kernel_size, stride),
                nn.Tanh(),
            )

    def unsqueeze_noise(self, noise):
        '''
        Function for completing a forward pass of the generator: Given a noise tensor, 
        returns a copy of that noise with width and height = 1 and channels = z_dim.
        Parameters:
            noise: a noise tensor with dimensions (n_samples, z_dim)
        '''
        return noise.view(len(noise), self.z_dim, 1, 1)

    def forward(self, noise):
        '''
        Function for completing a forward pass of the generator: Given a noise tensor, 
        returns generated images.
        Parameters:
            noise: a noise tensor with dimensions (n_samples, z_dim)
        '''
        x = self.unsqueeze_noise(noise)
        return self.gen(x)

def get_noise(n_samples, z_dim, device='cpu'):
    '''
    Function for creating noise vectors: Given the dimensions (n_samples, z_dim)
    creates a tensor of that shape filled with random numbers from the normal distribution.
    Parameters:
        n_samples: the number of samples to generate, a scalar
        z_dim: the dimension of the noise vector, a scalar
        device: the device type
    '''
    return torch.randn(n_samples, z_dim, device=device)

Let’s Test

# UNQ_C2 (UNIQUE CELL IDENTIFIER, DO NOT EDIT)
'''
Test your make_gen_block() function
'''
gen = Generator()
num_test = 100

# Test the hidden block
test_hidden_noise = get_noise(num_test, gen.z_dim)
test_hidden_block = gen.make_gen_block(10, 20, kernel_size=4, stride=1)
test_uns_noise = gen.unsqueeze_noise(test_hidden_noise)
hidden_output = test_hidden_block(test_uns_noise)

# Check that it works with other strides
test_hidden_block_stride = gen.make_gen_block(20, 20, kernel_size=4, stride=2)

test_final_noise = get_noise(num_test, gen.z_dim) * 20
test_final_block = gen.make_gen_block(10, 20, final_layer=True)
test_final_uns_noise = gen.unsqueeze_noise(test_final_noise)
final_output = test_final_block(test_final_uns_noise)

# Test the whole thing:
test_gen_noise = get_noise(num_test, gen.z_dim)
test_uns_gen_noise = gen.unsqueeze_noise(test_gen_noise)
gen_output = gen(test_uns_gen_noise)

Discriminator

The second component you need to create is the discriminator.

Let’s Code

class Discriminator(nn.Module):
    '''
    Discriminator Class
    Values:
        im_chan: the number of channels in the images, fitted for the dataset used, a scalar
              (MNIST is black-and-white, so 1 channel is your default)
    hidden_dim: the inner dimension, a scalar
    '''
    def __init__(self, im_chan=1, hidden_dim=16):
        super(Discriminator, self).__init__()
        self.disc = nn.Sequential(
            self.make_disc_block(im_chan, hidden_dim),
            self.make_disc_block(hidden_dim, hidden_dim * 2),
            self.make_disc_block(hidden_dim * 2, 1, final_layer=True),
        )

    def make_disc_block(self, input_channels, output_channels, kernel_size=4, stride=2, final_layer=False):
        '''
        Function to return a sequence of operations corresponding to a discriminator block of DCGAN, 
        corresponding to a convolution, a batchnorm (except for in the last layer), and an activation.
        Parameters:
            input_channels: how many channels the input feature representation has
            output_channels: how many channels the output feature representation should have
            kernel_size: the size of each convolutional filter, equivalent to (kernel_size, kernel_size)
            stride: the stride of the convolution
            final_layer: a boolean, true if it is the final layer and false otherwise 
                      (affects activation and batchnorm)
        '''
        #     Steps:
        #       1) Add a convolutional layer using the given parameters.
        #       2) Do a batchnorm, except for the last layer.
        #       3) Follow each batchnorm with a LeakyReLU activation with slope 0.2.
        #       Note: Don't use an activation on the final layer

        # Build the neural block
        if not final_layer:
            return nn.Sequential(
                nn.Conv2d(input_channels, output_channels, kernel_size, stride),
                nn.BatchNorm2d(output_channels),
                nn.LeakyReLU(0.2, inplace=True)

            )
        else: # Final Layer
            return nn.Sequential(
                nn.Conv2d(input_channels, output_channels, kernel_size, stride)
            )

    def forward(self, image):
        '''
        Function for completing a forward pass of the discriminator: Given an image tensor, 
        returns a 1-dimension tensor representing fake/real.
        Parameters:
            image: a flattened image tensor with dimension (im_dim)
        '''
        disc_pred = self.disc(image)
        return disc_pred.view(len(disc_pred), -1)

• The Discriminator evaluates whether an input image is real or fake using convolutional layers.

Intermediate layers include:

• Convolution, BatchNorm, and LeakyReLU.
• The final layer predicts a single scalar value without activation.

Training

Now you can put it all together!

• criterion: the loss function
• n_epochs: the number of times you iterate through the entire dataset when training
• z_dim: the dimension of the noise vector
• display_step: how often to display/visualize the images
• batch_size: the number of images per forward/backward pass
• lr: the learning rate
• beta_1, beta_2: the momentum term
• device: the device type

criterion = nn.BCEWithLogitsLoss()
z_dim = 64
display_step = 500
batch_size = 128
# A learning rate of 0.0002 works well on DCGAN
lr = 0.0002

# These parameters control the optimizer's momentum, which you can read more about here:
# https://distill.pub/2017/momentum/ but you don’t need to worry about it for this course!
beta_1 = 0.5 
beta_2 = 0.999
device = 'cuda'

# You can tranform the image values to be between -1 and 1 (the range of the tanh activation)
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,)),
])

dataloader = DataLoader(
    MNIST('.', download=False, transform=transform),
    batch_size=batch_size,
    shuffle=True)

Initialize your generator, discriminator, and optimizers.

# Initialize the generator model and move it to the specified device (e.g., GPU or CPU)
gen = Generator(z_dim).to(device)

# Define the optimizer for the generator using the Adam optimizer
# Learning rate (lr) and betas (beta_1, beta_2) are hyperparameters
gen_opt = torch.optim.Adam(gen.parameters(), lr=lr, betas=(beta_1, beta_2))

# Initialize the discriminator model and move it to the specified device
disc = Discriminator().to(device)

# Define the optimizer for the discriminator using the Adam optimizer
# Similar hyperparameters (lr, beta_1, beta_2) are used for consistency
disc_opt = torch.optim.Adam(disc.parameters(), lr=lr, betas=(beta_1, beta_2))

# Function to initialize weights of the model
# Convolutional layers (Conv2d and ConvTranspose2d) weights are initialized with a
# normal distribution having mean 0 and standard deviation 0.02
# BatchNorm2d layers have their weights initialized the same way, with biases set to 0
def weights_init(m):
    if isinstance(m, nn.Conv2d) or isinstance(m, nn.ConvTranspose2d):
        torch.nn.init.normal_(m.weight, 0.0, 0.02)
    if isinstance(m, nn.BatchNorm2d):
        torch.nn.init.normal_(m.weight, 0.0, 0.02)
        torch.nn.init.constant_(m.bias, 0)

# Apply the custom weight initialization function to the generator and discriminator models
gen = gen.apply(weights_init)
disc = disc.apply(weights_init)

GAN Training Loop

# n_epochs Number of times the entire dataset is passed through the models.
n_epochs = 50

# cur_step: Tracks the current step in training (increments every batch).
cur_step = 0

# mean_generator_loss: Cumulative average loss for the generator over 
# display_step iterations.
mean_generator_loss = 0

# mean_discriminator_loss: Cumulative average loss for the discriminator 
# over display_step iterations.
mean_discriminator_loss = 0

 # training the models in each pass.
for epoch in range(n_epochs):
    # Dataloader returns the batches

# Returns batches of real data (real) and optional labels (_, ignored here)
    for real, _ in tqdm(dataloader):

        cur_batch_size = len(real)
        real = real.to(device)

        ## Update discriminator ##

# Clears previous gradients to avoid accumulation during backpropagation.
        disc_opt.zero_grad()

# Creates random noise as input to the generator.
        fake_noise = get_noise(cur_batch_size, z_dim, device=device)

# Passes the noise through the generator to produce fake data.
        fake = gen(fake_noise)

# The discriminator predicts whether the fake data is real or not. 
# The .detach() prevents gradients from flowing back into the generator.
        disc_fake_pred = disc(fake.detach())

# Measures how far the discriminator's predictions are from the correct 
# label (0 for fake).
        disc_fake_loss = criterion(disc_fake_pred, torch.zeros_like(disc_fake_pred))

# The discriminator predicts on real data.
        disc_real_pred = disc(real)

# Measures how far the discriminator's predictions are from the correct label (1 for real).
        disc_real_loss = criterion(disc_real_pred, torch.ones_like(disc_real_pred))


# Combines the loss for real and fake predictions, averaged for stability.
         disc_loss = (disc_fake_loss + disc_real_loss) / 2

        # Keep track of the average discriminator loss
        mean_discriminator_loss += disc_loss.item() / display_step
        # Update gradients
        disc_loss.backward(retain_graph=True)
        # Update optimizer
        disc_opt.step()

        ## Update generator ##

# Clears previous gradients for the generator.
        gen_opt.zero_grad()

# New random noise is passed through the generator.
        fake_noise_2 = get_noise(cur_batch_size, z_dim, device=device)
        fake_2 = gen(fake_noise_2)
        disc_fake_pred = disc(fake_2)

# The generator aims to "fool" the discriminator, so its goal is to 
# maximize the discriminator’s prediction for fake data being real (1).
        gen_loss = criterion(disc_fake_pred, torch.ones_like(disc_fake_pred))

# Computes gradients and updates the generator's parameters.
        gen_loss.backward()
        gen_opt.step()

        # Keep track of the average generator loss
        mean_generator_loss += gen_loss.item() / display_step

        ## Visualization code ##

# Outputs the average losses for the generator and discriminator at every 
# display_step.
        if cur_step % display_step == 0 and cur_step > 0:
            print(f"Epoch {epoch}, step {cur_step}: Generator loss: {mean_generator_loss}, discriminator loss: {mean_discriminator_loss}")
            show_tensor_images(fake)
            show_tensor_images(real)
            mean_generator_loss = 0
            mean_discriminator_loss = 0
        cur_step += 1

That’s it Next is SN-GANs

Thanks

[1] Git: https://github.com/kru2710shna