Understanding CNN-Based GAN Architectures: DCGAN, Cycle GAN, Pix2Pix GAN, and StyleGAN

A Convolution neural network (CNN) architecture is used within a Generative Adversarial Network (GAN) mainly for image related tasks.

A CNN-based GAN model was implemented using the following techniques:

1) DCGAN (Deep Convolutional GAN)

2) Cycle GAN

3) Pix2Pix GAN

4) Style GAN

1) Deep Convolution GAN(DCGAN):

Basically, the GAN model consists of two functions, such as Generator and Discriminator. By integrating GAN with a CNN model, the following task was performed:

1. Generator: Generate images using CNN layers, where the generated images are exactly like the original image.

2. Discriminator: Discriminator classifies the generated images from original images with the help of CNN layers.

CNN layers play a crucial role in detecting the edges, patterns, and preserving the spatial relationships in images.

These overall processes are called Deep Convolution GAN(DCGAN).

Architecture of DCGAN

DCGAN is highly useful in healthcare for synthetic image generation, augmentation, enhancement, privacy preserving AI research, especially when medical datasets are limited.

2. Cycle GAN:

Cycle GAN is yet another approach to use a GAN model with CNN, where image-image translation was performed without paired data. Cycle GAN performs tasks such as Forward translation and backward reconstruction.

1. In forward translation, two generator networks were used. Both generators use the convolution layer, residual blocks, and transposed convolution layer. One generator converts domain A images to domain B images. For instance, CT images to MRI. Another generator converts domain B images to domain A images. Here, it converts MRI images to CT images.
2. In backward reconstruction, two discriminator networks were used. Both Discriminator networks invoke the convolution layer, Leaky ReLu, and pathGAN CNN architecture to segregate original and generated synthetic images.

Architecture of CycleGAN

Key features of Cycle GAN:

· Unpaired learning, as no paired datasets are needed.

· Preserve the original structure of the image through the concept of cycle consistency.

• Two generators and two discriminators use the concept of CNN architecture to learn the image features.

· As Adversarial training was provided, it leads to the generation of reality images.

3)Pix2Pix GAN model:

Pix2Pix is a CNN-based GAN model that performs image-to-image translation that converts Google Maps to Google Earth images. Pix2Pix invokes a Conditional GAN architecture. This GAN model uses one generator and one discriminator, where it implements a pretrained CNN model known as the U-Net model for generating the images.

1. The generator network uses the U-net architecture. The U-Net architecture consists of three parts. The first encoder path extracts feature from the images using seven convolution layers, batch normalization, and leaky ReLu. The middle bottleneck path stores the important features of images. The final Decoder path reconstructs the target image by using a transposed convolution layer, batch normalization, and ReLU activation function. This path generates high-resolution output image.

The Skip Connection path connects the encoding path with the decoding path. These skip connections help to preserve fine details, organ boundaries, and spatial information.

2. The Discriminator network uses PatchGAN discriminator for segregating the original image from fake images. PatchGAN splits the images into small patches. Here, it uses Convolution layer, batch normalization, leaky ReLu, PatchGAN, sigmoid output.

Architecture of Pix2Pix GAN

Key features of Pix2Pix:

· Implements image translation using paired training data.

· Uses U-Net generator for preserving the details, and PatchGan discriminator for identifying realistic images.

· Skip connection ensures the boundaries of the images.

4) StyleGAN:

Style GAN is a generative model developed by Nvidia that generates highly realistic images by controlling image features at each level, like texture and lightning. Style GAN separates style from content, as a result changes in the appearance of generated images.

Style GAN also makes use of generator and discriminator networks along with various other layers.

The model starts with a random latent vector that contains random image information. The second component mapping network converts latent vector into intermediate vector. Here, the model learns a disentangled styled representation. The third component style vector controls the style features like texture, color, lightning etc.

The model uses a style GAN generator that uses CNN layers, Upsampling, AdaIN normalization, and Noise injection. The most important component known as AdaIN (Adaptive Instance Normalization) that controls style at different layers. Random noise is added at different layers that creates small variation, fine textures, and natural details.

The fine-tuned generated image resolution was enhanced using progressive upsampling. The model uses a discriminator network to segregate real and fake images. The discriminator network invokes CNN-based network for separation.

Architecture of StyleGAN

Key Features:

· Learn style representation

· Control the image attributes using Style Vector.

· Inject style into the generator network using AdaIN.

· Add realistic variations using noise injection.

· Generate an image using a CNN and the resolution was improved using upsampling.

· CNN Discriminator detects fake images.

Comparison of DC GAN, Cycle GAN, Pix2PIX, Style GAN models