From Noise to Faces: Building a Denoising Diffusion Probabilistic Model (DDPM) from Scratch

A hands-on guide to implementing the theory behind Stable Diffusion’s core engine using PyTorch.

If you’ve been following the AI art revolution, you’ve heard of Stable Diffusion, DALL-E, and Midjourney. But what is actually happening under the hood? While these models seem like magic, the core technology powering many of them is the Denoising Diffusion Probabilistic Model (DDPM) .

In this article, I’ll walk you through my journey of building a DDPM from scratch to generate high-quality human faces. We’ll break down the math, the architecture, and the code — no black boxes, just PyTorch.

1. The Core Idea: What is Diffusion?

Imagine an artist who only knows how to remove “noise.” You show them a canvas of pure static TV noise. Step by step, they gently remove a little bit of noise, refine the shapes, and eventually — after several hundred steps — a face emerges. That is diffusion in a nutshell.

The process has two stages:

• Forward Process (Training): We take a real face image and gradually add Gaussian noise over T steps (e.g., 200 steps). Eventually, the image turns into pure random noise.
• Reverse Process (Sampling): We train a neural network to predict the noise we added at each step. Once trained, we start with pure noise and run the process backward to “denoise” a face into existence.

2. The Architecture: U-Net with Attention

The brain of the operation is a U-Net. Standard U-Nets are good at segmentation, but for diffusion, we need to tell the network how much noise has been added.

The architecture features:

• Downsampling and Upsampling blocks to capture global features.
• Residual connections to preserve fine details.
• Attention mechanisms to understand facial features like eyes and mouths.
• Time Embeddings: We pass the current timestep (t) to the network so it knows whether it is removing heavy noise (early steps) or doing fine detail work (late steps).

3. Data Preprocessing

Quality in, quality out. I used a dataset of 30,000 celebrity faces. The preprocessing pipeline was simple:

• Resize images to 128x128.
• Random horizontal flipping for augmentation.
• Normalize pixel values to the range [-1, 1] (standard for DDPMs).

4. The Training Loop

Training was the most intensive part. Here is the simplified logic:

1. Pick a random image from the dataset.
2. Pick a random timestep t between 1 and 200.
3. Generate random Gaussian noise.
4. Corrupt the image by mixing the real image and the noise based on a cosine schedule (which determines how much noise is added at step t).
5. Feed the noisy image and the timestep t into the U-Net. The U-Net tries to predict the noise we added.
6. Calculate the loss (Mean Squared Error between predicted noise and actual noise) and backpropagate.

5. The Results: Watching Faces Emerge

After training for 20 epochs (and several hours of GPU time on an NVIDIA T4), the model began to understand the latent space of human faces.

Progression of Sampling:

• Step 0: Pure noise.
• Step 100: Fuzzy shapes and shadow patterns start to look like hair and face outlines.
• Step 50: Symmetry and specific facial features (eyes, nose, mouth placement) begin to solidify.
• Step 0 (Final): A sharp, novel human face that does not exist in the original dataset!

6. Evaluation Metrics: PSNR & SSIM

To quantitatively measure how well our generated images matched the distribution of real faces, I used:

• PSNR (Peak Signal-to-Noise Ratio): Measures pixel-level accuracy.
• SSIM (Structural Similarity Index): Measures the perceived quality of the structure (edges, textures).

The scores confirmed that the model effectively learned the distribution of the training data.

7. Making it Interactive with Gradio

Theory is great, but seeing is believing. I wrapped the model in a Gradio interface. You can click a button, and the model will generate a new random face in real-time.

Takeaways: Building a DDPM from scratch is a rite of passage for deep learning engineers. It bridges the gap between simple GANs and the complex multi-modal models dominating the industry today. While the training is computationally heavy, the mathematical elegance of “learning to remove noise” is surprisingly simple and incredibly powerful.