Day 11: Diffusion Models — How AI Dreams in Pictures and Video

For the first ten days of this challenge, we focused heavily on text. We learned how AI reads, searches, and writes. But over the last two years, the internet has been flooded with stunning AI-generated photography and cinematic video clips.

How does a machine that only understands numbers suddenly know how to direct a science-fiction film or paint a masterpiece? It relies on a completely different branch of AI architecture known as Diffusion.

Under the Hood: The Art of Removing Static

If I asked you to draw a dog, you would likely start with a blank piece of paper, sketch an outline, and color it in. You add details layer by layer.

AI does the exact opposite.

During its training phase, engineers take millions of real photos and slowly add digital “noise” to them — like the fuzzy, snowy static on an old analog television — until the original photo is completely destroyed and only static remains. The AI watches this happen step-by-step. This is called Forward Diffusion.

Eventually, the AI learns how to run this process in reverse. This is Reverse Diffusion, and it is the core of all modern image and video generation.

When you type a prompt like “A futuristic science fiction cinematic city scene,” the AI starts with a canvas of 100% pure static. Because it has studied millions of images of cities and spaceships being destroyed by static, it knows how to reverse the math. It slowly subtracts the noise, pixel by pixel, carving away the static that doesn’t look like a sci-fi city, until a brand-new, never-before-seen image is left behind.

AI generates images by pulling shapes like these out of pure static..

When generating video, the AI simply performs this exact same static-removal process dozens of times per second, ensuring that the new image it generates slightly logically follows the one before it to create smooth motion.

Real-World Applications

1. Free AI Video Generators: Content creators are using tools like Invideo or open-source platforms to type a simple text script and instantly generate highly realistic B-roll footage or creative cinematic scenes without ever picking up a camera.
2. Concept Art and Storyboarding: Film directors and video game designers use diffusion models to rapidly prototype what a character or environment should look like before hiring a team of 3D artists to build it.
3. Medical Imaging Enhancement: Hospitals use diffusion models not to create fake images, but to take low-resolution, “noisy” MRI scans and mathematically remove the static to give doctors a crystal-clear view of the patient.

The Counter-Intuitive Nuance

A massive misconception is that AI image generators are just high-tech collage machines that cut and paste existing artists’ work from Google Images.

This is mathematically false. The AI does not have a database of images stored inside it. It only stores the mathematical rules for how static turns into shapes. Every image or video it generates is constructed from scratch at the pixel level.

However, this is also why AI notoriously struggles with human hands, often drawing six fingers. Hands are incredibly complex, overlapping shapes. Because the AI doesn’t actually know what a “hand” structurally is — it only knows the statistical pattern of pixels that usually make up a hand — it sometimes miscalculates the pattern when clearing away the static.

About the Author

Shoaib Alam is a Machine Learning Engineer and AI researcher with 3+ years of experience in Natural Language Processing (NLP) and Large Language Models (LLMs). An M.Tech graduate from IIT Gandhinagar, he focuses on making AI accessible across linguistic boundaries, highlighted by his recent publication at EMNLP 2024 on multilingual representation learning. Through both his technical research and content creation, Shoaib is passionate about bridging the gap between state-of-the-art AI innovations and real-world, inclusive applications.