DeepDream 101: How Neural Networks Dream

“All the world is made of faith, trust, and pixie dust”

~ J.M. Barrie, Peter Pan

I was on an introductory quest to understand research papers, when I came across one that stayed with me for a different reason.

Immediately, I was hit with mind-boggling, slightly nightmarish images. Despite trying my hardest to ignore these abominations and just move on with what I had set out to do, I was stuck. Those endless dog-sloth-cyborgs emerging from seemingly nothing were really getting to me.

Figure 1: The unnerving visuals of feature optimization from the Distill.pub dataset

A Brief History of DeepDream

It’s 2015. After the introduction of *AlexNet (2012), neural networks have grown deeper, faster, and wildly accurate. Researchers know that the networks they’re training to classify images are able *to classify them. The final answer is there, but what’s the point if we aren’t able to see the working?

This is the exact idea behind wanting to visualise how a network ‘thinks’. Convolutional Neural Networks (CNNs) extract higher and higher-level features of images at each layer, and then spit out an identification. But knowing what happens at each layer; what the network sees, or what it wants to see, is integral to identifying biases in training data and optimizing results.

It all started with this paper.

Unlike the introduction of transformers, there were no formal definitions or architecture diagrams in the article above. The writers basically said, “Hey, check out this thing I did that shows me what my neural nets are thinking”, and people could implement it using knowledge they already had.

DeepDream occupies a strange place in AI history. It’s extremely famous, historically important, widely cited, and influential in interpretability. Yet, it has no canonical implementation or standard benchmark. It is a true community-driven project that captured public attention through demonstration rather than publication.

Neural networks are a black box, and before this concept, they were essentially left up to faith and trust.

How DeepDream Works

implementation documented here

Building a network consists of defining an architecture and training it in order to build a combination of weights that optimize identification. It does this by creating feature maps that are basically a matrix of the image and contain values as to how much a layer gets activated by that particular feature. Each layer becomes sensitive to increasingly complex patterns.

For example, let’s take this popular image:

Figure 2: Original Image of a Labrador Retriever

A trained neural network, like InceptionV3, would run it through a structural hierarchy where different layers extract different levels of abstraction:

Figure 3: The hierarchical nature of a Convolutional Neural Network. Shallow layers react to local pixel changes like edges and orientation. As data travels deeper into the InceptionV3 architecture, these edges are mathematically combined into complex textures, shapes, and eventually, whole object parts.

Let’s take a very early feature map from this process:

Figure 4: mixed0 feature map 0

As you can see, this map looks like it’s trying to identify edges and lines in the image. If a neuron fires when it sees an edge or other textures, what would make it fire even more strongly? What would activate it even more?

Hence, DeepDream takes an upside-down approach. Instead of changing weights like in standard training, it changes pixels of the input image to maximize activation.

The term ‘Gradient Ascent’ is what all this fuss is about. Its training counterpart, Gradient Descent, is a function that computes loss using weights and tries to minimize it. This is how we get the weights that help identify images. In DeepDream, using gradient ascent, this “loss” (or rather, the activation of the network’s own internal representations) is maximized. Instead of asking “What object is in this image?”, we ask the network: “Whatever you think you see in this image, show me more of it.”

To actually pull this off, the DeepDream algorithm relies on a beautifully orchestrated, dual-loop pipeline:

• The Inner Loop (The Core Ascent): The algorithm takes an image, runs a forward pass through the network, measures the targeted layer’s activation, computes the gradients, and updates the image raw pixels. This tight cycle repeats dozens or hundreds of times at a fixed resolution, steadily amplifying and forcing out whatever microscopic textures the network initially noticed.
• The Outer Loop (The Octave Scale): Once the inner loop completes, the image is scaled up or down, moving across what are called “octaves.” The inner gradient ascent loop then runs all over again at this new resolution. Repeating this macro-cycle 3 to 10 times is the secret sauce: it forces the network to inject complex visual detail at fine, medium, and coarse structural scales simultaneously.

The final image is the result of letting these nested cycles run to completion, leaving behind a landscape where the network’s learned internal representations are permanently etched into the pixels.

Figure 5: An architectural comparison of optimization paths. Standard training (left) uses Gradient Descent to tweak updateable model weights to minimize classification loss on a fixed input image. In contrast, the DeepDream algorithm (right) uses Gradient Ascent to freeze and padlock the pretrained CNN weights, instead utilizing backpropagation (d/dv) to iteratively update and optimize raw input image pixels to maximize selected layer activations (like mixed7).

Trust me, I was pretty confused at this stage too. To cut through the confusion, I decided to build it from scratch. First, I tried to implement the most minimal version of DeepDream on that image in a baseline notebook.

1. The minimal approach (deepdream.ipynb)

Using TensorFlow, I loaded a pretrained InceptionV3 model with its ImageNet weights, explicitly leaving off the classification head. The goal was simple: get inside the hidden layers. I picked a single intermediate layer — mixed7 — to serve as my feature visualization target.

Using tf.GradientTape for automatic differentiation, I defined a minimal loss function as the mean activation of that chosen layer. Then, instead of updating model weights, the gradients were calculated with respect to the image itself. I applied basic gradient normalization to stabilize the step size and iteratively updated the raw pixel values.

Figure 6: Minimal DeepDream output

Not what you were expecting, is it? Kind of a let-down. This image has a lot more noise and ‘blurriness’ compared to Google’s famous examples. Without any constraints, the gradient ascent loop quickly cheats — it finds high-frequency, jagged pixel adjustments that technically satisfy the mathematical loss function but look like complete static to human eyes.

2. Improvements (deepdream2.ipynb)

To fix this, I moved to a second notebook to progressively implement a series of algorithmic improvements.

a. Dreaming across multiple layers

Instead of maximizing activations from a single isolated layer like mixed7, I modified the loss function to pool activations across a combination of layers: mixed3, mixed5, and mixed7 simultaneously.

Because lower layers capture simple lines, intermediate layers capture repeated patterns, and higher layers encode complex semantic parts, optimizing them all at once allowed a richer visual hierarchy to emerge. The static began to organize. The visual patterns became far more diverse, showing a stronger interaction between low-level textures and higher-level shapes.

Figure 7: Multilayer DeepDream output

b. Dreaming at multiple scales (octaves)

Even with multiple layers, the image quality needed a massive boost. The real game-changer was implementing octave processing — a multi-scale optimization technique.

What exactly does upscaling mean here? Instead of running gradient ascent purely on the massive, original high-resolution photo, we downsample the image to a much smaller size first. We run a few steps of DeepDream on that small image so the network can easily formulate large, macro-scale structures. Then, we upscale the image, inject the lost detail back in, and run gradient ascent again to carve out fine, micro-scale textures.

Figure 8: Octaval DeepDream ouput

Octave processing produced the largest qualitative improvement of all modifications. The output shifted dramatically from blurry, noisy artifacts to beautifully sharp, recognizable structures — bringing the experiment closer to Google’s iconic results.

The Art Angle

While splitting an image into octaves gets us remarkably close to Google’s classic results, it only scratches the surface of what the algorithm can actually do. If you leave the script running on its default settings, the process can feel a bit like a wild, uncontrollable trip — you are entirely at the mercy of whatever patterns the network randomly chooses to amplify.

But engineers and creators quickly realized that DeepDream didn’t have to be a chaotic, one-note trick. By hijacking the optimization loops, adjusting the mathematical objectives, and changing how the image was fed back into the network, they found ways to make the outputs incredibly clean, directed, and complex.

This technical evolution took center stage at the historic 2016 Google Arts & Culture DeepDream Exhibition in San Francisco, co-hosted with the Gray Area Foundation. There, the code officially became a digital paintbrush. Instead of letting the network run wild, creators pioneered distinct algorithmic sub-methods to bend the code to their precise artistic intent:

1. Guided DeepDream

The default version of DeepDream maximizes the overall energy of an entire layer, meaning the network will surface whatever it is most biased toward (which, as we know, usually means dogs). Guided DeepDream introduces strict constraints.

• How it works: Instead of letting the loss function look at a whole layer generally, you rewrite the objective function to target a highly specific activation class from the ImageNet dataset — like “violins,” “towers,” or “arches.”
• The Tech Behind It: The gradient ascent loop is heavily penalized if it introduces pixels that don’t match the statistical signature of your chosen target. In artist Mike Tyka’s pieces from the 2016 show, like *Style is Violins*, the network’s internal representation of stringed instruments was explicitly targeted. The algorithm deliberately suppressed its usual puppy-eye tendencies, forcing beautiful, sweeping wooden curves and parallel string lines out of ordinary backgrounds.

2. Iterative DeepDream Zooming

If you’ve ever seen a DeepDream video that looks like an endless, hypnotic tunnel where patterns morph into other patterns forever, you’ve seen iterative zooming.

• How it works: You run a few steps of gradient ascent on an image, zoom into the output by a tiny fraction (say, 1% or 2%), crop it back to the original dimensions, and then feed that modified image right back into the DeepDream algorithm as the starting point for the next frame.
• The Tech Behind It: This creates a recursive feedback loop. The tiny structures the network hallucinated in Frame 1 are magnified in Frame 2, forcing the network to look deeper into its own newly created textures. It drives the model into an escalating cycle of algorithmic pareidolia, turning micro-textures like animal fur into macro-structures like a sprawling castle or a mountain range over time.

3. DeepDream Style Transfer

Right around the time DeepDream was capturing the public’s imagination, Neural Style Transfer (NST) was being developed. The artists at the 2016 exhibition sat right at the intersection of these two breakthroughs.

• How it works: Instead of simply asking a layer to maximize its own internal features, artists used the statistical correlations between different layers (known mathematically as a Gram Matrix) to capture the stylistic texture of one artwork and apply it to the structural content of another image.
• The Tech Behind It: This allowed creators to separate the style of a painting from the content of a photograph, proving that a human operator could control the aesthetic texture of the network’s dream rather than just accepting raw algorithmic noise.

Algorithmic Pareidolia and Human Perception

Going back to the First Image In This Article, can you guess why dogs start to formulate out of thin air when DeepDream is applied to images? It depends greatly on the training data. The dataset that Inception was trained on, called ImageNet, had quite a lot (more than 120) of different dog breeds and not enough of other animals to balance it out. So, an enormous chunk of internal weights and hidden layers became hyper-optimized to detect fur, floppy ears, wet noses, and round eyes . Hence, DeepDream tried to teach the image how to ‘be more dog’.

This insight reveals a bias in the training data, which is essential to understand how to train it better. While it is true that this concept resulted in a wave of different forms of AI art being produced, the main motive has always been to pinpoint what exactly is setting off the neurons in a network, and possibly using it to improve training.

But there is a deeper, almost unsettling realization here. When we look at these surreal, multi-eyed digital hallucinations and feel a strange sense of familiarity, we aren’t just seeing the network’s bias. We are catching a reflection of our own minds.

Have you ever spotted diagrams in blobs of water on the bathroom floor? Seen faces in clouds or mountains? Human beings are evolutionarily hardwired to see meaningful patterns…

:)

…in random shapes. This psychological phenomenon is called pareidolia occurs because our own biological visual cortex operates much like a CNN. Our brains constantly process low-level lines and textures, assembling them into mid-level shapes, and pushing them to high-level semantic concepts so we can instantly spot a threat or a friendly face in the dark.

When DeepDream injects patterned noise into an image, both the artificial network and the human brain looking at the screen try to do the exact same thing: find meaning in the chaos. As artist Memo Akten noted during the 2016 exhibition, the tool serves as a perfect mirror. It reveals that our own perception of reality isn’t just a passive recording of the world — it is an active simulation, a controlled hallucination that we rely on every single day to make sense of the noise around us.

Ultimately, its true legacy isn’t just generating trippy art; it is scientific. It stripped away the blind reliance on “faith and trust” by forcing the model to reveal its inner workings visually. DeepDream is the pixie dust that allows us to step into a neural network and get clarity on our model.

References and Interesting Links

(in no particular order)

https://distill.pub/2017/feature-visualization/#enemy-of-feature-vis

https://abhishekmishra13k.medium.com/using-ai-to-generate-art-an-introduction-to-googles-deepdream-algorithm-b71972b87b95

https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf

https://medium.com/@siddheshb008/alexnet-architecture-explained-b6240c528bd5

https://www.tensorflow.org/tutorials/generative/deepdream

https://medium.com/hackernoon/deep-dream-with-tensorflow-a-practical-guide-to-build-your-first-deep-dream-experience-f91df601f479

https://deepdreamgenerator.com/

https://medium.com/@nettricegaskins/dream-variations-langston-hughes-dreamtime-deepdream-9080c19dfbf5

https://artsandculture.google.com/story/deepdream-the-art-of-neural-networks-gray-area/gAVBUUSCYZ_FNQ?hl=en

https://vimeo.com/132700334