Visualizing ReLU Topology in Neural Networks (Thinking Out of BlackBox Why and How ReLU works)

We all know we need to add non linear activation functions to neural nets to make it work otherwise it is just a Linear Regression with extra steps though what bugged me was ReLU(probably the most famous activation function) how come just adding a kink at origin make it predict all kinds of complex shaped decision boundaries in hyperspace. That’s what we will be exploring in this article.

Introduction: The Illusion of Smoothness

we often Think of Neural Networks as smooth continuous function approximators, but in actuality the ReLU function divides the input space into Shattered Crystal like convex Polyhedrons. we will be trying to visualize these structures in this article.

Part 1: Address of a Data point (some prerequisite concepts and terms)

So the first question should be how does the network define these regions that we are talking about? It gives every single point in the input space a binary address.

Consider a single neuron, it calculates z = w.x + b

if z > 0, ReLU acts a identity function{f(x) = x} and we call this neuron “Active”.

if z ≤ 0, ReLU kills it and we call it “Inactive”.

For an input x, if we track this active and Inactive states for each layer’s neurons and stack them together, we get a binary code called Binary Vector which is essentially the address of that point in our input space. I’ll be using address and Binary Vector interchangibly here since they are the same thing in this context.

For example let’s take a network with only 2 hidden layers with 1 neuron each.

say w_1=1 , b_1 = -1 and w_2 = -2 , b_2 = 3

if x = 3

for layer 1: y_1 = w_1 * x + b_1 = 1(3) -1 = 2

so, y_1 = 2 > 0 (Hence Active, s_1= 1)

for Layer 2: input will be output of layer 1 i.e. y_1

y_2 = w_2 * y_1 + b_2 = -2(2) + 3 = -1

y_2 = -1 < 0 (Hence Inactive, s_2 = 0)

so our Binary Vector = [s1 s2] = [1, 0]

For a network with H hidden neurons, this vector s(x) is the “digital fingerprint” of that input region. All input points that share the exact same binary fingerprint live in the same convex polyhedron. Within this region, the network behaves as a single, fixed affine linear transformation.

Python class for method to compute Binary Vector

class ReLUNetwork:
  def __init__(self, layer_sizes):
    self.weights = []
    self.biases = []
    self.layer_sizes = layer_sizes

    np.random.seed(32)
    for i in range(len(self.layer_sizes)-1):
      #He initialization
      scale = np.sqrt(2.0 / self.layer_sizes[i])
      W = np.random.randn(self.layer_sizes[i+1], self.layer_sizes[i]) * scale
      b = np.random.randn(self.layer_sizes[i+1]) * 0.1
      self.weights.append(W)
      self.biases.append(b)

  def get_binary_vector(self, x):
    activation = x
    binary_code = []

    for i in range(len(self.weights)-1):
      W = self.weights[i]
      b = self.biases[i]
      z = np.dot(W, activation) + b

      layer_bits = (z>0).astype(int)
      binary_code.extend(layer_bits)

      #apply ReLU for next layer
      activation = np.maximum(0, z)
    return tuple(binary_code), activation

Part 2: Visualizing the Stained Glass

Different Polyhedrons in

This is the space distribution I get in 2D through a simple Network like this

ReLUNetwork([2, 5, 5,1])

Notice the structure:

1. Convexity: Every colored region is a convex polygon.
2. The Cuts: The boundaries are straight lines (hyperplanes) created by individual neurons turning on or off.
3. Complexity: As we add layers, these simple linear cuts intersect to form increasingly complex shapes.

This is the Polyhedral Decomposition of the input space.

code for visualization

from enum import unique
def visulaize_polyhedra(net, x_min=-2, x_max=2, resolution = 400):
  xx, yy = np.meshgrid(
      np.linspace(x_min, x_max, resolution),
      np.linspace(x_min, x_max, resolution),
    )

  grid_points = np.c_[xx.ravel(), yy.ravel()]

  signatures = []
  for point in grid_points:
    sig, _ = net.get_binary_vector(point)
    signatures.append(sig)

  unique_sigs = list(set(signatures))
  sig_map = {sig : i for  i, sig in enumerate(unique_sigs)}

  z_values = np.array([sig_map[s] for s in signatures])
  z_values = z_values.reshape(xx.shape)

  plt.figure(figsize = (10, 10))
  plt.imshow(
      z_values,
      extent=(x_min, x_max, x_min, x_max),
      origin='lower',
      cmap = 'tab20b',
      interpolation = 'nearest'
    )
  plt.title(f"Polyhedron Decomposition \n {len(unique_sigs)} unique regions identified", fontsize=15)
  plt.xlabel("input_dimension 1")
  plt.ylabel("input_dimension 2")
  plt.colorbar(label = 'polyhedron ID')
  plt.show()

Part 3: The Neural Metric (Hamming Distance)

Now, This is the point where everything will make sense now, why define and find those seemingly useless Binary Vectors. In real world we use Euclidean distances(measuring straight lines). Now, we use Hamming Distance, Simply counting how many bits differ between two addresses.

Now this is the Holy Grail of this Article

single ReLU basically divides the space into 2 regions active and inactive as we discussed it generates a hyperplane of equation w.x + b = 0. and our bits in binary vector or address vector depend on which side of this hyperplane we lie.

Now here it comes : Two polyhedra share a wall (a facet) if and only if their binary vectors differ by exactly one bit. BOOOOOM!!!

This implies that the Hamming distance is a proxy for the “Geodesic Distance” on the network’s internal graph. If two points have a Hamming distance of 5, you have to cross roughly 5 “walls” to walk from one to the other.

Each data point is a Polyhedron space and edges represent sharing of faces

from inspect import signature
import networkx as nx

def build_dual_graph(net, x_min=-2, x_max=2, resolution=50):
  xx, yy = np.meshgrid(
      np.linspace(x_min, x_max, resolution),
      np.linspace(x_min, x_max, resolution)
  )

  grid_points = np.c_[xx.ravel(), yy.ravel()]

  unique_sig = set()

  for point in grid_points:
    sig, _ = net.get_binary_vector(point)
    unique_sig.add(sig)

  unique_sig = list(unique_sig)

  G = nx.Graph()

  for i, sig in enumerate(unique_sig):
    G.add_node(i, signature=sig)

  for i in range(len(unique_sig)):
    for j in range(i+1, len(unique_sig)):
      diff = sum(b1 != b2 for b1,b2 in zip(unique_sig[i], unique_sig[j]))
      if diff == 1:
        G.add_edge(i,j)
  return G

small_net = ReLUNetwork([2, 5, 5, 1])
G = build_dual_graph(small_net)

plt.figure(figsize=(10,8))
pos = nx.spring_layout(G, seed = 42)
nx.draw(G, pos, node_size = 50, node_color='blue', edge_color = 'gray', with_labels = False)
plt.title(f"The Dual Graph of the Neural Network\n(Nodes = Polyhedra, Edges = Shared Facets)\nTotal Regions: {len(G.nodes)}")
plt.show()

Part 4: Experiment to see how well Hamming Distance preserves the Euclidean distances information

In this experiment, Let’s see how well hamming Distance can preserve the structural information of a circle. I just sampled a circle using 100 points and generated the distance maps first using Euclidean distance and next using Hamming Distance.

Interpretation: The white diagonal you see means 0 distance (distance of a point to itself). Then as we move outward the distance increases as distance of point 0 say P_0 with P_i increases as i Increases but starts to repeat the pattern because P_0 is close to higher index points like p_90, p_99, p_100. And Hamming distance preserved this repeating band like structure. also Hamming Distance map starts to resemble more and more like Euclidean distance map as we increase the complexity of the neural net.

Python code

from scipy.spatial.distance import pdist, squareform

def generate_circle_data(n_points = 100, radius = 1.0, noise = 0.05):
  t  = np.linspace(0, 2*np.pi, n_points)

  x = radius * np.cos(t)
  y = radius * np.sin(t)

  data = np.c_[x,y] + np.random.randn(n_points, 2)*noise
  return data

def get_hamming_matrix(net, data):
  signatures= []
  for point in data:
    sig, _ = net.get_binary_vector(point)
    signatures.append(sig)
  signatures = np.array(signatures)

  dists = pdist(signatures, metric = 'hamming') * signatures.shape[1]
  return squareform(dists)

deep_net = ReLUNetwork([2, 50,100, 50, 1])
circle_data = generate_circle_data(n_points=100)

hamming_matrix = get_hamming_matrix(deep_net, circle_data)
euclidean_matrix = squareform(pdist(circle_data, metric='euclidean'))

fig, axes = plt.subplots(1, 3, figsize = (18, 5))
axes[0].scatter(circle_data[:,0], circle_data[:,1], c=np.arange(len(circle_data)), cmap='twilight')
axes[0].set_title("input Manifold (Circle)")
axes[0].axis("equal")

im1 = axes[1].imshow(euclidean_matrix, cmap = 'twilight')
axes[1].set_title("Euclidean distance Matrix")
plt.colorbar(im1, ax = axes[1])

im2 = axes[2].imshow(hamming_matrix, cmap='twilight')
axes[2].set_title("neural Hamming Distance Matrx")
plt.colorbar(im2, ax = axes[2])

plt.show()

Now if you want to move beyond toy datasets and shallow networks, you can read the original paper from where I got the content for this article.

[embed]ReLU Neural Networks, Polyhedral Decompositions, and Persistent Homolog A ReLU neural network leads to a finite polyhedral decomposition of input space and a corresponding finite dual graph…arxiv.org

The authors experimented with ResNet-50 trained on ImageNet, using images of a miniature poodle, a Persian cat, and a Saluki (basically cats and dogs), interpolated between them, the Hamming distance still recovered the underlying topological signal. This suggests that Deep Learning models might not just be memorizing textures. they are building a robust, discrete geometric map of the data manifold. Which is pretty cool to think about.

Conclusion:

By shifting our perspective from weights and biases to polyhedra and bit-vectors we gain a new set of tools to analyze them. We can map their decision boundaries, measure their complexity by counting regions, and even use topology to see if they have learned the true shape of the data.

The Black Box is actually full of geometric crystals. we just needed the right math to see them.

Complete Code Link — https://github.com/23Tarandeep57/ReLU-Topology-Exploration

References

• Liu, Y., & Cole, C., et al. (2023). ReLU Neural Networks, Polyhedral Decompositions, and Persistent Homology. arXiv:2306.17418.