WHAT DO ELEPHANTS AND IRRATIONAL ROTATIONs HAVE IN COMMON?

Some Systems are Forgetful; Others Aren’t

The mathematics of ergodicity and mixing

Via Engin Akyurt.

Imagine you drop some dye into a bathtub full of water and stir it in a perfect circle for an hour. After this hour, in theory, the dye will still be in a ring. Now do the same thing but stir chaotically: reverse directions, figure eights, you get the idea. After an hour, the water will be the same color throughout.

Why is it that, even though we were stirring for the same amount of time, with water and dye with the same chemical properties, we get quite distinct qualitative results?

Now, I’m not great at chemistry, and luckily for me, it turns out this isn’t much of a chemistry question — it’s a math question.

In a branch of mathematics called ergodic theory, we ask, “What does it mean for a dynamical system to mix”?

In this article, we’ll explore the ideas of ergodicity and build to the stronger idea of mixing. We’ll also touch on how ergodic theorists quantify the seemingly intangible idea of information generation within chaotic (or even stochastic) systems.

This framework applies in contexts ranging from the statistical mechanics of gases to the firing patterns of neurons. It can even inform more philosophical discussions of long-run statistical behavior and chaos versus randomness.

Some Set-Up

In ergodic theory, we’re concerned primarily with measure-preserving transformations: dynamical systems that conserve some notion of volume or probability as they evolve. The canonical setting (and I think the most intuitive setting) is a probability space: a set of states X equipped with a measure assigning probabilities to regions. We consider this together with a map T from X that leaves the measure unchanged.

This is not as restrictive as it might sound initially. There are plenty of examples of transformations/maps that are measure-preserving:

• Rotations of a circle preserve arc length.
• Automorphisms of a torus preserve area.
• Hamiltonian flows in classical mechanics preserve the Liouville measure on phase space.

The world is full of measure-preserving systems!

A cool concept surrounding these measure-preserving systems is Poincaré recurrence: for almost every starting point, the orbit will eventually return to (arbitrarily close to) where it began. No matter how complicated the dynamics, measure-preserving systems can’t “escape”. They are forced to revisit the past because the total volume of the space is finite. The set of non-returning points has measure zero and is, for statistical purposes, basically invisible.

But there’s a more interesting question to explore when it comes to how the system distributes itself over time, and how that works when time approaches infinity.

Ergodicity

A measure-preserving system is ergodic if it cannot be decomposed into two positive-measure regions that never exchange points. More precisely, the only essentially T-invariant sets are those of measure zero or one.

In other words, there are no “trapped” regions.

This brings us to the Birkhoff Ergodic Theorem:

Statement of the Birkhoff Pointwise Ergodic Theorem.

This is the more mathematically formal statement of the ergodic hypothesis from statistical mechanics: a gas in a box, given enough time, visits all energetically accessible configurations with equal frequency.

Via Wikipedia.

You don’t need to average over an ensemble of many systems; one system, observed for long enough, samples everything.

The time average is the ensemble average.

Rational vs. irrational rotation on a circle. Rational rotation is not ergodic, whereas irrational rotation is!

For the map of an irrational rotation on the circle, no point ever returns to exactly where it started because we’re not rotating by an integer. The orbit of any single point visits every arc of the circle, eventually spending proportional time in regions of proportional length. This is ergodic.

A rational rotation, in contrast, is not ergodic. Every orbit is finite, so it just cycles through exactly q points and then repeats. The orbit of any starting point is permanently confined to q equally spaced locations, never visiting the vast majority of the circle. The space has been decomposed into invariant regions, and the orbit of a single point is trapped in one of them. The time average is not the space average.

Mixing: A Stronger Condition

Ergodicity guarantees that orbits spread out on average. But it says nothing about how quickly this might happen, or about the instantaneous distribution at any particular time. An irrational rotation is ergodic, but at step n, the orbit is at the single point x + n*alpha. In other words, it hasn’t spread out at all. It is simply visiting a structured, deterministic sequence of positions that collectively fill the circle over time.

Mixing demands something stronger: that the system becomes randomized asymptotically.

The left side asks: what fraction of B eventually maps into A after n steps? The right side is what you would expect if the event “point is in A after n steps” were statistically independent of the event “point started in B.” Mixing says these two events become asymptotically independent as n goes to infinity. In other words, the system forgets its initial condition.

Going back to our stirring analogy, we can understand this as the dye spreading out such that the eventual distribution of dye in region A is just the size of A, regardless of where we initially put the dye.

What do elephants and irrational rotation have in common? They never forget. 🥁

In our example of the irrational rotation, we can see that this is very much not mixing.

A rotation is a rigid isometry: it never distorts or stretches. A blob of dye placed on the circle stays a blob, rotating — it does not spread out. The long-run visit frequency implies ergodicity, but at any given time, the blob is still coherent and localized, not diffused throughout the space.

But the doubling map given by 2x mod 1 (take x, multiply by 2, then delete everything to the left of the decimal) is mixing. Each application of the doubling map stretches the circle by a factor of two.

![The doubling map on 0, 1). Via B.S-Boer.

The doubling map on [0, 1). Via B.S-Boer.

On the circle, a blob of any initial length eventually wraps around the entire circle and is essentially uniform. The system has forgotten where it started.

The key difference between ergodicity isn’t a question of how far orbits travel, but whether the map sufficiently distorts the geometry of the space. Rotations are isometries, meaning they preserve shape. But the doubling map is expansive; it stretches, causing nearby points to separate exponentially and information about the initial condition to be lost.

Functional Analysis + Examples

The same analysis applies across a surprisingly diverse set of systems, all accessible via functional analysis. Now, when I first started exploring ergodic theory, I’d never even dipped my toe into functional analysis, which is a fairly intimidating topic. However, functional analysis makes discussions of examples of ergodicity and mixing feasible (though we need to introduce a few things first).

The Koopman Operator

The idea behind the Koopman Operator is that, instead of watching how individual points move, you watch how measurements change. If you have some observable (for example, a physicist might be concerned with the temperature at a given location), the Koopman operator asks: what does that same observable look like one step later? So, it takes a function and returns the function composed with one step of the dynamics.

Crucially, the Koopman operator is linear. Dynamics on a space of points can be complicated and nonlinear, but the Koopman operator always acts linearly on functions, which means we can use all the tools of linear algebra and functional analysis to explore the complexities of ergodicity and mixing.

Within this framework, we can redefine ergodicity by saying that the only functions the operator leaves completely unchanged are the constant functions. This is compatible with our previous examples as well; if measuring something gives you the same number no matter when you measure it, that number has to be the same everywhere.

Mixing says that correlations between any two measurements decay to zero over time. In other words, knowing the value of one observable now tells you nothing about a different observable far in the future.

What makes functional analysis useful in studying the dynamical systems we are preoccupied with (rotations, expanding maps, toral automorphisms) there is a natural basis of functions — the Fourier basis, the complex exponentials — that the Koopman operator acts on very simply.

Each basis function gets multiplied by a single number, its eigenvalue.

Checking ergodicity and mixing reduces to asking what those eigenvalues do!

Rotations and Doubling

For an irrational rotation, each Fourier basis function gets multiplied by a complex number of absolute value exactly one. That number never decays to zero, so correlations never die, which implies that the map is not mixing. But, for a non-constant basis function, that number is also never equal to one, so no non-constant function stays fixed, which means the map is ergodic.

For the doubling map, the operator sends each basis function to a strictly higher-frequency one. Frequencies keep climbing with each iteration of the map, and high-frequency oscillations average out to zero in any inner product, so correlations decay, and the system mixes.

In future articles, I’ll discuss two other important examples: hyperbolic toral automorphisms and subshifts of finite type.

The pattern across all of these examples is essentially the same: ergodicity comes from the leading eigenvalue being simple, and mixing comes from all other eigenvalues being strictly smaller.

Zooming Out: Entropy

The concepts of ergodicity and mixing are helpful for understanding the long-run behavior of dynamical systems, but they are still somewhat qualitative in nature. Two mixing systems can still be very different dynamically.

The coin-flip process with a fair coin and the coin-flip process with four equally likely symbols are both mixing, yet they generate information at different rates.

The quantitative measure that distinguishes these two systems is called entropy: a single non-negative number measuring how quickly a system produces new information. Metric entropy is zero for irrational rotations, which are completely predictable. It is one bit per step for the doubling map. It grows with the number of symbols for a Bernoulli shift.

What makes entropy cool is that it is a complete invariant for Bernoulli shifts, meaning that two such systems are measurably equivalent if and only if they have the same entropy.

This is called Ornstein’s theorem, proved in 1970, and it is one of the most interesting results of twentieth-century mathematics when it comes to understanding our universe via mathematics.

I could write many articles about Ornstein’s theorem, but for now, we can stop at the idea that some systems forget, and some do not. The ones that forget are the ones that mix, and the rate at which they forget is measured by their entropy!

Sources and Further Reading

1. Shifts and Ergodic Theory, Antonio Donald | University of Chicago
2. Introduction to Dynamical Systems, Michael Brin and Garrett Stuck

My related articles

[embed]Math Misconceptions: Chaos vs. Randomness Is anything truly random?sciencespectrumu.com

[embed]A Simple Look at Simple Random Walks A drunken man always returns home, but a drunken bird will eventually be lost.thequantasticjournal.com

P.S. To the Medium Staff, I will love you forever if you add LaTeX rendering to your UI. Please, please, please! ❤

If you enjoyed this article and would like to help support me through graduate school, you can buy me a coffee or drop a comment below. Thank you so much for your time and attention!