What does it take to make a system unpredictable?
If you watch a newspaper blowing in the wind, its flopping will be pretty unpredictable. The forces that govern it are constantly changing, maybe even randomly changing: the breeze fluctuates, the pages flap, the creases spin. As onlookers we would decide that the system is governed by some really complicated rules and forces, and we’d be right. It’s easy to give a system the appearance of randomness when its inputs are (or might as well be) random. But what’s the deeper nature of chaos? In chaotic dynamics, we find that the appearance of randomness, unpredictability, and intricacy of a system’s behavior can arise from a few simple deterministic rules. The strict definition of chaos has been somewhat controversial, but a working standard is:
Chaos is non-periodic behavior in a deterministic system with high sensitivity to initial conditions.
The nature of chaos is that simple rules can give rise to unpredictable and subtle behavior.
We’ll be looking at the Lorenz system, a famous system of differential equations which comes from a 1963 paper Edward Lorenz of MIT published in the Journal of Atmospheric Sciences . The publication was momentous, and the Lorenz system helped spur the development modern chaos theory. In his paper, Lorenz derives a system of differential equations as a simplification of an existing fluid-convection model, with intent to study convection in the atmosphere:
where σ, r, and b are positive parameters related to the physics of convection. We’ll usually use σ = 10, b = 8/3, and r = 28, since Lorenz originally studied the system with those values, but we’ll give a little special treatment to the value of r later on. Notice that the equations don’t look terribly complicated; there are only two nonlinear terms (the xz and xy), and there are only three equations. Likewise the system is clearly deterministic—it involves no random inputs. Yet we’ll see that the system exhibits subtle and chaotic behavior.
Some background before things get too chaotic
Dynamical systems are generally analyzed in terms of their states, which are just lists of variables that describe the system at any point in time. If your system is described by n variables, then you can imagine an n-dimensional space of all possible states—this space is referred to as phase-space, and a particular state of the system is just a point in the phase-space. If your system starts in a particular state, the dynamics will cause the system to follow some trajectory through phase-space as its state changes. A trajectory is just a solution to the system equations. (Technically, each point along a trajectory could serve as the initial conditions for another solution. But since those solutions have the same path through phase-space, they’re referred to as a single trajectory.)
For example, if the system you’re working with is a mass on a spring (in one dimension), you might describe the state by the position and the velocity of the mass. Both are functions of time, and the state of your system is some point in phase-space called S = (x, v). The variables x and v are functions of time, so your system will trace out a trajectory in phase-space as it moves through time. In this case, the trajectory is an ellipse, as shown in the Figure below.
The analysis of phase-space trajectories is central to understanding nonlinear dynamics and chaos, partially because the behavior of these systems can’t generally be “solved” completely. It’s often much more useful (and easier) to figure out the qualitative behavior of a system in terms of the footprint it leaves in phase-space, especially for chaotic systems.
Returning to the Lorenz system, our goal is to have a phase-space portrait of the solutions to the system equations, and to understand its structure in terms of chaos. If we were to find that all trajectories converged on a point, or orbited around a point, for example, that would immediately show us how the system behaves. The question is: if we start a trajectory at a given point in phase-space, what happens next?
From direct numerical integration of the Lorenz equations, the plot in Figure 2 is our answer. The trajectory shown outlines the limiting set for the system, called an attractor, toward which all trajectories are attracted. Remember, what you’re seeing isn’t really the attractor itself, but rather one particular solution that outlines it—similar to how iron filings will line up to outline an unseen magnetic field.
Figure 2. (Click to enlarge) Lorenz discovered this chaotic attractor after simulating his convection model. Notice that the trajectory swings irregularly between the two “wings” of the butterfly-shaped attractor. Plot generated using σ = 10, b = 8/3, r = 28.
I’ve also plotted the output of just the x-component so you can get a feel for what the non-periodic solutions of this chaotic system really look like.
Figure 3. One of the three non-periodic state variables generated in the Lorenz system. The irregular oscillations in the positive- and negative-x-regions correspond to the trajectory lingering respectively on the right and left wings of the attractor in Figure 2. Although there are strong reasons to believe the solution is non-periodic, nobody has proven the non-periodicity totally rigorously. Plot generated using σ = 10, b = 8/3, r = 28.
We can learn a whole lot about the system and its attractor with some analysis. First, it’s useful to think about what happens to volumes in phase-space (the following derivation will follow section 9.2 in Strogatz ). If we have some volume in phase-space (meaning some blob of possible initial conditions) bounded by a surface , the volume will expand in time as all the points in evolve under the Lorenz equations. The velocity of each point on the surface is denoted by and let be a unit-vector normal to . Then in a time , a patch of area on the surface will add a volume to the volume .
Integrating over all patches of area on the surface, we get
Then, after subtracting and dividing by , the divergence theorem gives
So we see that volumes in phase-space evolve as , and draw an important conclusion: volumes in phase-space contract toward zero. That means that the volume of the attractor itself must be zero since any blob of initial conditions is drawn to the attractor, but its volume decreases exponentially.
Another tactic for understanding the system is to look for points which are fixed in phase-space for all time, called fixed points. If the system starts at (or reaches) a fixed point, it will stay there forever. The fixed points occur when
For the Lorenz system, (x, y, z) = (0, 0, 0) is a fixed point, and there are two other fixed points at C+ = and C– = which exist only when r > 1 (check for yourself).
The fact that C+ and C– only exist for r > 1 means that the system behaves very differently for r ≤ 1 and r > 1. If we were to tune r continuously starting from some tiny number, then something serious would have to happen to the system dynamics when we reached r = 1. That event is called a bifurcation, and r = 1 is called a bifurcation value of r. There are many types of bifurcations, but what they all have in common is this: as we change a system parameter through a bifurcation value, the system undergoes sudden qualitative changes in behavior. In our case, the system “sprouts” two new fixed points, C+ and C–, as r increases through 1.
The actual effects of the fixed points on the solutions of the differential equations (that is, on the trajectories through phase-space) are not obvious without numerical integration. If you’re interested in how the analysis is done, take a look at Section 9.2 in Strogatz. Meanwhile, we’ll get some intuition for what these particular fixed points do from numerical integration of the Lorenz equations. It turns out that for r < 1, all trajectories are attracted to the origin. The case for an r-value not too much greater than 1 is shown below.
Figure 5. The Lorenz system with r = 10. Each color represents a different initial state. For r not too much greater than 1, the fixed points C+ and C– are attractive for all trajectories. In this case the system is pretty boring (and certainly not chaotic).
The system actually has yet another bifurcation at r ≈ 24.74. In this bifurcation, the location of the fixed points is unchanged—what does change, though, is their effect on the trajectories. For r > 24.74, trajectories which approach C+ and C– are actually repelled from those fixed points. Meanwhile the fixed point at the origin also repels trajectories. When looking for chaos in this system, we’ll want to use a value of r beyond this second bifurcation.
Let’s summarize what we know so far:
- All trajectories will tend toward some limiting set in phase-space which has zero volume. This is the attractor we glimpsed from our numerical integration.
- Trajectories are repelled from all three fixed points, so they must keep moving forever.
But we haven’t actually proven that the Lorenz system is truly chaotic, according to our definition above. To show that it really is, we still need to show that the system has “sensitive” dependence on initial conditions. Figure 6 below shows what happens to two trajectories which start off with almost identical initial conditions.
Figure 6. (Click to enlarge) Solutions to the Lorenz system with initial conditions (0, 1.00, 0) [blue] and (0, 1.01, 0) [red]. Notice that while both trajectories adhere to the attractor, their separation at any given point in time may be very large.
Below I’ve plotted the x-outputs for the trajectories shown above to make it easy to see that the two solutions really do have drastically different behavior.
For chaotic systems in general, if two points start off with a separation in phase-space, after a time t their separation will be . From numerical results, the magnitude of two trajectories’ separation is found to be roughly exponential in time:
This is not a precise relationship, just a rough model of behavior observed in lots of chaotic systems. The true relationship between and is complicated and depends on where the trajectories start, and for the Lorenz system is limited by the maximum separation between two points on the attractor. Anyway, the rate of separation of points in phase-space is roughly described by λ, which is called the Lyapunov exponent of the system. For the Lorenz system λ is found to be about 0.9. The fact that the Lyapunov exponent is positive means that the system is very sensitive to initial conditions—initially proximate solutions will diverge exponentially quickly. In fact, in our definition of chaos, what I meant by “high sensitivity to initial conditions” was actually that the system should have a positive Lyapunov exponent! The attractor in Figure 2 is called a strange attractor due to its sensitive dependence on initial conditions, and we can conclude that the Lorenz system really is chaotic.A note about Lyapunov exponents: the separation of points in phase-space is actually an n-dimensional vector. The initial separation may be along any (or all) of the n dimensions of the phase-space, and each direction will have a different empirical Lyapunov exponent. In practice, the largest Lyapunov exponent of the system dominates the separation of trajectories, so we forget about the others and simply call that one the Lyapunov exponent.
As a final puzzling thought, let’s think about the long-term fate of trajectories on the Lorenz attractor. There is an important Existence and Uniqueness Theorem which tells us that each point in phase-space corresponds to a single unique solution (that is, a single unique trajectory) of the Lorenz system. That means that no two trajectories can ever cross, or else the point of intersection would itself serve as the initial conditions for two different solutions, violating the Theorem! Likewise, no trajectory can ever intersect itself. If it did, it would have to enter some closed loop which always brings it back to the point where it intersected itself, meaning it would be periodic. Keep in mind that every single point in the infinite phase-space can serve as the initial conditions for a trajectory. So the implications of the Existence and Uniqueness Theorem here are dramatic:
Every possible trajectory stays within a bounded attracting region in phase-space forever without ever reaching the same point twice and without ever reaching any of the same points that any other trajectory ever reaches in infinite time.
It’s like every trajectory is playing an infinite game of “Snake” with itself and every other trajectory simultaneously, and wins!
And oh yeah, as if this strange attractor weren’t already strange enough, it’s also a roughly 2.05-dimensional fractal—but more on that later! In the meantime, check out some cool attractors that arise in other chaotic systems.
References and Further Reading
 Lorenz, Edward N. “Deterministic Nonperiodic Flow.” Journal of The Atmospheric Sciences V. 20, pp. 130-141. March 1963.
 Strogatz, Stephen H. Nonlinear Dynamics and Chaos. Chapters 6, 9.