June 11, 2019

The Return of Zeta

MSRI Journal
The Mark of Zeta

The Return of Zeta

The Riemann hypothesis is widely considered the outstanding unsolved problem in mathematics. Generations of mathematicians have been lured into hunting for a proof of this celebrated conundrum. All have so far failed.

In the late 1990s, however, a cautious optimism took hold among mathematicians tangling with the problem. New approaches showed promise, potentially bringing a proof of the Riemann hypothesis within reach.

One of the most exciting possibilities involved an astonishing, unexpected connection between the distribution of prime numbers and the energy levels of excited atoms. The vehicle is a branch of mathematics known as random matrix theory.

The Riemann hypothesis was first proposed in 1859 by the German mathematician Georg Friedrich Bernhard Riemann (1826-1866). It concerns the so-called zeta function, which encodes a great deal of information about the seemingly haphazard distribution of prime numbers among the integers.

Gaussian integers are complex numbers of the form a + bi, where a and b are integers. The Gaussian primes are Gaussian integers that cannot be factored in a nontrivial way into a product of other Gaussian integers. In this weaving, the dark blue squares represent Gaussian primes. Department of Mathematics, University of Nebraska, Lincoln, Nebraska.

Mathematicians know that the Riemann zeta function (zeta(s) = 1 + 1/2s + 1/3s + ... + 1/ns + ...) is zero when the complex number s is a negative even number. Riemann conjectured that all other solutions of the equation have the form 1/2 + bi. In other words, the so-called zeros of the Riemann zeta function lie along a line in the complex plane at positions specified by the number b. Knowing precisely where those zeros lie (the values of b) would give accurate estimates of the distribution of primes.

The Riemann hypothesis is "one of the finest examples of the extraction of order from chaos in the whole of mathematics," Philip J. Davis (1923-2018) and Reuben Hirsch wrote in The Mathematical Experience.

In a sense, if the zeros of the zeta function are like musical notes, then prime numbers are chords, and theorems about these entities are symphonies, quantum chaologist Michael Berry has remarked. The Riemann hypothesis imposes a pleasing harmony on the zeta-zero notes, he added.

One of the first inklings of a connection between number theory and quantum mechanics came in 1972. Hugh L. Montgomery had discovered a formula that describes the average spacing between consecutive zeros (values of b) of the zeta function. During a visit to the Institute for Advanced Study in Princeton, New Jersey, he showed his result to quantum physicist Freeman Dyson, when they happened to meet at afternoon tea.

Dyson immediately recognized it as identical to the result obtained for so-called random matrix models, which are used to describe the energy levels of large atoms or heavy nuclei. By an amazing coincidence, Dyson was one of just a handful of physicists in the world who had done such calculations and could appreciate Montgomery's work.

The fundamental equation of quantum mechanics is known as the Schrödinger equation. It describes the behavior of microscopic entities such as atoms, electrons, and nuclei. Solving the equation for a given quantum system means determining its so-called eigenvalues—whole numbers that correspond to energy levels. Such a set of eigenvalues is termed the system's spectrum.

Equivalently, physicists can represent a given quantum system by a matrix—a rectangular array of whole numbers. Solving that matrix gives the system's eigenvalues.

Calculating the eigenvalues of a simple system, such as a lone electron trapped in a rectangular box or an electron orbiting a single proton (as in a hydrogen atom), can be done by any beginning student of quantum mechanics. Determining the energy levels of a heavy nucleus or an atom with many electrons is immensely more difficult.

"One way to understand a complicated object is to study properties of a random thing which matches it in a few salient features," mathematician and statistician Persi Diaconis noted.

In the 1970s, theoretical physicists started to use sets of random matrices (with certain specified properties) as mathematical models of quantum systems. In a random matrix, the rows and columns are filled with numbers randomly selected from a normal, or Gaussian, distribution. (Imagine randomly choosing numbered balls from an urn, where certain numbers are represented more often than others.)

By solving a sufficiently large random matrix, physicists can determine a system's "typical" eigenvalues. Combining the results of many such calculations, they can come up with statistical laws about those eigenvalues.

If the random matrices belong to a class of matrices known as the Gaussian Unitary Ensemble (GUE), physicists obtain good estimates of the average spacing between consecutive energy levels of heavy atomic nuclei and other complex quantum systems.

It turns out that the spacings between consecutive zeros of the zeta function also appear to behave statistically like the spacings between consecutive eigenvalues of these large, random matrices. Indeed, this observation also suggests that the infinitely many zeros specified in the Riemann hypothesis are irregularly distributed in a particular way along the line 1/2 + bi.

In the late 1990s, large-scale computations of zeros of the zeta function by Andrew M. Odlyzko and others furnished numerical evidence that random matrices, like those used for modelling quantum systems, accurately represent the behavior of the zeta function.

These computational results gave mathematicians a hint that random matrix theory could provide an avenue to a proof of the Riemann hypothesis.

There was also new hope for physicists, who generally have a hard time computing the eigenvalues and experimentally measuring the energy levels of complex quantum systems. The Riemann zeta function itself could provide a new way to do such calculations in quantum mechanics.

In particular, the zeros of the zeta function can be interpreted as energy levels in the quantum version of a conventional, or classical, system that behaves chaotically. Such a simple mathematical model would allow physicists to test ideas about how to bridge the apparently incompatible chaotic and quantum-mechanical descriptions of the microscopic world.

Why the Riemann zeta function so convincingly mimics a quantum system without being one remains a matter of conjecture and mystery. Whatever the reason, physicists and mathematicians have opened a new window on the fascinating interplay of order and chaos—in nature and among integers.

Originally posted June 28, 1999.

Solitaire-y Sequences
A Song About Pi
Row Your Boat
Juggling By Design
Averting Instant Insanity
Matrices, Circles, and Eigenthings
Lunar Shadows
MSRI Reflections

No comments: