Godel and the End of Physics

This lecture is the intellectual property of Professor S.W.Hawking. You may not reproduce, edit, translate, distribute, publish or host this document in any way with out the permission of Professor Hawking.

Note that there may be incorrect spellings, punctuation and/or grammar in this document. This is to allow correct pronunciation and timing by a speech synthesiser.
In this talk, I want to ask how far can we go in our search for understanding and knowledge. Will we ever find a complete form of the laws of nature? By a complete form, I mean a set of rules that in principle at least enable us to predict the future to an arbitrary accuracy, knowing the state of the universe at one time. A qualitative understanding of the laws has been the aim of philosophers and scientists, from Aristotle onwards. But it was Newton's Principia Mathematica in 1687, containing his theory of universal gravitation that made the laws quantitative and precise. This led to the idea of scientific determinism, which seems first to have been expressed by Laplace. If at one time, one knew the positions and velocities of all the particles in the universe, the laws of science should enable us to calculate their positions and velocities at any other time, past or future. The laws may or may not have been ordained by God, but scientific determinism asserts that he does not intervene to break them.

At first, it seemed that these hopes for a complete determinism would be dashed by the discovery early in the 20th century; that events like the decay of radio active atoms seemed to take place at random. It was as if God was playing dice, in Einstein's phrase. But science snatched victory from the jaws of defeat by moving the goal posts and redefining what is meant by a complete knowledge of the universe. It was a stroke of brilliance whose philosophical implications have still not been fully appreciated. Much of the credit belongs to Paul Dirac, my predecessor but one in the Lucasian chair, though it wasn't motorized in his time. Dirac showed how the work of Erwin Schrodinger and Werner Heisenberg could be combined in new picture of reality, called quantum theory. In quantum theory, a particle is not characterized by two quantities, its position and its velocity, as in classical Newtonian theory. Instead it is described by a single quantity, the wave function. The size of the wave function at a point, gives the probability that the particle will be found at that point, and the rate at which the wave function changes from point to point, gives the probability of different velocities. One can have a wave function that is sharply peaked at a point. This corresponds to a state in which there is little uncertainty in the position of the particle. However, the wave function varies rapidly, so there is a lot of uncertainty in the velocity. Similarly, a long chain of waves has a large uncertainty in position, but a small uncertainty in velocity. One can have a well defined position, or a well defined velocity, but not both.

This would seem to make complete determinism impossible. If one can't accurately define both the positions and the velocities of particles at one time, how can one predict what they will be in the future? It is like weather forecasting. The forecasters don't have an accurate knowledge of the atmosphere at one time. Just a few measurements at ground level and what can be learnt from satellite photographs. That’s why weather forecasts are so unreliable. However, in quantum theory, it turns out one doesn't need to know both the positions and the velocities. If one knew the laws of physics and the wave function at one time, then something called the Schrodinger equation would tell one how fast the wave function was changing with time. This would allow one to calculate the wave function at any other time. One can therefore claim that there is still determinism but it is determinism on a reduced level. Instead of being able accurately to predict two quantities, position and velocity, one can predict only a single quantity, the wave function. We have re-defined determinism to be just half of what Laplace thought it was. Some people have tried to connect the unpredictability of the other half with consciousness, or the intervention of supernatural beings. But it is difficult to make either case for something that is completely random.

In order to calculate how the wave function develops in time, one needs the quantum laws that govern the universe. So how well do we know these laws? As Dirac remarked, Maxwell's equations of light and the relativistic wave equation, which he was too modest to call the Dirac equation, govern most of physics and all of chemistry and biology. So in principle, we ought to be able to predict human behavior, though I can't say I have had much success myself. The trouble is that the human brain contains far too many particles for us to be able to solve the equations. But it is comforting to think we might be able to predict the nematode worm, even if we can't quite figure out humans. Quantum theory and the Maxwell and Dirac equations indeed govern much of our life, but there are two important areas beyond their scope. One is the nuclear forces. The other is gravity. The nuclear forces are responsible for the Sun shining and the formation of the elements including the carbon and oxygen of which we are made. And gravity caused the formation of stars and planets, and indeed, of the universe itself. So it is important to bring them into the scheme.

The so called weak nuclear forces have been unified with the Maxwell equations by Abdus Salam and Stephen Weinberg, in what is known as the Electro weak theory. The predictions of this theory have been confirmed by experiment and the authors rewarded with Nobel Prizes. The remaining nuclear forces, the so called strong forces, have not yet been successfully unified with the electro weak forces in an observationally tested scheme. Instead, they seem to be described by a similar but separate theory called QCD. It is not clear who, if anyone, should get a Nobel Prize for QCD, but David Gross and Gerard ‘t Hooft share credit for showing the theory gets simpler at high energies. I had quite a job to get my speech synthesizer to pronounce Gerard's surname. It wasn't familiar with apostrophe t. The electro weak theory and QCD together constitute the so called Standard Model of particle physics, which aims to describe everything except gravity.

The standard model seems to be adequate for all practical purposes, at least for the next hundred years. But practical or economic reasons have never been the driving force in our search for a complete theory of the universe. No one working on the basic theory, from Galileo onward, has carried out their research to make money, though Dirac would have made a fortune if he had patented the Dirac equation. He would have had a royalty on every television, walkman, video game and computer.

The real reason we are seeking a complete theory, is that we want to understand the universe and feel we are not just the victims of dark and mysterious forces. If we understand the universe, then we control it, in a sense. The standard model is clearly unsatisfactory in this respect. First of all, it is ugly and ad hoc. The particles are grouped in an apparently arbitrary way, and the standard model depends on 24 numbers whose values can not be deduced from first principles, but which have to be chosen to fit the observations. What understanding is there in that? Can it be Nature's last word? The second failing of the standard model is that it does not include gravity. Instead, gravity has to be described by Einstein's General Theory of Relativity. General relativity is not a quantum theory unlike the laws that govern everything else in the universe. Although it is not consistent to use the non quantum general relativity with the quantum standard model, this has no practical significance at the present stage of the universe because gravitational fields are so weak. However, in the very early universe, gravitational fields would have been much stronger and quantum gravity would have been significant. Indeed, we have evidence that quantum uncertainty in the early universe made some regions slightly more or less dense than the otherwise uniform background. We can see this in small differences in the background of microwave radiation from different directions. The hotter, denser regions will condense out of the expansion as galaxies, stars and planets. All the structures in the universe, including ourselves, can be traced back to quantum effects in the very early stages. It is therefore essential to have a fully consistent quantum theory of gravity, if we are to understand the universe.

Constructing a quantum theory of gravity has been the outstanding problem in theoretical physics for the last 30 years. It is much, much more difficult than the quantum theories of the strong and electro weak forces. These propagate in a fixed background of space and time. One can define the wave function and use the Schrodinger equation to evolve it in time. But according to general relativity, gravity is space and time. So how can the wave function for gravity evolve in time? And anyway, what does one mean by the wave function for gravity? It turns out that, in a formal sense, one can define a wave function and a Schrodinger like equation for gravity, but that they are of little use in actual calculations.

Instead, the usual approach is to regard the quantum spacetime as a small perturbation of some background spacetime; generally flat space. The perturbations can then be treated as quantum fields, like the electro weak and QCD fields, propagating through the background spacetime. In calculations of perturbations, there is generally some quantity called the effective coupling which measures how much of an extra perturbation a given perturbation generates. If the coupling is small, a small perturbation creates a smaller correction which gives an even smaller second correction, and so on. Perturbation theory works and can be used to calculate to any degree of accuracy. An example is your bank account. The interest on the account is a small perturbation. A very small perturbation if you are with one of the big banks. The interest is compound. That is, there is interest on the interest, and interest on the interest on the interest. However, the amounts are tiny. To a good approximation, the money in your account is what you put there. On the other hand, if the coupling is high, a perturbation generates a larger perturbation which then generates an even larger perturbation. An example would be borrowing money from loan sharks. The interest can be more than you borrowed, and then you pay interest on that. It is disastrous.

With gravity, the effective coupling is the energy or mass of the perturbation because this determines how much it warps spacetime, and so creates a further perturbation. However, in quantum theory, quantities like the electric field or the geometry of spacetime don't have definite values, but have what are called quantum fluctuations. These fluctuations have energy. In fact, they have an infinite amount of energy because there are fluctuations on all length scales, no matter how small. Thus treating quantum gravity as a perturbation of flat space doesn't work well because the perturbations are strongly coupled.

Supergravity was invented in 1976 to solve, or at least improve, the energy problem. It is a combination of general relativity with other fields, such that each species of particle has a super partner species. The energy of the quantum fluctuations of one partner is positive, and the other negative, so they tend to cancel. It was hoped the infinite positive and negative energies would cancel completely, leaving only a finite remainder. In this case, a perturbation treatment would work because the effective coupling would be weak. However, in 1985, people suddenly lost confidence that the infinities would cancel. This was not because anyone had shown that they definitely didn't cancel. It was reckoned it would take a good graduate student 300 years to do the calculation, and how would one know they hadn't made a mistake on page two? Rather it was because Ed Witten declared that string theory was the true quantum theory of gravity, and supergravity was just an approximation, valid when particle energies are low, which in practice, they always are. In string theory, gravity is not thought of as the warping of spacetime. Instead, it is given by string diagrams; networks of pipes that represent little loops of string, propagating through flat spacetime. The effective coupling that gives the strength of the junctions where three pipes meet is not the energy, as it is in supergravity. Instead it is given by what is called the dilaton; a field that has not been observed. If the dilaton had a low value, the effective coupling would be weak, and string theory would be a good quantum theory. But it is no earthly use for practical purposes. 

In the years since 1985, we have realized that both supergravity and string theory belong to a larger structure, known as M theory. Why it should be called M Theory is completely obscure. M theory is not a theory in the usual sense. Rather it is a collection of theories that look very different but which describe the same physical situation. These theories are related by mappings or correspondences called dualities, which imply that they are all reflections of the same underlying theory. Each theory in the collection works well in the limit, like low energy, or low dilaton, in which its effective coupling is small, but breaks down when the coupling is large. This means that none of the theories can predict the future of the universe to arbitrary accuracy. For that, one would need a single formulation of M-theory that would work in all situations. 

Up to now, most people have implicitly assumed that there is an ultimate theory that we will eventually discover. Indeed, I myself have suggested we might find it quite soon. However, M-theory has made me wonder if this is true. Maybe it is not possible to formulate the theory of the universe in a finite number of statements. This is very reminiscent of Godel's theorem. This says that any finite system of axioms is not sufficient to prove every result in mathematics. 

Godel's theorem is proved using statements that refer to themselves. Such statements can lead to paradoxes. An example is, this statement is false. If the statement is true, it is false. And if the statement is false, it is true. Another example is, the barber of Corfu shaves every man who does not shave himself. Who shaves the barber? If he shaves himself, then he doesn't, and if he doesn't, then he does. Godel went to great lengths to avoid such paradoxes by carefully distinguishing between mathematics, like 2+2 =4, and meta mathematics, or statements about mathematics, such as mathematics is cool, or mathematics is consistent. That is why his paper is so difficult to read. But the idea is quite simple. First Godel showed that each mathematical formula, like 2+2=4, can be given a unique number, the Godel number. The Godel number of 2+2=4, is *. Second, the meta mathematical statement, the sequence of formulas A, is a proof of the formula B, can be expressed as an arithmetical relation between the Godel numbers for A-  and B. Thus meta mathematics can be mapped into arithmetic, though I'm not sure how you translate the meta mathematical statement, 'mathematics is cool'. Third and last, consider the self referring Godel statement, G. This is, the statement G can not be demonstrated from the axioms of mathematics. Suppose that G could be demonstrated. Then the axioms must be inconsistent because one could both demonstrate G and show that it can not be demonstrated. On the other hand, if G can't be demonstrated, then G is true. By the mapping into numbers, it corresponds to a true relation between numbers, but one which can not be deduced from the axioms. Thus mathematics is either inconsistent or incomplete. The smart money is on incomplete.

What is the relation between Godel’s theorem and whether we can formulate the theory of the universe in terms of a finite number of principles? One connection is obvious. According to the positivist philosophy of science, a physical theory is a mathematical model. So if there are mathematical results that can not be proved, there are physical problems that can not be predicted. One example might be the Goldbach conjecture. Given an even number of wood blocks, can you always divide them into two piles, each of which can not be arranged in a rectangle? That is, it contains a prime number of blocks.

Although this is incompleteness of sort, it is not the kind of unpredictability I mean. Given a specific number of blocks, one can determine with a finite number of trials whether they can be divided into two primes. But I think that quantum theory and gravity together, introduces a new element into the discussion that wasn't present with classical Newtonian theory. In the standard positivist approach to the philosophy of science, physical theories live rent free in a Platonic heaven of ideal mathematical models. That is, a model can be arbitrarily detailed and can contain an arbitrary amount of information without affecting the universes they describe. But we are not angels, who view the universe from the outside. Instead, we and our models are both part of the universe we are describing. Thus a physical theory is self referencing, like in Godel’s theorem. One might therefore expect it to be either inconsistent or incomplete. The theories we have so far are both inconsistent and incomplete.

Quantum gravity is essential to the argument. The information in the model can be represented by an arrangement of particles. According to quantum theory, a particle in a region of a given size has a certain minimum amount of energy. Thus, as I said earlier, models don't live rent free. They cost energy. By Einstein’s famous equation, E = mc squared, energy is equivalent to mass. And mass causes systems to collapse under gravity. It is like getting too many books together in a library. The floor would give way and create a black hole that would swallow the information. Remarkably enough, Jacob Bekenstein and I found that the amount of information in a black hole is proportional to the area of the boundary of the hole, rather than the volume of the hole, as one might have expected. The black hole limit on the concentration of information is fundamental, but it has not been properly incorporated into any of the formulations of M theory that we have so far. They all assume that one can define the wave function at each point of space. But that would be an infinite density of information which is not allowed. On the other hand, if one can't define the wave function point wise, one can't predict the future to arbitrary accuracy, even in the reduced determinism of quantum theory. What we need is a formulation of M theory that takes account of the black hole information limit. But then our experience with supergravity and string theory, and the analogy of Godel’s theorem, suggest that even this formulation will be incomplete.

Some people will be very disappointed if there is not an ultimate theory that can be formulated as a finite number of principles. I used to belong to that camp, but I have changed my mind. I'm now glad that our search for understanding will never come to an end, and that we will always have the challenge of new discovery. Without it, we would stagnate. Godel’s theorem ensured there would always be a job for mathematicians. I think M theory will do the same for physicists. I'm sure Dirac would have approved.

Thank you for listening.