About the SA Blog Network

The Curious Wavefunction

The Curious Wavefunction

Musings on chemistry and the history and philosophy of science
The Curious Wavefunction Home

“The Perfect Theory”: The story of general relativity, and what makes something a science

The views expressed are those of the author and are not necessarily those of Scientific American.

Email   PrintPrint

Until the development of radio astronomy - exemplified here by the Very Large Array telescope in New mexico - general relativity was considered more speculation than fact (Image: NRAO)

Pedro Ferreira’s book “The Perfect Theory: A Century of Geniuses and the Battle over General Relativity” essentially tells us what other people did with Einstein’s general theory of relativity after he developed it. While one chapter is devoted to Einstein’s hard struggle with learning the non-Riemannian geometry and building the field equations that define the theory, the book really takes off after 1917 when a series of men and women discovered the awesome implications of these equations. The book is a fast read and it does a very good job portraying the colorful personalities and exciting discoveries unearthed by general relativity.

By 1919 the theory had been well-established as part of the scientific enterprise, especially after it retrodicted the correct value of the perihelion of mercury and predicted the bending of starlight observed by Arthur Eddington, a discovery that splashed Einstein’s name on the front pages of the world’s leading newspapers. Eddington was Einstein’s heir, thoroughly learning the theory and grasping its implications for stellar structure. Ironically he did not dare to take these implications to their logical conclusion. That task was left to a young Indian astrophysicist named Subrahmanyan Chandrasekhar who paved the way toward the discovery of black holes by considering what happens when stars run out of fuel and collapse under gravitational contraction. Famously Eddington rebuked Chandrasekhar’s findings and revealed himself to be much like Einstein, a revolutionary in young age and a reactionary in old age.

The story of black holes is one important thread that the book follows. Chandrasekhar’s ideas were further developed by Lev Landau, Fritz Zwicky and Robert Oppenheimer in the 30s. Oppenheimer’s story is especially interesting since he was the one who theoretically discovered black holes but later completely dissociated himself from them, showing no interest in general relativity until the end of his life. In fact Oppenheimer’s view of relativity was similar to that of the vast majority of physicists who were caught up in the revolutions in nuclear and quantum physics in the 30s and 40s. Quantum mechanics and particle physics were the new frontiers; relativity was a speculative backwater.

It was the eminent Princeton University physicist John Wheeler who picked up where Oppenheimer had left off. Wheeler is really the father of modern relativity since he was the one who rejuvenated interest in the topic in the 50s and 60s. Many of his students like Jacob Bekenstein and Kip Thorne became leaders in the field. In Britain the field was fathered by Dennis Sciama, whose students Roger Penrose and Stephen Hawking led the way in understanding singularities and the Big Bang. Hawking especially forged a very important link between information, relativity, thermodynamics and quantum mechanics through his exploration of what we now call the “black hole information paradox”.

Hawking’s work on singularities connects to the second major thread of the book, this time involving the applications of general relativity to the entire universe. The story begins right after Einstein developed his framework when Russian bomber pilot Alexander Friedmann and Belgian priest Georges Lemaitre found out that one of the solutions of the equations would be an expanding universe. In a famous mistake which Einstein called “the greatest blunder of my life”, Einstein had found this solution but, based on the observation of a locally static universe, had applied a fudge factor – a “cosmological constant“-  to halt the expansion which turned out to have great significance almost eight decades later. Lemaitre and Friedmann’s story logically leads to that of Edwin Hubble who in 1929 observed the redshifting of galaxies, thereby inaugurating one of the great eras in the exploration of the cosmos. This era culminated in the discovery of dark matter and dark energy and the transformation of cosmology into a precision science, all of which has opened up frontiers undreamt of by Einstein. And Ferreira hopes there’s much more in store than can flow from those beautiful equations.

Ferreira is quite adept at describing these two main threads. One of the most important aspects of the development of relativity was the shot in the arm which the theory received from experimental observations of distant objects by radio telescopes made by Martin Ryle, Jocelyn Bell and others. In fact the book underscores the fact that without these observations relativity would have continued to be considered mathematical doodling at worst and speculative science at best. The grounding of relativity in the real world through the discoveries of quasars, pulsars, neutron stars and black holes makes the paramount significance of experimental evidence in lending respectability to a theory quite clear. Personally I would have appreciated it if Ferreira had also considered some other evidence for general relativity, such as the observation of frame-dragging by Gravity Probe B, a technical marvel and a jaw-dropping exercise in accurate measurement if there ever was one.

The last part of the book concerns the quest over the last four decades to combine general relativity with quantum mechanics, an effort that was started by Wheeler and his student Bryce DeWitt in the 60s. The same techniques of field theory that led to such spectacular successes in particle physics – culminating in the Standard Model – failed abysmally when applied to relativity. One possible way out is string theory whose virtue is that gravity emerges naturally from the theoretical framework. Another promising framework is loop quantum gravity. The problem with string theory, as well known by now, is that it makes no testable predictions and its solution space is so vast that virtually anything can be accommodated in its expansive embrace. In science, a theory that can explain anything and everything is usually considered a theory that can explain nothing.

One thing that again struck me is how important experiment and observation are for actually taking a theory from a realm of fanciful speculation to hard reality. It’s worth comparing the progress of quantum mechanics, general relativity and string theory in this context. Quantum mechanics was developed in the 1920s and immediately explained scores of previously confusing experimental facts. Its success only grew in the 30s and 40s as it was applied to solid-state physics, chemistry and nuclear physics, always amply supported by experiment. The philosophical conundrums in the theory – which we still struggle with – did not harm the theory because of its great experimental success. In contrast, general relativity was developed about ten years earlier. By 1940 or so it had two major experimental predictions to its credit: the bending of starlight and the expansion of the universe. But even by the late 1950s it had not become part of mainstream physics and was considered more mathematics than physics, mainly because the experimental evidence was lacking. As mentioned above, it was only the development of radio astronomy that really put the whole framework on a firm pedestal.

Thus it took quantum mechanics no time at all and relativity almost forty years to become respectable, even when there were two astonishing experimental observations which the latter had successfully predicted. The great difference was the experimental evidence, copious in case of the former and spotty and only slowly emerging in case of the latter. Compared to this, string theory has been around for about forty years and there is still no unambiguous experimental evidence in its favor. Purely on a historical basis this might hint that it may be on the wrong track. There’s a reason why Feynman said that the only true test of a scientific theory is experiment.

Ashutosh Jogalekar About the Author: Ashutosh (Ash) Jogalekar is a chemist interested in the history and philosophy of science. He considers science to be a seamless and all-encompassing part of the human experience. Follow on Twitter @curiouswavefn.

The views expressed are those of the author and are not necessarily those of Scientific American.

Rights & Permissions

Comments 18 Comments

Add Comment
  1. 1. M Tucker 4:20 pm 02/5/2014

    “In contrast, general relativity was developed about ten years earlier.”

    I disagree that general relativity is the older theory. The work done in the 20’s would not have been possible without the work done by Max Planck 20 years earlier. Einstein’s photoelectric effect paper would not have been possible without Planck’s work. The demonstration of the quantization of energy began it all and it was Planck’s monumental achievement. The work done in the 20’s was achieved by the constant development of new mathematical techniques applied to resolve the experimental results and rigorously explain the atomic models all beginning prior to general relativity.

    Link to this
  2. 2. curiouswavefunction 4:47 pm 02/5/2014

    Quantum theory did have its roots in Planck’s work, but I am talking about the development of quantum mechanics as a complete theory and a practical tool. Until the 1920s we had what was called the “old” quantum theory which was successful but failed to explain many observations like the Stark and Zeeman effects and the structure of multi-electron atoms. It was also not paired up with special relativity. You could apply quantum mechanics to practical problems only with the work of Heisenberg, Born, Schrodinger and Dirac; for instance Heitler and London could perform their calculation even on the very simple hydrogen molecule only in the 20s. In contrast, the basic framework of general relativity was complete by 1917 and Friedmann could derive cosmological models from it in 1922 (this is explicated in Walter Isaacson’s book for instance).

    Link to this
  3. 3. M Tucker 6:06 pm 02/5/2014


    Yes, exactly. Quantum mechanics was experiment driven and developed over many years. The first big theoretical work was done by Planck. Because it was experiment driven it took a while to develop the mathematical framework.

    General relativity was pure theory that sprang from the genius of Einstein and experiments had to be performed in order to verify it.

    Then with Yukawa and Dirac we see more evidence of theory preceding discovery. That is why I think so many are willing to take mathematical theory as verification for inflation theory and the multiverse and even the correctness of string theory.

    Link to this
  4. 4. LeeJRickard 6:46 pm 02/5/2014

    Chandra had a very insightful discussion of the qualities of General Relativity that made it particularly attractive, in his Schwarzschild lecture in 1986. There’s a good reprinting in his little book Truth and Beauty.

    Link to this
  5. 5. rloldershaw 6:47 pm 02/5/2014

    Since the advent of the very elegant theory of General Relativity, physics has suffered because of its inability to bring a fundamental symmetry into its theories: relativity of scale.

    Weyl, Einstein, Dirac and a host of others tried repeatedly to accomplish this, but it never seemed to work.

    However, if your emphasis is on studying nature, instead of studying mathematical models, then you can see how nature accomplishes this.

    Nature cannot have continuous conformal symmetry because that strongly violates our empirical knowledge of nature.

    But discrete conformal symmetry does not need to conflict with empirical results.

    If the laws of physics, especially General Relativity, are made more symmetric so as to incorporate discrete conformal symmetry, then you get a new and completely different understanding of nature in terms of a discrete self-similar hierarchy that has no bounds, i.e., a discrete fractal paradigm for all of nature.

    With this new paradigm you can unify GR and QM, explain the fine structure constant, demystify h-bar, resolve the vacuum energy density crisis, predict the exact nature of the dark matter, retrodict the masses of all particles (including the electron), and have a proper understanding of the hierarchy of Planck scales.

    This new paradigm predicted pulsar-planets, and it predicted the hundreds of billions of unbound planetary-mass objects recently inferred as roaming free throughout the Galaxy. It makes an exact prediction for the dark matter mass spectrum.

    I have a website (link below) that serves as a teaching resource for this new paradigm.

    Discrete Scale Relativity/Fractal Cosmology

    Link to this
  6. 6. jtdwyer 12:29 am 02/6/2014

    Nicely done – interesting and readable!
    BTW, I have a reading problem – quickly losing interest in any book I attempt to read: perhaps you could condense other interesting books on physics?

    Link to this
  7. 7. cshbar 8:07 am 02/6/2014

    The comments about loop quantum gravity and string theory are are just horrendously lazy and stupid. Its not journalism to read one guy’s popular book and adopt its slogans like religious edicts. The author clearly has not done his homework.

    Nobody has any business trying to explain the state of quantum gravity research unless they understand the first thing about it themselves, which the author clearly doesn’t. The first thing worth knowing about quantum gravity is that there is a dimensionful constant that matters – the Planck scale – at which the characteristic physics of whatever the correct theory is should really come into full effect. So maybe the author better look up on wikipedia what the value of that scale is and compare that to the collision energy of the LHC if he has some trouble understanding why experiments haven’t been able to offer much guidance on the unification of gravity and quantum mechanics.

    That is only the most basic problem. Its crazy to suggest a big problem with string theory due to lack of predictions, and not even bother to notice that Loop Quantum Gravity doensn’t predict anything at all. Thats just what people say who have been trained like Pavlov’s dog by Peter Woit to sling this criticism at string theory and never apply the standard elsewhere. Exactly what is the author claiming LQG is supposed to predict?

    Furthermore, and most importantly, the problem with string theory isn’t that it lacks predictivity. The problem is that, like any sensible theory of physics, it can only make predictions once a configuration of its ingredients is chosen. So to make a prediction, as model builders do all the time, one only has to propose a configuration that could plausibly reduce to the Standard Model at low energies. That is exactly analogous to what Higgs, Weinberg and the other fathers of the Standard Model did. They made specific model-building proposals based on the known evidence, they didn’t stand around whining about the “vast solution space” of quantum field theory, which is enromously more accommodating than string theory’s even.

    So in short, please learn the basics from some actual scientists, not popular authors. Maybe try reading the series “Quantum field theory, string theory and predictions” by Matt Strassler, since you clearly don’t understand the first thing about which you’re writing.

    Link to this
  8. 8. Curious Wavefunction in reply to Curious Wavefunction 10:04 am 02/6/2014

    cshbar: Clearly you have a bone to pick with Peter Woit. I did not claim that loop quantum gravity makes testable predictions; I only mentioned it as another possibility that some people are working on. Please read the post again. Also, the kind of “predictions” you are talking about are either postdictions or highly vague predictions that can be made to agree with pretty much anything. It’s also very convenient to say that your theory makes predictions at an energy scale which may never be achieved in practice. All these are serious problems. In any case, if you think you have a problem with Pedro Ferreira’s book you should probably email him since he is the author of the book, not me.

    Link to this
  9. 9. cshbar 11:31 am 02/6/2014

    Curious Wavefunction: I haven’t read the book ; my criticisms were directed at you. Based on your reply, it seems you still haven’t understood why what you write is nonsense, so perhaps I will try to explain it again.

    The diversity of solutions of string theory is the one and only reason anyone was ever able to claim it “makes no testable predictions”. So when you write “it makes no testable predictions *and* its solution space is so vast that virtually anything can be accommodated” its really clear you are confusing the diversity of solutions with a lack of ability to compute things in general. And that is a serious confusion, and a deeply misleading idea to propagate, so you should correct it. If you pick up any string theory textbook, like Polchinski’s, you can learn in detail how to calculate the predicted results for all kinds of possible experiments. There are regimes of the theory where calculation becomes impossible, much like in the Standard Model, but there is nothing vague here. If you understand string theory, then you understand how to calculate with it, and that means deriving predictions from the theory. If there is any confusion about this point in the science-following public I hold responsible all the bloggers and authors who have sown this confusion with statements like yours.

    You say “In science, a theory that can explain anything and everything is usually considered a theory that can explain nothing.” Well thats a cute slogan except that it couldn’t possibly have come from anyone familiar with the subtlety of these issues in real, established science. Because the most successful scientific framework in history is quantum field theory, which can explain not just any phenomena we have observed, but almost any phenomena we could ever imagine observing. If we were to take this snide comment seriously none of the progress that has been made in particle physics over the past century would have been possible. Because instead of using the framework of QFT to make predictions, all of our heroes should have just been paralyzed by the ability of QFT to “explain anything”. What you fail to understand is that predictions require not just a broad overall framework of physical laws but specific model building choices. And that is exactly what modern string theorists (AND field theorists) working on model-building do. They propose models that reproduce the verified behavior low energies, and modify the behavior in some way at higher energies that have not yet been tested.

    As for your “It’s also very convenient to say that your theory makes predictions at an energy scale which may never be achieved in practice.” Here again, reviewing some absolute basics is in order. String theory says nothing about the value of the Planck scale; this comes from experiment. So its not “convenient” for anyone in science, its very inconvenient, but this is an observed fact about the universe, not a feature of string theory, as anyone commenting on this should know.

    And it may be true you didn’t claim LQG makes testable predictions. Those couple sentences in your article strongly implied it. I took note of this because it seems emblematic of the phenomenon where bloggers write about string theory by mimicking what its critics in the popular press say, instead of anyone who actually knows what they’re talking about. If you were aware that LQG has no universal predictions either, then why not state it? If you think LQG is promising, then why not state why? All you seem to be doing here is regurgitating some conventional wisdom using the exact same sequences of words as we always hear. But the conventional wisdom, as you’ve recited it, is deeply flawed, and its not reflective of any kind of consensus among people who actually work on these topics. So if you’re as unfamiliar with these matters as you seem, I think your audience would be better served if you simply left that whole part out.

    Link to this
  10. 10. Shoshin 1:30 pm 02/6/2014

    “One thing that again struck me is how important experiment and observation are for actually taking a theory from a realm of fanciful speculation to hard reality.”

    Excellent insight. You hit the nail on the head. Too bad most climate “scientists” have yet to understand and embrace this issue.

    Link to this
  11. 11. curiouswavefunction 4:04 pm 02/6/2014

    cshbar: Firstly, let’s keep the tone here civil, please. And secondly, sorry but I still don’t think you get it. Quantum field theory can explain many things but as a prediction, not as a postdiction. If you make your assumptions and constraints loose enough so that your “theory” can account for virtually any scenario then it’s not a good theory. Just witness the “explanation” of the fundamental constants based on M-theory’s postulation of multiple universes (documented in the latest Tegmark book for instance). Or consider the difference between good and bad climate models. Even Matt Strassler mentions some of the vague predictions and postdictions of string theory on his blog. In any case, this is a post about Pedro Ferreira’s book and not about string theory so we are getting sidelined here. If you want to learn about these issues or debate them I would suggest you read Peter Woit’s book or blog. Thanks for commenting.

    Link to this
  12. 12. elautin 5:35 pm 02/6/2014

    From the article:

    “…and predicted the bending of starlight observed by Arthur Eddington, a discovery that splashed Einstein’s name on the front pages of the world’s leading newspapers.”
    Questions from an interested layperson:
    The verification of Einstein’s prediction by Eddington did splash Einstein’s name in the newspapers. Is it true that this experiment was delayed by the inconvenience of World War 1? That Einstein had made an initial miscalculation in his prediction, and had the eclipse measurements been made the first time, Einstein’s predictions would have been wrong? (Even though he could have shown the calculation mistake after the measurement, credibility would have been a problem—even though the theory was correct.) That the delay permitted Einstein to correct his error that was then verified by Eddington. Is it true that Eddington’s measurements verified the predictions, in fact and in retrospect the measurements were not accurate enough and the verification misleading? That actually correct verification happened years later (many times)?

    “…one of the solutions of the equations would be an expanding universe. In a famous mistake which Einstein called “the greatest blunder of my life”, Einstein had found this solution but, based on the observation of a locally static universe, had applied a fudge factor – a “cosmological constant“- to halt the expansion which turned out to have great significance almost eight decades later.”
    Einstein used the “cosmological constant” to halt the “collapse of the universe due to gravity (actually, I guess, due to curved space-time). The cosmological constant was an expansile force that would counteract gravity precisely—resulting in the steady state universe that was known at the time. Retrospectively, the steady state is unstable. The recent observation of the accelerating expanding universe uses the new cosmological constant, “dark energy.” Is it at all reasonable to correlate this with Einstein’s cosmological constant and to then say that Einstein’s blunder was an insight by Einstein. One that he didn’t live long enough to learn was “true.?”

    Link to this
  13. 13. jtdwyer 8:03 pm 02/6/2014

    As I understand, Einstein’s discovery that his original field equations produced incorrect results and his subsequent search though his notes for any equation that produced correct results amounts to parametric fine-tuning more than theoretical work…
    - However, in defense of his preference for a stable ‘universe’ it should be noted that, at that time the entire universe was then considered to fit within the boundaries of what we now consider to be the Milky Way galaxy. The observed universe could reasonably be considered to be gravitationally bound! Intergalactic spacetime was undiscovered!
    - I think it’s most unfortunate that the abandoned cosmological fudge factor had historical precedence when the supernovae search teams – attempting to more precisely constrain the cosmological deceleration parameter – discovered the unexplained discrepancy between distance estimates derived from type Ia supernova luminosity and those derived from their host galaxies redshift – applied to then standard cosmological models. Those researchers simply derived a ‘cosmological constant’ parameter value and inserted it into their model to fit its predictions to observations.
    - As a result, the apparent reacceleration of universal expansion is, because of Einstein’s early attempt to fine tune the ‘cosmos’ (actually, the milky way), most commonly presumed to be the physical product of a constant vacuum energy density!
    - I only wonder how vacuum energy density can possibly remain constant within a dimensionally expanding intergalactic spacetime spacetime and the variably contracted gravitational spacetimes of large scale bound structures (such as galaxies)!
    - This convenient reinterpretation of Einstein’s fudge factor may yet turn out to be ‘his’ greatest blunder!

    Link to this
  14. 14. anumakonda.jagadeesh 9:13 am 02/7/2014

    Very interesting.
    Dr.A.Jagadeesh Nellore(AP),India

    Link to this
  15. 15. sfagnew 12:46 am 02/10/2014

    This is all well and good, but somehow, gravity is not enough to explain the structure of galaxies and galaxy clusters. The greatest failing of general relativity today is that it does not explain either dark matter or dark energy. Does this book even mention that? No theory or model ever will explain all of the observations of our complex universe. Nevertheless, it seems rather obvious that there is something seriously wrong with general relativity and its treatment of gravity and matter and time and action. Science knows that there must be a quantum gravity, but so far, there has not been a quantum gravity. This is really a shame and not just an oddity and so general relativity has several large gaping holes that science seems content to overlook for the time being…

    Link to this
  16. 16. curiouswavefunction 5:56 pm 02/10/2014

    Yes, the book does mention all this. And on that point it’s more ambiguous; on one hand it says that dark matter and dark energy open up new opportunities for GR, on the other it also says that perhaps we need a new theory surpassing GR that may explain these things.

    Link to this
  17. 17. deSitter 4:49 am 02/19/2014

    GR was born with a correct prediction – the advance of the perihelion of Mercury’s orbit. You left that out.

    I think I will not be reading this book, which sounds like more of the same semi-literate boilerplate from the pen of a self-promoter that were are so used to seeing lately. And Einstein himself would hardly have called his theory “perfect” – in fact there is no conservation law for energy, and I’d say that was a pretty huge flaw.

    It’s really a shame that the days of literate and thoughtful scientists are over. There will never be another Gamow or even a Sagan. Maybe I should try my own hand. I certainly can do no worse.


    Link to this
  18. 18. bolonkib 10:22 pm 03/3/2014

    In “Warped Passages” by Dr. Lisa Randall, professor of quantum physics at Harvard University, that direction through the variously bent timespaces is officially defined for science to be the fifth dimension.

    Link to this

Add a Comment
You must sign in or register as a member to submit a comment.

More from Scientific American

Email this Article