About the SA Blog Network

The Curious Wavefunction

The Curious Wavefunction

Musings on chemistry and the history and philosophy of science
The Curious Wavefunction Home

Surprises in physics: From black bodies to the accelerating universe

The views expressed are those of the author and are not necessarily those of Scientific American.

Email   PrintPrint

Max Planck's revolutionary 1900 discovery that energy in the subatomic world exists as discrete packets marked the beginning of a century of spectacular surprises in physics (Image: Wikipedia Commons)

Surprises rank high on the list of things that make science a source of everlasting delight. When it comes to being surprised scientists are no different from the general public. Just like children on their birthdays being surprised by unexpected gifts, scientists revel in the surprises that nature whips up in front of them. Surprises in science point to something deeper: the mystery, excitement and things unseen and unknown that always keep the scientific enterprise interesting, amusing and profound.

Scientific surprises are not always tantamount to important findings. For instance the experimental detection of the Higgs boson was a great achievement but it was not exactly surprising since the theoretical prediction had been made much earlier. The theoretical prediction itself emerged as a logical extension of ideas which were then in the air, and no fewer than six individuals contributed to its genesis. The fact is that some discoveries in science are important, some are surprising and some are both. It is this third category that is the most memorable, and rare is the scientist who finds himself or herself the beneficiary of a discovery that is both surprising and important.

Thus it’s worth taking a look back and charting some of the most important surprises in science, those that either forced us to rethink a lot of our assumptions or, in rare cases, those that truly changed our view of the world. Here I present a list of surprises in physics drawn from roughly the last one hundred years. Physics is in equal parts a theoretical and an experimental science, so its surprises come from both arenas. Feel free to note other surprises which I might have missed, and remember that surprising is not the same as important and important is not the same as surprising (and therefore this list excludes many of the most important discoveries in physics).

1900 – Max Planck: The beginning of quantum mechanics – Max Planck’s solution to the thorny theoretical problem of blackbody radiation started what was surely the greatest revolution in physics since Isaac Newton’s time. Planck was famously trying to explain the dependence of the intensity of radiation emitted by a black body on its frequency and temperature. The conservative German physicist was essentially struggling with what we today call “curve fitting”, finding the right equation for a graph of experimental data. He realized that the only way he could do this was to imagine a formula with a new constant called h and the assumption that energy was emitted by the blackbody only in certain discrete units. Planck intended his solution as a mathematical fix and not a representation of reality, and it was only when Albert Einstein appeared on the scene that he realized the nature of the radical transformation in our view of reality that he had inaugurated. The age of the quantum had dawned.

1905 – Albert Einstein: Special relativity – In 1905 Einstein wrote five papers that changed the face of physics, but perhaps only one of these can be called truly surprising. This was his famous paper setting out the principles of the special theory of relativity. There were of course many profound surprises in the theory – including time dilation and length contraction – but the biggest fundamental surprise was probably the iron rule that required the speed of light to be constant. The breakthrough was counterintuitive, bold and clearly revolutionary.

1909 – Ernest Rutherford: The atomic nucleus – Until Rutherford’s time the structure of the atom was nebulous, the best guess coming from J. J. Thomson who imagined it as a “plum pudding” with negatively charged electrons uniformly embedded in a positively charged sphere. It was in Manchester that Rutherford, along with two associates and a 70 pound grant from the Royal Institution, performed his famous gold foil experiment in which he shot alpha particles at a thin gold foil. If Thomson’s uniform and homogeneous atomic model had been correct the particles should have been scattered equally in all directions. Instead a very few of them came right back at the experimenters. The novelty and surprise of the result is best captured by Rutherford himself: “It was almost as incredible as if you had fired a 15 inch shell at a piece of tissue paper and it came right back at you”. Rutherford’s discovery signaled the beginning of nuclear physics.

1911 – Heike Kamerlingh Onnes: Superconductivity – Until 1911 there were divergent views regarding the behavior of matter at very low temperature. Kamerlingh Onnes from Leiden settled the debate in 1911 by finding, to his utter surprise, that mercury at 4.2 degrees Kelvin lost all resistance to the flow of current. The discovery was so surprising and significant that Onnes received a Nobel Prize only two years later. Superconductivity continues to be a frontier of physics research, both pure and applied.

1913 – Niels Bohr: The quantum theory of the atom – Bohr’s surprise was the postulation that electrons can occupy only certain energy levels in atoms and emit photons with energies defined by Planck’s formula when they make transitions. This really marked the beginning of the weird, probabilistic world of quantum mechanics, encapsulated by the question, “How does the electron know when to make a transition”?

1917 – Albert Einstein: General relativity – Einstein followed up his annus mirabilis with his crowning achievement – a theory of gravitation – in 1917. General relativity sealed the seamless meld of space and time and postulated gravity not as Newton’s force but as a property of spacetime itself, arising from the curvature of spacetime by the presence of mass.

1919 – Arthur Eddington: The bending of starlight – Leading an expedition to the island of Principe lying off the West coast of Africa, Eddington confirmed one of the most startling predictions of general relativity, the bending of starlight by the gravitational field of a massive object. This was not strictly a surprise since Einstein had predicted it, but in science any theory is only as good as the data that supports it. The discovery not only splashed Einstein’s face all over the world’s papers but also signified a rare bond of scientific friendship between two nations which had just ended a devastating war with each other.

1926 – Werner Heisenberg: The Uncertainty Principle – Heisenberg’s Uncertainty Principle – which has since then been hijacked and affixed to remote and bizarre notions in popular culture – drew a wedge between our assumptions of determinism and the nature of reality. Along with the phenomenon of quantum entanglement it essentially captures all the weirdness about the quantum world. Since then we have been struggling with the legacy of the true nature of quantum mechanics.

1928 – Paul Dirac: The Dirac equation – The Dirac equation is one of the true glories of theoretical physics. In a single line that can be neatly stated on a cocktail napkin, it combines special relativity with quantum mechanics. But most surprisingly, it proposes a new form of matter – antimatter. Dirac found this implication of the equation so unsettling that for some time he assumed that the missing twin of the electron predicted by the theory must be a proton. But in 1932 Carl Anderson found the elusive positron in cosmic rays. Both Dirac and Anderson got well-deserved Nobel Prizes for prediction and discovery.

1929 – Edwin Hubble: The Expanding Universe – Just as the Dirac equation is one of the true glories of theoretical physics, so is Hubble’s discovery of an expanding universe in the subsequent year one of the true glories of astronomy. Again, the possibility of an expanding universe has been conjectured by Russian Alexander Friedmann and Belgian priest Georges Lemaitre. But this possibility was so bizarre that even Einstein could not take it seriously at first. It took Hubble and his painstaking studies of nebulae at the Mount Wilson observatory in California to etch the truth in stone.

1938 – Otto Hahn and Fritz Strassman: Nuclear fission – Both Enrico Fermi’s group and the husband-wife duo of Irene and Frederic Joliot-Curie narrowly missed discovering fission in the mid-thirties. Curiously, the chemist and physicist Ira Noddack (who celebrates her birthday today) had hypothesized the phenomenon in 1934 but was ignored; one wonders if everyone would have taken her more seriously had she been a male scientist, preferably German or English. Yet it was a female scientist, Lisa Meitner, who interpreted Hahn and Strassman’s novel breaking up of uranium into barium and calculated the startling magnitude of energy release, paving the way toward both the peaceful and the destructive uses of atomic energy. Fission was thought to be so improbable that great theorists like Oppenheimer and Bethe had done calculations arguing against its existence. And yet here it was, an elegant theory demolished by an ugly, consequential fact.

1939 – Robert Oppenheimer and Hartland Snyder: Black holes – In the same issue of the journal Physical Review in which Niels Bohr and John Wheeler published their famous liquid drop model of fission, Oppenheimer and his student Hartland Snyder published the first description of what we now call black holes. The paper’s conclusions were surprising and bizarre and followed on the heels of a decade’s worth of groundbreaking research on the implications of general relativity for stellar evolution by Subrahmanyan Chandrasekhar, Lev Landau, Fritz Zwicky and others. Curiously, Oppenheimer never followed up on this paper and in fact displayed a remarkable indifference to general relativity for the rest of his life.

1947 – Willis Lamb: The Lamb Shift – The Lamb Shift was one of those subtle experimental measurements that lay the foundation for a whole new field. In 1947 Lamb and Retherford measured a tiny, surprising difference in energy levels of the electron in the hydrogen atom; the Dirac equation predicted identical energies for both these levels. This tiny difference concealed a wealth of important aspects of quantum field theory, and feverish and pathbreaking theoretical work explaining the shift culminated in the creation of the strikingly accurate theory of quantum electrodynamics (QED) by Schwinger, Feynman, Dyson and Tomonaga by the end of the decade.

1956 – Yang and Lee: Parity violation – Parity is one of the most basic properties of the atomic world. In physics a parity transformation is an operation that changes the sign of the spatial coordinates of an object. For instance a parity transformation on a right-handed coordinate system will turn it into a left-handed one. Until 1956 it was rightly and reasonably believed that nature should not care whether we use right or left handed coordinate systems and that the laws of physics should be the same for both. Yet in 1956, Chen Ning Yang and Tsung Dao Lee showed that parity was not conserved for certain particles which decayed through the weak nuclear force, so that the products of their decay depended on whether their spins were aligned parallel or antiparallel to a magnetic field. This result was confirmed by C. S. Wu in an elegant experiment at Columbia University. The discovery that at a very fundamental level nature actually cares about parity was so radical that Yang and Lee received a Nobel Prize only one year later. Wu was left out.

1964 – Penzias and Wilson: The cosmic microwave background – The discovery of an expanding universe had given at least some scientists the inkling of a purported “big bang” origin for the universe. But until 1964 this was just a hypothesis, so much so that the Cambridge astrophysicist Fred Hoyle had coined the phrase “Big Bang” as a derogatory dig at the theory. 1964 changed all that. Arno Penzias and Robert Wilson’s discovery – made using a small Bell Labs antenna in Holmdel, New Jersey – of the remnant of the big bang in the form of a steady background radiation hum was one of the most significant and surprising discoveries in twentieth century science, and one which has led to at least two Nobel Prizes. With Penzias and Wilson our current view of the universe acquired a firm foundation.

1964 – John Bell: Bell’s Theorem – Einstein had never been happy with quantum mechanics, all the way until his death, partly because his 1933 EPR paradox paper seemed to imply non-local, seemingly faster-than-light communication of information between particles. In spite of quantum mechanics’s spectacular predictive power, Einstein was convinced that there must be some kind of “hidden variables” which could explain what seem like contradictory and bizarre properties of the quantum world and bring locality into the picture. In 1964 an obscure Irish physicist named John Bell laid these questions to rest with a remarkable and surprising theorem that can be derived and stated using high school algebra. Because of Bell’s obscurity it seemed to come out of left field. In short, Bell’s Theorem said that no local, hidden variable theory of quantum mechanics could explain its basic postulates. The theorem thus established quantum weirdness – and especially entanglement – as a fundamental part of the physical universe. An equally surprising development was the set of experiments by Alain Aspect, Stuart Freedman, John Clauser and others in the 70s and 80s confirming Bell’s Theorem.

1973 – Gross, Politzer and Wilczek: Asymptotic freedom – Asymptotic freedom refers to the strange weakening of the force between quarks as they come near. This is a surprising and totally counterintuitive mechanism of particle interaction since in case of every other force (gravitation, electromagnetism with opposite charges and the weak force), the interactions become stronger with decreasing distance. The discovery of asymptotic freedom put the quantum theory of the strong force (called quantum chromodynamics) on a sure footing.

1986 – Bednorz and Mueller: High temperature superconductivity – In 1986 the world of physics and materials science was shaken by the discovery of ceramic materials like yttrium copper barium oxide which become superconducting at “high” temperatures, in some cases above the boiling point of liquid nitrogen (77 degrees kelvin) and in others as high as 138 degrees kelvin. This not only caused a revolution in our understanding of superconductivity (these materials were ceramics, not metals) but also led to dreams of practically harnessing the phenomenon. The dream has not quite come true yet, but ceramic superconductivity continues to be a thriving field of physics research. IBM’s Alex Muller and George Bednorz received a Nobel Prize in the very next year; so groundbreaking and surprising was the discovery.

1998 – Perlmutter, Reiss, Schmidt and others: The accelerating universe – Even after Hubble discovered the expanding universe, for the longest time scientists believed that the rate of expansion was slowing. Observations of distant supernovae in the early 90s completely turned this picture on its head. It was found that the universe was not only expanding but the expansion was speeding up. Einstein’s cosmological constant was resurrected and a new entity, dark energy, was postulated as a placeholder to account for the process. This was by any definition a surprising and counterintuitive discovery wherein experiment drove our understanding of reality without any major theoretical input.

A few observations emerge from this partial list of surprising discoveries in physics. The most important is that while lone individuals made discoveries in the early period, most findings in the later period were the work of groups of researchers or at least research duos. Many times more than one team made the same discovery. The discoveries were also a mix of theoretical and experimental work; in many cases theory and experiment were equally important since it’s one thing to postulate a surprising result but quite another to actually observe it in front of your eyes. I also find it interesting that no truly surprising fundamental discovery has emerged in the twenty-first century yet, although we are just getting started.

One thing is clear from the list, that physics will continue to make surprising discoveries and that our understanding of the universe will never be complete. We can all look forward to that.

Ashutosh Jogalekar About the Author: Ashutosh (Ash) Jogalekar is a chemist interested in the history and philosophy of science. He considers science to be a seamless and all-encompassing part of the human experience. Follow on Twitter @curiouswavefn.

The views expressed are those of the author and are not necessarily those of Scientific American.

Rights & Permissions

Comments 7 Comments

Add Comment
  1. 1. jtdwyer 5:07 am 02/26/2014

    Again, nicely done – I especially like the balanced report of the accelerating universe. To clarify a couple of fine points (as best I understand), the universal expansion of spacetime is still considered to have slowed from early periods until around 5-7 billion years ago, when observations of type Ia supernovae conflicted with the results of then standard cosmological models. The discoverers used the convenient cosmological constant model parameter (which had been by convention been set to zero for the past nearly 70 years) _and_ changed (for observations between about 2-5 billion light years away) their deceleration parameter to a _negative_ value! Unfortunately, without theoretical basis that established the largely undefined cosmological constant, which produced an effect within the model somewhat like antigravity might, to be the default cause of the identified acceleration of universal spacetime expansion.
    - To be brief, there are other potential explanations – including one suggested by the discoverers, but then lost in the shuffle. Since the density of matter diminishes as the universe expands, the probability that light from distant supernovae might be dimmer than models predicted (as observed) not because of their greater than expected distance but by increased intergalactic dust. Recently I’ve become increasingly interested in an area of active research regarding cosmological modeling.
    - In recent years researchers have identified that the universe exhibits a pronounced ‘large scale structure’ of filaments, composed of galaxies, galaxy clusters and inferred dark matter, separating enormous bubble-like voids, which contain very little matter. One of the guiding principles of cosmology is that the universe contains a homogeneous distribution of mass and energy. As a result, the dynamics of the universe can be described by a discrete set equations whose parameters are represent averages of universal characteristics, such as mass-energy density. However, the discovery of large scale structure with varying characteristics seem to imply that voids, for example, with very low mass-energy density might expand at much faster rates than the average. As a result, the relatively smaller scale ‘large scale structures’ could affect the universal average rate of expansion. As structure has developed in the universe, its smaller scale effects could have caused the universal rate of expansion to increase… Please see the review article: “Does the growth of structure affect our dynamical models of the universe?”

    Link to this
  2. 2. jtdwyer 5:35 am 02/26/2014

    I also have to point out that you’ve omitted the ‘discovery’ of dark matter. Despite there being no actual detection of any of several hypothesized exotic dark matter particles (that is not predicted by the standard model of particle physics) – its presence is most commonly inferred from discrepancies between identified gravitational effects and those predicted by models of exceedingly complex, compound configurations of many billions of discretely interacting masses. For an overview, see
    - The critical issue is that, after more than 30 years, the physical existence of dark matter has not been definitively confirmed. Many if not most physicists are convinced of its existence, however, and will vigorously defend their belief in it. However, so many’s careers are deeply vested in its existence – over the past several decades many thousands have published research considering is nature and cosmological effects. It is now the center-post of the current standard Lambda-Cold Dark Matter cosmological model and is absolutely critical to the interpretation of most cosmological observations. In summary – its had an enormous impact on cosmology, astrophysics particle physics and other disciplines.
    - Still, I contend that its presence was initially established primarily to explain the rotation of spiral galaxies by using faulty gravitational evaluations. IMO, it was erroneously expected that a vast distribution of discretely interacting masses within a planar spiral galaxy disk should exhibit rotational velocities that diminish as a function of radial distance, just like planets that almost exclusively interact with the Sun… Dark matter was employed to explain the observational deviation with that expectation. Please see my brief, informal essay “Inappropriate Application of Kepler’s Empirical Laws of Planetary Motion to Spiral Galaxies…”

    Link to this
  3. 3. bigbopper 9:50 am 02/26/2014

    Nice article. However I would quibble with the notion that something can be a “surprise” if it has been predicted on theoretical grounds. Thus the discovery of the CMB shouldn’t really have been a surprise as it had been predicted. Likewise the bending of starlight had been predicted by GR and its confirmation came as no surprise to Eddington and colleagues. Indeed, even the Lorentz transformations were well known prior to the advent of SR, although Einstein was the first to show that they arose naturally from the two simple postulates of SR.

    A few other minor quibbles: 1) quantum mechanics really began with Einstein’s 1905 paper on the photoelectric effect, and his 1907 paper on the specific heat of solids at low temperatures. 2) No list of the great surprises of 20th century physics should omit the discovery of dark matter.

    Link to this
  4. 4. rshoff2 1:33 pm 02/26/2014

    jtdwyer: I have to say that you have included a great comment that helps me understand what thoughts are already out there. Satisfying my curiosity….

    Your points seem very reasonable and they just make sense.

    Link to this
  5. 5. jtdwyer 4:48 pm 02/26/2014

    rshoff2 – Thanks very much!
    I just noticed that I’d double-copied the wikipedia dark matter link – of course, it should be <%)

    Link to this
  6. 6. M Tucker 6:03 pm 03/6/2014

    I love the list Ash and I always enjoy your posts on the history of science. Some of my favorites not on the list:

    X-rays – 1895 Wilhelm Rontgen (1845-1923). Maybe not the first to have observed X-rays while experimenting with cathode rays but he was shocked by his observations and so was the scientific community. He was certainly the first to do a detailed study of them and publish his work along with the stunning evidence of a photo showing the bones of his wife’s hand. This really got late 19th century and early 20th century science working on overdrive.

    Radioactivity – What else might also emit X-rays? Could phosphorescent elements emit X-rays? That is what Antoine Henri Becquerel (1852-1908) asked himself. In 1896 he found something extraordinary and surprising and definitely new, the radioactivity of uranium. It led directly to the work of the Curie’s and Rutherford.

    Electron – J J Thompson, working with cathode ray as others before him, in 1897 found the very first subatomic particle and measured its mass.

    1895 was the beginning of what would prove to be a shocking period of new discoveries and whole new branches of science.

    All those mentioned above won Nobel Prizes. Rontgen was honored with the very first physics prize in 1901.

    And one of my very favorite, especially since we are sort of celebrating one of the most beloved of the science popularizers, Carl Sagan, with the new Cosmos series, the guy who said, “We are all made of star stuff.”

    But how do we know what stars are made of? How do we know what the sun is made of? When it is said that the majority of the matter in the universe is hydrogen and helium who should we credit with this?

    We did not get to this understanding until 1925 when Cecilia Payne (1900-1979) presented it as part of her PhD work. It was so shocking that when reviewing her work the great astronomer Henry Norris Russell (of Russell diagram fame) persuaded her to remove mention of it in her paper. Cecilia earned her PhD from Radcliffe College, now part of Harvard, but Russell is usually given the credit for the discovery. He later showed that Cecilia’s original work was correct and that the sun did not have a composition similar to Earth or some other imagined composition. A discovery so surprising and revolutionary that it had to be suppressed made by a woman who it is mostly forgotten today. After marriage she changed her name to Cecilia Payne-Gaposchkin.

    We should also remember Fred Hoyle for also contributing to the knowledge that “we are made of star stuff.” We generally do and actually ignore the guy who was given the Nobel Prize for it. It seems a little unfair to William Fowler who no one really remembers but Hoyle got such a bad reputation vigorously defending the steady state theory that I think it is good we give him credit for nucleosynthesis.

    Other surprising theories:
    Darwins theory – Caused controversy when it was first published and still causing controversy 155 years later.

    Plate Tectonics – violently opposed until overwhelming evidence from several different lines of investigation demonstrated it to be a reality.

    Sorry Ash, I know your post is about physics but I couldn’t help myself.

    I could add more because science is mostly about important surprises.

    Link to this
  7. 7. Ashutosh Jogalekar in reply to Ashutosh Jogalekar 11:26 am 03/19/2014

    Thanks for your list; I agree with all your recommendation. More is always better!

    Link to this

Add a Comment
You must sign in or register as a member to submit a comment.

More from Scientific American

Email this Article