What separates good from bad troublemakers? Productive provocateurs from mere contrarians, bullshit artists, attention-seekers? This is the personalized equivalent of philosophy's demarcation problem, which involves telling genuine from pseudo-science. Lee Smolin, a 59-year-old physicist at the Perimeter Institute in Canada, has always struck me as a good--even necessary--troublemaker. I first interviewed him in the early 1990s about loop quantum gravity. Conceived with Abhay Ashtekar, Carlo Rovelli and others, loop quantum gravity is an attempt to solve an abiding conundrum, the incompatibility of quantum mechanics with general relativity, Einstein's theory of gravity. Smolin has contributed to string theory, a more popular quantum-gravity model, but in his controversial 2006 book The Trouble with Physics, he deplored the dominance of string theory and argued that physics needs more diverse, creative thinking. Smolin has presented creative ideas in physics and cosmology in four other books, including, most recently, The Singular Universe, which he co-authored. In his writings, Smolin invokes the philosophical, social and historical dimensions of physics in a way that seems both old-fashioned and fresh. I've recently posted Q&As with physicists George Ellis, Carlo Rovelli, Edward Witten, Garrett Lisi and Paul Steinhardt, and their remarks got me wondering what Smolin is up to. At the end of our email exchange, he urges young scientists to seek "that hot zone where you are in equal parts a rebel and a conservative." Smolin lives in that zone.

Smolin: "Fundamental physics and cosmology have to transform themselves from a search for timeless laws and symmetries to the investigation of hypotheses about how laws evolve."

Horgan: Why did you become a physicist?

Smolin: I decided to become a physicist one spring evening when I was 17, as a result of reading Einstein’s autobiographical notes. He wrote that quantum mechanics needed a completion, which must include its unification with general relativity. At that moment I was a high school dropout planning to study architecture; but suddenly I was seized with the idea that I could follow Einstein, become a theoretical physicist and work on those two problems. That has defined my life.

I remember thinking at the time that I knew people who were smarter than me, who were better at mathematics, but that maybe I still had a small chance to discover something because nature is much smarter than all of us.

I then read everything by Einstein I could get my hands on, starting with his papers on special and general relativity. (My interest in using curved surfaces in architecture had already driven me to study differential geometry.) I then read Mach and Newton and, much later Leibniz.) To understand Einstein’s unhappiness with quantum mechanics I read books and papers by Bohr, Heisenberg and Schrodinger, as well as the Einstein-Podolsky-Rosen (EPR) paper. Thus, before I ever took a formal physics course I was immersed in the original papers and writings of the great early 20th century physicists. This gave me a perspective on the development of physics as a process of development of ideas lasting centuries. From this perspective, what happens to be popular this year or this decade seems less important than the long struggle to understand what space, time, matter and motion are. This has given me my own compass.

Horgan: Do you—or did you ever--believe in a final theory of physics, as defined by Stephen Hawking?

Smolin: I personally was always interested in the next step, which is challenge enough. I’d be happy to move science from an understanding of the first three minutes after the big bang to the stage where we have tested some hypotheses about the last three minutes before the big bang.

Beyond that, I trust in the tradition and community of science to take care of the long term. Meanwhile, let’s focus on quantum theory and relativity and their relation; if we succeed it will take generations to sort out the ramifications. We are still engaged in finishing the revolution Einstein started. This is, not surprisingly, a long process. The last revolution of comparable magnitude took 140 years from Copernicus’s book to Newton’s. When that is achieved we will have a picture as changed from our present understanding as Newton’s was from Galileo’s. Let’s wait till then to assess what to do from there.

When you know the history of physics you are painfully aware of how much each era overestimates the scope of its understanding. Science progresses well, but at each stage, we embrace metaphysical fantasies that seem motivated by the science but come to look silly when people come to know more in the future. As Brian Eno once said, “Nothing so dates an era as its conception of the future.”

Horgan: Have the problems you identified in The Trouble with Physics gotten worse?

Smolin: I identified two kinds of problems: scientific issues and sociological issues. The scientific issues I described which trouble string theory have not progressed much. While there has been development of the mathematics related to string theory and also progress on applications of an idea (AdS/CFT) )that originally arose in string theory to other fields, the obstacles to taking string theory seriously as a fundamental theory of nature remain unsolved. Among these are the lack of a complete definition of the theory, the landscape problem, the absence of any falsifiable predictions for doable experiments and the failure to prove key conjectures such as finiteness.

Nor has much changed with the sociological issues. What I’ve understood since the book was published is that these infect many fields of research and scholarship from economics to computer science to education to medicine, just to mention a few. I received many communications from experts in these and other fields who got in touch to say that their fields were troubled by the same sociological issues I addressed in my book.

The problems are rooted in the way the career and funding structures of the academy reward me-too science, lack of courage, entrenchment of failed research programs, legacy building, empire building, narrowness, defensive strategies and groupthink. These should be of concern to anyone in a position to craft incentives for academics, such as officers of funding agencies and foundations, university leaders and administrators, private donors. Many spoke to me and are concerned and a few are making efforts to craft incentives that reward high risk/high payoff, transformational science and avoid groupthink, low risk/low payoff and me-too science. I can mention the Templeton Foundation and FQXi as leaders in this field. But not nearly enough is being done. A first step would be to divert 10% of research funding to transformational high risk/high payoff research.

A danger to avoid is capture of new initiatives by empire builders and legacy seekers, who divert resources aimed for transformational science to support well established but failed research directions. This is easily done by focusing on narrow notions of excellence that reward technical virtuosity over the invention of new ideas. The problem is that new ideas are always fragile; in addition scientist who follow incremental low risk strategies are often better academic politicians than those of us who focus on high risk strategies because--their research being less challenging--they have more time and effort to devote to the game of academic influence.

Some readers of Trouble criticized me for not focusing on the sociological issues in my own fields of quantum gravity, so let me say here that they exist there too--albeit in a somewhat different form due to the fragility of the support these fields receive. Anyone who knows me in that context knows that I am the perpetual champion of opening up conferences and research groups to diverse points of view and research programs. It remains the case that LQG researchers have done much more to reach out to and include string theorists in our conferences and research groups than the reverse.

Horgan: Could physics be in trouble in part because it's bumping up against fundamental limits?

Smolin: No, I think we are nowhere near any limits on knowledge. There seems to be a general problem with lack of risk-taking in the zeitgeist, in that compared to the century before 1980 innovation has slowed on many fronts, from transportation and energy to styles of music. There are some exceptions to this, such as consumer electronics, but this slowdown of innovation should be of broad concern.

Horgan: Edward Witten still insists that string theory is the best candidate for a unified theory. Comment?

Smolin: I believe that the basic insight on which string theory is based may capture part of the truth. This is that the physics of quantum gauge theories can be re-expressed in terms of the dynamics of extended objects, first of all strings. This is in fact the same insight that is at the heart of loop quantum gravity. We have good evidence that quantum gauge fields describe the interactions of the elementary particles, and we also know since the work of Ashtekar that the deepest description of general relativity is as a gauge field, so this insight is likely fundamental.

Having said that, I also suspect that string theory has gone badly wrong and its present expression of this insight lacks essential features necessary to describe nature. First of all, it is lacking background independence. Other aspects, including supersymmetry, may be dead ends--fruitful for mathematics but irrelevant for physics.

Of course the biggest problem string theory faces is the lack of contact with experiment. There seems no path to do better so long as they stay within the current framework of ideas. The landscape issue seems real, which is why I identified it and invented cosmological natural selection to address it in the early 90’s.

There have been claims to test applications of an idea inspired by string theory to other domains of physics but these don’t test the hypothesis that string theory is a fundamental theory of nature.

Loop quantum gravity is the expression of the same idea in a background independent framework. It gets further, and, as an approach to quantum gravity, is now progressing at a faster rate than string theory, but new ideas are needed there too.

I am sure that to make real progress with the project of unification we need fresh ideas and novel principles. So I believe string theory and LQG could both develop in surprising directions which still capture their common essential insight.

Meanwhile, beyond the string versus loops conversation, there are new approaches springing up which many young people are developing. These include diverse ideas such as the study of scattering amplitudes, shape dynamics, group field theory, relative locality.

Horgan: But loop quantum gravity could still turn out to be the correct quantum-gravity theory?

Smolin: LQG as I said is based on the same basic idea as string theory, and it develops it in a background independent context, which I believe must be correct. So that is two essential insights it incorporates. It has several key results which support its plausibility including getting black hole entropy right for generic black holes and getting the emergence of general relativity in the classical limit right. And it is a very vital field, with a strong sense of progress and lots of amazing young people. Around 200 researchers attend our conferences.

So I think there is a good chance that LQG captures important aspects of the truth. It's certainly worth more exploration.

Having said that, I believe there are key ideas missing too. Some of these are technical, others concern the role of time, which is the subject of my last two books. And LQG lacks so far a clean way to connect its basic result-the discreteness of quantum geometry-with definite experimental predictions. There is a rough picture of how this could work which involves seeing the effects of the discreteness on the propagation of photons and neutrinos over cosmological distances, and observations of the required sensitivity are underway. But this needs to be finally tied down.

Horgan: Do you ever wonder whether physicists' psychological needs—and not some intrinsic property of nature—are driving the quest for a unified theory?

Smolin: I think there is a compelling argument for a unification of gravity with quantum physics. This is the existence of experimentally accessible phenomena that require such a unification to describe them. One can prove this just by dimensional analysis based on the constants of nature. So I am sure there are quantum gravitational phenomena, and they will have to be understood as part of completing the revolution Einstein started.

There are some lazy ideas about unification that reflect uncritical thinking, such as the idea that the more fundamental a phenomena is the more symmetry it must have. When you think seriously about the problem you realize it must be exactly the opposite. Roger Penrose use to say this, and indeed the insight that the most fundamental theory can have no symmetries goes back to Leibniz. I also think we need to move on from the tired idea that mathematical beauty is a key to unification. Mathematics is our most useful tool, but the idea that it should be prophetic has done a lot of harm.

Horgan: Do we have to accept the paradoxes of quantum mechanics, or could they be dispelled by a deeper theory?

Smolin: I am convinced that the 90-year failure to give a satisfactory “interpretation” of quantum mechanics, except in strictly operational terms, means that the measurement problem cannot be solved without a completion of quantum mechanics. That is, the problem is not with how we talk about quantum physics, but with an incomplete understanding of the dynamics of quantum systems. Quantum mechanics gives a very useful, approximate description of small subsystems of the universe but cannot as it stands be extended to a theory of the whole universe. The problem of finding a sensible extension of quantum physics to cosmology is the same as the problem of finding its completion.

Throughout my career I have proposed possible completions of quantum mechanics. Some of these were based on matrix models, more recently I proposed that the quantum state corresponds to an ensemble of similar systems in the universe; this idea is inspired by Leibniz’s principle of the identity of the indiscernible.

Horgan: Why hasn't the acceleration of universe—arguably the most important discovery in physics of the past 30 years--led to more theoretical advancement?

Smolin: At one level there is no problem, in that the acceleration of the universe’s expansion is easily described by adding a cosmological constant to Einstein’s equations, just as Einstein proposed in 1917. The problem is just with the value of that constant—it's ridiculously tiny. This is an extreme example of the basic problem that plagues the standard model of particle physics, which is that we don’t understand the reason for the value of any of the roughly 30 parameterize we need to write the laws of physics.

I am convinced that the answer to all these puzzles must be that these constants evolve, so the explanation for their values must be historical. Indeed cosmological natural selection gives a plausible explanation for the observed value of the cosmological constant.

Horgan: Some leading physicists, such as Tegmark, Susskind, and Greene, espouse multiverse theories plus the anthropic principle as a kind of final framework for cosmology. Comment?

Smolin: This is a sleigh of hand by which they hope to convert an explanatory failure into an explanatory success. If we don’t understand the values the fundamental constants take in our universe, just presume our universe is a member of an infinite and unobservable ensemble of universes each with randomly chosen parameters. Our universe has the values it does because those make it hospitable to life.

There is so much wrong with this as a scientific hypothesis. As I have explained in detail in three of my books and several papers, it is hard to see how it could make any falsifiable predictions for doable experiments. Claims to the contrary are fallacious, as I and others have explained in detail. I won’t impose the details on your readers but just mention that these criticisms have not been answered.

What we have to do is to propose mechanisms by which the laws and constants may have evolved which imply falsifiable predictions by which they can be checked. I have proposed two: cosmological natural selection and the principle of precedence.

Horgan: You suggest above, and in your new book, that the laws of nature "evolve." Won't that hypothesis make physics and cosmology even more flexible and hence less falsifiable?

Smolin: No, the key lesson of cosmological natural selection (CNS) is that it makes falsifiable predictions for real observations. In fact the predictions I published in 1992 have held up. To mention one: there can be no neutron stars heavier than twice the mass of the sun. Current limits come close; the heaviest well-measured neutron star is at 1.9 solar masses, but so far none go over.

At first I had the same intuition your question expresses. But it's wrong; making laws evolve increases the falsifiability of science because it increases the number of hypotheses that can be checked because they imply falsifiable predictions. The reason is that the additional hypotheses concern the processes by which evolution took place. Since these processes would have taken place in the past they imply predictions which are checkable by real observations. This point is discussed in detail in my books Life of the Cosmos and Time Reborn.

One way to reconcile evolving laws with falsifiability is by paying attention to large hierarchies of time scales. The evolution of laws can be slow in present conditions, or only occur during extreme conditions that are infrequent. On much shorter time scales and far from extreme conditions, the laws can be assumed to be unchanging.

As Roberto Mangabeira Unger and I argue in our new book The Singular Universe, the most important discovery cosmologists have made is that the universe has a history. We argue this has to be extended to the laws themselves. Biology became science when the question switched from listing the species to the dynamical question of how species evolve. Fundamental physics and cosmology have to transform themselves from a search for timeless laws and symmetries to the investigation of hypotheses about how laws evolve.

Horgan: You seem to hope that more imagination and creativity can knock physics out of its current impasse. But isn't physics suffering from a surfeit of imagination and a deficit of data?

Smolin: More data would always be nice. Imagination is great too, but imagination is not productive when you are asking the wrong question. There is no use wondering what symmetry unifies the elementary particles and forces if Penrose and Leibniz are right that the more fundamentally we understand nature the less symmetry the laws will have. Nor is it fruitful to look for principles to frame timeless laws when the real story is that the laws evolved, and so are to be explained historically.

Horgan: In 2002, I bet Michio Kaku $1,000 that by 2020, "no one will have won a Nobel Prize for work on superstring theory, membrane theory, or some other unified theory describing all the forces of nature.” Do you regret not having bet me that $1000?

Smolin: No, I would have been on your side. Here is another bet I offered: that quantum gravity effects on the propagation of light (such as a breaking or deformation of Lorentz invariance) will be tested at the Planck scale before classical gravitational waves are observed. Had I gotten a taker I would have won because some possible deviations from Lorentz invariance are already bounded above the Planck scale by observations of gamma ray bursts by the Fermi satellite, while LIGO has till now seen nothing.

Horgan: Roger Penrose has suggested that a unified theory might also help solve the problem of consciousness. Do you agree?

Smolin: Roger is a deep and original thinker, who stands head and shoulders above almost everyone now living in the lasting importance of his thought and insights. Nonetheless, in this one regard I suspect he is overstepping the limits of present science. He is absolutely right that consciousness is real and that its role in nature is a physics problem. But I suspect physics needs to progress a lot more before we will have the vocabulary to frame useful hypotheses about consciousness.

Horgan: Does neuroscience offer a better future than physics for bright would-be scientists?

Smolin: Neurosciences are a fabulous area to work in, ripe for great discoveries. I’ve always felt this and indeed the only alternative to a career in physics that ever attracted me was a brief flirtation in college with neuroscience. But that is a field which is as bedeviled by outdated metaphysical baggage as physics is. In particular, the antiquated idea that any physical system that responds to and processes information is isomorphic to a digital programmable computer is holding back progress.

Physics is also ripe for great discoveries. The best advice I would give to would be scientists is to do what you most love, make sure you master the tools and technicalities and then try to get to that hot zone where you are in equal parts a rebel and a conservative.

Further Reading:

“If You Want More Higgs Hype, Don’t Read This Column.” http://blogs.scientificamerican.com/cross-check/2012/07/04/if-you-want-more-higgs-hype-dont-read-this-column/

“Cosmic Clowning: Stephen Hawking’s “new” theory of everything is the same old CRAP.” http://blogs.scientificamerican.com/cross-check/2010/09/13/cosmic-clowning-stephen-hawkings-new-theory-of-everything-is-the-same-old-crap/

"Is speculation in multiverses as immoral as speculation in subprime mortgages?" http://blogs.scientificamerican.com/cross-check/2011/01/28/is-speculation-in-multiverses-as-immoral-as-speculation-in-subprime-mortgages/

Photo credit Edge.org: http://edge.org/conversation/think-about-nature.

*New, revised headline is much, much, much better than original: Troublemaker Lee Smolin Questions If Physics Laws Are "Timeless".