I'm a believer in wishful thinking, in the power of our hopes to become self-fulfilling. I even believe that war is going to end! But at some point, if wishful thinking diverges too sharply from what we can reasonably expect from reality, it morphs into denial or delusion. David Deutsch's hope that science will keep making the world more and more comprehensible is an example of delusional thinking.

Deutsch expresses this hope in his remarkable new book The Beginning of Infinity: Explanations That Transform the World (Viking, 2011). A physicist and pioneer of quantum computation, Deutsch is a radically original and intrepid thinker. He reminds me of such quirky iconoclasts as Freeman Dyson and Roger Penrose, who keep veering away from the mainstream of physics into strange, uncharted territory. Deutsch's book is worth reading simply for his forceful rejection of traditional, materialistic reductionism and his defense of the power of ideas, which transcend any particular physical instantiation and yet exert a powerful influence over the world.

But Deutsch's primary goal is to lay out a vision of science as infinite, open-ended, in its infancy. Deutsch at heart is really a philosopher, or what used to be called a "pure" scientist. (Does anyone use that quaint term anymore?) The primary purpose of science, he contends, is not power or prediction; it is understanding. I use the term "understanding" rather than "truth" advisedly: Deutsch is a hard-core Popperian, who believes that absolute, final truth is unattainable; even our most robust current explanations can in principle always be falsified, as new data emerge. (I disagree with this quasi-postmodernism, but that's a topic for another column.)

Deutsch rejects the view that our capacity for understanding might be bounded. He quotes the astrophysicist Martin Rees, who once wrote: "Just as a chimpanzee can't understand quantum theory, it could be that there are aspects of reality that are beyond the capacity of our brains." Accepting this view, Deutsch argues, is akin to regressing to belief in magical, superstitious, pseudo-explanations of the world, such as those that attribute natural phenomena to the whims of gods. Actually, Rees's view is more "scientific" than Deutsch's. Science itself tells us that we are animals, designed by natural selection for reproducing, not for understanding black holes or protein-folding. Denying this admittedly disturbing insight about human evolution seems far more irrational and regressive than embracing it.

Science, and especially physics, is clearly bumping up against cognitive as well as physical and economic limits. Quantum mechanics, Deutsch's own specialty, is a case in point. As Martin Rees notes, chimps can't grok quantum mechanics. Most humans, and even most physicists, can't either. Physicists have embraced quantum mechanics not because it makes sense but because it works: Quantum theories predict the outcomes of experiments with extraordinary accuracy, and as a result the theories have yielded a host of powerful technologies, from lasers to semiconductors.

Quantum mechanics makes reality less rather than more comprehensible, because it subverts conventional notions about causes and effects, time and space, objective versus subjective reality. That was why Einstein, one of its founders, never accepted this radical new physics. Deutsch nonetheless dismisses the old quip, sometimes attributed to Bohr: "If you think you've understood quantum mechanics then you don't." (This paradox reminds me of what Lao Tzu said of mystical truth: Those who know do not speak. Those who speak do not know.)

To Deutsch, quantum mechanics makes perfect sense. He is an adamant advocate of the many worlds interpretation, which addresses the puzzling fact that quantum mechanics offers only possibilities and probabilities, not certainties. The many worlds interpretation holds that although we only see a particle such as an electron follow one particular path, it actually follows every path allowed it by quantum mechanics—in other universes. Although Deutsch finds many worlds to be a compelling explanation, vanishingly few other physicists do. If you are virtually alone in finding a given explanation compelling, is it really an explanation?

Some modern physicists share the hope of Einstein that the weirdness of quantum mechanics will vanish when it yields to a more fundamental theory, ideally one that unifies quantum mechanics and general relativity, our theory of gravity. But the leading contender for such a unified theory, string theory, simply compounds our incomprehension. For decades I've been asking physicists what a string is. Is it an object? A form of energy? A process? Or what? No one has given me a satisfactory answer (although Scientific American's own George Musser puts in a valiant effort in my conversation with him on Bloggingheads.tv). Strings seem to be a set of purely mathematical formalisms with no real-world counterpart.

I was once sympathetic toward the romantic notion that science is infinite. I became a science journalist because I saw science as our most potent source of revelation about reality. Science, ideally, transforms our "Whaa…?" of puzzlement into an "Aha!" of illumination. Early in my career, physics in particular seemed poised to answer the deepest mysteries of existence. What is reality made of? Where did it come from? Why do we find ourselves in this reality rather than some other one? Why does anything exist at all?

I shared the hope of the physicist John Wheeler, one of my all-time favorite scientists, that one day scientists will solve—really solve—the riddle of reality. Deutsch is apparently a Wheeler fan, too. Deutsch quotes Wheeler saying, "Behind it all is surely an idea so simple, so beautiful, that when we grasp it—in a decade, a century or a millennium—we will all say to each other, 'How could it have been otherwise?'" I predict that the opposite will happen. The farther science progress, the more we will descend toward utter befuddlement.

Photo courtesy of Viking