About the SA Blog Network



Opinion, arguments & analyses from the editors of Scientific American
Observations HomeAboutContact

Quantum Chaos: After a Failed Speed Test, the D-Wave Debate Continues

The views expressed are those of the author and are not necessarily those of Scientific American.

Email   PrintPrint

Photo courtesy of D-Wave Systems Inc.

How hard can it be to determine whether a computer works as promised? Step one: turn it on. Step two: Try to solve some problems. If it doesn’t work, it doesn’t work. Right?

Things are never so simple in the real world, of course. And on the highly contested frontiers of quantum computing, matters are more complex still.

Many of you are no doubt familiar with the saga of the Canadian company D-Wave, which claims to have created the first commercially available quantum computer. Short version: In 2007, at a press conference in Mountain View, California, the CEO Geordie Rose switched on what he said was a 16-bit adiabatic quantum computer and invited the audience to watch it solve a Sudoku puzzle. The backlash from the academic quantum-computing research community was ferocious and immediate; experts lined up to criticize D-Wave’s bold claims. But D-Wave stuck around, and in time, they began attracting window shoppers. In 2009, they partnered with a research team at Google. In 2011, they announced the launch of D-Wave One, a $10 million machine they billed as the world’s first commercially available quantum computer. Lockheed Martin bought the company’s next machine, the 512-qubit D-Wave Two, and made it available to researchers outside the company.

All the while, D-Wave, its allies, and its critics have been sparring, publishing study after study that attempt to figure out what’s going on inside that black box.

The latest entry is out today in the new issue of Science. (The paper has been out in preprint since January.) A group of researchers from EHT Zurich, Google, Microsoft, the University of Southern California and the University of California at Santa Barbara gained access to Lockheed’s D-Wave machine and subjected it to a series of tests. They wanted to see whether the machine exhibited quantum speedup—that is, whether it was any faster than a classical computer running the same sorts of problems.

The first thing to know about the D-Wave Two is that it is not a general quantum computer—it’s what is called a quantum annealer, which should, in theory, be capable of solving certain types of optimization and sorting problems exponentially faster than a classical computer. To test it, the authors of the Science paper tested 1,000 randomly chosen cost-function problems on both the D-Wave Two and a classical annealer.

They found that, overall, the D-Wave did not exhibit quantum speedup on the set of problems used. The group phrased its findings very diplomatically. “This does not mean the device cannot have quantum speedup,” lead author Matthias Troyer says. “It just means that in the tests we conducted, [quantum speedup] was not there.”

Naturally, the people at D-Wave are sensitive about how this paper is being received. Colin Williams, director of business development for D-Wave, repeatedly emphasized to me in a phone interview that Troyer’s results only dealt with a specific class of problems. “The paper that’s coming out in Science today doesn’t say anything fundamental about the scaling of quantum annealing,” he says.

Moreover, he says, Troyer’s group chose an inappropriate set of problems to perform this test. “The problem ensemble was too easy,” he says. Given harder problems, he says, the D-Wave machine would have had a chance to distinguish itself. Williams points to a recent paper by Helmut Katzgraber for support, along with a blog post on benchmarking the D-Wave Two by the Google Quantum A.I. Lab team. Williams also says that problems of the sort outlined in a recent talk by Itay Hen of USC could point to better benchmarking problems.

Any outsider who wades into the D-Wave controversy probably wonders: Why, again, is it so hard to settle this debate? Troyer explained it well in an email, so I’ll quote him at length:

It is particularly difficult here because they [D-Wave] use qubits with very short coherence times of the order of a few nanoseconds, while the total time to perform of one annealing run is 20 microseconds. The qubits are thus coherent for only a fraction of the total time, and this raises the question whether they are “coherent enough” or “quantum enough” to show a quantum speedup.

The situation is made even more complex in that we do not know theoretically if even perfectly coherent qubits would offer any advantage (quantum speedup) for the type of optimization problems that there are interested in. Thus, if one is unlucky, even perfect qubits might not offer any advantage, while if one is lucky one might see quantum speedup even with imperfect qubits with short coherence times.

Because of both these reasons—unknown potential for quantum speedup and unknown effect of short coherence times—one needs to experimentally investigate the potential for quantum speedup. That’s what we aimed for in our paper, where we developed a methodology to reliably detect any potential quantum speedup.

Colin Williams told me that “in the next six months or so” we would see new results showing that the D-Wave machine is indeed faster than a classical annealer, given the right problems. If D-Wave does show quantum speedup, it will be a big deal—a “huge advance,” Troyer says. Even so, quantum computing will still face a slog in the years ahead. “There are big challenges to making this a real device even if there is quantum speedup,” Troyer says. “To make it useful as a product, they have a long way to go after [quantum speedup] is shown.”

I mentioned Troyer’s paper the other evening to a physicist friend who happened to be in town. He’s finishing up a postdoc in Europe and thinking about what to do next. He just turned down a job doing quantum computing research for a big private player because the timing wasn’t right—he would have had to leave his postdoc early, and, he says, a job like that can wait a few months. There is plenty of time. It’s going to be a while before anyone has a practical quantum computer.

I suspect Troyer would agree.

About the Author: Seth is a senior editor. Follow on Twitter @seth_fletcher.

The views expressed are those of the author and are not necessarily those of Scientific American.

Rights & Permissions

Comments 9 Comments

Add Comment
  1. 1. jtdwyer 5:33 pm 06/19/2014

    Thanks for the update – I hadn’t bothered following this since I discovered that the vendor’s performance improvement claims had been based on a comparison of solving a similar problem on a network of 6 IBM desktop computers running a similar mathematical solution -implemented in different software. Some benchmark! This is hardly the technological alternative for a multi-million dollar ‘quantum-computer’ that executes a single (albeit complex) instruction – requiring a substantial (unspecified) supporting configuration of ‘ordinary’ computers and software. I think at least that an optimized hardware/software configuration of ‘ordinary’ supercomputers, given a comparable budget, would be a more valid comparison.
    “Why, again, is it so hard to settle this debate?”
    The most straightforward answer is simply that there is too much profit potential involved…

    Link to this
  2. 2. m123456 2:35 am 06/20/2014


    I recall reading the paper and apparently the very last test, which was hardest of all, the D-wave was faster.

    This is why they are saying, lets give the machine some harder problems.

    They also noted the coherent issue, for which the D-wave researchers are investigating and improving.

    The tests as they mentioned did not invalidate the concept that it was a quantum computer, what they did say in laymans terms is that it wasn’t very good yet.

    Now if you had invented the first quantum computer, would you expect it to work perfectly… This thing is so new its at the bleeding edge. Can improvements be made to it…nobody see’s why not.

    Which is why you are left with the situation with improvements this machine may offer the very first signs of significant quantum speed-up. Stay tuned.

    Link to this
  3. 3. jtdwyer 7:20 am 06/20/2014

    Thanks, but my point is, faster than what: a $10k workstation or a comparably priced supercomputer using algorithm optimized for parallel array computations?
    Meanwhile, the company has been pumping outrageous performance claims and hyperbole to the press – and potential investors and customers. In the meantime, I do not feel that my tax dollars should be used to purchase production machines – ostensibly to solve real problems – as they are merely funding the private development of speculative, over-hyped ‘technology of the future’.

    Link to this
  4. 4. jtdwyer 7:54 am 06/20/2014

    OK, I scanned the paper – – to find buried in the text that “Our classical reference CPU is an 8-core Intel Xeon E5-2670 CPU…”
    One to eight of the 8 processor ‘cores’ were varyingly used in comparison tests.

    For comparison shoppers with $10M to apply to their random spin glass computational problems, I find that an HP DL360p Gen8 Intel Xeon E5-2670 Sandy Bridge-EP 2.6GHz (Turbo Boost up to 3.3GHz) 2MB L2 Cache 20MB L3 Cache LGA 2011 115W 8-Core Server Processor Kit 654786-B21 can be purchased for $2,139.99 [!]

    These comparisons are utterly meaningless, as a much more powerful ‘classical’ supercomputer configuration could undoubtedly be acquired for substantially less than $10M – one that (using properly optimized parallel software) could _far_ outperform the baseline Intel workstation processor cores.
    What point is there in comparing the performance of a full-race Ferrari to a motor scooter? They are not realistic alternatives for solving the same problem (competing in Grand Prix races)!

    Link to this
  5. 5. m123456 8:57 am 06/20/2014


    I doubt it can out-perform a ZX81. But truly I was not expecting the very first attempt at a quantum computer to come out and suddenly render every problem solved… Did you?

    I have read a few excellent articles about the field and most of them believe too many issues still need to be resolved with other architectures. This architecture may or may not work, but its a build, test and see scenario.

    I am happy they built it, i’m happy it has quantum properties..which are not fully understood or optimised and that improvements in both cost and capabilities are down the road.

    I do not in total agreement with you see this as anything more than a research endeavour at this stage.

    Link to this
  6. 6. rufusgwarren 4:33 pm 06/20/2014

    Sounds like it works, but only if you want the answer to the universe and everything; wait to easy, find the question.

    Link to this
  7. 7. VanScar 4:14 pm 06/24/2014

    I wonder if they find the cat dead or alive, when they will open the machine.

    Link to this
  8. 8. hkraznodar 5:58 pm 06/27/2014

    For a company that is essentially a research endeavor to market an experimental model at 500 times the fair market price for a production model is essentially criminal. Who cares if it is quantum or not? Does it provide the service it is marketed as providing? The obvious answer is no. The cat is dead and we can smell it rotting.

    Quantum computing is supposed to use subatomic particles and thus require very little power per flop. So how does this machine compare to current processors power use? What about the much smaller material amounts leading to microscopic super computers? If it were actually quantum they would be howling it from the roof tops. This sounds to me like it is merely a better organized conventional computer.

    Link to this
  9. 9. TheeDefiler 12:02 am 07/9/2014


    Just want to ask you how far have we come from the beginning of computers. they started out as a whole freakin room with one transistor. Technology isnt instantaneous its gradual.

    Quantum computers are way way low on their development process we dont know that much about them.

    Plus that 10 mill is for a QUANTUM COMPUTER not a classic computer. Just FYI

    Link to this

Add a Comment
You must sign in or register as a member to submit a comment.

More from Scientific American

Scientific American Dinosaurs

Get Total Access to our Digital Anthology

1,200 Articles

Order Now - Just $39! >


Email this Article