This article was published in Scientific American’s former blog network and reflects the views of the author, not necessarily those of Scientific American
As IBM preps for its next big man–machine showdown (and latest high-tech publicity stunt), Scientific American took a brief, informal, unscientific poll on Monday of 26 print and online staffers to determine whether there was a consensus on who would win this week's Jeopardy! tournament, by how much and why. We'll look at the poll's results shortly, but first a look at the contestants who will match wits and watts over the next three evenings.
The men:
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Ken Jennings broke the Jeopardy! record for the most consecutive games played, winning 74 games in a row during the 2004-2005 season and earning more than $2.5 million in prize money.
Brad Rutter has won more money on Jeopardy! than any other player in the game's history, having accumulated more than $3.2 million as a result of his original appearance in 2000 as well as three subsequent tournament wins in 2001, 2002 and 2005.
The machine:
Watson, named for IBM founder Thomas J. Watson, is a set of advanced natural language processing, information retrieval, knowledge representation and reasoning, and machine learning technologies running on a massively parallel high-performance computer. IBM scientists built Watson in the past four years with a specific purpose: To rival a human's ability to answer questions posed in natural language with speed, accuracy and confidence. Watson also makes use of IBM's DeepQA technology for hypothesis generation, massive evidence gathering, analysis and scoring.
The ground rules:
Watson will answer Jeopardy! clues by analyzing the clue to figure out what the clue is asking for, searching its millions of pages of information (documents, books and encyclopedias) scanned into its memory for potential answers, then applying various analytics to determine a level of confidence in a given answer. If Watson's confidence in its top answer is high enough, it will attempt to ring in for the clue, according to IBM. Watson will play using this knowledge base and will not be connected to the Internet. If the computer malfunctions, IBM is permitted to get it back to "normal operating standards," according to the company, whose technicians are not permitted to make any other changes to Watson's system between games or within a particular game. Watson is programmed to have a dynamic game strategy, which means it can base its betting strategy on game conditions as they unfold.
Watson's challenge:
Watson faces a very different test than the one that Deep Blue faced when it first went head to head with chess champion Gary Kasparov 15 years ago. Deep Blue lost their first showdown in 1996 but returned the following year to win a six-game match by two wins to one (with three draws). IBM acknowledges that chess is a game with a limited space of possibilities and options. One of the primary challenges Watson faces with Jeopardy! is simply understanding the questions, which often include puns and other word play that provide contestants with context that may be lost on Watson. During games, Watson must also use a mechanical button clicker in order to buzz in with the correct answer.
The results of Scientific American's poll:
Scientific American staffers were divided in terms of who would win. Half chose one of the human contestants, with more than half of those staffers choosing Jennings by name and the rest nonspecifically saying "a human" would win. (Sadly, no one singled out Rutter as the inevitable winner.) One of the staffers cited "the associative accuracy of the human players" trumping "raw processing power paired with guesswork of Watson." However, that same staffer then conveniently hedged his bet by proclaiming, "Even if Watson wins, I predict it will be because of human effort."
Staffers in Watson's corner were swayed largely by the supercomputer's victory in an abbreviated demo match against Jennings and Rutter a month ago at the IBM Thomas J. Watson Research Center in Yorktown Heights, N.Y. The computer ended the demo with $4,400, as Jennings took second place with $3,400. During the demo, every question was answered correctly—the decisive factor came down to timing.
One of Watson's fans touted its "nerves of steel! (Or at least of silicon.)" Others doubted that its human opponents had the consistent brain-to-eye-to-hand coordination required to consistently buzz in first over a three-day game. Another staffer noted that, even if Watson isn't quicker on the draw, the machine would "take advantage of the final Jeopardy! question, which gives a lot of time (for a computer) and is not a button race." Another commented, "I give the edge to brute force computing over the pattern-recognition finesse of the human mind."
An air of cynicism pervaded many comments. "I think that this shows that AI has made an incremental advance, though not a monumental one," one staffer responded. "This is still not as if a machine is passing a true Turing test. This is brute-force computing at its best. The type of reasoning involved has nothing to do with the way humans reason. Jeopardy! is fairly formulaic. If it weren't, the computer would be sunk. This whole thing is essentially a stunt to advertise the company's data-mining technology." Another staffer added, "I'm not entirely convinced Ken Jennings isn't a robot," insinuating that Watson wasn't the only artificially intelligent competitor.
The predicted margin of victory on either side varied widely from several hundred dollars to a $20,000 smack down of the competition (no real consensus there either).
The grand prize for the Watson competition will be $1 million, with second place earning $300,000, and third place earning $200,000. Rutter and Jennings say they will donate half of their winnings to charity—Rutter to Lancaster County Community Foundation and Jennings to VillageReach. IBM has pledged to donate all of its winnings to World Vision and World Community Grid.
Image of Watson courtesy of IBM