About the SA Blog Network



Opinion, arguments & analyses from the editors of Scientific American
Observations HomeAboutContact

Oh, the humanity: Jeopardy! champs aim to take down IBM’s Watson computer

The views expressed are those of the author and are not necessarily those of Scientific American.

Email   PrintPrint

IBM, gameAs IBM preps for its next big man–machine showdown (and latest high-tech publicity stunt), Scientific American took a brief, informal, unscientific poll on Monday of 26 print and online staffers to determine whether there was a consensus on who would win this week’s Jeopardy! tournament, by how much and why. We’ll look at the poll’s results shortly, but first a look at the contestants who will match wits and watts over the next three evenings.

The men:
Ken Jennings broke the Jeopardy! record for the most consecutive games played, winning 74 games in a row during the 2004-2005 season and earning more than $2.5 million in prize money.

Brad Rutter has won more money on Jeopardy! than any other player in the game’s history, having accumulated more than $3.2 million as a result of his original appearance in 2000 as well as three subsequent tournament wins in 2001, 2002 and 2005.

The machine:
Watson, named for IBM founder Thomas J. Watson, is a set of advanced natural language processing, information retrieval, knowledge representation and reasoning, and machine learning technologies running on a massively parallel high-performance computer. IBM scientists built Watson in the past four years with a specific purpose: To rival a human’s ability to answer questions posed in natural language with speed, accuracy and confidence. Watson also makes use of IBM’s DeepQA technology for hypothesis generation, massive evidence gathering, analysis and scoring.

The ground rules:
Watson will answer Jeopardy! clues by analyzing the clue to figure out what the clue is asking for, searching its millions of pages of information (documents, books and encyclopedias) scanned into its memory for potential answers, then applying various analytics to determine a level of confidence in a given answer. If Watson’s confidence in its top answer is high enough, it will attempt to ring in for the clue, according to IBM. Watson will play using this knowledge base and will not be connected to the Internet. If the computer malfunctions, IBM is permitted to get it back to "normal operating standards," according to the company, whose technicians are not permitted to make any other changes to Watson’s system between games or within a particular game. Watson is programmed to have a dynamic game strategy, which means it can base its betting strategy on game conditions as they unfold.

Watson’s challenge:
Watson faces a very different test than the one that Deep Blue faced when it first went head to head with chess champion Gary Kasparov 15 years ago. Deep Blue lost their first showdown in 1996 but returned the following year to win a six-game match by two wins to one (with three draws). IBM acknowledges that chess is a game with a limited space of possibilities and options. One of the primary challenges Watson faces with Jeopardy! is simply understanding the questions, which often include puns and other word play that provide contestants with context that may be lost on Watson. During games, Watson must also use a mechanical button clicker in order to buzz in with the correct answer.

The results of Scientific American‘s poll:
Scientific American staffers were divided in terms of who would win. Half chose one of the human contestants, with more than half of those staffers choosing Jennings by name and the rest nonspecifically saying "a human" would win. (Sadly, no one singled out Rutter as the inevitable winner.) One of the staffers cited "the associative accuracy of the human players" trumping "raw processing power paired with guesswork of Watson." However, that same staffer then conveniently hedged his bet by proclaiming, "Even if Watson wins, I predict it will be because of human effort."

Staffers in Watson’s corner were swayed largely by the supercomputer’s victory in an abbreviated demo match against Jennings and Rutter a month ago at the IBM Thomas J. Watson Research Center in Yorktown Heights, N.Y. The computer ended the demo with $4,400, as Jennings took second place with $3,400. During the demo, every question was answered correctly—the decisive factor came down to timing.

One of Watson’s fans touted its "nerves of steel! (Or at least of silicon.)" Others doubted that its human opponents had the consistent brain-to-eye-to-hand coordination required to consistently buzz in first over a three-day game. Another staffer noted that, even if Watson isn’t quicker on the draw, the machine would "take advantage of the final Jeopardy! question, which gives a lot of time (for a computer) and is not a button race." Another commented, "I give the edge to brute force computing over the pattern-recognition finesse of the human mind."

An air of cynicism pervaded many comments. "I think that this shows that AI has made an incremental advance, though not a monumental one," one staffer responded. "This is still not as if a machine is passing a true Turing test. This is brute-force computing at its best. The type of reasoning involved has nothing to do with the way humans reason. Jeopardy! is fairly formulaic. If it weren’t, the computer would be sunk. This whole thing is essentially a stunt to advertise the company’s data-mining technology." Another staffer added, "I’m not entirely convinced Ken Jennings isn’t a robot," insinuating that Watson wasn’t the only artificially intelligent competitor.

The predicted margin of victory on either side varied widely from several hundred dollars to a $20,000 smack down of the competition (no real consensus there either).

The grand prize for the Watson competition will be $1 million, with second place earning $300,000, and third place earning $200,000. Rutter and Jennings say they will donate half of their winnings to charity—Rutter to Lancaster County Community Foundation and Jennings to VillageReach. IBM has pledged to donate all of its winnings to World Vision and World Community Grid.

Image of Watson courtesy of IBM


Rights & Permissions

Comments 1 Comment

Add Comment
  1. 1. S. Pelech - Kinexus 3:21 pm 02/17/2011

    IBM’s Watson can answer Trivial Pursuit-type questions put to it very effectively, but it cannot yet ask questions in a thoughtful way that permits Watson to create new knowledge. However, this could be achieved in the foreseeable future and will probably require the additional sensory and effector abilities so that it can actively acquire new data. The HAL 9000 computer in the film "2001: A Space Odyssey" probably still serves as the most recognizable paradigm for an intelligent machine. Incidentally, HAL 9000 got its name from taking the letters in IBM and changing then backward by a single letter in the English alphabet.

    Watson is rightly viewed as a major advancement in "artificial intelligence" (AI for short). Watson will certainly spur better interaction and retrieval of information from future computers by humans.

    Interestingly, the Watson affair forces us to review the definition of "intelligence" itself. This remains a somewhat ambiguous term. I am inclined to accept the more broad definition: "The ability of a system to adapt to a changing environment in a way that ensures its survival and propagation." This might first appear to be restricted to biological systems, but I think otherwise. As computers advance in their cognitive abilities, it is going to become exceeding difficult to view them as possessing "artificial intelligence." Perhaps "synthetic intelligence" may be more apt phraseology as this type of intelligence is an off-spring of the human imagination that will ultimately exceed its creator.

    Link to this

Add a Comment
You must sign in or register as a member to submit a comment.

More from Scientific American


Get All-Access Digital + Print >


Email this Article