The DeepQA Research Team - Watson's Sparring Record
To evaluate the performance of Watson's live game play and strategy, the DeepQA team ran two rounds of "sparring" matches against players who had appeared on the Jeopardy! show. The games were held at the IBM T.J. Watson Research Center in New York, where IBM built a mock Jeopardy! studio to conduct the games and hired professional actor Todd Crain to play the host.
The sparring games were actual episodes from the Jeopardy! season, provided by Jeopardy!. To ensure that the games were previously unseen by all players, Jeopardy! provided the games after they had been taped but before they had aired on the syndicated show.
On each sparring day, four to five Jeopardy! players visited IBM for the day and played against Watson in randomly selected pairs. From November 2009 through April 2010 Watson played 73 games against players that had won at most two games of Jeopardy!.
From September 2010 through December 2010 Watson played 55 games against players that had appeared in a Jeopardy! Tournament of Champions match. These players represent some of the best Jeopardy! players in the world. In this round of sparring, Watson proved that it was competitive against champion players.
You can watch highlights from the sparring games in this video: