IBM demonstrates Watson supercomputer in Jeopardy practice match

We're at IBM's HQ in upstate NY, where IBM will pit its monstrous Watson project (in the middle buzzer spot) against two Jeopardy greats, Ken Jennings and Brad Rutter. Watson has been in development for four years, and this is its first big public practice match before it goes on national TV in February for three matches against these giants of trivia. Unlike IBM's Deep Blue chess project in the 90s, which was pretty much pure math, Watson has to deal with the natural language and punny nature of real Jeopardy questions. IBM, ever the salesman, has thrown gobs of its fancy server hardware at the project, with 10 racks full of IBM Power 750 servers, stuffed with 15 terabytes of RAM and 2,880 processor operating at a collective 80 teraflops. IBM says it would take one CPU over two hours to answer a typical question, so this massive parallel processing is naturally key -- hopefully fast enough to buzz in before Ken and Brad catch on to the human-oriented questioning. We'll update this post as the match begins, and we'll have some video for you later in the day.

12:04PM Now we're going to get to meet "Watson." That's it for the liveblog, we'll have video and more information later on!

12:04PM Alex asks his own question: Why Jeopardy? And answers (how fitting): Jeopardy has always been a technology-forward quiz show. Computer displays, first quiz show in HD. "And if you need another example of how forward-looking we are, I've been hosting the program for the past 9 years without a mustache."

12:02PM Ken: So Watson isn't going to push us out an airlock. Alex: Can Watson sing Daisy?

12:01PM Q: Are we close to a HAL 900 scenario. A: That's science fiction. IBM sees Watson more like the computer on Star Trek.

12:01PM Ken and Brad haven't played in five years, they're a little rusty. Watson apparently doesn't feel any pity.

11:58AM IBM is talking about health care again. "I don't want to be overdramatic, but we can really save lives."

11:57AM Q: Brad and Ken, is there any part of you that wants Watson to win, for the sake of human progress? A: Ken: Human Progress? Which side are you on?! Brad: This is an aspect of human development that I would live for us not to reach just yet. Ken: Playing Jeopardy is something that I feel special about, personally. IBM: A word of warning, the technology doubles in speed every 18-24 months.

11:55AM Also has an ability to look at answers that it missed and can look at the right answer and learn what it was missing.

11:51AM Q: Does Watson have a recursion process for knowing "I need to learn more about this category." A: Watson does have self-assessment. None of the algorithms around natural language are perfect, so there's always error and uncertainty. It knows that it's not perfect at this, so it adjusts its algorithms accordingly.

11:50AM Context and visual clues are also very important for precision.

11:50AM "Watson can literally read all of the health care text in seconds, but we also want it to be able to read X-Rays and test reports, so a lot of work still to be done."

11:49AM Q: Is there any chance of putting a Watson-like interface on the web. A: We'll see about that. Right now more into "vertical" applications like health care, "empowering decision makers."

11:47AM Beyond question answering, there's obviously a lot of strategy involved in a game of Jeopardy and Watson seems well acquainted with it.

11:46AM We're putty in Watson's hands.

11:46AM IBM: Watson does not have emotions, but Watson knows that humans have emotions.

11:46AM Alex brings up the question of psychological games. Ken: Watson can't be psyched out. IBM: That's true, Watson does not have emotions. Ken: He can't love.

11:44AM Brad answers a question about how much they need to know about a question before the buzz in. Brad notes that Watson just did really good at a children's books category, but neither Brad or Watson has children. Ken notes that the retrieval process is trivial for a computer and difficult for a human, while the comprehension process is much easier for Ken than Watson.

11:43AM Alex explains that the order of the clues helps the answerer "learn" as they go down through a category. Figuring out the style of the category or something like that.

11:41AM The category is a "context" and Watson tries to learn in that context, but the specific category might not be a very good predictor of how well Watson will do.

11:40AM Q: Are there any categories that Watson is better or worse at? (Ken: I'd like to know the answer to that as well!) A: Yes.

11:40AM Alex: "but after Jeopardy, Watson will be available for sale on eBay."

11:39AM IBM doesn't have any estimate of how much it's spent on this.

11:39AM On Jeopardy's part, the questions were written just like usual, with the writers having no knowledge of Watson. They then pulled a pool of 30 games randomly to get a set for the practice matches, and merely pulled out the audio and video and picture clues.

11:38AM A lot of Watson's AI is oriented around regular natural language and reasoning stuff, but there's also some custom Jeopardy-specific stuff.

11:37AM Yeah, we're talking Terminator.

11:37AM Q: Humans, do you feel any pressure competing with Watson? A: Brad isn't worried about Watson, but afraid of "Watson's progeny when they come back from the future to kill me." Ken says he was warned by a friend, "Remember John Henry.' Ken's response: "Screw that, remember John Connor!"

11:36AM We're still flushed from that match. Wow.

11:35AM This will be part of the IBM portfolio in analytics in the future, but no exact time of when we'll see any of it on the market. Obviously, the hardware is standard stuff that's available in the world from your local IBM rep. IBM "can't imagine a single industry where there isn't a potential to transform it."

11:34AM Q: How much of this is based on actual commercial software IBM already has out there? A: "A lot of new code."

11:34AM Alex Trebek seems really into this project. He's running the QA and fielding half of them.

11:33AM We're getting some questions answered now in a panel form. Watson is fed the question instantly via plain text, but he has to mechanically press a buzzer, just like those pesky meat bags.

11:30AM Oooh, Alex Trebek is out!

11:30AM There was barely a moment to breath. Not that Watson needs to breath.

11:29AM OK, our first break. Watson is in the lead with $4,400, Ken has $3,400, and Brad has $1,200.

11:28AM Watson is back, up to $3,400 now. We don't even understand these questions, much less know the answers. Er, strike that, reverse it.

11:28AM A third for Ben, but Watson is still in the lead.

11:28AM Ooh, Ken just got the first pre-Watson buzz in on the $2k question on the "Chicks Dig Me" topic. Now Ben gets two. Where you at, Watson?

11:27AM Ken kicks it off... and Watson just got the first two questions right. It's all a blur. Third question right. HE'S DESTROYING. Four now.

11:26AM And here we go!

11:26AM Brad is out, a destroyer of worlds at Jeopardy. And now Ken, a destroyer of universes. They've already played a bit of practice with Watson this morning, and now they're here for a practice round.

11:24AM Harry Friedman, executive producer of Jeopardy, is up to thank everybody. Proud to be a part of this "adventure." Ken and Brad are giving 50% of their winnings to charity, while Watson will be giving away all of its winnings. Who's the moral actor now, huh?

11:21AM Get ready for Watson to start answering the tech support helpline.

11:20AM Watson now averages 3 seconds per answer.

11:19AM IBM calls its tech "DeepQA," a "massively parallel probabilistic evidence-based architecture."

11:16AM Challenges: broad domain, complex language, high precision, accurate confidence, high speed.

11:15AM Dr. David Ferrucci, lead researcher on the project, is up talking a bit more about how Watson actually works.

11:14AM IBM would like to encourage us to think about the applications beyond Jeopardy, like health care. Really doesn't take much imagination to see that this is a big deal, to be honest.

11:10AM There's clearly a quasi-religious atmosphere here, that the Watson project was only something you could succeed at if you really believed it was possible -- and many didn't.

11:09AM The history of the project includes some pretty hilariously bad answers. In 2007 an "I Love Lucy" question resulted in the answer "Song."

11:08AM IBM has managed to beat Jeopardy winners before, but beating world champions is a huge undertaking, and IBM itself sounds pretty up in the air as to how well Watson will do.

11:07AM IBM calls Watson one of its "grand challenges." An advance of "miles" in an artificial intelligence industry that typically advances by "inches."