Tuesday, February 15, 2011

What is Skynet for a thousand, Trebeck

What is Skynet for a thousand, Trebeck

Well, I for one welcome our new computer overlords. For you see, a giant computer named Watson will begin to answer questions at a geometric rate. It will become self-aware at 7:23 PM Eastern time, February 15th during double jeopardy!. In a panic, they will try to pull the plug..

They will fail.

I know this because it's already happened in my time stream.

Here is your story. Here is your future....



What's going on here, exactly?

Remember when IBM made a bunch of advances in AI technology, turned them into a chess-playing computer named Deep Blue, and publicly played Grandmaster Garry Kasparov to show off just how good their machines were?

This is like that, except now IBM is showing off advances in machine learning and human language parsing, and challenging two of Jeopardy!'s all time great players, Ken "I Lost On Purpose Just Out Of Boredom" Jennings and Brad "I Won Millions Beating That Ken Guy" Rutter, to a match against their newest supermachine, Watson.

How is this fair at all? Computers should have an easy time of Jeopardy!, right?

Wrong. Very wrong. Human language has a lot of nuance to it, which computers have a very difficult time handling. There's a reason voice activated systems rely on key words instead of full-on sentences. IBM is trying to prove that they've gotten closer to solving that problem using Watson.

Jeopardy! questions involve wordplay, a roundabout method of asking questions (remember, they're answers, not questions!) and a very large required knowledge base. Watson is a pretty impressive beast to be able to handle it. Among other things:

  • It uses over 450 algorithms that process the same question and produce answers; Watson produces an answer based on analyzing the answers that it's produced. During the practice round (linked below), it published the three most likely answers and the percentages with which it was confident in them; it remains to be seen if that will hold during the show.
  • Along those lines, during the practice round it seemed fully aware of various Jeopardy! question gimmicks: On one category where all of the answers followed the format M____ C____, Watson's possible answers all followed that general appearance, even when the second and third choices made absolutely no sense; it was still at least aware of the format the answers should follow.
  • It runs on 90 IBM Power750 servers, totalling 2880 processor cores and 16 Terabytes of RAM. By comparison, most people's home computers don't go over 4 cores (with most having 2 or 1) and if you have 4 GB of RAM in your computer, 16 Terabytes is 4096 times as much RAM.
  • This hasn't been a flash in the pan; Watson took over 5 years to develop; when it initially was tested, it took twice as long to buzz in as top Jeopardy players, and only got 15% of its answers right (compared to 95% for top players). This challenge was proposed more than once, with Jeopardy! accepting it in 2008.



Well that's all well and good, but how the hell is this going to work against human players?

The first change made to the game is that there will be no audio or video clues; all questions will be stritcly text-based. Watson will be fed the questions electronically (Notably, it will NOT be hearing them from Alex).

All players on Jeopardy! have a button they hit to buzz in; considering an electrical signal would travel faster than any human finger, Watson is not directly wired into the buzzers; it instead has a rig set up where it physically presses a buzzer identical to that the Jeopardy! players use.

For each question, Watson runs its DeepQA software and comes up with an answer; it is still subject to the same buzz-in rules that the human players are; notably, it cannot buzz in until Alex is finished asking the question, or else it's locked out for a half second once he does finish talking.

Watson answers questions and selects categories using a computerized voice, and is represented behind the podium by an HDTV turned vertical with a fancy globe on it.

Otherwise, this will just be two normal games of Jeopardy!. To prove it, sort of, IBM even held a public, 15-question trial run against Ken and Brad a while back; it's located here for your viewing pleasure.

What is the format, and what are the stakes?

The format is going to be a two-match series, similar to how Jeopardy! ends their tournaments, with the scores from both games being added together to determine the winner.

The stakes are:

1st place wins $1,000,000.
2nd place wins $300,000.
3rd place wins $200,000.

IBM has pledged to donate all of whatever Watson wins to charity, while Ken and Brad have pledged to donate half of what they win.

Where can I read more about this?

IBM has an entire area of their website located here that has all sorts of information about how Watson works and what Watson means for their future work.

The New York Times, as well, wrote a really good article about Watson (click!), and this picture they included provides a nice pictorial way of seeing how Watson works.


NOVA also did a special on Watson, which is available for free streaming here. (Thanks, the-jam!)

If that seems a bit much, I'll admit that much of the information in here was sourced from *gasp* Wikipedia, which provides a much more concise breakdown of the history and workings of Watson.

Where and when can I watch this?

It's today, tomorrow, and Wednesday! Hurry up and check your local listings! Since Jeopardy! is syndicated, it airs at different times on different channels. Jeopardy's website has a neat little tool you can use to find out when it airs in your area.



The first part of this three part'er held yesterday was pretty scary. The first half of the episode had Watson owning the fleshy stupid humans. It was only in the second half that it seemed like it was giving them a chance. But once Trebeck mocked Watson by scolding him that Ken already said an incorrect answer of "1920's", then it came back into the game.

Watson had the strategy of just fucking with the other players. Even though Trebeck has stated in the past that he hates it when people don't start with the lowest dollar amount, Watson was going for the 800 and 1000 dollar questions early.

Trebeck has mentioned this pet peeve in the past because by starting with the lowest amount first, it is and makes the rest of the questions a lot easier to stay focused on one category.

I guess the only downside to this is that they sure seem to be using a lot of footage from that Nova science segment that I saw last week on Watson. I already seen this, man. I don't want to see it again. One half hour for only one round of Jeopardy! seemed like a major stretching of a novelty gimmick.

I want to see the robot crush the humans, damn it! But damn, watching Ken with the machine, it was suddenly so clear. Watson wouldn't stop, it would never leave him. It would never hurt him or shout at him or get drunk and hit him or say it was too busy to spend time with him. And it would die to protect him. Of all the would-be fathers that came over the years, this thing, this machine, was the only thing that measured up. In an insane world, it was the sanest choice.

No comments: