Recommend
4 
 Thumb up
 Hide
12 Posts

BoardGameGeek» Forums » Everything Else » Chit Chat

Subject: I'll take Things That Are Awesome for $1000, Alex. rss

Your Tags: Add tags
Popular Tags: [View All]
Ken H.
United States
Amherst
Ohio
flag msg tools
badge
Avatar
mbmbmbmbmb

Computer Program to Take On ‘Jeopardy!’

By JOHN MARKOFF
Published: April 26, 2009


YORKTOWN HEIGHTS, N.Y. — This highly successful television quiz show is the latest challenge for artificial intelligence.

What is “Jeopardy”?

That is correct.

I.B.M. plans to announce Monday that it is in the final stages of completing a computer program to compete against human “Jeopardy!” contestants. If the program beats the humans, the field of artificial intelligence will have made a leap forward.

I.B.M. scientists previously devised a chess-playing program to run on a supercomputer called Deep Blue. That program beat the world champion Garry Kasparov in a controversial 1997 match (Mr. Kasparov called the match unfair and secured a draw in a later one against another version of the program).

But chess is a game of limits, with pieces that have clearly defined powers. “Jeopardy!” requires a program with the suppleness to weigh an almost infinite range of relationships and to make subtle comparisons and interpretations. The software must interact with humans on their own terms, and fast.

Indeed, the creators of the system — which the company refers to as Watson, after the I.B.M. founder, Thomas J. Watson Sr. — said they were not yet confident their system would be able to compete successfully on the show, on which human champions typically provide correct responses 85 percent of the time.

“The big goal is to get computers to be able to converse in human terms,” said the team leader, David A. Ferrucci, an I.B.M. artificial intelligence researcher. “And we’re not there yet.”

The team is aiming not at a true thinking machine but at a new class of software that can “understand” human questions and respond to them correctly. Such a program would have enormous economic implications.

Despite more than four decades of experimentation in artificial intelligence, scientists have made only modest progress until now toward building machines that can understand language and interact with humans.

The proposed contest is an effort by I.B.M. to prove that its researchers can make significant technical progress by picking “grand challenges” like its early chess foray. The new bid is based on three years of work by a team that has grown to 20 experts in fields like natural language processing, machine learning and information retrieval.

Under the rules of the match that the company has negotiated with the “Jeopardy!” producers, the computer will not have to emulate all human qualities. It will receive questions as electronic text. The human contestants will both see the text of each question and hear it spoken by the show’s host, Alex Trebek.

The computer will respond with a synthesized voice to answer questions and to choose follow-up categories. I.B.M. researchers said they planned to move a Blue Gene supercomputer to Los Angeles for the contest. To approximate the dimensions of the challenge faced by the human contestants, the computer will not be connected to the Internet, but will make its answers based on text that it has “read,” or processed and indexed, before the show.

There is some skepticism among researchers in the field about the effort. “To me it seems more like a demonstration than a grand challenge,” said Peter Norvig, a computer scientist who is director of research at Google. “This will explore lots of different capabilities, but it won’t change the way the field works.”

The I.B.M. researchers and “Jeopardy!” producers said they were considering what form their cybercontestant would take and what gender it would assume. One possibility would be to use an animated avatar that would appear on a computer display.

“We’ve only begun to talk about it,” said Harry Friedman, the executive producer of “Jeopardy!” “We all agree that it shouldn’t look like Robby the Robot.”

Mr. Friedman added that they were also thinking about whom the human contestants should be and were considering inviting Ken Jennings, the “Jeopardy!” contestant who won 74 consecutive times and collected $2.52 million in 2004.

I.B.M. will not reveal precisely how large the system’s internal database would be. The actual amount of information could be a significant fraction of the Web now indexed by Google, but artificial intelligence researchers said that having access to more information would not be the most significant key to improving the system’s performance.

Eric Nyberg, a computer scientist at Carnegie Mellon University, is collaborating with I.B.M. on research to devise computing systems capable of answering questions that are not limited to specific topics. The real difficulty, Dr. Nyberg said, is not searching a database but getting the computer to understand what it should be searching for.

The system must be able to deal with analogies, puns, double entendres and relationships like size and location, all at lightning speed.

In a demonstration match here at the I.B.M. laboratory against two researchers recently, Watson appeared to be both aggressive and competent, but also made the occasional puzzling blunder.

For example, given the statement, “Bordered by Syria and Israel, this small country is only 135 miles long and 35 miles wide,” Watson beat its human competitors by quickly answering, “What is Lebanon?”

Moments later, however, the program stumbled when it decided it had high confidence that a “sheet” was a fruit.

The way to deal with such problems, Dr. Ferrucci said, is to improve the program’s ability to understand the way “Jeopardy!” clues are offered. The complexity of the challenge is underscored by the subtlety involved in capturing the exact meaning of a spoken sentence. For example, the sentence “I never said she stole my money” can have seven different meanings depending on which word is stressed.

“We love those sentences,” Dr. Nyberg said. “Those are the ones we talk about when we’re sitting around having beers after work.”


*****

I'm not a big Jeopardy fan, but I think AI is a very interesting topic. This should be pretty interesting to follow.

I also like the chit chat meme re/ "I never said she stole my money". Never really noticed how difficult that would be for a computer. At least when you guys riff on it, you underline the relevant words.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Diane Close
United States
Twin Cities
Minnesota
flag msg tools
badge
Avatar
mbmbmbmbmb
Rubric wrote:
At least when you guys riff on it, you underline the relevant words.


Yes, we do underline the relevant words!
3 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Timothy Hunt
United States
St Louis
Missouri
flag msg tools
badge
Avatar
mbmbmbmbmb
Yes, we do underline the relevant words!
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Mystery McMysteryface
United States
Florida
flag msg tools
Avatar
mbmbmbmbmb
Yes, we do underline the relevant words!
2 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ian Klinck
Canada
Toronto
Ontario
flag msg tools
Avatar
mbmbmbmbmb
Yes, we do underline the relevant words!





Doh!
7 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Richard S
United States
Rensselaer
New York
flag msg tools
badge
Avatar
mbmbmbmbmb
iklinck wrote:
Yes, we do underline the relevant words!





Doh!


Yes, we can.







The question is will we?
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Paul Szilagyi
United States
Parma
Ohio
flag msg tools
Avatar
mbmbmbmbmb
See, as an avid Jeopardy fan, my biggest gripe wouldn't even be the massive amount of data that can be "read" into the computer's memory, it would be the speed with which the computer could operate its "signaling device".

If the computer "buzzes in" with the right answer 85% of the time, and it's a fraction of a second faster than its human counterparts, there'll be no contest.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Canada
Abby
BC West Coast
flag msg tools
publisher
badge
Avatar
mbmbmbmbmb
I just hope it remembers to answer in the form of a question.

Seriously tho, I think the A.I. has a shot. I've worked with ALICE bots before and they're not bad at this sort of thing.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Ken H.
United States
Amherst
Ohio
flag msg tools
badge
Avatar
mbmbmbmbmb
ZeroZilla wrote:
See, as an avid Jeopardy fan, my biggest gripe wouldn't even be the massive amount of data that can be "read" into the computer's memory, it would be the speed with which the computer could operate its "signaling device".


Yeah, I was wondering if the AI would check it's data FIRST, to see if it knows the answer, and then buzz in. Or, if it will just automatically buzz in as fast as possible and then look up the answer with the extra few seconds you get at that point.

I also though it's a bit of a cop out that they will feed it the questions in electronic form. A much more impressive accomplishment would be to have it interpret Alex Trebec's voice.
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Paul Szilagyi
United States
Parma
Ohio
flag msg tools
Avatar
mbmbmbmbmb
That'll be a feature of the WatsonPRO model, coming out about six months after the original, and costing twice as much.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
The Steak Fairy
United States
Columbia
South Carolina
flag msg tools
Games? People still play games??
badge
Specious arguments are not proof of trollish intent.
Avatar
mbmbmbmbmb
I've had zero respect for IBM since they named their second-generation WSYIWYG system "BETTY" which meant "Better than YODA." YODA was of course the name of the first generation of the system.
 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Drew Heath
United States
Galesburg
Illinois
flag msg tools
badge
Avatar
mbmbmbmbmb
There's a whole slew of common Jeopardy categories that are too esoteric for a database to have a chance though...

Category: Internet Memes

A. Lesbians at Starbucks

Spoiler (click to reveal)
Q: What is "2 Girls, 1 Cup"?


No AI is ever gonna get that...
1 
 Thumb up
 tip
 Hide
  • [+] Dice rolls
Front Page | Welcome | Contact | Privacy Policy | Terms of Service | Advertise | Support BGG | Feeds RSS
Geekdo, BoardGameGeek, the Geekdo logo, and the BoardGameGeek logo are trademarks of BoardGameGeek, LLC.