AUDIE CORNISH, HOST:
It's been billed as a breakthrough in artificial intelligence - a computer in England has fooled human beings into thinking it is a 13-year-old boy, not by the way it looks, but by the way it chats through instant messaging. NPR's Aarti Shahani reports some analysts are unimpressed by this digital trickery.
AARTI SHAHANI, BYLINE: A team from the University of Reading put the computer through a test - it's called the Turing Test - and to pass it, the computer had to fool people.
WILLIAM COHEN: So this did happen. So the computer fooled some human judges. It fooled a third of them.
SHAHANI: William Cohen, a computer scientist at Carnegie Mellon, followed the competition with curiosity. The test was different from the way it was originally conceived. Back in the 1950s, the assignment was for the computer to answer fairly adult questions. Say...
COHEN: questions about poetry, like, you know, shall I compare you to a summer's day?
SHAHANI: The Russian-made program decided to make the computer a 13-year-old boy from Ukraine who goes by the name Eugene Goostman. And as a kid who speaks English only as a second language, Eugene managed to send whimsical messages in five-minute-long chats, and also to lower the judge's expectations.
COHEN: You're also sort of coming up with, you know, a plausible way of not exposing the weaknesses that a computer program is going to have. So, you know, it's very impressive to sort of see how clever these programs can be.
SHAHANI: Computer scientist, Scott Aaronson at MIT, doesn't think Eugene is that clever.
SCOTT AARONSON: It doesn't seem like this bot does better than any of the other chat bots for the last 50 years.
SHAHANI: He remembers a chat bot back in the 1960s named Eliza, it pretended to be a psychoanalyst. People would pour their hearts out to it and it would mimic back the last phrase in the form of a question. Sound familiar? Aaronson just tried out a version of Eugene that he found on the web, and he got nonsense responses.
AARONSON: For example, you know, when I asked it whether a shoebox is bigger than Mt. Everest, it said, I can't make a choice right now. I should think it out later.
SHAHANI: Then Eugene tried to get cutesy in the face of another factual question.
AARONSON: How many legs does a camel have? It said something between two and four - maybe three - smiley face.
SHAHANI: Researchers have not yet released the transcripts from this weekend's test, but Peter Norvig, Director of Research at Google, is skeptical of its commercial worth. Though he does note the entertainment value of a chat bot could be improving.
PETER NORVIG: It's progress to go from a horrible first date to a good first date - so, you know, I'm not saying there's nothing there.
SHAHANI: Norvig is not moving up the date he expects humans to surrender to our computer overlords. Aarti Shahani, NPR News.
CORNISH: You are listening to ALL THINGS CONSIDERED from NPR News. Transcript provided by NPR, Copyright NPR.
