h2g2 The Hitchhiker's Guide to the Galaxy: Earth Edition

The Turing test is inherrently flawed because:
(a) It assumes that intelligence is either present or absent, and doesn't allow for degrees of intelligence.
(b) It assumes that Humans are a benchmark of intelligence - but doesn't nail that concept more precisely. People vary to a huge degree in their own intelligence - from the blinding to the depressing.
(c) It assumes conversations show intelligence. I find that a conversation is a good way as any to find a lack of intelligence .

...on the other hand, I can't think of a better test myself, so no disrespect to the man!

These arguments also undermine Searle's Chinese Room Thought Experiment used to attack the possibility of Artificial Intelligence or Neural Networks every creating anything sentient.

The key issue is that humans are seemingly incapable defining intelligence outside of what we experience from each other and pets.An alien or computer may have 'intelligence' in a sense, but not one we understand.

Maybe computers already do...

'ChatBots' are not intelligent, yet they can score 20% on the Test.
When Tomorrows World had the Turing Test on their lab program, the person scored 70%.

I feel that Turing's original statement still holds - define Intelligence, and we will be able to measure it.

I hadn't heard it expressed that way but I guess it all relates to his Universal Machine concept of anything which could be expressed mathematically will be solved by the machine if it has enough time and tape (memory). Or something like that...

The problem remains expressing the problems... i.e. what is the question? Though in this case we don't know the answer like Arthur did.

I like the idea of AI... and I feel it is definately possible.

A computer is in base 2: on and off.
A person is in base 4: the four bases of DNA.

Talking of DNA...

well - it depends - our program is in base 4, but our input, output, and storage are all in a non-based format.....

My personal irritation at the test is this - if I took the turing test in China I would probably very rapidly fail. And english and chinese people are both human. If you want to know if computers are intelligent, you're going to have to meet them halfway - by the time they can go all the way by themselves, they'll be more intelligent...

Yes: but we become more than just our genes - but at the beginning we are only our genes. We become more than our programming.

As can computers - programs which learn from experience are here *now*.

Absolutely - I just gave mine a 'reet good kicking', which may seem harsh, but it won't be giving me any of that "file not found" cheek any more

hi - I just had a lecture on the subject, so I know a little more now...

The original test compared the success rate of a man pretending to be a woman (or vica versa) with the success rate of a machine pretending to be human. It measured this over a large number of trials, each trial being a fixed time length.

{the underlying assumption is that the difference between man and woman is less than that between machine and human}

So, if a man/woman fooled the judge 45% of the time, while a computer fooled the judge 15% of the time, then the computer is less intelligent than the man/woman. If a cockroach fooled the judge never at all, then the cockroach is less intelligent than the computer.

In fact, in the really original test you placed a man in competition with a woman, (or a machine in competition with a human) and they would take turns to try and help the judge, or hinder the judge. So in fact, the conversation would be 3-way. Sometimes both would be trying to pretend to be a machine, sometimes both would be trying to pretend to be a human. {or man/woman, respectively}

The judge could be human or machine - provided it has some accuracy at performing the task of distinguishing the two. The higher the intelligence of the judge, the more difficult the test becomes, for both man/woman, and human/machine.

Similarly Turing didn't intend the test to be a measure of intelligence - just a measure of how good machines where at performing a certain task. It just so happens to be a task that many people believe requires intelligence - but his aim appears to habe had more to do with defining a measure for how well the AI research has progressed.

Incidentally, Turing predicted that by 2000 a human judge would have only a 70% success rate over a period of five minutes (the man/woman success rate was around 60%). He was wrong...

the turing test is a one-way test. it would be phenomenally difficult for a machine to pass the full-blown unrestricted version of the test - so difficult i think it would be fair to say the machine was intelligent. failing the test wouldn't prove anything - just cos you can't talk doesn't mean you can't think.

you could argue about what "intelligence" actually is, but that would be whiney philosophy undergrad type of conversation. afterall who gives a toss if the machine is "truly" intelligent - if it passes the test, it passes the test, and that's pretty damn impressive.

I was fairly sure that the original paper described it containing three participants - human, machine, judge. I don't see, incidentally, how this would be any harder to pass than the one-way version.

Agreed - failing either test doesn't prove that something isn't intelligent - if I handcuffed and gagged you you'd fail the test, but you wouldn't have stopped being intelligent.

It still only comes down to one question:

what is intelligence?

For the answer to 'what is intelligence' you should look at psychology, starting with Binet and so on. The argument as to whether it is a single trait (Allport?) or a combination goes on. It is undeniable that individuals who perform well in one kind of 'intelligence' test (as long as it is not too culture dependant) tend to do well over a range of tests. Intelligence is a very emotive subject and evidence is frequently buried or pooh-poohed by the establishment in the interests of promoting a publicly acceptable image of science. It is difficult to think of 'intelligence' without self-consciousness (Piaget et al.) so perhaps the question of artificial self-consciousness is more critical?

For the answer to 'what is intelligence' you should look at psychology, starting with Binet and so on. The argument as to whether it is a single trait (Allport?) or a combination goes on. It is undeniable that individuals who perform well in one kind of 'intelligence' test (as long as it is not too culture dependant) tend to do well over a range of tests. Intelligence is a very emotive subject and evidence is frequently buried or pooh-poohed by the establishment in the interests of promoting a publicly acceptable image of science. It is difficult to think of 'intelligence' without self-consciousness (Piaget et al.) so perhaps the question of artificial self-consciousness is more critical?

To understand the concept of self-consciousness you need to be intelligent... a real problem

All right, I heard someone mention the Chinese Room, so it gives me an excuse to present one of the best refutations of any logical fallacy I've ever seen!

The Chinese Room was a idea that John Searle came up with to defeat
the Turing Test and to support his view of AI. I have to confess that I think it is really flawed, and my opinion of Searle is not improved by the fact that he and Dennett behave like children towards each other in major philosophical journals and even in books, and not too maturely towards their doctoral candidates (see the link to Dr. Ina Roy on my homepage).

Basically, a person who knows English, or any other language totally unrelated to Chinese, is locked in a room. They are given a long list of symbols/combinations of symbols, and appropriate responses to such symbols, generated by a native Chinese speaker. Then, they are fed stories in Chinese and comprehension questions, and they answer according to the lexicon they were given, causing outsiders to think they are a native Chinese speaker.

Searle's conclusion is that the Turing test is thus flawed, and it shows nothing about whether a machine is conscious. Douglas R. Hofstadter provides an incredible refutation. The first problem with this theory is one of physical & time scale: that list would have to be so long, and the time it took to access it would be so long, that the questions would never get answered. This is surmountable, though, given the law in computer science that processing speed & memory capacity double every six months or whatever it is. A bigger fallacy, though, is that Searle is asking you to map your consciousness onto the HUMAN in the room--not the whole system. The machine by necessity includes the whole system, and so does a human brain. Therefore, any machine capable of passing the Turing Test might not entirely be conscious, but it would have achieved a major step towards such.

Kathy

In fact, the list would almost certainly have to be infinite. After all, the set of all sequences of related questions would be infinite...

The entry on Searle's Argument (which I dislike as it is far too pro-Searle) is at http://www.h2g2.com/A399549. If you wish to see it modified to be more balanced, do go and post there, and hopefully the sub-ed will change the tone somewhat.

Okay, everyone go check out my posts there under "Why Searle is Wrong." And if you think he's wrong, feel free to join me in my crusade!

Kathy