Turing test

From Wikipedia, the free encyclopedia

Jump to: navigation, search
For the Doctor Who novel named after the test, see The Turing Test (novel).
For the opera named after the test, see under the composer, Julian Wagstaff.

The Turing test is a proposal for a test of a machine's capability to demonstrate intelligence. Described by Alan Turing in the 1950 paper "Computing machinery and intelligence," it proceeds as follows: a human judge engages in a natural language conversation with one human and one machine, each of which try to appear human; if the judge cannot reliably tell which is which, then the machine is said to pass the test. In order to keep the test setting simple and universal (to explicitly test the linguistic capability of the machine instead of its ability to render words into audio), the conversation is usually limited to a text-only channel such as a teletype machine as Turing suggested or, more recently, IRC or instant messaging.

Contents

[edit] History

The test was inspired by a party game known as the "Imitation Game", in which a man and a woman go into separate rooms, and guests try to tell them apart by writing a series of questions and reading the typewritten answers sent back. In this game, both the man and the woman aim to convince the guests that they are the other. Turing proposed a test employing the imitation game as follows: "We now ask the question, 'What will happen when a machine takes the part of A in this game?' Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? These questions replace our original, 'Can machines think?'"[1] Later in the paper he suggested an "equivalent" alternative formulation involving a judge conversing only with a computer and a man.

Turing originally proposed the test in order to try to replace the emotionally charged and (for him) meaningless question "Can machines think?" with a more well-defined one. The advantage of the new question, he said, was that it "drew a fairly sharp line between the physical and intellectual capacities of a man."[2]

Turing's paper considered nine common objections, which include all the major arguments against artificial intelligence that have been raised in the years since his paper was first published. (See Computing Machinery and Intelligence.)[3]

[edit] Strengths of the test

The power of the Turing test derives from the fact that it is possible to talk about anything. Turing wrote "the question and answer method seems to be suitable for introducing almost any one of the fields of human endeavor that we wish to include."[4] John Haugeland adds that "understanding the words is not enough; you have to understand the topic as well."[5]

In order to pass a well designed Turing test, the machine would have to use natural language, to reason, to have knowledge and to learn. The test can be extended to include video input, as well as a "hatch" through which objects can be passed, and this would force the machine to demonstrate the skill of vision and robotics as well. Together these represent almost all the major problems of artificial intelligence.[6]

[edit] Weaknesses of the test

The test has been criticized on several grounds.

[edit] Human intelligence vs. intelligence in general

Image:Weakness of Turing test 1.jpg

The test is explicitly anthropomorphic. It only tests if the subject resembles a human being. It will fail to test for intelligence under two circumstances:

  • It tests for many behaviors that we may not consider intelligent, such as the susceptibility to insults or the temptation to lie. A machine may very well be intelligent without being able to chat exactly like a human.
  • It fails to capture the general properties of intelligence, such as the ability to solve difficult problems or come up with original insights. If a machine can solve a difficult problem that no person could solve, it would, in principle, fail the test.

Russell and Norvig argue that the anthropomorphism of the test prevents it from being truly useful for the task of engineering intelligent machines. They write: "Aeronautical engineering texts do not define the goal of their field as 'making machines that fly so exactly like pigeons that they can fool other pigeons.'"[7]

[edit] Real intelligence vs. simulated intelligence

The test is also explicitly behaviorist or functionalist: it only tests how the subject acts.

A machine passing the Turing test may be able to simulate human conversational behaviour but the machine might just follow some cleverly devised rules. Two famous examples of this line of argument against the Turing test are John Searle's Chinese room argument and Ned Block's Blockhead argument.

Even if the Turing test is a good operational definition of intelligence, it may not indicate that the machine has consciousness, or that it has intentionality. Perhaps intelligence and consciousness, for example, are such that neither one necessarily implies the other. In that case, the Turing test might fail to capture one of the key differences between intelligent machines and intelligent people.

In the words of science popularizer Larry Gonick, "I personally disagree with this criterion, on the grounds that a simulation is not the real thing."[citation needed]

[edit] Imitation game vs. standard Turing test

There has been some controversy over which of the alternate formulations of the test Turing intended. (Moor, 2003) Sterret argues that two distinct tests can be extracted from Turing's 1950 paper, and that, pace Turing's remark, they are not equivalent. The test that employs the party game and compares frequencies of success in the game is referred to as the "Original Imitation Game Test" whereas the test consisting of a human judge conversing with a human and a machine is referred to as the "Standard Turing Test". Sterrett agrees that the Standard Turing Test (STT) has the problems its critics cite, but argues that, in contrast, the Original Imitation Game Test (OIG Test) so defined is immune to many of them, due to a crucial difference: the OIG Test, unlike the STT, does not make similarity to a human performance the criterion of the test, even though it employs a human performance in setting a criterion for machine intelligence. A man can fail the OIG Test, but it is argued that this is a virtue of a test of intelligence if failure indicates a lack of resourcefulness. It is argued that the OIG Test requires the resourcefulness associated with intelligence and not merely "simulation of human conversational behaviour". The general structure of the OIG Test could even be used with nonverbal versions of imitation games (Sterrett 2000).

Still other writers (Genova (1994), Hayes and Ford (1995), Heil (1998), Dreyfus (1979)) have interpreted Turing to be proposing that the imitation game itself is the test, without specifying how to take into account Turing's statement that the test he proposed using the party version of the imitation game is based upon a criterion of comparative frequency of success in that imitation game, rather than a capacity to succeed at one round of the game.

[edit] Predictions and tests

Turing predicted that machines would eventually be able to pass the test. In fact, he estimated that by the year 2000, machines with 109 bits (about 119.2 MiB) of memory would be able to fool 30% of human judges during a 5-minute test. He also predicted that people would then no longer consider the phrase "thinking machine" contradictory. He further predicted that machine learning would be an important part of building powerful machines, a claim which is considered to be plausible by contemporary researchers in Artificial intelligence.

By extrapolating an exponential growth of technology over several decades, futurist Ray Kurzweil predicted that Turing-test-capable computers would be manufactured around the year 2020, roughly speaking. See the Moore's Law article and the references therein for discussions of the plausibility of this argument.

As of 2007, no computer has passed the Turing test as such. Simple conversational programs such as ELIZA have fooled people into believing they are talking to another human being, such as in an informal experiment termed AOLiza. However, such "successes" are not the same as a Turing Test. Most obviously, the human party in the conversation has no reason to suspect they are talking to anything other than a human, whereas in a real Turing test the questioner is actively trying to determine the nature of the entity they are chatting with. Documented cases are usually in environments such as Internet Relay Chat where conversation is sometimes stilted and meaningless, and in which no understanding of a conversation is necessary. Additionally, many internet relay chat participants use English as a second or third language, thus making it even more likely that they would assume that an unintelligent comment by the conversational program is simply something they have misunderstood, and don't recognize the very non-human errors they make. See ELIZA effect.

The Loebner prize is an annual competition to determine the best Turing test competitors. Although they award an annual prize for the computer system that, in the judges' opinions, demonstrates the "most human" conversational behaviour (with learning AI Jabberwacky winning in 2005 and 2006, and A.L.I.C.E. before that), they have an additional prize for a system that in their opinion passes a Turing test. This second prize has not yet been awarded. The creators of Jabberwacky have proposed a personal Turing Test: the ability to pass the imitation test while attempting to specifically imitate the human player, with whom the AI will have conversed at length before the test. [1].

Trying to pass the Turing test in its full generality is not, as of 2005, an active focus of much mainstream academic or commercial effort. Current research in AI-related fields is aimed at more modest and specific goals.

There is an ongoing $10,000 bet at the Long Bet Project between Mitch Kapor and Ray Kurzweil about the question whether a computer will pass a Turing Test by the year 2029. The bet specifies the Turing Test in some detail.

[edit] Variations of the Turing test

A modification of the Turing test, where the objective or one or more of the roles have been reversed between computers and humans, is termed a reverse Turing test.

Another variation of the Turing test is described as the Subject matter expert Turing test where a computer's response cannot be distinguished from an expert in a given field.

As brain and body scanning techniques improve it may also be possible to replicate the essential data elements of a person to a computer system.[citation needed] The Immortality test variation of the Turing test would determine if a person's essential character is reproduced with enough fidelity to make it impossible to distinguish a reproduction of a person from the original person.

The Minimum Intelligent Signal Test proposed by Chris McKinstry, is another variation of Turing's test, but where only binary responses are permitted. It is typically used to gather statistical data against which the performance of artificial intelligence programs may be measured.

Another variation of the reverse Turing test is implied in the work of psychoanalyst Wilfred Bion (1979), who was particularly fascinated by the "storm" that resulted from the encounter of one mind by another. Carrying this idea forward, R. D. Hinshelwood (2001) described the mind as a "mind recognizing apparatus," noting that this might be some sort of "supplement" to the Turing test. To make this more explicit, the challenge would be for the computer to be able to determine if it were interacting with a human or another computer. This is an extension of the original question Turing was attempting to answer, but would, perhaps, be a high enough standard to define a machine that could "think" in a way we typically define as characteristically human.

Another variation is the Meta Turing test, in which the subject being tested (for example a computer) is classified as intelligent if it itself has created something that the subject itself wants to test for intelligence

[edit] Practical Applications

Russell and Norvig note that "AI researchers have devoted little attention to passing the Turing Test",[8] since there are easier ways to test their programs: by giving them a task directly, rather than through the roundabout method of first posing a question in a chat room populated with machines and people. Turing never intended his test to be used as a real, day-to-day measure of the intelligence of AI programs. He wanted to provide a clear and understandable example to help us discuss the philosophy of artificial intelligence.[9]

Real Turing tests, such as the Loebner prize, don't usually force programs to demonstrate the full range of intelligence and are reserved for testing chatterbot programs. But it is still very hard.

CAPTCHA is a form of Reverse Turing test. When, for example, logging on to a website, the user is presented with a word or number in a distorted graphic image and asked to enter it. If the value entered does not match what is expected, then the user is rejected. This is intended to prevent automated systems from using the site. The assumption is that software sufficiently sophisticated to read the distorted image accurately either does not exist or is not available to the average user, so any system that is able to do so must be a human being.

Also another usefullness of the turing test, is to verify if players on multiplayer games are real players or computer programs acting like a player, the only way found for now to circunvent the test is to create a way to call a human to take over the reins of the conversation before the other person notices.

[edit] References in Popular Fiction

This section may stray from the topic of the article into the topic of another article, artificial intelligence in fiction.
Please help improve this section or discuss this issue on the talk page. (help)
Arthur C. Clarke used the term Turing test in the science-fiction novel 2001: A Space Odyssey (1968), where it is applied to the computer HAL 9000.

Merlin's Ghostwheel project in Roger Zelazny's Amber is mentioned to be capable of passing the Turing Test.

In the Philip K. Dick novel Do Androids Dream of Electric Sheep? and in the Ridley Scott movie Blade Runner, replicants are subjected to a Voight-Kampf test, intended to discover whether a person is a real human or a robot based on their reaction to a series of hypothetical situations intended to provoke sympathetic, or at least empathetic, responses. In the post-apocalyptic setting of the novel, computers were able to pass a Turing Test long ago.

In William Gibson's novel Neuromancer there is a government-run "Turing Registry" which registers and monitors AIs so that they do not become too smart or powerful.

Ian McDonald's 'River of Gods is all about artificial intelligence and constantly references the Turing test. AI's are constantly tested so that any 3.0 AI, or an AI with human intelligence can be destroyed. However, as any truly intelligent AI knows, any AI smart enough to pass a Turing test is smart enough to fail it.

[edit] See also

[edit] Notes

  1. ^ Turing, p. 1
  2. ^ Turing 1950, p. 2
  3. ^ Turing 1950 and see Russell & Norvig 2003, p. 948 where comment "Turing examined a wide variety of possible objections to the possibility of intelligent machines, including virtually all of those that have been raised in the half century since his paper appeared."
  4. ^ Turing 1950 under "Critique of the New Problem"
  5. ^ Haugeland 1985, p. 8
  6. ^ Russell and Norvig write "These six disciplines represent most of AI". Russell & Norvig 2003, p. 3
  7. ^ Russell & Norvig 2003, p. 3
  8. ^ Russell & Norvig 2003, p. 3
  9. ^ Turing 1950 under The Imitation Game, where he writes "Instead of attempting such a definition I shall replace the question by another, which is closely related to it and is expressed in relatively unambiguous words."

[edit] References

[edit] External links

cs:Turingův test da:Turing-test de:Turing-Test et:Turingi test es:Prueba de Turing eo:Testo de Turing fa:آزمون تورینگ fr:Test de Turing gl:Proba de Turing ko:튜링 테스트 it:Test di Turing he:מבחן טיורינג lv:Tjūringa tests hu:Turing-teszt nl:Turing-test ja:チューリング・テスト no:Turingtest pl:Test Turinga pt:Teste de Turing ro:Testul Turing ru:Тест Тьюринга simple:Turing test sk:Turingov test sv:Turingtest th:การทดสอบของทัวริง tr:Turing Testi uk:Тест Тьюрінга zh:圖靈試驗

Retrieved from "http://india.smashits.com/wikipedia/Turing_test"
Views
Personal tools

Toolbox